Electric power artificial intelligence model system and working method

Abstract

The present application provides an electric power artificial intelligence model system and a working method. The electric power artificial intelligence model system comprises an algorithm model database, a user data management module, an intelligent recommendation module, an intelligent transaction module and a feedback and optimization module. The intelligent transaction module is configured to perform automatic charging on the basis of an algorithm model selected by a user and according to a use condition of the user, determine a final settlement cost on the basis of the identification effect of the algorithm model and a scoring result fed back by the user, and automatically generate a transaction order.

Description

Power artificial intelligence model system and working method

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on October 20, 2023, with application number 202311363718.2, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] This application belongs to the field of electric power artificial intelligence technology, for example, it relates to an electric power artificial intelligence model system and working method. Background Art

[0003] The power AI model system integrates multiple model algorithms and provides modular algorithm services to businesses and individuals based on the system's own hardware resources. The platform integrates a large number of AI algorithm models. For example, within the power sector's AI operations platform, it offers a variety of AI algorithm models for equipment inspection, meter data recognition, and power equipment fault detection. Users can use the platform to trial, trade, and evaluate different algorithm models.

[0004] However, power AI model systems often have the following problems:

[0005] (1) In the related art, the power artificial intelligence model system will match the recommendation algorithm model for the user based on the keywords input by the user. However, the keywords input by the user may contain multiple different keywords, and the importance of these keywords is different. The related art does not consider the proportion of the importance of different keywords. Therefore, it may lead to inaccurate recommendation results and the inability to achieve accurate recommendations.

[0006] (2) Every model cannot be used without input data. When users call or use the model, they need to upload their own data to the system server. These data are unique to the enterprise or individual and have confidentiality requirements. Due to the openness of computer networks and the interconnectedness of terminals, the security of personal data is difficult to guarantee. In addition, the openness of the system makes it impossible to effectively guarantee the data security of the model itself, and it is impossible to prevent malicious attacks, dissemination and use of the model by the outside world. Therefore, it may lead to the leakage of model data and cause unpredictable economic losses.

[0007] (3) In related technologies, the system's billing methods are often divided into two modes: fixed billing and single billing. In the single billing mode, cumulative billing is performed based on the number of times the user calls the model. However, it is impossible to identify whether the user successfully uses the model after calling it or whether the user encounters anomalies during the use of the model. As long as the user uses the model, he or she needs to pay the same fee. This single billing method cannot perform differentiated fee calculations based on user needs and actual usage. At the same time, the model's effectiveness and user feedback are not taken into account when making the final settlement of fees, which can easily lead to unreasonable and inaccurate billing and settlement, affecting user experience.

[0008] (4) After the algorithm model transaction is completed, users usually evaluate the effectiveness of the model. The system can optimize and adjust the model based on the user's feedback and evaluation information; however, it cannot be guaranteed that all feedback and evaluation information is true and valid (for example, when users submit feedback at will after trying the model), which will cause a large amount of noise and interference data, affecting the final effect evaluation of the model.

[0009] Summary of the Invention

[0010] This application provides an electric power artificial intelligence model system and working method. The system can make accurate model intelligent recommendations to users based on user demand information and model weights, and at the same time perform differentiated billing and tiered settlement for users' use of algorithm models. It also conducts confidence assessment on user feedback evaluation information to obtain accurate user evaluation of the effectiveness of the algorithm model.

[0011] A power artificial intelligence model system includes: an algorithm model database configured to store power artificial intelligence algorithm models; a user data management module configured to obtain basic user information, historical usage trace information of the algorithm model, and operating habit information of the user, and store the information in the user database; an intelligent recommendation module configured to construct a keyword feature vector based on user demand information input, calculate similarity between the keyword feature vector and the feature vector matrix of multiple algorithm models in the algorithm model database, obtain a list of recommended algorithm models, and automatically estimate the usage fee of each recommended algorithm model; wherein the elements in the feature vector of each algorithm model are the product of the model label and the label weight; an intelligent transaction module configured to automatically bill the user based on the algorithm model selected by the user and the user's usage, determine the final settlement fee based on the recognition effect of the algorithm model and the user feedback rating results, and automatically generate a transaction order; and a feedback optimization module configured to receive user feedback evaluation information on the algorithm model, evaluate the confidence of the feedback evaluation information based on the user's operational behavior data during the feedback evaluation and the user's operational habit model, and optimize the algorithm model based on all feedback evaluation information of the algorithm model and the confidence evaluation results.

[0012] In one implementation, the system also includes: a model calling module, which is configured to respond to a user's model calling request, compress the target algorithm model, encrypt the target algorithm model using a temporary key, and transmit the encrypted algorithm model to the user.

[0013] In one implementation, the model calling module is configured to compress the target algorithm model in the following manner, encrypt the target algorithm model with a temporary key, and then transmit the encrypted algorithm model to the user: decrypt based on the temporary key sent by the user to obtain user identification information and the target algorithm model that the user wants to call; perform user identity authentication based on the user identification information, and after the verification is passed, obtain the key of the target algorithm model and decrypt it, compress the decrypted target algorithm model, and then encrypt the target algorithm model based on the temporary key and transmit it to the user; wherein, the user generates a temporary key during this login to the system based on the login information, the current system time and the user identification information.

[0014] In one implementation, the model calling module is further configured to replace the current system time with the time period in which the user uses the model, and update the temporary key; and use the updated key to encrypt the compressed target algorithm model.

[0015] In one implementation, the intelligent recommendation module is configured to construct a keyword feature vector based on the demand information input by the user in the following manner, calculate the similarity between the keyword feature vector and the feature vector matrix of multiple algorithm models in the algorithm model database, and obtain a recommended algorithm model list: extract keywords from the user's business demand information and quantify them to form a keyword feature vector; construct a feature vector for each algorithm model based on the labels of multiple algorithm models in the algorithm model database and the weights of the labels; calculate the similarity between the keyword feature vector and the feature vector of each algorithm model; sort the matching algorithm models based on the similarity, select a set number of algorithm models with the highest similarity, and obtain a recommended algorithm model list.

[0016] In one implementation, the intelligent recommendation module is further configured to determine the weight corresponding to the label of each algorithm model in the following manner: obtaining keywords related to the label of the algorithm model in historical browsing records, search records or demand records to form a keyword library; using the word frequency-inverse document frequency method to calculate the word frequency and inverse document frequency of the algorithm model label in the keyword library to obtain the weight value corresponding to the label of the algorithm model.

[0017] In one implementation, the similarity is represented by the cosine value of the angle between two vectors.

[0018] In one implementation, the intelligent recommendation module is configured to automatically estimate the usage fee of each recommended algorithm model in the following manner: obtain the number of requests, data processing volume, model call time and resource usage time data of a type of algorithm model within a historical set time period, and preprocess them as historical feature data; based on the regression model, use the PolynomialFeatures method to perform diversified processing on the historical feature data; use the processed historical feature data to train the polynomial regression model to obtain a billing prediction model for this type of algorithm model; obtain a billing prediction model that belongs to the same type as the recommended algorithm model, and predict the usage fee of the model based on the historical usage data of the recommended algorithm model.

[0019] In one implementation, the intelligent transaction module is configured to automatically bill based on the user's usage in the following manner: calculate the initial cost based on preset cost data and attribute data; obtain user type and usage frequency information to determine the discount ratio when the user uses the algorithm model; obtain the graphics processing unit (GPU) resource usage time, central processing unit (CPU) resource usage time, storage resource usage time and the time when the task abnormality occurs when the user uses the algorithm model this time, and calculate the usage fee; calculate the final fee for using the algorithm model this time based on the initial cost, usage fee and discount ratio.

[0020] In one implementation, the intelligent transaction module is configured to determine the final settlement fee by combining the recognition effect of the algorithm model and the scoring results of user feedback in the following manner: when settling fees, obtain the user's scoring results of the selected algorithm model calling process and the recognition effect of the selected algorithm model; normalize the scoring results and the recognition effect, and perform weighted summation to obtain the final effectiveness value; determine the proportion of fee settlement based on the relationship between the final effectiveness value and the set target value; and perform the final fee settlement based on the final fee obtained by billing and the ratio of the fee settlement.

[0021] In one implementation, the intelligent trading module is further configured to, in response to the difference between the user feedback rating result and the model rating mean being greater than a set threshold, lower the weight of the user rating result in calculating the final performance value according to predetermined rules.

[0022] In one implementation, the final cost of using the selected algorithm model this time is: final cost information = (basic cost + GPU unit time cost * time + CPU unit time cost * time + storage size * each storage unit consumption cost - abnormal time * ratio * fixed cost) * manually configured weight * discount ratio; wherein, the manually configured weight is a set value.

[0023] In one implementation, the operational behavior data of the user during this feedback evaluation is all the operational data of the user after logging into the system, including the operational behavior data of the keyboard and mouse, and the operational behavior data of the called algorithm model.

[0024] In one implementation, the user's operation habit model includes the user's various operation behaviors and the user's quantitative value for each operation behavior, wherein the user is guided by a tutorial to perform behavioral operations of the usage model and the operation model to establish operation habits; the user's quantitative value for each operation behavior is determined based on the user's historical operation behavior after the tutorial guidance.

[0025] In one implementation, the feedback optimization module is configured to evaluate the confidence of the feedback evaluation information in the following manner: select the user's mouse click frequency, drag number and drag distance, keyboard key frequency, and page dwell time as evaluation indicators, and assign weights to each operation behavior; calculate the similarity between the operation behavior data during this feedback evaluation and the corresponding operation behavior quantification value in the operation habit model; perform weighted average calculation on the similarity of all operation behavior data to obtain the confidence of this feedback information.

[0026] In one implementation, the feedback optimization module is further configured to perform a secondary evaluation after evaluating the confidence of the current model feedback evaluation information; the feedback optimization module is configured to perform the secondary evaluation in the following manner: multiple anchor indicators are set in the questionnaire of the feedback evaluation information, and for the first feedback information to be evaluated, whether the current feedback information is reliable is judged based on the user's feedback on the multiple anchor indicators, and the current feedback information is unreliable and the confidence is low.

[0027] In one implementation, it also includes: a sample database, which is configured to store algorithm model application instance data in different scenarios, and the algorithm model application instance data includes: application scenarios, algorithm models, model application effects, usage fees and model optimization data.

[0028] A working method of an electric power artificial intelligence model system comprises: in response to a user's algorithm model call request, obtaining user basic information, user's historical usage trace information of the algorithm model, and operation habit information from a user database; constructing a keyword feature vector based on the demand information input by the user, calculating the similarity between the keyword feature vector and the feature vector matrix of multiple algorithm models in the algorithm model database, obtaining a recommended algorithm model list, and automatically estimating the usage fee of each recommended algorithm model; wherein the element in the feature vector of each algorithm model is the product of the model label and the label weight; based on the algorithm model selected by the user, automatically billing according to the user's usage, combining the algorithm model recognition effect and the user feedback rating result, determining the final settlement fee, and automatically generating a transaction order; receiving the user's feedback evaluation information on the algorithm model, based on the user's operation behavior data during this feedback evaluation, using the user's operation habit model, evaluating the confidence of this feedback evaluation information, and optimizing the algorithm model based on all the feedback evaluation information of the algorithm model and its confidence evaluation results.

[0029] In one implementation, after the confidence level of the model feedback evaluation information is evaluated, a secondary evaluation is performed: multiple anchor indicators are set in the questionnaire of the feedback evaluation information. For the feedback information to be evaluated, the reliability of the feedback information is judged based on the user's feedback on the multiple anchor indicators. If the feedback information is unreliable, the confidence level is low.

[0030] In one implementation, the method further includes: responding to a model call request from a user, decrypting based on a temporary key sent by the user to obtain user identification information and a target algorithm model that the user wants to call; performing user identity authentication based on the user identification information, and after the verification is passed, obtaining the key of the target algorithm model and decrypting it, compressing the decrypted target algorithm model, and then encrypting the target algorithm model based on the temporary key and transmitting it to the user; wherein the user generates a temporary key during this login to the system based on the login information, the current system time and the user identification information. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a schematic diagram of the structure of an electric power artificial intelligence model system provided in an embodiment of the present application;

[0032] FIG2 is a schematic diagram of a billing method provided in an embodiment of the present application;

[0033] FIG3 is a schematic diagram of the working process of an electric power artificial intelligence model system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0036] Example 1

[0037] According to Figure 1, this embodiment provides an electric power artificial intelligence model system, and the entire system is divided into an application layer, a platform layer, a network layer, and a resource layer; among them, the application layer includes the portal of the artificial intelligence operation system, background management, and service monitoring and other functional units; among them, the portal unit includes basic management, online experience, model collection, sample service, demand collection, algorithm model mall, and developer community, etc., the background management includes article management, demand management, and model management, etc., and the service monitoring includes operation service monitoring, model / sample service monitoring, and service resource performance monitoring, etc.

[0038] The platform layer is the service center of the entire artificial intelligence operation system, mainly including: algorithm model database, user data management module, intelligent recommendation module, intelligent transaction module, feedback optimization module, security protection module and sample database.

[0039] The network layer is mainly configured to realize the interaction of data and instructions, including power optical cables and dedicated networks.

[0040] The resource layer mainly includes case resources, sample resources, and model resources.

[0041] The different parts of the platform layer of this embodiment are composed as follows:

[0042] (1) An algorithm model database, configured to store artificial intelligence algorithm models; in this embodiment, the artificial intelligence algorithm models are algorithm models commonly used in the field of power systems (transmission, distribution or transformation), such as: a model configured to perform image recognition (face recognition, meter reading recognition), a model configured to perform line or equipment fault recognition, a model configured to perform text recognition, etc.

[0043] (2) A user data management module is configured to obtain basic user information, historical usage trace information of the algorithm model and operation habit information of the user, and store them in a user database; in this embodiment, basic user information includes basic user information, whether the user type is an enterprise or an individual, etc.; historical usage trace information of the algorithm model includes the algorithm models used by the user in the past, operation records, purchase records and usage or trial records when using each algorithm model, etc.; operation habit information of the user includes the frequency of mouse and keyboard clicks when the user uses the model, the operation process of the algorithm model, etc.

[0044] In this embodiment, users can be grouped, creating user groups based on user profiles and custom filtering criteria, and their user group membership can be determined based on user information. User grouping allows users to segment large amounts of user data to create more accurate, flexible, and convenient user groups, thereby enabling more refined operational work, such as intelligent recommendations based on algorithmic models.

[0045] User grouping is performed based on maintained user profiles, which primarily include basic user information, user statistics, and rule information. Basic information refers to the user's native information, including gender and age, which is entered by the user upon entering the system. Statistics are derived from analyzing user behavior, such as the time of the most recent purchase or visit. Rule information is based on business-defined rules, used to determine whether a user meets these rules. For example, a user who has made more than three purchases in the last 30 days is considered a high-frequency user.

[0046] (3) An intelligent recommendation module is configured to automatically match a list of recommended algorithm models from the algorithm model database based on user demand information, and automatically estimate the usage cost of each recommended algorithm model.

[0047] In this embodiment, the user's demand information is obtained (such as which type of algorithm model is needed, the accuracy requirements of the model, and the data provided by the user data management module, etc.), and keywords are extracted and quantified through word segmentation technology and semantic analysis to form a keyword feature vector; based on the labels of multiple algorithm models in the algorithm model database and the weights of the labels, the feature vector of each algorithm model is constructed; the similarity between the keyword feature vector and the feature vector of each algorithm model is calculated; the similarity calculation method can be implemented using relevant technologies.

[0048] The matching algorithm models are sorted based on similarity, and a set number of algorithm models with the highest similarity are selected to obtain a list of recommended algorithm models. For example, algorithm models with a similarity of 0.5-1 are sorted, with the highest similarity placed first.

[0049] In this embodiment, based on the algorithm model list, the usage fee of each model in the list is estimated; the usage fee of the algorithm model includes: the usage fee under the fixed fee model and the single usage fee under the single billing model. The single usage fee of each recommended model is automatically estimated by combining the historical usage trace information; the process is as follows:

[0050] Collect data related to the specified model service, including usage records, resource usage, time periods, resource application quotas, and other information. Clean and process the collected data, including removing outliers and filling missing values, to ensure data accuracy and completeness. If a feature value is missing, assign a default value to proceed to the next step. Obtain the number of requests, data processing volume, model call time, and resource usage time data from the cleaned data as historical feature data. This data can make the prediction model more accurate. Based on the regression model, use the PolynomialFeatures method to perform diversified processing on the historical feature data. Use the processed historical feature data to train the polynomial regression model. Use a portion of the historical feature data to evaluate the prediction model, comparing the difference between the prediction results and the actual cost. If the difference is significant, adjust the parameters or algorithm to optimize the model, and ultimately obtain a billing prediction model for this type of algorithm model. Obtain a billing prediction model that belongs to the same category as the recommendation model. Based on the historical usage data of this type of recommendation model, predict the single-time usage fee of the recommendation model.

[0051] The cost estimation of this embodiment takes the usage of the model into consideration, and can estimate a more accurate usage cost based on the usage characteristics of each type of model.

[0052] The intelligent recommendation module of this embodiment matches the user with a list of recommended models, and provides the recognition effect and estimated cost of each recommended model. Among them, the basic information of each model will include the recognition effect. This allows users to more intuitively understand various comprehensive information when selecting a model, which is conducive to selecting a model that is more suitable for them and reduces the time of model selection.

[0053] (4) The intelligent transaction module is configured to automatically bill users based on the algorithm model selected by the user and their usage. It determines the final settlement fee based on the algorithm model recognition effect and user feedback, and automatically generates transaction orders.

[0054] In this embodiment, a transaction order is automatically generated for the model selected by the user. Since the usage fee of the algorithm model includes: the usage fee under the fixed fee model and the single usage fee under the single billing model, if the user selects the fixed fee model, the fee is fixed and the fee settlement is carried out according to the preset fixed fee; if the user selects the single billing model, the single usage fee is calculated in real time based on the data when the user uses the model. In conjunction with Figure 2, the process is as follows:

[0055] (4-1) Calculate the initial cost based on the preset cost data and attribute data.

[0056] In this embodiment, when a manufacturer accesses the algorithm module, it is necessary to predefine labor cost, cycle cost or material cost data according to the system. When the modular algorithm service is initially connected, multiple attribute values ​​are preset for the modular algorithm service, including required resources, dependent services, etc.; after the above parameters are set, it can be connected to the platform; the initial cost is calculated based on the pre-set data, and the calculation formula is as follows: initial cost value = man_v*m+time_v*t+number of GPUs*g+men size*m2+message queue fixed fee; among them, man_v and time_v are the labor cost and cycle cost of maintaining the modular algorithm service respectively; m represents the initial discount coefficient of labor cost (if there is no discount, it is 1), t represents the initial discount coefficient of cycle cost (if there is no discount, it is 1), g represents the unit pricing of GPU use (such as the pricing of 1 GPU is 20), m2 represents the unit pricing of memory use (such as the pricing of 1G is 5), men size represents the minimum memory size required for the service, and the fixed fee is the configuration of the platform parameters, and new values ​​are maintained regularly based on market research.

[0057] (4-2) Obtain user type and usage frequency information to determine the discount ratio when the user uses the algorithm model; in this embodiment, the record of the user's use of the modular algorithm service is stored in a non-relational database (Not Only Structured Query Language, NOSQL) for offline task analysis. The main purpose of the analysis is to determine whether the user is an individual user or a company user (because individual users are generally non-commercial, a smaller granularity discount configuration will be made for non-company users in the manual configuration weight), usage frequency, applied GPU resources, CPU resources and memory resources, the duration of each service use and other dimensional information, based on which the user is labeled on this dimension.

[0058] Based on historical data information, the user's usage traces on the platform are calculated through statistical tasks to output the usage frequency. The discount rate of the fee is determined based on the usage frequency data. The calculation method is:

[0059] in, Sum(n)=F(1)+...+F(n); In the above formula, n represents the frequency of use of the modular algorithm service, F(n) is used to calculate the iteration value of each use of the modular algorithm service, and Sum(n) represents the total iteration value of each use.

[0060] In the above formula, n represents the frequency of service usage, F(n) is used to calculate the iterative value of each service usage, and F(n) is then used as an input parameter for calculating the discount. Sum(n) represents the sum of the iterative values ​​of each usage and is also mainly used as an input parameter for calculating the discount. This ensures that the discount is a decreasing value that tends to stabilize depending on the frequency of use.

[0061] (4-3) Obtain the GPU resource usage time, CPU resource usage time, storage resource usage time and the time when the task exception occurs when the user uses the algorithm model this time, and calculate the usage fee; in this embodiment, user operation information such as starting the service, pausing the service, and stopping the service are distributed to the real-time stream computing platform, and the fee information is calculated in real time in combination with the current resource usage information and pushed to the user. For example, from the time the user applies for resources and starts the service, the GPU usage price (pre-defined by the platform) and the memory usage price (pre-defined by the platform) and the CPU usage price (pre-defined by the platform) per second are multiplied by the usage time to obtain the real-time fee.

[0062] If a task exception occurs in the middle, the fee can be adjusted in proportion to the usage time (if an exception occurs, the real-time fee will be automatically reduced to 0.1) and the price can be calculated to obtain the final fee information.

[0063] Because the exception affects the user service experience, and although the exception occurs, it still occupies resources. Therefore, this part of the fee calculated according to the preset ratio needs to be deducted from the fee. The calculation method of this part of the fee is: exception time * ratio * fixed fee; among them, the exception time refers to the time from the time when the exception occurs after the algorithm service is enabled to the time when the service call is manually terminated or the service response is received. The ratio here is the system preset value, such as 0.1 for an exception time of 0 to 10 minutes, 0.15 for 10 to 20 minutes, etc., or it can be a fixed fee, such as charging per minute, the fee is 0.5 yuan per minute.

[0064] (4-4) Based on the initial cost, usage fee and discount ratio, the final cost of using the algorithm model is calculated.

[0065] Final cost information = (basic fee + GPU unit time cost * time + CPU unit time cost * time + storage size * consumption cost of each storage unit (calculated in MB) - abnormal time * ratio * fixed fee) * manual configuration weight * discount ratio.

[0066] The discount ratio is calculated based on the discount calculation formula. Manually configured weights are customized based on the service provider's promotional strategy for the algorithm service or tailored to different user types. For example, to promote a specific algorithm model, a smaller value can be configured to reduce the cost. GPU unit time fee, CPU unit time fee, and storage unit time fee are all pre-set. These basic metered fees are determined based on operational research and service consumption per unit time in a test environment, and are mandatory attributes configured when entering the algorithm model.

[0067] The final cost is discounted and the final calculated price is generated to obtain the final cost of using the algorithm model this time.

[0068] (4-5) When making fee settlement, obtain the user's rating results of the algorithm model calling process and the recognition effect of the algorithm model; normalize the two results, add them up by weight, and get the final performance value; determine the fee settlement ratio based on the relationship between the final performance value and the set target value; make the final fee settlement based on the final fee obtained by billing and the fee settlement ratio.

[0069] After using the model, users will fill out feedback and evaluation information. The feedback and evaluation information includes a scoring part and an evaluation part. The scoring part mainly scores the usage of the model, for example, the total score of the model usage; the evaluation part mainly includes some non-scoring options, such as: for a model's recognition result of an image, the user needs to provide feedback on whether the recognition is correct.

[0070] This step only obtains the feedback results of the scoring part; considering that user feedback is subjective, the weight of the model effect is greater than the weight of the scoring results of user feedback. For example, the weight of the model effect is 80%, and the weight of the scoring results of user feedback is 20%.

[0071] In addition, in order to prevent users from maliciously giving low ratings, the model will collect real-time statistics on the average of user ratings. If the current user rating deviates significantly from the average, the weight of the model effect will be dynamically increased, and the weight of the user feedback rating results will be reduced to ensure the accuracy of the final performance value. The increase and decrease strategies can be set by technicians in this field according to actual conditions.

[0072] The calculated final performance value is compared with the set performance target value (usually 1), and the settlement ratio is determined in a step-by-step manner based on the percentage of the final performance value to the performance target value. In this embodiment, the correspondence between different final performance values ​​and fee settlement ratios is determined in advance according to the principle that the closer the final performance value is to the set target value, the higher the fee settlement ratio. Based on the obtained final performance value and the corresponding relationship, the fee settlement ratio is obtained.

[0073] For example: if the final effect reaches 50% or less of the target value (fixed at 1), the settlement amount is 60% of the billing amount; if the final effect reaches 50%-60% of the target value, the settlement amount is 65% of the billing amount; if the final effect reaches 60%-70% of the target value, the settlement amount is 70% of the billing amount; if the final effect reaches 70%-80% of the target value, the settlement amount is 75% of the billing amount; if the final effect reaches 80%-85% of the target value, the settlement amount is 80% of the billing amount; if the final effect reaches 85%-90% of the target value, the settlement amount is 85% of the billing amount; if the final effect reaches 90%-95% of the target value, the settlement amount is 90% of the billing amount. The above are only examples, and those skilled in the art can set the rules according to actual conditions.

[0074] The model recognition effect gradually improves as the model is upgraded. Therefore, the better the model recognition effect, the higher the corresponding settlement fee.

[0075] The tiered fee settlement method of this embodiment fully considers factors such as the different recognition effects (recognition accuracy) of different models and the different satisfaction levels of different users after using the models, making the fee settlement more reasonable and improving the user experience.

[0076] (5) A feedback optimization module is configured to receive user feedback evaluation information on the algorithm model, evaluate the confidence of the feedback evaluation information based on the user's operational behavior data during the feedback evaluation, and utilize the user's operational habit model, and optimize the algorithm model based on all feedback evaluation information of the algorithm model and its confidence evaluation results.

[0077] In this embodiment, after the transaction is completed, the user's feedback evaluation information on the model is received. The feedback evaluation information here includes all the evaluation information of the scoring part and the evaluation part. Based on the user's operational behavior data during this feedback evaluation, the user's operational habit model is used to evaluate the confidence of the feedback evaluation information of this model, and the algorithm model is optimized based on the evaluation results.

[0078] Feedback evaluation information is collected through a questionnaire, which includes multiple evaluation indicators, each of which corresponds to a question. The user's operational behavior data during this feedback evaluation is all the user's behavior data after logging into the system, including:

[0079] Keyboard and mouse operation behavior data, such as: mouse single and double-click frequency, drag times and distance, keyboard keystroke frequency and tapping frequency, etc.

[0080] Operational behavior data of the called algorithm model, such as: length of time spent on the page, reading speed based on the amount of text on the page, page-turning habits, interface settings, etc.

[0081] In this embodiment, the user's operational habit model includes multiple operational behaviors of the user and a quantitative value for each operational behavior. A tutorial guides the user through the usage model and operational model to establish operational habits. The quantitative value for each operational behavior is determined based on the user's historical operational behavior after the tutorial. For example, the multiple operational behaviors include mouse click frequency, keyboard keystroke frequency, page turning speed, etc., which are determined based on the user's historical operational behavior, such as the user's typical operational behavior.

[0082] Through user training, the user can use the system carefully multiple times during the tutorial phase to establish their initial operating habits. After the initial operating habits are established, the details of each operation are analyzed. After long-term use, a user's operating habit model can be formed.

[0083] Generally speaking, users who use the model seriously will have significant differences in mouse movement speed, movement distance, click frequency, page dwelling, page switching, and many other aspects compared to users who use it informally, such as trials or browsing. The feedback evaluations given by the former are more credible than those given by the latter. For the same user, if the same person uses the system and logs in to his or her own account, then the account UID, corresponding operating habits, interface settings, etc. should all correspond one to one. If one or more of them do not correspond, it is considered a problem. It may be that someone else has logged in and used it, or it may be that the user did not operate seriously and provide feedback due to usage scenarios such as showing or browsing to others. When the behavioral habits displayed by a user in a single operation are inconsistent with their usual performance, the confidence level of the feedback given by the user for this operation is considered to be low.

[0084] Based on the above principles, this embodiment uses the user's operational behavior data during the feedback evaluation and the user's operational habit model to evaluate the confidence level of the model feedback evaluation information as follows:

[0085] (5-1) Select the user's mouse click frequency, drag times and drag distance, keyboard keystroke frequency, and page dwell time as evaluation indicators, and assign weights to each operation behavior.

[0086] (5-2) Calculate the similarity between the operational behavior data during this feedback evaluation and the corresponding quantitative values ​​of the operational behavior in the operational habit model; perform a weighted average calculation on the similarities of all operational behavior data to obtain the confidence level of this feedback information.

[0087] As a solution, this embodiment also conducts a secondary evaluation of each user's feedback information: by setting multiple anchor indicators in the performance feedback questionnaire, for the feedback information to be evaluated, the reliability of this feedback information is judged based on the user's feedback on the multiple anchor indicators. If the feedback information is unreliable, the confidence level is low.

[0088] Multiple anchor indicators can include indicators with clear and obvious answers, indicators with the same question and options but different wording, or indicators with the same question but different order of options, etc., without limitation. For indicators with clear and obvious answers, if the user chooses an incorrect answer, the confidence level of this feedback is considered low. For the latter two anchor indicators, if the chosen answer is inconsistent, the confidence level of this feedback is considered low.

[0089] The results of the secondary assessment are used as a verification method, and the two are cross-validated. If the results of the two assessments are inconsistent, the results of the secondary assessment shall prevail.

[0090] Based on all feedback and confidence assessment results of the algorithm model, the algorithm model is optimized. For example, if the evaluation information with high confidence contains more positive information, it indicates that the model's accuracy or performance is acceptable. If the evaluation information with high confidence contains more negative information, it indicates that the model's accuracy or performance cannot meet user needs and the algorithm model needs to be optimized. This process can promptly identify algorithm models that do not meet user needs and optimize the models in a timely manner.

[0091] (6) A model calling module is configured to respond to a model calling request from the user terminal, decrypt the temporary key sent by the user terminal to obtain the client identification information and the target algorithm model that the user terminal wants to call; obtain the key of the target algorithm model and decrypt it, compress the decrypted target algorithm model, and then encrypt the target algorithm model based on the temporary key and transmit it to the user terminal; wherein the user terminal generates a temporary key during this login to the system based on the customer login information, the current system time and the client identification information.

[0092] After the server parses and obtains the temporary key, it can also modify the time in the temporary key to the time period when the target algorithm model is available, and then update the temporary key; encrypt the compressed target algorithm model with the new temporary key, and then transmit it to the user end; in this way, the user can use the model only within the set time period.

[0093] In this embodiment, when the user calls the model, some input data is uploaded. To ensure the security of this data, a temporary key can be used to encrypt this input data. The input data is often image data. The process of encrypting the image data using a temporary key is as follows:

[0094] ① Convert the image data to RGB color mode to obtain an image matrix of l*h*3, where l is the number of rows of the image, h is the number of columns of the image, and 3 corresponds to the three channels of R, G, and B; convert the temporary key into a key matrix with a size of h*l.

[0095] ② Randomly generate a mask row vector. The mask row vector consists of randomly arranged l*h*3 integers ranging from 1 to l*h*3. Mask the elements in the image matrix according to the mask row vector to obtain a mask image. Concatenate each row of the image matrix to obtain an image row vector of length l*h*3, and simultaneously create a new image row vector. Use each element value in the mask row vector as a sequence number, and search the image row vector for the pixel value corresponding to that sequence number. Starting with the first element, write the new image row vector in sequence. Convert the new image row vector to a l*h*3 matrix to obtain the mask image. For example, if the third element in the mask row vector is 30, the 30th element in the image row vector will be used as the third element in the new image row vector.

[0096] ③Convert the mask row vector into a l*h*3 matrix to obtain the mask matrix.

[0097] ④ Multiply each layer of the mask matrix and mask image with the key matrix to obtain encrypted image data.

[0098] By encrypting the input image data with a temporary key, the security of user data during the user's use of the model can be fully guaranteed.

[0099] (7) A sample database is configured to store algorithm model application instance data in different scenarios. The algorithm model application instance data includes: application scenario, algorithm model, model application effect, usage fee and model optimization data.

[0100] The intelligent operations system of this embodiment can provide more accurate and comprehensive intelligent model recommendations based on user needs. The billing method of each algorithm model takes into account the differences in user usage, making the billing method more accurate and reasonable. The confidence level of user feedback evaluation information is assessed based on user operation habit data, realizing the credibility evaluation of user feedback information. Based on the correctness or inconsistency of the user's feedback on the anchor indicator, the system can provide an overall evaluation of the user's feedback information. Through the cross-certification of information at two levels, a secondary evaluation of user feedback information is achieved, improving the accuracy of user feedback information evaluation.

[0101] Example 2

[0102] In one or more embodiments, a working method of the artificial intelligence operation system in Example 1 is disclosed, and with reference to FIG3 , includes the following process:

[0103] In response to the user's algorithm model call request, the user's basic information, the user's historical usage trace information of the algorithm model and the operation habit information are obtained from the user database; based on the demand information input by the user, a keyword feature vector is constructed, and the similarity between the keyword feature vector and the feature vector matrix of multiple algorithm models in the algorithm model database is calculated to obtain a recommended algorithm model list, and the usage fee of each recommended algorithm model is automatically estimated; wherein, the element in the feature vector of each algorithm model is the product of the model label and the label weight; based on the algorithm model selected by the user, automatic billing is performed according to the user's usage, and the final settlement fee is determined by combining the algorithm model recognition effect and the user feedback rating results, and a transaction order is automatically generated; the user's feedback evaluation information on the algorithm model is received, and based on the user's operation behavior data during this feedback evaluation, the user's operation habit model is used to evaluate the confidence of this feedback evaluation information, and the algorithm model is optimized based on all feedback evaluation information of the algorithm model and its confidence evaluation results.

[0104] After evaluating the confidence of the model feedback evaluation information, a second evaluation is performed: multiple anchor indicators are set in the questionnaire of the feedback evaluation information. For the feedback information to be evaluated, the reliability of this feedback information is judged based on the user's feedback on the multiple anchor indicators. If the feedback information is unreliable, the confidence is low.

[0105] In addition, when the user calls the model, compression encryption is used to ensure the security of user and model data; the process is: in response to the user's model call request, the user identification information and the target algorithm model that the user wants to call are obtained by decryption based on the temporary key sent by the user; user identity authentication is performed based on the user identification information, and after the verification is passed, the key of the target algorithm model is obtained and decrypted, the decrypted target algorithm model is compressed, and then the target algorithm model is encrypted based on the temporary key and transmitted to the user; wherein, the user generates a temporary key during this login to the system based on the login information, the current system time and the user identification information.

[0106] The implementation process of the above method has been described in detail in Example 1 and will not be described in detail here.

[0107] This application proposes a method for optimizing and recommending electric power artificial intelligence models, which solves the problem that the recommendation method is not applicable to algorithm model recommendation in the electric power artificial intelligence model system. According to multiple algorithm model labels and their weights in the algorithm model library, a feature vector matrix is ​​obtained, and then the similarity between the feature vectors of keywords is calculated, and the algorithm models in the algorithm model library are sorted, thereby improving the accuracy of recommending algorithm models for electric power user groups with different businesses and realizing the needs of electric power users with different businesses.

[0108] This application proposes a method for online calling of an electric power artificial intelligence model, which performs data security protection based on compression encryption technology. A temporary key containing setting information is constructed each time a user logs in; user data and the required model are compressed and encrypted based on the temporary key, which solves the problem of easy leakage of user data and model data during the model calling process, ensures that each login has a unique key, implements one-time one-key, and ensures the security of calling the algorithm model; based on the temporary key, the client's model download request and the compressed model returned by the server are encrypted, so that only the user on the client and within the specified time can use the target model, ensuring the security of the model on the server.

[0109] This application proposes a differentiated billing and tiered fee settlement method for an electric power artificial intelligence model. It combines the specific usage of different users, takes into account task anomalies and discount ratios, and performs differentiated calculations on the usage fees of the algorithm model. At the same time, it calculates the discount ratio for each use of the modular algorithm service based on the frequency of use, thereby realizing automatic calculation of the discount ratio and ensuring that the fee discount is a decreasing value that tends to be stable depending on the frequency of use. When settling fees, this application comprehensively considers the use effect of the model and the user's rating results, determines the fee settlement ratio, realizes tiered fee settlement for different users and different models, and improves the accuracy and rationality of fee settlement.

[0110] This application proposes a dual credibility assessment method for user feedback evaluation information. Based on the operational behavior data of the user during the feedback evaluation, the user's operational habit model is used to conduct a primary assessment of the feedback evaluation information; then the feedback evaluation information is secondary assessed using a set anchor indicator; this solves the problem that a single evaluation result may be inaccurate; and cross-validates the dual evaluation results to improve the accuracy of the assessment of user feedback evaluation information.

[0111] This application also constructs a billing prediction model for the algorithm model, which estimates the model usage fee based on the historical usage trace information of the algorithm model. It can more accurately estimate the specific usage fee of the model and provide accurate data support for users to select models.

Claims

1. An electric power artificial intelligence model system, comprising: an algorithm model database configured to store power artificial intelligence algorithm models; A user data management module is configured to obtain basic user information, historical usage trace information of the algorithm model and operation habit information of the user, and store them in a user database; The intelligent recommendation module is configured to construct a keyword feature vector based on the demand information input by the user, calculate the similarity between the keyword feature vector and the feature vector matrix of multiple algorithm models in the algorithm model database, obtain a recommended algorithm model list, and automatically estimate the usage fee of each recommended algorithm model; wherein the element in the feature vector of each algorithm model is the product of the model label and the label weight; The intelligent transaction module is configured to automatically charge users based on the algorithm model selected by the user and the user's usage, determine the final settlement fee based on the recognition effect of the algorithm model and the scoring results of user feedback, and automatically generate a transaction order; The feedback optimization module is configured to receive the user's feedback evaluation information on the algorithm model, evaluate the confidence of the feedback evaluation information based on the user's operational behavior data during the feedback evaluation and using the user's operational habit model, and optimize the algorithm model based on all feedback evaluation information and confidence evaluation results of the algorithm model.

2. The electric power artificial intelligence model system according to claim 1, further comprising: The model calling module is configured to respond to the user's model calling request, compress the target algorithm model, encrypt the target algorithm model with a temporary secret key, and transmit the encrypted algorithm model to the user.

3. The electric power artificial intelligence model system according to claim 2, wherein: The model calling module is configured to compress the target algorithm model in the following manner, encrypt the target algorithm model with a temporary secret key, and transmit the encrypted algorithm model to the user: The user identification information and the target algorithm model that the user wants to call are obtained by decryption based on the temporary key sent by the user; user identity authentication is performed based on the user identification information, and after the authentication is passed, the key of the target algorithm model is obtained and decrypted, the decrypted target algorithm model is compressed, and then the target algorithm model is encrypted based on the temporary key and transmitted to the user; wherein the user generates a temporary key during this login to the system according to the login information, the current system time and the user identification information.

4. The electric power artificial intelligence model system according to claim 2, wherein: The model calling module is also configured to replace the current system time with the time period of the user using the model, update the temporary key; and use the updated key to encrypt the compressed target algorithm model.

5. The electric power artificial intelligence model system according to claim 1, wherein: The intelligent recommendation module is configured to construct a keyword feature vector based on the demand information input by the user in the following manner, calculate the similarity between the keyword feature vector and the feature vector matrix of multiple algorithm models in the algorithm model database, and obtain a recommended algorithm model list: Extracting keywords from the user's business demand information and quantifying them to form the keyword feature vector; Constructing a feature vector for each algorithm model according to the labels and label weights of multiple algorithm models in the algorithm model database; Calculate the similarity between the keyword feature vector and the feature vector of each algorithm model; The multiple algorithm models are sorted based on similarity, and a set number of algorithm models with the highest similarity are selected to obtain the recommended algorithm model list.

6. The electric power artificial intelligence model system according to claim 5, wherein: The intelligent recommendation module is further configured to determine the weight corresponding to the label of each algorithm model in the following manner: Obtain keywords related to the labels of the algorithm model in historical browsing records, search records, or demand records to form a keyword library; The word frequency-inverse document frequency method is adopted to calculate the word frequency and inverse document frequency of the algorithm model label in the keyword library, and obtain the weight value corresponding to the label of the algorithm model.

7. The electric power artificial intelligence model system according to claim 5, wherein: The similarity is characterized by the cosine value of the angle between the two vectors.

8. The electric power artificial intelligence model system according to claim 1, wherein: The intelligent recommendation module is configured to automatically estimate the usage cost of each recommended algorithm model in the following manner: Obtain the number of requests, data processing volume, model call time, and resource usage time data of a type of algorithm model within a historical set time period, and pre-process them as historical feature data; Based on the regression model, the PolynomialFeatures method is used to perform diversified processing on the historical feature data; Using the processed historical feature data to train a polynomial regression model to obtain a billing prediction model of the type of algorithm model; A billing prediction model belonging to the same category as the recommended algorithm model is obtained, and based on the historical usage data of the recommended algorithm model, the usage fee of the recommended algorithm model is predicted.

9. The electric power artificial intelligence model system according to claim 1, wherein: The smart transaction module is configured to automatically charge users according to their usage in the following manner: Calculate initial cost based on preset cost data and attribute data; Obtain user type and usage frequency information to determine the user's usage of the selected algorithm model. Deduction ratio; Obtain the GPU resource usage time, CPU resource usage time, storage resource usage time and task exception time when the user uses the selected algorithm model, and calculate the usage fee; Based on the initial cost, the usage fee and the discount ratio, the final cost of using the selected algorithm model this time is calculated.

10. The electric power artificial intelligence model system according to claim 9, wherein: The smart transaction module is configured to determine the final settlement fee by combining the recognition effect of the algorithm model and the scoring result of user feedback in the following manner: When performing fee settlement, obtaining the user's scoring result of the selected algorithm model calling process and the recognition effect of the selected algorithm model; After normalizing the scoring result and the recognition effect, weighted summing is performed to obtain a final effectiveness value; Determine the cost settlement ratio based on the relationship between the final performance value and the set target value; The final fee settlement is performed based on the ratio of the final fee obtained by billing to the fee settlement.

11. The electric power artificial intelligence model system according to claim 10, wherein: The intelligent trading module is also configured to, in response to the difference between the rating result of user feedback and the model rating mean being greater than a set threshold, lower the weight of the user rating result in calculating the final effectiveness value according to a predetermined rule.

12. The electric power artificial intelligence model system according to claim 10, wherein: The final cost of using the selected algorithm model this time is: Final cost information = (basic cost + GPU unit time cost * time + CPU unit time cost * time + storage size * each storage unit consumption cost - abnormal time * ratio * fixed cost) * manual configuration weight * discount ratio; Wherein, the manually configured weight is a set value.

13. The electric power artificial intelligence model system according to claim 1, wherein: The operational behavior data of the user during this feedback evaluation is all the behavioral data of the user after logging into the system, including the operational behavior data of the keyboard and mouse, and the operational behavior data of the called algorithm model.

14. The electric power artificial intelligence model system according to claim 1, wherein: The user's operation habit model includes the user's multiple operation behaviors and the user's quantitative value for each operation behavior, wherein the user is guided by a tutorial to perform behavioral operations on the usage model and the operation model to establish operation habits; the user's quantitative value for each operation behavior is determined based on the user's historical operation behavior after the tutorial guidance.

15. The electric power artificial intelligence model system according to claim 1 or 14, wherein: The feedback optimization module is configured to evaluate the confidence of the feedback evaluation information in the following manner: Select the user's mouse click frequency, drag times and drag distance, keyboard keystroke frequency, and page dwell time as evaluation indicators, and assign weights to each operation behavior; The similarity between the operation behavior data during this feedback evaluation and the corresponding operation behavior quantization value in the operation habit model is calculated; the similarity of all operation behavior data is weighted averaged to obtain the confidence of this feedback information.

16. The electric power artificial intelligence model system according to claim 1, wherein: The feedback optimization module is further configured to perform a secondary evaluation after evaluating the confidence of the model feedback evaluation information; The feedback optimization module is configured to perform secondary evaluation in the following manner: A plurality of anchor indicators are set in the questionnaire of the feedback evaluation information. For the feedback information to be evaluated, whether the feedback information is reliable is judged according to the feedback of the user on the plurality of anchor indicators. The feedback information is unreliable and has low confidence.

17. The electric power artificial intelligence model system according to claim 1, further comprising: The sample database is configured to store algorithm model application instance data in different scenarios, and the algorithm model application instance data includes: application scenario, algorithm model, model application effect, usage fee and model optimization data.

18. A working method of an electric power artificial intelligence model system, applied to the electric power artificial intelligence model system according to any one of claims 1 to 17, comprising: In response to the user's algorithm model call request, obtain the user's basic information, the user's historical usage trace information and operation habit information of the algorithm model from the user database; Construct a keyword feature vector based on the demand information input by the user, calculate the similarity between the keyword feature vector and the feature vector matrix of multiple algorithm models in the algorithm model database, obtain a recommended algorithm model list, and automatically estimate the usage fee of each recommended algorithm model; wherein the element in the feature vector of each algorithm model is the product of the model label and the label weight; Based on the algorithm model selected by the user, automatic billing is performed according to the user's usage. The final settlement fee is determined based on the recognition effect of the algorithm model and the scoring results of user feedback, and a transaction order is automatically generated; Receive user feedback and evaluation information on the algorithm model, evaluate the confidence of the feedback and evaluation information based on the user's operational behavior data during this feedback and evaluation and use the user's operational habit model, and optimize the algorithm model based on all feedback and evaluation information and confidence evaluation results of the algorithm model.

19. The working method of the electric power artificial intelligence model system according to claim 18, further comprising: After evaluating the confidence of the evaluation information fed back by this model, a second evaluation is conducted; The secondary assessment includes: A plurality of anchor indicators are set in the questionnaire of the feedback evaluation information. For the feedback information to be evaluated, whether the feedback information is reliable is judged according to the feedback of the user on the plurality of anchor indicators. The feedback information is unreliable and has low confidence.

20. The working method of the electric power artificial intelligence model system according to claim 18, further comprising: In response to the user's model call request, decrypt the temporary key sent by the user to obtain the user identification information and the target algorithm model that the user wants to call; User identity authentication is performed based on the user identification information. After the authentication is passed, the key of the target algorithm model is obtained and decrypted, the decrypted target algorithm model is compressed, and then the target algorithm model is encrypted based on the temporary key and transmitted to the user; wherein the user generates a temporary key during this login to the system based on the login information, the current system time and the user identification information.

Images