Artificial intelligence platform sample library management method and system

Abstract

The present invention discloses a method and system for managing a sample library of an artificial intelligence platform, the method comprising: S1, receiving massive electric power business data and data description information of the business data; S2, obtaining a first preprocessing method set for the business data according to the data description information, and preprocessing the electric power business data based on at least one preprocessing method in the first preprocessing method set; S3, obtaining an AI model associated with the business data; S4, obtaining format rule information of the AI ​​model for input data, S5, loading a second preprocessing method corresponding to the format rule of the data itself to perform a second preprocessing on the business data, and loading a data annotation method corresponding to the format rule of the annotation data to perform data annotation on the business data that has undergone the second preprocessing. The present invention realizes the centralized and effective utilization of a large amount of sample data scattered in various business systems, and manages them in the sample library of the artificial intelligence platform.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence platform sample library management method and system. Background Art

[0002] Artificial intelligence is the core part of the future smart grid. It is an important means to build an efficient, collaborative, interactive, shared, and "thing-thing-people" interconnected energy Internet. It is an inevitable choice for the development of the power system.

[0003] However, in the existing technologies, various AI applications or AI support capabilities are built in a scattered manner, and there is a lack of a unified sample management platform at the headquarters level, which leads to inconsistent interfaces between different systems, inconvenient data sharing, and a large waste of computing resources. Currently, a large amount of sample data is scattered in various business systems and has not been effectively utilized. It is necessary to organize and collect data from various business systems, and manage them in the sample library of the AI ​​platform to form high-quality sample data and sample labels, provide a sample basis for model training and iterative optimization, and deepen the application of the sample library of the power AI platform. Summary of the invention

[0004] In view of the problems existing in the above-mentioned prior art, the present invention provides an artificial intelligence platform sample library management method and system, which can realize the management of business data of various business systems and provide a sample basis for model training and iterative optimization.

[0005] The technical solution is as follows:

[0006] The first embodiment of the present application proposes an artificial intelligence platform sample library management method, comprising the following steps:

[0007] S1, receiving massive power business data and data description information of the business data;

[0008] S2, acquiring a first preprocessing method set for the business data according to the data description information, and preprocessing the power business data based on at least one preprocessing method in the first preprocessing method set;

[0009] S3, obtaining an AI model associated with the business data from an AI model library of the artificial intelligence platform;

[0010] S4, obtaining format rule information of the AI ​​model for input data, where the format rule includes the format rule of the data itself and the format rule of the annotation data carried by the data;

[0011] S5, loading a second preprocessing method corresponding to the format rule of the data itself to perform second preprocessing on the business data, and loading a data annotation method corresponding to the format rule of the annotated data to perform data annotation on the business data after the second preprocessing;

[0012] S6, storing the results of the first preprocessing, the second preprocessing and the data labeling of the business data respectively.

[0013] In one embodiment of the present application, the data description information includes but is not limited to application scenarios and data types of business data, and the data types include images, videos, voices, and texts.

[0014] In one embodiment of the present application, the S2 includes:

[0015] S21, recommending at least one candidate preprocessing method for the power business data according to the data description information of the power business data, to form a first preprocessing method set;

[0016] S22, recommending a candidate preprocessing method combination for the power business data according to the preprocessing method combination strategy adopted by the historical power business data;

[0017] S23, when it is determined to adopt the "one-key preprocessing" operation, a series of preprocessing operations are automatically performed on the power business data according to the optimal combination item in the candidate preprocessing method combination; otherwise, the power business data is preprocessed according to at least one preprocessing method selected from the first preprocessing method set.

[0018] In one embodiment of the present application, the step S21 recommends at least one candidate preprocessing method for the power business data according to the data description information of the power business data to form a first preprocessing method set, including:

[0019] S211, based on the data description information of the power business data, according to a preset fuzzy rule table, obtaining at least one candidate preprocessing method applicable to the power business data, and generating a first group of first preprocessing methods;

[0020] or

[0021] S212, based on the data description information of the electric power business data, determining the execution frequency distribution data of the preprocessing method of the historical electric power business data under the data description information, and generating a second group of first preprocessing methods based on the preprocessing methods whose execution frequencies meet the preset frequency conditions;

[0022] S213: Based on the first group of first preprocessing methods and the second group of first preprocessing methods, evaluate the confidence level that each first preprocessing method is suitable for the power business data, and determine a first preprocessing method set based on the confidence level.

[0023] In one embodiment of the present application, the step S213, based on the first group of first preprocessing methods and the second group of first preprocessing methods, evaluating the confidence that each first preprocessing method is applicable to the power business data, and determining the first preprocessing method set based on the confidence, includes:

[0024] Based on the first group of first preprocessing methods and the second group of first preprocessing methods generated for the historical power business data, the first probability of the preprocessing methods actually performed on the historical power business data belonging to the intersection of the first group of first preprocessing methods and the second group of first preprocessing methods, and the second probability of belonging to the first group of first preprocessing methods or the second group of first preprocessing methods respectively;

[0025] For the intersection and non-intersection preprocessing methods of the first group of first preprocessing methods and the second group of first preprocessing methods generated for the current power business data, determine the confidence level that each preprocessing method in the first group of first preprocessing methods and the second group of first preprocessing methods is applicable to the power business data based on the first probability and the second probability;

[0026] A first preprocessing method set is determined based on the preprocessing methods whose confidences meet a preset confidence threshold range.

[0027] In one embodiment of the present application, the step S3, obtaining an AI model associated with the business data from an AI model library of an artificial intelligence platform, includes:

[0028] S31, obtaining business data of an AI model that is explicitly associated with the business data to be analyzed, and determining an AI model that is explicitly associated with the business data to be analyzed based on the business data of the AI ​​model that is explicitly associated with the business data to be analyzed, wherein the explicit association is calculated based on the similarity of the business data;

[0029] S32, obtaining an AI model that is implicitly associated with the business data to be analyzed;

[0030] S33: Determine the AI ​​model associated with the business data based on the explicitly associated AI model and the implicitly associated AI model.

[0031] In one embodiment of the present application, the S32 includes:

[0032] The input and output data of the AI ​​model in the AI ​​model library of the artificial intelligence platform form the semantic features of the AI ​​model;

[0033] Based on the matching correlation between the semantic feature and the business data to be analyzed, it is determined whether the AI ​​model is associated with the business data to be analyzed.

[0034] The second embodiment of the present application proposes an artificial intelligence platform sample library management system, including:

[0035] A massive power business data aggregation module, used to receive massive power business data and data description information of the business data;

[0036] A first preprocessing method module, configured to obtain a first preprocessing method set for the business data according to the data description information, and preprocess the power business data based on at least one preprocessing method in the first preprocessing method set;

[0037] An associated AI model analysis module, used to obtain an AI model associated with the business data from an AI model library of an artificial intelligence platform;

[0038] A second preprocessing rule acquisition module is used to obtain format rule information of the AI ​​model for input data, where the format rules include format rules of the data itself and format rules of the annotation data carried by the data;

[0039] A second preprocessing rule execution module is used to load a second preprocessing method corresponding to the format rule of the data itself to perform a second preprocessing on the business data, and load a data annotation method corresponding to the format rule of the annotated data to perform data annotation on the business data after the second preprocessing;

[0040] The sample data classification management module is used to store the results of the first preprocessing, the second preprocessing and the data labeling of the business data respectively.

[0041] The third aspect of the present application provides an electronic device, including:

[0042] at least one processor; and

[0043] a memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the artificial intelligence platform sample library management method described in the above-mentioned first aspect embodiment.

[0045] The fourth aspect of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the artificial intelligence platform sample library management method described in the first aspect of the present application.

[0046] An artificial intelligence platform sample library management method and system of the present invention have the following beneficial effects: based on the preprocessing, data labeling, and sample data set management operations of the received massive power business data, a large amount of sample data scattered in various business systems can be centrally and effectively utilized, and each business data is sorted and collected, and managed in the artificial intelligence platform sample library to form high-quality sample data and sample labels, providing a sample basis for model training and iterative optimization, and deepening the application of the power artificial intelligence platform sample library. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a flowchart of the artificial intelligence platform sample library management method in an embodiment of the present application;

[0048] Figure 2 It is a structural diagram of the artificial intelligence platform sample library management system in the embodiment of the present application. DETAILED DESCRIPTION

[0049] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0050] An embodiment of the present invention provides an artificial intelligence platform sample library management method, comprising the following steps:

[0051] S1, receiving massive power business data and data description information of the business data;

[0052] S2, acquiring a first preprocessing method set for the business data according to the data description information, and preprocessing the power business data based on at least one preprocessing method in the first preprocessing method set;

[0053] S3, obtaining an AI model associated with the business data from an AI model library of the artificial intelligence platform;

[0054] S4, obtaining format rule information of the AI ​​model for input data, where the format rule includes the format rule of the data itself and the format rule of the annotation data carried by the data;

[0055] S5, loading a second preprocessing method corresponding to the format rule of the data itself to perform second preprocessing on the business data, and loading a data annotation method corresponding to the format rule of the annotated data to perform data annotation on the business data after the second preprocessing;

[0056] S6, storing the results of the first preprocessing, the second preprocessing and the data labeling of the business data respectively.

[0057] In the embodiment of the present application, the received massive power business data can be obtained based on local data upload, or can be obtained based on the data interface (service directory, Restful service, WS service, etc.) of the data center; for the massive image, voice, text and other structured data and unstructured data of the power business, the RDS database and OSS object storage (or relational database, HBase distributed storage system) of the State Grid Cloud Platform are used for storage. The platform stores data attribute information including name, source, type, path, business attributes, processing and annotation ratio, size and creation time, and stores the source file in the distributed file system.

[0058] At the same time, it provides standardized annotation functions for the collected image, voice and other business data, follows the preset unified sample data annotation specifications, supports automatic annotation and manual labeling of sample data, and realizes professional, standardized and refined annotation of data. At the same time, it supports multi-tenant mechanism, supports different users to complete the data annotation work together, realizes unified annotation of power business data, provides data support for model algorithm optimization and upgrading, and gives full play to the value of data.

[0059] In addition, data set management is performed on the labeled sample data, and management operations of training sets, test sets, and validation sets are performed on the sample data. In this application, sample data is formed after the first preprocessing of the business data, and model training samples with labeled data are obtained after the second preprocessing and data labeling. Based on the model training samples, based on different uses in the iterative training process of the AI ​​model, they can be divided into training sets, test sets, and validation sets. In this application, data ratios of training sets, test sets, and validation sets can be adjusted, samples can be re-divided multiple times during cross-validation, and new samples can be added.

[0060] Based on the above preprocessing, data labeling, and sample data set management, we can achieve centralized and effective utilization of a large amount of sample data scattered in various business systems, organize and collect data from various business systems, and manage them in the sample library of the artificial intelligence platform to form high-quality sample data and sample labels, providing a sample basis for model training and iterative optimization, and deepening the application of the sample library of the power artificial intelligence platform.

[0061] The above data description information includes but is not limited to the application scenarios and data types of business data, and the data types include images, videos, voices, and texts.

[0062] Furthermore, the above S2, obtaining a first preprocessing method set for the business data according to the data description information, and preprocessing the power business data based on at least one preprocessing method in the first preprocessing method set, comprises the following steps:

[0063] S21, recommending at least one candidate preprocessing method for the power business data according to the data description information of the power business data, to form a first preprocessing method set;

[0064] S22, recommending a candidate preprocessing method combination for the power business data according to the preprocessing method combination strategy adopted by the historical power business data;

[0065] S23, when it is determined to adopt the "one-key preprocessing" operation, a series of preprocessing operations are automatically performed on the power business data according to the optimal combination item in the candidate preprocessing method combination; otherwise, the power business data is preprocessed according to at least one preprocessing method selected from the first preprocessing method set.

[0066] In the embodiment of the present application, the data preprocessing function provides corresponding data preprocessing methods according to different data types. For example, for image type data, the preprocessing methods that can be used include denoising, defogging, enhancement, smoothing, sharpening, sampling, etc. For voice type data, the preprocessing methods that can be used include filtering, quantization, etc. For text type data, the preprocessing methods that can be used include character replacement, space removal, text desensitization, etc. In this application, single file processing and one-click operation of multiple operation modes can be performed. Users can customize the data preprocessing scheme according to data characteristics.

[0067] Furthermore, in the above S21, at least one candidate preprocessing method is recommended for the power business data according to the data description information of the power business data to form a first preprocessing method set, including:

[0068] S211, based on the data description information of the power business data, according to a preset fuzzy rule table, obtaining at least one candidate preprocessing method applicable to the power business data, and generating a first group of first preprocessing methods;

[0069] or

[0070] S212, based on the data description information of the electric power business data, determining the execution frequency distribution data of the preprocessing method of the historical electric power business data under the data description information, and generating a second group of first preprocessing methods based on the preprocessing methods whose execution frequencies meet the preset frequency conditions;

[0071] S213: Based on the first group of first preprocessing methods and the second group of first preprocessing methods, evaluate the confidence level that each first preprocessing method is suitable for the power business data, and determine a first preprocessing method set based on the confidence level.

[0072] In an embodiment of the present application, a set of preprocessing methods suitable for received power business data is generated based on three methods. Based on a preset fuzzy rule table, a group of preprocessing methods corresponding to a determined application scenario of the power business data and a determined data type can be generated. A group of preprocessing methods can also be determined based on high-frequency execution methods in preprocessing methods used in historical power business data. The preprocessing methods generated by the above two methods can also be mutually verified to evaluate the confidence that each preprocessing method is suitable for the power business data. The preprocessing method groups generated by the two generation methods are combined and evaluated to obtain the best first preprocessing method set.

[0073] Further, the above S213, based on the first group of first preprocessing methods and the second group of first preprocessing methods, evaluates the confidence level of each first preprocessing method being applicable to the power business data, and determines the first preprocessing method set based on the confidence level, including:

[0074] Based on the first group of first preprocessing methods and the second group of first preprocessing methods generated for the historical power business data, the first probability of the preprocessing methods actually performed on the historical power business data belonging to the intersection of the first group of first preprocessing methods and the second group of first preprocessing methods, and the second probability of belonging to the first group of first preprocessing methods or the second group of first preprocessing methods respectively;

[0075] For the intersection and non-intersection preprocessing methods of the first group of first preprocessing methods and the second group of first preprocessing methods generated for the current power business data, determine the confidence level that each preprocessing method in the first group of first preprocessing methods and the second group of first preprocessing methods is applicable to the power business data based on the first probability and the second probability;

[0076] A first preprocessing method set is determined based on the preprocessing methods whose confidences meet a preset confidence threshold range.

[0077] In an embodiment of the present application, based on the preprocessing method execution options in the historical power business data, the actual execution rates of the preprocessing methods that belong to both preprocessing method sets and the actual execution rates of the preprocessing methods that belong exclusively to a certain preprocessing method set are determined, and the execution rate records in the historical data and the preprocessing method sets generated based on the two generation methods for the current power business data are combined to perform confidence assessment, thereby improving the precise adaptability of the first preprocessing method set to the current power business data and further realizing the inductive management of preprocessing methods for massive power business data.

[0078] In addition, for the above S22, based on the preprocessing method combination strategy adopted by the historical power business data, a candidate preprocessing method combination is recommended for the power business data. Specifically, a method similar to the above step S212 can be adopted, based on the preprocessing method combination strategy adopted by the historical power business data with the same data description information, and through the frequency of execution of the preprocessing method combination strategy, a high-frequency execution preprocessing method combination strategy is determined as the candidate preprocessing method combination recommended for the current power business data. Furthermore, in the above S23, the optimal combination item can be the execution preprocessing method combination strategy with the highest execution frequency.

[0079] In addition, the above S22 can also be implemented by the following steps:

[0080] Based on the current power business data, the quality of the current power business data is evaluated through a variety of evaluation indicators, such as the clarity of image data, the noise data size of image / text / voice data, the data validity or data redundancy of image / text / voice data, etc., to obtain the power business data quality evaluation data, and based on the data quality evaluation results of the historical power business data and the corresponding preprocessing methods that have been executed, determine one or more preprocessing methods that match the current power business data to form a preprocessing method combination strategy. For example, if the noise index of the historical power business data is greater than 30%, the noise reduction method A is used, then when the noise index of the current power business data is greater than 30%, the noise reduction method A can also be used.

[0081] Furthermore, the above S3, obtaining the AI ​​model associated with the business data from the AI ​​model library of the artificial intelligence platform, includes the following steps:

[0082] S31, obtaining business data of an AI model that is explicitly associated with the business data to be analyzed, and determining an AI model that is explicitly associated with the business data to be analyzed based on the business data of the AI ​​model that is explicitly associated with the business data to be analyzed, wherein the explicit association is calculated based on the similarity of the business data;

[0083] S32, obtaining an AI model that is implicitly associated with the business data to be analyzed;

[0084] S33: Determine the AI ​​model associated with the business data based on the explicitly associated AI model and the implicitly associated AI model.

[0085] Furthermore, the above S32, obtaining an AI model that is implicitly associated with the business data to be analyzed, includes the following steps:

[0086] S321, forming semantic features of the AI ​​model based on input and output data of the AI ​​model existing in the AI ​​model library of the artificial intelligence platform;

[0087] S322: Determine whether the AI ​​model is associated with the business data to be analyzed based on the matching correlation between the semantic feature and the business data to be analyzed.

[0088] In the embodiment of the present application, the direct correlation analysis is not performed directly based on the AI ​​model input data and the business data to be analyzed, but based on the semantic features of the AI ​​model. In the present application, the semantic features characterize the specific application scenarios and application effects of the model. For example, there is a semantic feature of an AI model for analyzing transmission line faults based on the collected transmission line images. If the business data to be analyzed is a tower image, a conductor image, an anti-vibration hammer image, etc., they can all be associated with the AI ​​model. It can be understood that the business data and annotated data processed based on the above steps S4 and S5 can be used as training samples in the new AI model training process. The new AI model can be an AI model that does not exist in the original AI model library of the artificial intelligence platform. That is, in the present application, based on the analysis of the AI ​​model with explicit association with the business data to be analyzed and the AI ​​model with implicit association, it is possible to further obtain the AI ​​model that can be applied to the business data to be analyzed and the new AI model that may be expanded and applied, and generate corresponding training samples for the two types of AI models.

[0089] The present application also provides an artificial intelligence platform sample library management system, including:

[0090] A massive power business data aggregation module, used to receive massive power business data and data description information of the business data;

[0091] A first preprocessing method module, configured to obtain a first preprocessing method set for the business data according to the data description information, and preprocess the power business data based on at least one preprocessing method in the first preprocessing method set;

[0092] An associated AI model analysis module, used to obtain an AI model associated with the business data from an AI model library of an artificial intelligence platform;

[0093] A second preprocessing rule acquisition module is used to obtain format rule information of the AI ​​model for input data, where the format rules include format rules of the data itself and format rules of the annotation data carried by the data;

[0094] A second preprocessing rule execution module is used to load a second preprocessing method corresponding to the format rule of the data itself to perform a second preprocessing on the business data, and load a data annotation method corresponding to the format rule of the annotated data to perform data annotation on the business data after the second preprocessing;

[0095] The sample data classification management module is used to store the results of the first preprocessing, the second preprocessing and the data labeling of the business data respectively.

[0096] It should be noted that the aforementioned explanation of the artificial intelligence platform sample library management method is also applicable to the artificial intelligence platform sample library management system of this embodiment, and will not be repeated here.

[0097] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0098] Specifically, the electronic device includes:

[0099] at least one processor; and

[0100] a memory communicatively connected to at least one processor; wherein,

[0101] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the aforementioned artificial intelligence platform sample library management method.

[0102] The electronic device described above is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0103] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the electronic device via a network.

[0104] Specifically, the above-mentioned readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to enable the computer to execute the aforementioned artificial intelligence platform sample library management method.

[0105] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A method for managing a sample library of an artificial intelligence platform, characterized in that: include: S1, receiving massive power business data and data description information of the business data; S2, acquiring a first preprocessing method set for the business data according to the data description information, and preprocessing the power business data based on at least one preprocessing method in the first preprocessing method set; S3, obtaining an AI model associated with the business data from an AI model library of the artificial intelligence platform; S4, obtaining format rule information of the AI ​​model for input data, where the format rule includes the format rule of the data itself and the format rule of the annotation data carried by the data; S5, loading a second preprocessing method corresponding to the format rule of the data itself to perform second preprocessing on the business data, and loading a data annotation method corresponding to the format rule of the annotated data to perform data annotation on the business data after the second preprocessing; S6, storing the results of the first preprocessing, the second preprocessing and the data labeling of the business data respectively; The S2 comprises: S21, recommending at least one candidate preprocessing method for the power business data according to the data description information of the power business data, to form a first preprocessing method set; S22, recommending a candidate preprocessing method combination for the power business data according to the preprocessing method combination strategy adopted by the historical power business data; S23, when it is determined to adopt the "one-key preprocessing" operation, automatically performing a series of preprocessing operations on the power business data according to the optimal combination item in the candidate preprocessing method combination, otherwise, preprocessing the power business data according to at least one preprocessing method selected from the first preprocessing method set; The step S21, recommending at least one candidate preprocessing method for the power business data according to the data description information of the power business data, to form a first preprocessing method set, includes: S211, based on the data description information of the power business data, according to a preset fuzzy rule table, obtaining at least one candidate preprocessing method applicable to the power business data, and generating a first group of first preprocessing methods; or S212, based on the data description information of the electric power business data, determining the execution frequency distribution data of the preprocessing method of the historical electric power business data under the data description information, and generating a second group of first preprocessing methods based on the preprocessing methods whose execution frequencies meet the preset frequency conditions; S213, evaluating the confidence level of each first preprocessing method being applicable to the power service data based on the first group of first preprocessing methods and the second group of first preprocessing methods, and determining a first preprocessing method set based on the confidence level; The step S3, obtaining an AI model associated with the business data from an AI model library of the artificial intelligence platform, includes: S31, obtaining business data of an AI model that is explicitly associated with the business data to be analyzed, and determining an AI model that is explicitly associated with the business data to be analyzed based on the business data of the AI ​​model that is explicitly associated with the business data to be analyzed, wherein the explicit association is calculated based on the similarity of the business data; S32, obtaining an AI model that is implicitly associated with the business data to be analyzed; S33: Determine the AI ​​model associated with the business data based on the explicitly associated AI model and the implicitly associated AI model.

2. The method for managing a sample library of an artificial intelligence platform according to claim 1, characterized in that: The data description information includes application scenarios and data types of business data, and the data types include images, videos, voices, and texts.

3. The method for managing a sample library of an artificial intelligence platform according to claim 1, characterized in that: The step S213, evaluating the confidence level of each first preprocessing method being applicable to the power service data based on the first group of first preprocessing methods and the second group of first preprocessing methods, and determining the first preprocessing method set based on the confidence level, includes: Based on the first group of first preprocessing methods and the second group of first preprocessing methods generated for the historical power business data, the first probability of the preprocessing methods actually performed on the historical power business data belonging to the intersection of the first group of first preprocessing methods and the second group of first preprocessing methods, and the second probability of belonging to the first group of first preprocessing methods or the second group of first preprocessing methods respectively; For the intersection and non-intersection preprocessing methods of the first group of first preprocessing methods and the second group of first preprocessing methods generated for the current power business data, determine the confidence level that each preprocessing method in the first group of first preprocessing methods and the second group of first preprocessing methods is applicable to the power business data based on the first probability and the second probability; A first preprocessing method set is determined based on the preprocessing methods whose confidences meet a preset confidence threshold range.

4. The method for managing a sample library of an artificial intelligence platform according to claim 1, characterized in that: The S32 includes: The input and output data of the AI ​​model in the AI ​​model library of the artificial intelligence platform form the semantic features of the AI ​​model; Based on the matching correlation between the semantic feature and the business data to be analyzed, it is determined whether the AI ​​model is associated with the business data to be analyzed.

5. An artificial intelligence platform sample library management system, characterized in that: include: A massive power business data aggregation module, used to receive massive power business data and data description information of the business data; A first preprocessing method module, configured to obtain a first preprocessing method set for the business data according to the data description information, and preprocess the power business data based on at least one preprocessing method in the first preprocessing method set; An associated AI model analysis module, used to obtain an AI model associated with the business data from an AI model library of an artificial intelligence platform; A second preprocessing rule acquisition module is used to obtain format rule information of the AI ​​model for input data, where the format rules include format rules of the data itself and format rules of the annotation data carried by the data; A second preprocessing rule execution module is used to load a second preprocessing method corresponding to the format rule of the data itself to perform a second preprocessing on the business data, and load a data annotation method corresponding to the format rule of the annotated data to perform data annotation on the business data after the second preprocessing; A sample data classification management module is used to store the results of the first preprocessing, the second preprocessing and the data labeling of the business data respectively; The first preprocessing method module is used to perform the following steps: S21, recommending at least one candidate preprocessing method for the power business data according to the data description information of the power business data to form a first preprocessing method set; S22, recommending a candidate preprocessing method combination for the power business data according to the preprocessing method combination strategy adopted by the historical power business data; S23, when it is determined to adopt the "one-key preprocessing" operation, automatically performing a series of preprocessing operations on the power business data according to the optimal combination item in the candidate preprocessing method combination, otherwise, preprocessing the power business data according to at least one preprocessing method selected from the first preprocessing method set; The S21, recommending at least one candidate preprocessing method for the power business data according to the data description information of the power business data to form a first preprocessing method set, includes: S211, based on the data description information of the power business data, according to a preset fuzzy rule table, obtaining at least one candidate preprocessing method applicable to the power business data, and generating a first group of first preprocessing methods; or, S212, based on the data description information of the power business data, determining the execution frequency distribution data of the preprocessing methods of the historical power business data under the data description information, and generating a second group of first preprocessing methods based on the preprocessing methods whose execution frequencies meet the preset frequency conditions; S213, evaluating the confidence level of each first preprocessing method applicable to the power business data based on the first group of first preprocessing methods and the second group of first preprocessing methods, and determining the first preprocessing method set based on the confidence level; The associated AI model analysis module obtains the AI ​​model associated with the business data from the AI ​​model library of the artificial intelligence platform, including: S31, obtaining the business data of the associated AI model that is explicitly associated with the business data to be analyzed, and determining the AI ​​model explicitly associated with the business data to be analyzed based on the business data of the associated AI model that is explicitly associated, wherein the explicit association is calculated based on the similarity of the business data; S32, obtaining the AI ​​model that is implicitly associated with the business data to be analyzed; S33, determining the AI ​​model associated with the business data based on the explicitly associated AI model and the implicitly associated AI model.

6. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the artificial intelligence platform sample library management method described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the artificial intelligence platform sample library management method according to any one of claims 1 to 4.

Images