A method for training AI systems based on big data

Abstract

The present invention discloses a method for training an AI system based on big data, including: obtaining basic data for training an AI system, establishing a basic database based on the basic data; receiving an AI system to be trained sent by an AI developer; obtaining attribute information of the AI ​​system to be trained; obtaining target data based on the attribute information in the basic database; and performing a training task on the AI ​​system to be trained based on the target data. This solution can effectively ensure the universality of training data in the AI ​​system training industry, reduce the waste of computing power caused by the non-universality of training data, and can also form a universal third-party training system that is independent of AI system developers and users, further lowering the threshold of the AI ​​system training industry, and is conducive to the advancement of AI system technology in today's society.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an AI system training method based on big data. Background Art

[0002] At present, artificial intelligence (AI) has been widely used in all walks of life, greatly improving the level of human production and life. Since AI system technology has just started and is developing slowly, it is not enough to meet human needs, so the training of AI systems is particularly important. The current popular AI system training method is: developers independently obtain the data required for training AI systems and conduct targeted training on AI systems.

[0003] This training method requires a high level of technical skills from developers. The data obtained by developers is often not rich enough, and the performance of the trained AI system is low. In addition, when developers train AI systems, they are more targeted and lack systematicity. The data they obtain is less universal and cannot be applied to train other AI systems, resulting in large differences in AI system training, which is not conducive to the formulation of unified standards for AI system training industry, and AI technology progresses slowly. In summary, the current problems of difficulty in obtaining training data for AI systems and poor universality need to be solved urgently. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems in the above-mentioned technology to a certain extent. To this end, the purpose of the present invention is to propose an AI system training method based on big data, provide common training data for AI systems, and promote the standardization of the AI ​​system training industry.

[0005] To achieve the above object, the present invention proposes an AI system training method based on big data, comprising:

[0006] Acquire basic data for training the AI ​​system, and establish a basic database based on the basic data;

[0007] Receive AI systems to be trained sent by AI developers;

[0008] Obtaining attribute information of the AI ​​system to be trained;

[0009] Acquiring target data in the basic database according to the attribute information;

[0010] Execute a training task on the AI ​​system to be trained according to the target data.

[0011] Preferably, the acquiring of basic data for training the AI ​​system and establishing a basic database based on the basic data includes:

[0012] Get the data source sent by the user;

[0013] Preprocess the data source, extract data for training the AI ​​system and mark it as basic data;

[0014] A basic database is established according to the basic data.

[0015] Preferably, the step of obtaining the data source sent by the user includes:

[0016] Provide users with extended services including big data storage, cloud deployment and data processing;

[0017] Acquire the data source sent by the user in the extended service.

[0018] Preferably, the step of establishing a basic database based on the basic data further includes:

[0019] The basic data is divided into levels and categories, and the basic data is stored in the basic database according to the division results.

[0020] Preferably, the attribute information includes level and category.

[0021] Preferably, the level is the level of training service required by the AI ​​system to be trained, including at least one of basic training, advanced training, and customized service;

[0022] Preferably, the application category is the category of the AI ​​system application, including at least one of recognition, classification, image segmentation, and defect detection.

[0023] Preferably, before receiving the AI ​​system to be trained sent by the AI ​​developer, the method further includes:

[0024] Collect information released by AI developers;

[0025] Performing semantic recognition on the information to obtain a semantic recognition result;

[0026] Determining whether the information includes AI training requirements according to the semantic recognition result;

[0027] When determining AI training needs, identify the AI ​​developers who need AI training services.

[0028] Preferably, when performing a training task on the AI ​​system to be trained according to the target data, it also includes:

[0029] Obtaining application and feedback information of the AI ​​system to be trained on the target data;

[0030] Determining abnormal data based on the application and feedback information;

[0031] Analyzing the abnormal data;

[0032] The basic database is updated according to the analysis results.

[0033] Preferably, establishing a standard data correction model to correct the basic database includes:

[0034] (1) extracting standard data of each category under each level based on the basic data divided into levels and categories, and establishing a standard data set based on the standard data;

[0035] (2) constructing a first-order data correction model, and correcting the standard data set based on the first-order data correction model to obtain a first-order corrected data set;

[0036] (3) setting a feature data extraction algorithm, extracting standard feature data of the standard data set and first-order feature data of the first-order correction data set, and comparing the standard feature data with the first-order feature data to obtain first-order difference information;

[0037] (4) adjusting the parameters of the primary data correction model according to the first-order difference information to obtain a secondary data correction model, and correcting the standard data set based on the secondary data correction model to obtain a second-order corrected data set;

[0038] (5) extracting the second-order feature data of the second-order corrected data set, comparing the standard feature data with the second-order feature data, and obtaining second-order difference information;

[0039] (6) calculating a first similarity between the first-order feature data and the standard feature data;

[0040] (7) calculating a second similarity between the second-order feature data and the standard feature data;

[0041] (8) comparing the first similarity with the second similarity to obtain a comparison result;

[0042] (9) When it is determined that the comparison result is that the first similarity is greater than the second similarity, return to (2) and reconstruct the primary data correction model; otherwise, adjust the secondary data correction model according to the second-order difference information to obtain a tertiary data correction model;

[0043] Perform iterative training until the i-th order feature data is equal to the standard feature data;

[0044] (10) using the i-level data correction model corresponding to the i-order characteristic data as a standard correction model;

[0045] (11) Correcting the basic database according to the standard correction model.

[0046] Preferably, the modifying the basic database according to the standard modification model comprises:

[0047] Establishing a sorter grid according to the level and category of the basic data;

[0048] deploying a cluster of mapping unit sorters in the sorter grid;

[0049] implanting the standard correction model in the mapping unit sorter cluster;

[0050] The sorter grid is connected to the mapping grid;

[0051] The mapping grid includes mapping units, the mapping units are connected to the addresses of the basic data in the basic database, and the mapping unit sorter cluster is connected to the addresses of the basic data through the mapping units;

[0052] The standard correction model corrects the universality of the basic database by correcting the connection relationship between the mapping unit sorter cluster and the mapping unit without changing the basic data.

[0053] Preferably, after performing the training task on the AI ​​system to be trained according to the target data, the method further includes:

[0054] Test the trained AI system and obtain the test results;

[0055] The attribute information of the trained AI system is modified according to the test results.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. In the present invention, by establishing a basic database and providing training to AI system developers based on the basic database, data sharing in the AI ​​system training industry is guaranteed, so that AI system developers do not need to obtain training data by themselves, which reduces the burden on AI system developers and lowers the threshold for AI development.

[0058] 2. In the present invention, by enhancing the universality of the basic database, universal training services are provided to AI system developers based on the basic database, which is conducive to promoting the formation of unified standards in the AI ​​training service industry.

[0059] 3. In the present invention, by realizing universal correction of the basic database without changing the basic data, the frequent reading and writing of the basic database is reduced, the consumption of computer computing power is alleviated, and more computing power resources can be allocated to AI system training, which is conducive to the overall progress of the AI ​​training industry.

[0060] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0061] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0063] Figure 1 is a flow chart of an AI system training method based on big data according to an embodiment of the present invention;

[0064] Figure 2 is a flow chart of providing extended services to users to obtain data sources according to one embodiment of the present invention;

[0065] Figure 3 is a flow chart of updating a basic database according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0067] like Figure 1 As shown, the embodiment of the present invention proposes an AI system training method based on big data, including steps S1-S5:

[0068] S1. Obtain basic data for training the AI ​​system, and establish a basic database based on the basic data;

[0069] S2. Receive the AI ​​system to be trained sent by the AI ​​developer;

[0070] S3. Obtaining attribute information of the AI ​​system to be trained;

[0071] S4, acquiring target data according to the attribute information in the basic database;

[0072] S5. Execute a training task on the AI ​​system to be trained according to the target data.

[0073] The working principle of the above technical solution: For example, the character recognition AI system, also known as the OCR AI system, is mainly used to recognize words and patterns containing characters. The existing OCR AI systems based on deep neural networks are all obtained by developers training neural networks and collecting and organizing word patterns by themselves. Due to the uneven level of AI system developers, some developers obtain fewer word patterns, and the performance of the trained AI system is low. The AI ​​system training method based on big data described in the present invention can improve this problem. By collecting and organizing a large amount of word pattern data, establishing a word pattern basic database, receiving the OCR AI system training commission, according to the level and category attribute information of the OCR AI system, the target data of the same level and category in the basic database that is suitable for training the OCR AI system is obtained, that is, the word pattern that is pre-acquired and processed and stored in the word pattern basic database, and the word pattern data is used to train the OCR AI system.

[0074] This process is similar to the operation mode of human schools. The AI ​​system to be trained is the "student", and the developer of the AI ​​system is the parent of the "student". The content of this solution is the establishment and operation of the "school". The AI ​​training platform established according to this solution is the "school" deployed on the network cloud. The above-mentioned "school" is the software and hardware foundation including computers and AI training system software for implementing the present invention. The process of developers sending the AI ​​system to the AI ​​training platform for training is the process of "parents" sending "students" to the "school" for education. In this process, the "classroom" where the "students" learn is the software running the AI ​​training platform.

[0075] Of course, it is impossible to educate "students" only by "classroom", and "classroom" also needs the intervention or cooperation of "educators". The "educators" are composed of two parts, one of which is the "lecturer" who directly educates the "students", that is, the basic data in this scheme; the other part is the "professor" who is responsible for formulating the "teaching plan". The above-mentioned "professor" is a human staff member who implements this scheme, which is included in the AI ​​system training method based on big data proposed in the present invention. The above-mentioned "formulating 'teaching plan'" is the process of formulating AI training content and obtaining training data. The above-mentioned "professor" and "lecturer" together constitute "educators". In summary, the AI ​​system training method based on big data proposed in the present invention is that the "professor" establishes a "school", formulates a "teaching plan", recruits "lecturers" and screens "lecturers", and establishes a "lecturer" team. "Parents" do not need to educate "students" in person, but only need to send the students to the "school". The "school" arranges "lecturers" to educate "students" according to the "teaching plan" pre-formulated by the "professor", and returns the "school" to the "parents" after the education is completed.

[0076] The beneficial effects of the above technical solutions are: providing general AI system training data, so that the client who needs AI training services does not have to provide test data, saving the client's time, reducing trouble, improving efficiency, reducing the burden on AI system developers, and lowering the threshold for AI system development. At the same time, providing general artificial intelligence training services will help form unified standards for the AI ​​system training industry.

[0077] According to some embodiments of the present invention, obtaining basic data for training an AI system and establishing a basic database based on the basic data includes:

[0078] Get the data source sent by the user;

[0079] Preprocess the data source, extract data for training the AI ​​system and mark it as basic data;

[0080] A basic database is established according to the basic data.

[0081] The working principle of the above technical solution is: obtain the data source sent by the user, screen and correct the data source, obtain data suitable for training the AI ​​system, and establish a basic database.

[0082] The beneficial effects of the above technical solution are: establishing a basic database suitable for training AI systems, which is used to provide AI system training services to developers, so that developers do not have to search for data on their own, further lowering the threshold of the AI ​​system development industry.

[0083] like Figure 2As shown, the embodiment of the present invention proposes a method for providing an extended service to a user to obtain a data source, including steps S21-S24:

[0084] S21. Provide users with extended services including data storage and data processing;

[0085] S22, receiving a data source sent by the user in the extended service;

[0086] S23, performing data preprocessing on the data source, and screening data in the data source that can be used to train the AI ​​system;

[0087] S24. The data is taken as basic data and stored in a basic database.

[0088] The working principle of the above technical solution: For example, provide users with extended services such as data bank services, obtain the data source sent by users in the service, and establish a basic database. Generally speaking, AI products also have a common demand, that is, a large amount of industrial data (industrial big data) requires storage services and data processing services. In addition to solving the problem of AI product training, the present invention can also provide other extended services including big data storage, cloud deployment and data processing, such as providing users with data bank services in the form of network storage space such as network disks, obtaining data stored by users in network disks, pre-processing the data, obtaining data that can be used to train AI, such as photo images of different objects, images containing text information, and other data, and storing the data in the basic database.

[0089] The beneficial effects of the above technical solution are: by providing extended services to users, while satisfying the users' needs for extended services, the difficulty of obtaining data sources is reduced, which is conducive to the establishment of a basic database and ensures the diversity of data in the basic database.

[0090] According to some embodiments of the present invention, establishing a basic database based on the basic data further includes: dividing the basic data into levels and categories, and storing the basic data in the basic database according to the division results.

[0091] The working principle of the above technical solution is: establish a grade and classification algorithm to divide the grade and classification of the basic database. This process is similar to the lecturer rating and classification link in the human school operation model. This solution proposes a big data-based AI system training method for "school", and the basic data is "lecturer". After recruiting "lecturers", the "school" arranges teaching work according to the level of the "lecturer" and the "subjects" taught by the "lecturer", forming a "lecturer team" with clear division of labor, in which different categories of basic data are "lecturers teaching different subjects", and different levels of basic data are "lecturers teaching different grades".

[0092] The beneficial effects of the above technical solution are as follows: the basic database is divided into levels and categories, which facilitates the management of the basic data, the storage and reading of the basic data, and is conducive to the subsequent updating and maintenance of the basic database.

[0093] According to some embodiments of the present invention, the basic data is divided into levels and categories based on matrix operations, including:

[0094] Setting the number of levels into which the basic data needs to be divided;

[0095] Setting a level determination index for classifying the basic data;

[0096] Setting a level association function, wherein the level association function is a function including an association relationship between the level number and the level determination index;

[0097] According to the grade association function, establishing a first association relationship between the grade number and the grade determination index;

[0098] According to the first association relationship, a first transition matrix X is established:

[0099]

[0100] Among them, U x and V y is a parameter in the first association relationship, the number of rows of the first transition matrix is ​​the same as the number of levels;

[0101] Setting a basic data level determination algorithm, wherein the level determination algorithm is a function including a correlation relationship between the basic data and the level determination index;

[0102] According to the grade determination algorithm, establishing a second association relationship between the basic data and the grade determination index;

[0103] According to the second association relationship, the number of columns of the second transition matrix is ​​set to be the same as the number of levels, and a second transition matrix Y is established:

[0104]

[0105] Among them, M i and N j is a parameter in the second association relationship;

[0106] According to the first transition matrix X and the second transition matrix Y, a level determination matrix Z is obtained, wherein the level determination matrix Z includes a third association relationship formed by the first association relationship and the second association relationship.

[0107]

[0108] Among them, R e and S f is the parameter in the third association relationship, and U x 、V y 、M i 、N j Related;

[0109] Classifying the basic data according to the third association relationship;

[0110] Similarly, a third transition matrix including the number of categories and category determination indicators is set to construct the correlation between the number of categories and the category determination indicators, a category judgment algorithm is set to construct a fourth transition matrix including the correlation between basic data and category determination indicators, a category determination matrix including the correlation between basic data and categories is obtained based on the third transition matrix and the fourth transition matrix, and the categories of the level are divided according to the category determination matrix.

[0111] The working principle of the above technical solution is: the basic data is divided into levels and categories according to the meaning of matrix operations.

[0112] The beneficial effects of the above technical solution are: using matrix operations to divide the levels and categories of basic data, obtaining the association relationship between the level and basic data, and between the category and basic data, so as to facilitate obtaining basic data according to the association relationship.

[0113] According to some embodiments of the present invention, when acquiring target data in the basic database based on the attribute information, the attribute information includes a level and a category, the level being the level of training service required for the AI ​​system to be trained, including at least one of basic training, advanced training, and customized services, and the application category being the category of the AI ​​system application, including at least one of recognition, classification, image segmentation, and defect detection.

[0114] The working principle of the above technical solution is as follows: the basic database has been divided into levels and categories. When obtaining target data according to the attribute information in the basic database, the attribute information including the level and category of the AI ​​system to be trained is obtained, and the basic data under the corresponding category of the basic database is matched according to the level and category of the AI ​​system. The basic data is marked as target data and extracted. Taking the image recognition AI system as an example, before training the AI ​​system, the category of the AI ​​system (i.e., the image recognition AI system) is preferentially identified, and the image recognition AI system classification standard is set as the image recognition accuracy rate. The image recognition AI system with an image recognition accuracy rate higher than 50% and lower than 80% is a second-level AI system, and the image recognition AI system with an image recognition accuracy rate higher than 80% and lower than 90% is a third-level AI system. The attribute information of the image recognition AI system includes its historical image recognition accuracy rate of 57%. The training system determines that the AI ​​system is a second-level AI system, extracts the second-level basic data from the basic database as the target data, and trains the AI ​​system. This process is similar to the teaching in accordance with students' aptitude in the operation mode of human schools. After the training system as a "school" receives the AI ​​system to be trained as a "student", it determines the training plan according to the "half-care education" (i.e., the AI ​​system is not completely trained by the platform built according to this solution, and the AI ​​developer will use other methods to train the AI ​​system) or "full-care education" (i.e., the AI ​​system is completely trained by the platform built according to this solution) required by the "parents", and matches the "student" with a "lecturer of the same grade" (i.e., basic data of the same level as the AI ​​system) according to the "grade" in the "file" (i.e., the attribute information of the AI ​​system) to educate the "student". Similarly, according to the "subject" selected by the "student" (i.e., the category of data required for training the AI ​​system, for example, the data required for training the image recognition AI system is image data), arrange a "lecturer of the same subject" to teach the "student".

[0115] The beneficial effects of the above technical solution are as follows: when acquiring target data for training an AI system, basic data of the same level and category as the AI ​​system are matched first, thereby eliminating unnecessary retrieval, speeding up the acquisition of target data and saving computer computing resources.

[0116] According to some embodiments of the present invention, before receiving the AI ​​system to be trained sent by the AI ​​developer, the method further includes:

[0117] Collect information released by AI developers;

[0118] Performing semantic recognition on the information to obtain a semantic recognition result;

[0119] Determining whether the information includes AI training requirements according to the semantic recognition result;

[0120] When determining AI training needs, identify the AI ​​developers who need AI training services.

[0121] The working principle of the above technical solution: For example, the process of determining AI developers who need AI training services is also similar to a part of human school operations. The "school" collects the information released by AI developers who are "parents" to determine whether there is "school information" involved. If so, it determines that the "parent" needs the "education" (i.e. AI system training services) services provided by the school.

[0122] The beneficial effects of the above technical solution are: actively and targetedly searching for developers who need AI training services and providing AI training services, so that this technical solution can be promoted, promoting the generalization of AI system training services, and contributing to the formulation of unified standards for the AI ​​system training industry.

[0123] According to some embodiments of the present invention, Figure 3 As shown, the embodiment of the present invention proposes a universal correction method for a basic database, including steps S31-S34:

[0124] S31. Provide users with extended services including data storage and data processing;

[0125] S32, receiving a data source sent by the user in the extended service;

[0126] S33, performing data preprocessing on the data source, and screening data in the data source that can be used to train the AI ​​system;

[0127] S34. The data is taken as basic data and stored in a basic database.

[0128] The working principle of the above technical solution: For example, when the AI ​​system to be trained is a mushroom classification AI system, the photo image containing mushrooms in the basic database is the target data, and the mushroom classification AI system is trained using the photo image. During the training process, the feedback of the mushroom classification AI system to the photo image is the application information. For example, the recognition result and abnormal error information output by the mushroom classification AI system all belong to the application information, so that the image photo of the application information output by the mushroom classification AI system is the abnormal data, for example, data that the mushroom classification AI system cannot read during the training process; the abnormal data is analyzed, and the basic database is updated according to the analysis results. For example, if the information output by the mushroom classification AI system is that the image is too large, the image is shielded in real time, and the photo image is compressed after the training is completed.

[0129] The beneficial effects of the above technical solution are as follows: by obtaining application information and feedback information during the AI ​​system training process, the basic database is corrected for universality according to the application information and feedback information, thereby enhancing the universality of the basic database, ensuring the universality of AI training data within the AI ​​system training industry, and promoting the formation of unified standards within the AI ​​training service industry.

[0130] According to some embodiments of the present invention, establishing a standard data correction model to correct the basic database includes:

[0131] (1) extracting standard data of each category under each level based on the basic data divided into levels and categories, and establishing a standard data set based on the standard data;

[0132] (2) constructing a first-order data correction model, and correcting the standard data set based on the first-order data correction model to obtain a first-order corrected data set;

[0133] (3) setting a feature data extraction algorithm, extracting standard feature data of the standard data set and first-order feature data of the first-order correction data set, and comparing the standard feature data with the first-order feature data to obtain first-order difference information;

[0134] (4) adjusting the parameters of the primary data correction model according to the first-order difference information to obtain a secondary data correction model, and correcting the standard data set based on the secondary data correction model to obtain a second-order corrected data set;

[0135] (5) extracting the second-order feature data of the second-order corrected data set, comparing the standard feature data with the second-order feature data, and obtaining second-order difference information;

[0136] (6) calculating a first similarity between the first-order feature data and the standard feature data;

[0137] (7) calculating a second similarity between the second-order feature data and the standard feature data;

[0138] (8) comparing the first similarity with the second similarity to obtain a comparison result;

[0139] (9) When it is determined that the comparison result is that the first similarity is greater than the second similarity, return to (2) and reconstruct the primary data correction model; otherwise, adjust the secondary data correction model according to the second-order difference information to obtain a tertiary data correction model;

[0140] Perform iterative training until the i-th order feature data is equal to the standard feature data;

[0141] (10) using the i-level data correction model corresponding to the i-order characteristic data as a standard correction model;

[0142] (11) Correcting the basic database according to the standard correction model.

[0143] The working principle of the above technical solution is as follows: for example, based on the basic data divided into levels and categories, the standard data of each category under each level is extracted, and a standard data set is established based on the standard data. For example, the basic data is divided into two levels, the first and second levels, and divided into three categories, the first category, the second category, and the third category. The first and second category standard data sets are the most representative data sets in the first and second category basic data. The standard data set can be represented in a form similar to coordinates, with the level as the horizontal coordinate and the category as the vertical coordinate. The first and second category standard data sets can be represented as (1, 2); a standard data set and a basic data correction model are set, the basic data model is trained using the standard data set, the basic data correction model is corrected according to the training results, and iterative training is performed until the data correction model can accurately identify all the data in the standard data set.

[0144] The beneficial effects of the above technical solution are: establishing a standard data correction model, correcting the basic database, enhancing the versatility of the basic database, and facilitating the sharing of training data within the AI ​​system training industry.

[0145] According to some embodiments of the present invention, correcting the basic database according to the standard correction model further includes:

[0146] Establishing a sorter grid according to the level and category of the basic data;

[0147] deploying a cluster of mapping unit sorters in the sorter grid;

[0148] implanting the standard correction model in the mapping unit sorter cluster;

[0149] The sorter grid is connected to the mapping grid;

[0150] The mapping grid includes mapping units, the mapping units are connected to the addresses of the basic data in the basic database, and the mapping unit sorter cluster is connected to the addresses of the basic data through the mapping units;

[0151] The standard correction model corrects the universality of the basic database by correcting the connection relationship between the mapping unit sorter cluster and the mapping unit without changing the basic data.

[0152] The working principle of the above technical solution is as follows: For example, a sorter grid is established according to the level and category of the basic data. For example, the basic data is divided into two levels, the first level and the second level, and further divided into three categories, the first category, the second category, and the third category. Then a 2×3 sorter grid is established, with the level as the horizontal axis and the category as the vertical axis. Any sorter grid can be represented by its corresponding category and level. For example, the sorter grid of the first level and the second category can be represented as sorter grid (1,2);

[0153] A mapping unit sorter cluster is deployed in each sorter grid. The mapping unit sorter cluster includes at least one mapping unit sorter. Each mapping unit sorter can also be represented by coordinates. For example, the seventh mapping unit sorter in the sorter grid (1,2) can be represented as mapping unit sorter (1,2,7).

[0154] A corresponding standard correction model is implanted in each mapping unit sorter cluster, for example, a standard data correction model (1,2) is implanted in the mapping unit sorter cluster in the sorter grid (1,2);

[0155] The sorter grid is connected to the mapping grid, and the mapping grid is a grid of p rows and d columns, where p and d are any natural numbers. Any mapping grid can also be represented by coordinates. For example, the second row and fourth column of the mapping grid can be represented as a mapping grid (2,4);

[0156] The mapping grid includes at least one mapping unit, and any mapping unit can also be represented in the form of coordinates. For example, the eighth mapping unit in the mapping grid (2,4) can be represented as mapping unit (2,4,8);

[0157] The mapping unit is connected to the address of at least one basic data in the basic database. For example, the mapping unit (2,4,8) is connected to the basic data with the address data. The connection can be expressed as {(2,4,8),data};

[0158] The mapping unit sorter is connected to the mapping units in the mapping grid. For example, the mapping unit sorter (1,2,7) is connected to the mapping unit (2,4,8). The connection can be represented as {(1,2,7),(2,4,8)};

[0159] The mapping unit sorter is indirectly connected to the address of the basic data through the mapping unit. For example, with the mapping unit (2,4,8) as the relay station, the mapping unit sorter (1,2,7) is indirectly connected to the basic data with the address data. The indirect connection relationship can be expressed as {(1,2,7), (2,4,8), data};

[0160] The standard correction model corrects the universality of the basic database without changing the basic data by correcting the connection relationship between the mapping unit sorter cluster and the mapping unit. Taking the indirect connection {(1,2,7), (2,4,8), data} as an example, when the mapping grid (1,2) receives feedback information that the basic data with the address data has poor universality, the data correction model in the mapping grid (1,2) controls the mapping unit sorter (1,2,7) to cut off the direct connection relationship between the mapping unit sorter (1,2,7) and the mapping unit (2,4,8), and then cuts off the indirect connection relationship between the mapping unit sorter (1,2,7) and the basic data with the address data, thereby shielding the basic data with the address data, and realizing the universality correction of the basic database without changing the basic data.

[0161] For example, the above plan is that the "school" selects "lecturers" that the "students" are unwilling to accept based on the "students'" feedback on the "lecturers", ensuring that the "lecturer team" is more easily accepted by the "students".

[0162] The beneficial effects of the above technical solution are as follows: by controlling the mapping unit sorter through the standard correction model, the basic database can be universally corrected without changing the basic data, and the non-universal basic data can be retained to facilitate the subsequent correction and reuse of the basic data, thereby improving the utilization rate of the data source and, at the same time, reducing the number of times the basic data is read and written and reducing the loss of computer computing power.

[0163] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for training an AI system based on big data, characterized in that: include: Acquire basic data for training the AI ​​system, and establish a basic database based on the basic data; Receive AI systems to be trained from AI developers; Obtaining attribute information of the AI ​​system to be trained; Acquiring target data in the basic database according to the attribute information; Execute a training task on the AI ​​system to be trained according to the target data; The step of establishing a basic database according to the basic data further comprises: Dividing the basic data into levels and categories, and storing the basic data in the basic database according to the division results; Establishing a standard data correction model to correct the basic database includes: (1) extracting standard data of each category under each level based on the basic data divided into levels and categories, and establishing a standard data set based on the standard data; (2) constructing a first-order data correction model, and correcting the standard data set based on the first-order data correction model to obtain a first-order corrected data set; (3) setting a feature data extraction algorithm, extracting standard feature data of the standard data set and first-order feature data of the first-order correction data set, and comparing the standard feature data with the first-order feature data to obtain first-order difference information; (4) adjusting the parameters of the primary data correction model according to the first-order difference information to obtain a secondary data correction model, and correcting the standard data set based on the secondary data correction model to obtain a second-order corrected data set; (5) extracting the second-order feature data of the second-order corrected data set, comparing the standard feature data with the second-order feature data, and obtaining second-order difference information; (6) calculating a first similarity between the first-order feature data and the standard feature data; (7) calculating a second similarity between the second-order feature data and the standard feature data; (8) comparing the first similarity with the second similarity to obtain a comparison result; (9) When it is determined that the comparison result is that the first similarity is greater than the second similarity, return to (2) and reconstruct the primary data correction model; otherwise, adjust the secondary data correction model according to the second-order difference information to obtain a tertiary data correction model; Perform iterative training until the i-th order feature data is equal to the standard feature data; (10) using the i-level data correction model corresponding to the i-order characteristic data as a standard correction model; (11) Correcting the basic database according to the standard correction model.

2. The AI ​​system training method based on big data according to claim 1, characterized in that: The acquiring of basic data for training the AI ​​system and establishing a basic database according to the basic data includes: Get the data source sent by the user; Preprocess the data source, extract data for training the AI ​​system and mark it as basic data; A basic database is established according to the basic data.

3. The AI ​​system training method based on big data according to claim 2, characterized in that: The obtaining of the data source sent by the user includes: Provide users with extended services including big data storage, cloud deployment and data processing; The data source sent by the user in the extended service is obtained.

4. The AI ​​system training method based on big data according to claim 1, characterized in that: The attribute information includes level and category.

5. The AI ​​system training method based on big data according to claim 4, characterized in that: Also includes: The level is the level of training services required by the AI ​​system to be trained, including at least one of basic training, advanced training, and customized services; The category is the category of application of the AI ​​system, including at least one of recognition, classification, image segmentation, and defect detection.

6. The AI ​​system training method based on big data according to claim 1, characterized in that: Before receiving the AI ​​system to be trained sent by the AI ​​developer, it also includes: Collect information released by AI developers; Performing semantic recognition on the information to obtain a semantic recognition result; Determining whether the information includes AI training requirements according to the semantic recognition result; When determining AI training needs, identify the AI ​​developers who need AI training services.

7. The AI ​​system training method based on big data according to claim 1, characterized in that: When performing a training task on the AI ​​system to be trained according to the target data, it also includes: Obtaining application and feedback information of the AI ​​system to be trained on the target data; Determining abnormal data based on the application and feedback information; Analyzing the abnormal data; The basic database is updated according to the analysis results.

8. The AI ​​system training method based on big data according to claim 1, characterized in that: The step of correcting the basic database according to the standard correction model comprises: Establishing a sorter grid according to the level and category of the basic data; deploying a cluster of mapping unit sorters in the sorter grid; implanting the standard correction model in the mapping unit sorter cluster; The sorter grid is connected to the mapping grid; The mapping grid includes mapping units, the mapping units are connected to the addresses of the basic data in the basic database, and the mapping unit sorter cluster is connected to the addresses of the basic data through the mapping units; The standard correction model corrects the universality of the basic database by correcting the connection relationship between the mapping unit sorter cluster and the mapping unit without changing the basic data.

Images