Model optimization methods, devices and storage media

Abstract

This application provides a model optimization method, device, and storage medium. The method involves testing the accuracy of a first model pre-trained on a first dataset using a second dataset to obtain test results. Based on these test results, a third dataset is extracted from the second dataset, where the test results correspond to datasets inaccurate in the second dataset. Scene information for each sample in the third dataset is analyzed, and an optimization scheme is generated based on this scene information. This optimization scheme is used to update the first model to generate a second model. This approach improves the model's recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model optimization method, device and storage medium. Background Technology

[0002] Automated machine learning (AML) technology has been widely researched and applied due to its ability to effectively alleviate the shortage of talent in model development. Currently, AML technology utilizes artificial intelligence platforms to evaluate and optimize model performance by identifying easily confused labels. Specifically, this method involves identifying commonalities among the target objects in samples with incorrect identification results to determine the cause of the misidentification. Based on these causes, the samples are then optimized to improve model performance. However, since the accuracy of model identification is influenced by multiple factors, relying solely on the commonalities among the target objects in incorrectly identified samples is insufficient to significantly improve model accuracy. Summary of the Invention

[0003] This application provides a model optimization method, device, and storage medium, which aims to generate a model optimization scheme by analyzing the impact of scene information of samples on the model recognition accuracy, and then optimize the model according to the optimization scheme to improve the model's recognition accuracy.

[0004] In a first aspect, embodiments of this application provide a model optimization method, including:

[0005] Obtain the first model pre-trained based on the first dataset;

[0006] The accuracy of the first model was tested using the second dataset, and the test results were obtained.

[0007] Based on the test results, a third dataset is extracted from the second dataset, wherein the third dataset is the dataset in the second dataset for which the test results are inaccurate;

[0008] The scene information of all samples in the third dataset is analyzed, and an optimization scheme is generated based on the scene information. The optimization scheme is used to update the first model to generate the second model.

[0009] Secondly, embodiments of this application provide a model optimization device, characterized in that it includes a memory and a processor;

[0010] The memory is used to store computer programs;

[0011] The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the model optimization method as described in the first aspect above.

[0012] Thirdly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps of the model optimization method described in the first aspect above.

[0013] This application provides a model optimization method, device, and storage medium. The method involves testing the accuracy of a first model pre-trained on a first dataset using a second dataset to obtain test results. Based on these test results, a third dataset corresponding to samples with inaccurate test results is extracted from the second dataset. Scene information for all samples in the third dataset is analyzed, and an optimization scheme is generated based on this scene information. This optimization scheme is used to update the first model to generate a second model. By analyzing the impact of sample scene information on model recognition accuracy, an optimization scheme is generated, and the model is then optimized according to this scheme to improve its recognition accuracy.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit the disclosure of the embodiments of this application. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the architecture of a model optimization device disclosed in an embodiment of this application;

[0017] Figure 2 This is a schematic diagram of the architecture of the model optimization system provided in the embodiments of this application;

[0018] Figure 2 This is a schematic diagram of the architecture of the model optimization system provided in the embodiments of this application;

[0019] Figure 3 This is a schematic diagram illustrating the implementation flow of a model optimization method provided in an embodiment of this application;

[0020] Figure 4 yes Figure 3 The detailed implementation flowchart of S305 in China;

[0021] Figure 5 yes Figure 4 The first detailed implementation flowchart of S412 in China;

[0022] Figure 6 yes Figure 4 The second specific implementation flowchart of S412;

[0023] Figure 7 yes Figure 4 The third detailed implementation flowchart of S412;

[0024] Figure 8 This is a schematic diagram illustrating the implementation flow of a model optimization method provided in another embodiment of this application;

[0025] Figure 9 This is a schematic diagram of an application scenario of the model optimization method provided in the embodiments of this application;

[0026] Figure 10 yes Figure 9 The diagram shows the distribution of scene information in the third dataset in the application scenario shown.

[0027] Figure 11 This is a schematic block diagram of a model optimization device provided in one embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0029] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0030] It should be noted that the model optimization method, device and storage medium provided in this application are used to effectively optimize the model and improve the model's recognition accuracy.

[0031] Currently, artificial intelligence (AI) has received widespread attention from academia and industry, demonstrating performance exceeding that of ordinary humans in many application areas. For example, the application of AI technology in machine vision (such as facial recognition, image classification, and object detection) has resulted in machine vision accuracy exceeding that of humans. AI technology also has good applications in natural language processing and recommendation systems.

[0032] Machine learning is a core method for achieving artificial intelligence. Computers build a model based on existing data to solve a technical problem, and then use this model to reason about unknown data to obtain a result. This method is analogous to a computer learning a certain ability (such as cognitive ability, discrimination ability, classification ability, etc.) like a human; therefore, it is called machine learning.

[0033] In existing technologies, after training an initial model, an AI platform uses the model to infer results from an evaluation dataset. Then, it determines optimization strategies for the model based on the commonalities between the inference results and the samples in the evaluation dataset. However, since the commonalities of the samples are a rather general concept, they cannot accurately provide information affecting the model's recognition accuracy, thus limiting the information available for model optimization.

[0034] This application discloses a model optimization method, device, and storage medium. The model optimization method can analyze the scene information of samples in the evaluation dataset through the model, and generate an optimization scheme for the model based on the analysis results. This can more effectively optimize the model and improve the model's recognition accuracy.

[0035] To better understand the model optimization method, device, and storage medium disclosed in the embodiments of this application, the scenario architecture used in the embodiments of this application is described below. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of the architecture of a model optimization device disclosed in an embodiment of this application. Figure 1 As shown, the model optimization device 10 integrates a model optimization system 111. The model optimization device 10 can be a server or a terminal device. The server can be a remote server, a cloud server, or a server cluster, etc., that can be used to optimize the model. The terminal device can be a laptop, PAD, handheld device, robot, or smart wearable device, etc. The model optimization system 111 integrated on the model optimization device 10 is an application program.

[0036] It is understood that the model optimization method described in the embodiments of this application can be applied to all application scenarios in which a model optimization system 111 is integrated in the model optimization device 10 and the model is optimized by the model optimization system 111.

[0037] It should be understood that a training system 112 and a testing system 113 may also be integrated into the model optimization device 10. Correspondingly, the training system 112, the testing system 113, and the model optimization system 111 can be integrated into a single computer program. In the model optimization device 10, these programs serve as the same application to train the initial model, test the first model, and then optimize the first model based on the test results. Alternatively, the training system 112, the testing system 113, and the model optimization system 111 can also be integrated into the model optimization device 10 as three different applications, with the model optimization device 10 performing its respective functions by calling the corresponding applications.

[0038] It should be understood that when the model optimization device 10 has high computing power, such as being a server or server cluster, the corresponding training system 112, testing system 113, and model optimization system 111 are all integrated into the model optimization device 10. When the computing power of the model optimization device 12 is limited, it is possible to consider integrating only the training system 112 and model optimization system 111, or the testing system 113 and model optimization system 111, or only model optimization system 111, into the model optimization device 10. Correspondingly, the remaining systems can be integrated into an artificial intelligence platform.

[0039] For example, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the architecture of the model optimization system provided in an embodiment of this application. Figure 2 As shown, the model optimization system 20 includes an artificial intelligence platform 11 and a model optimization device 10. It should be noted that the artificial intelligence platform 11 can be an entity deployed in a cloud environment to provide cloud services to various terminal devices. The cloud environment includes a cloud data center and a cloud service platform. The cloud data center includes a large amount of basic resources owned by the cloud service provider, such as the initial model; the computing resources included in the cloud data center can be a large number of computing devices, such as servers or server clusters.

[0040] In this embodiment, for example, the artificial intelligence platform 11 may include a training system 112 and a testing system 113. The training system 112 and the testing system 113 can provide model training and testing services through the artificial intelligence platform 11.

[0041] It should be understood that this embodiment schematically deploys the training system 112 and the testing system 113 on the artificial intelligence platform 11. In practical applications, the artificial intelligence platform 11 may include only the training system 112 or the testing system 113. Corresponding to different computing power of terminal devices, it is possible to choose to deploy both the training system 112 and the testing system 113 on the artificial intelligence platform, or to choose to deploy only one of the training system 112 or the testing system 113 on the artificial intelligence platform. Specifically, it can be determined according to the computing power of the model optimization device 10 in the actual application scenario to ensure the optimization efficiency of the first model.

[0042] In some embodiments, the training system 112 is used to train a preset initial model in the artificial intelligence platform 11 based on a first dataset to generate a first model. The first dataset may consist of sample images acquired by the model optimization device 10, or sample images pre-acquired by other devices and stored in the artificial intelligence platform.

[0043] The first dataset is not limited to images, text, audio, video, etc.; no single limitation is made here.

[0044] In the embodiments of this application, the first model is a model pre-trained by the training system 112 using automated machine learning modeling technology. Since the training process of the first model relies entirely on automated machine learning modeling technology, and the algorithm of automated machine learning modeling technology is fixed, it cannot flexibly address factors affecting the model's recognition accuracy during training, resulting in recognition errors in the trained first model. Therefore, in this embodiment, the first model needs to be tested by the testing system 113, and a corresponding optimization scheme needs to be generated based on the test results to ensure the optimization effect on the first model.

[0045] Specifically, the testing system 113 is used to test the accuracy of the first model trained by the training system 112 based on the first dataset, and to test the model based on the second dataset, thereby obtaining test results. The second dataset may consist of sample images acquired by the model optimization device 10, or sample images pre-acquired by other devices and stored in the artificial intelligence platform.

[0046] It should be understood that the first dataset and the second dataset may include different numbers of sample images or other sample data, and this is not a unique limitation.

[0047] The model optimization device 12 acquires the test results obtained by the test system 113, extracts the third dataset from the second dataset based on the test results, analyzes the scene information of all samples in the third dataset, generates an optimization scheme based on the scene information, and optimizes the first model based on the optimization scheme to obtain the second model.

[0048] For example, the scene information includes, but is not limited to, at least one of image scene information, light scene information, and density scene information.

[0049] It should be understood that both the training system 112 and the testing system 113 are program codes with different functions integrated on the artificial intelligence platform 11, and can also be referred to as applications running on the artificial intelligence platform 11. In some embodiments, the testing system 113 and the model optimization system 111 can be integrated into a piece of program code, and can be simultaneously integrated into a target application on the model optimization device 10, for the purpose of completing the testing and optimization functions of the first model.

[0050] Please see Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the implementation flow of a model optimization method provided in an embodiment of this application. The model optimization method provided in this embodiment can be implemented by... Figure 1 The model optimization method shown is completed on device 10. Details are as follows:

[0051] S301, Obtain the first model pre-trained based on the first dataset.

[0052] The first dataset can be, but is not limited to, pre-collected sample images; it is not the only example and is not a specific limitation. These sample images correspond to image data within a preset application scenario. For example, if the preset application scenario is to identify the safety attire of construction workers, then the corresponding image data for that application scenario would be construction site images containing workers, collected by a preset image acquisition device. In this embodiment, the first model is a model obtained by pre-training an initial model using the first dataset based on automated machine learning modeling techniques.

[0053] For example, during the process of training the initial model using the first dataset according to the automatic machine learning model technology, the training can be terminated based on the change of the preset loss function corresponding to the initial model or based on the convergence status of the model function of the initial model as training progresses.

[0054] For example, when the value of the preset loss function is less than a preset function threshold, such as 0.3, the initial model training is considered complete, and the trained model is taken as the first model. Alternatively, when the rate of change of each preset parameter value of the model function is less than a preset rate of change, the model function converges, and the initial model training is considered complete, thus obtaining the first model.

[0055] For example, the process of obtaining the first dataset may include: establishing sample index information for the model, obtaining a preset number of sample images from a predetermined database based on the sample index information, and generating the first dataset.

[0056] S302, Perform model accuracy testing on the first model based on the second dataset to obtain test results.

[0057] In some embodiments, the test results include: results that accurately identify both the category and location of the target object, results that incorrectly identify the category of the target object, results that incorrectly identify the location of the target object, and results that fail to identify the target object; wherein, the results that incorrectly identify the category of the target object and the results that incorrectly identify the location of the target object are collectively referred to as misidentification results.

[0058] It should be understood that the second dataset corresponds to a dataset containing the same sample data as the first dataset. That is, the sample data includes a training sample set and a test sample set, with the first dataset being the training sample set and the second dataset being the test sample set. The training sample set and the test sample set may contain different numbers of sample data, which may be images.

[0059] S303, Based on the test results, extract a third dataset from the second dataset, wherein the third dataset is the dataset in the second dataset for which the test results are inaccurate.

[0060] For example, extracting a third dataset from the second dataset based on the test results may include: extracting misidentification results from the test results, obtaining samples corresponding to the misidentification results from the second dataset, and obtaining the third dataset.

[0061] It should be understood that the third dataset includes at least one of the following: samples corresponding to the results of incorrect category identification of the target object, samples corresponding to the results of incorrect location identification of the target object, and samples corresponding to the results of missing target object identification.

[0062] S304, Analyze the scene information of all samples in the third dataset, generate an optimization scheme based on the scene information, and use the optimization scheme to update the first model to generate the second model.

[0063] The scene information may include, but is not limited to, at least one of the following: image scene information, target object density scene information, target object integrity scene information, target object size scene information, and target object and background similarity scene information.

[0064] In some embodiments, analyzing the scene information of all samples in the third dataset and generating an optimization scheme based on the scene information includes:

[0065] Determine at least one scenario corresponding to all samples in the third dataset, wherein each scenario in the at least one scenario corresponds to a scenario information list, and the scenario information list includes at least one scenario information; obtain the first scenario information corresponding to the at least one scenario, and generate the optimization scheme based on the first scenario information.

[0066] It should be understood that different samples in the third dataset may correspond to different scenarios, and the same sample may correspond to multiple scenarios. In this embodiment, the scenario information corresponding to each sample is recorded in a scenario information list.

[0067] For example, the scene information list includes scene categories and sample quantity. For instance, the scene information list might be displayed as: Image Scene Information, 10; indicating 10 samples where the test results are inaccurate due to the influence of image scene information. Here, image scene information includes, but is not limited to, scene information where the image is unclear due to the image acquisition device or image acquisition environment. Another example is: The scene information list might be displayed as: Target Object Integrity, 18; indicating 18 samples where the test results are inaccurate due to the incompleteness of the target object. Here, target object integrity can be measured using target object completeness.

[0068] In some embodiments, determining at least one scene corresponding to all samples in the third dataset includes: extracting scene index values ​​of all samples in the third dataset under a preset scene; comparing the scene index values ​​with the index thresholds under the preset scene respectively; if a sample has a scene index value that meets the index thresholds under the preset scene, then determining the preset scene as the image scene of the corresponding sample.

[0069] For example, the scene index value is used to measure the degree of influence of a sample on the recognition result of the first model under a preset scene. For example, the scene index value includes, but is not limited to, illumination intensity, target object density, foreground and background image color similarity, target completeness, and target size. The index threshold under the preset scene refers to a scene index critical value. When the scene index value of a sample in the corresponding scene is greater than the corresponding scene index critical value, it affects the accuracy of the recognition result of the first model. Alternatively, depending on the preset scene, when the scene index value of a sample in the corresponding scene is less than the corresponding scene index critical value, it affects the accuracy of the recognition result of the first model.

[0070] It should be understood that the scene index threshold value may not be limited to one. For example, when the scene index value is the light intensity value, there are corresponding first scene index threshold values ​​and second scene index threshold values. Specifically, when the light intensity value is less than the first scene index threshold value, it is determined to be an image scene with insufficient light, and when the light intensity value is greater than the first scene index threshold value, it is determined to be an image scene with too much light.

[0071] In some embodiments, before obtaining the first scene information corresponding to the at least one scene and generating the optimization scheme based on the first scene information, the method further includes: obtaining the second scene information corresponding to the at least one scene. It should be understood that the recognition result of the first model is affected by at least one scene, which may include multiple scene information or only one scene information. For example, in an application scenario for detecting the clothing of construction workers, the first scene information is the density of the target object, and the second scene information is the integrity of the target object. Similarly, in an application scenario for recognizing a specific animal such as an owl, the scene information is only the posture information of the target object; in this case, both the first and second scene information are the posture information of the target object. Since the recognition result of the first model may be affected by multiple scene information simultaneously, this application, in order to ensure that the influence of each scene information is fully considered during the optimization process of the first model, obtains the first scene information and the second scene information from at least one scene information before generating the optimization scheme for the first model. By analyzing the correlation between the first scene information and the second scene information, it ensures that the influence of all scene information on the first model is covered, thereby improving the accuracy of the optimization of the first model.

[0072] For example, such as Figure 4 As shown, Figure 4 yes Figure 3 A flowchart illustrating the specific implementation of S305 in China. Figure 4 It can be seen that S305 can optionally include S411 and S412. Details are as follows:

[0073] S411, determine the association between the first scene information and the second scene information.

[0074] S412, Determine the optimization scheme based on the aforementioned correlation.

[0075] It should be understood that the first scene information and the second scene information can be completely correlated, meaning the second scene information includes all scene information in the first scene information; or the first scene information and the second scene information can be partially correlated, meaning the second scene information includes some scene information in the first scene information; or the first scene information and the second scene information can be completely uncorrelated, meaning the second scene information does not include any scene information in the first scene information. Corresponding to different correlation relationships, different optimization schemes are adopted to improve the optimization effect of the first model, resulting in a second model with higher recognition accuracy.

[0076] In some embodiments, the association relationship is that the second scene information includes all scene information in the first scene information. For example, as... Figure 5 The above, Figure 5 yes Figure 4 The first detailed implementation flowchart of S412 is as follows:

[0077] S511, if the second scene information includes all scene information in the first scene information, then optimize all samples in the third dataset according to the second scene information.

[0078] Optionally, optimizing all samples in the third dataset based on the second scenario information includes: adjusting the scenario indicator values ​​of all samples in the third dataset under the corresponding scenario according to the second scenario information based on the second scenario information; and completing the optimization of all samples in the third dataset when the scenario indicator values ​​of all samples in the third dataset under the corresponding second scenario information meet the preset indicator threshold.

[0079] It should be understood that the preset indicator thresholds can be pre-set according to the scene information and can be modified as the scene information changes and as it affects the model's recognition accuracy. Furthermore, depending on the scene information, a first indicator threshold and a second indicator threshold can be set for different scenes. Specifically, the setting of indicator thresholds is not limited here.

[0080] S512, based on the optimized third dataset, analyzes and updates the corresponding target algorithm to form the target computation framework.

[0081] The target algorithm is the model architecture of the first model. The target computation framework is a model architecture that matches the optimized third dataset by adjusting the model architecture corresponding to the target algorithm.

[0082] It should be understood that the model architecture of the first model includes, but is not limited to, neural network models, random forest models, or decision tree models. For example, if the first model is a neural network model, the model architecture of the neural network model includes the number of convolutional layers, the function expression for each convolutional layer, the function expression for the connection layers, the loss function expression, and the output function expression.

[0083] S513, Update the first model according to the target computing framework.

[0084] Specifically, updating the first model according to the target computing framework includes: inputting the optimized third dataset into the target computing framework for training. The process of training the target computing framework can refer to existing model training processes and will not be elaborated here.

[0085] In some embodiments, the association relationship is that the second scene information includes a portion of the scene information in the first scene information; for example, such as Figure 6 As shown, Figure 6 yes Figure 4 The second detailed implementation flowchart of S412 is as follows:

[0086] S611, if the second scene information includes part of the scene information in the first scene information, then the scene corresponding to the first scene information and the scene corresponding to the second scene information are combined, and a first optimization strategy is generated based on the combined scene.

[0087] In this embodiment, the intersection of the scene corresponding to the first scene information and the scene corresponding to the second scene information is determined, and the scene information within the intersection and the scene information outside the intersection are merged to complete the combination of the scene included in the first scene information and the scene included in the second scene information.

[0088] It should be noted that the first optimization strategy includes, but is not limited to, the first data supplementation strategy. For example, the first optimization strategy may also include an optimization scheme for the model algorithm of the first model, which will not be detailed here.

[0089] The first data supplementation strategy includes the proportion of sample images to be supplemented in each merged scene. Specifically, the proportion of sample images to be supplemented in each merged scene is determined based on the distribution of sample images in each merged scene, and the number of samples to be supplemented in each scene is determined based on the proportion of sample images to be supplemented in each merged scene.

[0090] For example, based on the distribution of sample images of each scene after merging, the proportion of sample images of each scene included in the merged dataset to the total samples in the third dataset is determined, and the determined proportion of sample images of each scene included in the merged dataset to the total samples in the third dataset is the proportion of sample images of each scene that need to be supplemented after merging.

[0091] S612, Update the first model according to the first optimization strategy.

[0092] For example, updating the first model according to the first optimization strategy includes: obtaining a first supplementary dataset according to the first data supplementation strategy, and updating the first model according to the first supplementary dataset.

[0093] The process of obtaining a first supplementary dataset according to the first data supplementation strategy includes: acquiring sample images for each scene according to the proportion of the merged sample images to be supplemented, thus obtaining the first supplementary dataset. Specifically, the sample images can be obtained from a predetermined database, where sample index information for each scene is pre-established, and sample images for each scene are obtained from the database based on the sample index information. This improves the efficiency of sample image acquisition.

[0094] The sample index information can be any preset number, letter, or combination of number and letter, etc., without any specific restrictions.

[0095] It should be understood that updating the first model based on the first supplementary dataset includes: inputting the first supplementary dataset into the first model, and training the first model based on the first supplementary dataset. The process of training the first model can refer to existing model training processes.

[0096] For example, the end of updating the first model can be determined by monitoring changes in the loss function of the first model or by monitoring the convergence state of the first model.

[0097] For example, if the value of the loss function of the first model is detected to be less than a preset loss threshold, then the update of the first model is determined to be complete. Or, if the rate of change of the parameters of the first model within a preset time period is detected to be less than a preset rate of change threshold, then the update of the first model is determined to be complete, and the optimized second model is obtained. The preset loss threshold can be set by the user or obtained from historical data, and is not limited to a single value here.

[0098] In some embodiments, the association relationship is that the second scene information does not include the first scene information; for example, such as Figure 7 The above, Figure 7 yes Figure 4 The third detailed implementation flowchart of S412 is as follows:

[0099] S711, if the second scene information does not include the first scene information, then generate a second optimization strategy based on the scene corresponding to the first scene information and generate a third optimization strategy based on the scene corresponding to the second scene information.

[0100] For example, the second optimization strategy is a second data supplementation strategy. It should be understood that the second optimization strategy includes, but is not limited to, a second data supplementation strategy. For example, the second optimization strategy may be different optimization schemes for the model algorithm of the first model, which will not be described in detail here.

[0101] In this embodiment, the second data supplementation strategy is the number of samples in each application scenario corresponding to the first scenario information that needs to be supplemented. For example, based on the distribution of sample images in each scenario corresponding to the first scenario information, the proportion of sample images in each scenario corresponding to the first scenario information that needs to be supplemented is determined; and based on the proportion of sample images in each scenario corresponding to the first scenario information that needs to be supplemented, the number of samples in each application scenario corresponding to the first scenario information that needs to be supplemented is determined.

[0102] For example, based on the distribution of sample images in each scene corresponding to the first scene information, the proportion of sample images in each scene corresponding to the first scene information to the total samples in the third dataset is determined, and the determined proportion of sample images in each scene corresponding to the first scene information to the total samples in the third dataset is the proportion of sample images in each application scene corresponding to the first scene information that needs to be supplemented.

[0103] For example, the third optimization strategy is a third data supplementation strategy. It should be understood that the third optimization strategy includes, but is not limited to, a third data supplementation strategy. For instance, the third optimization strategy could be an optimization scheme for the model algorithm of the first model, the details of which will not be elaborated here.

[0104] The third data supplementation strategy refers to the number of samples in each application scenario corresponding to the second scenario information that needs to be supplemented. The process of determining the number of samples in each application scenario corresponding to the second scenario information is the same as the process of determining the number of samples in each application scenario corresponding to the first scenario information, and will not be repeated here.

[0105] S712, the first model is updated according to the second optimization strategy and the third optimization strategy.

[0106] For example, updating the first model according to the second optimization strategy and the third optimization strategy includes: obtaining a target supplementary dataset according to the second data supplementation strategy and the third data supplementation strategy, and updating the first model according to the target supplementary dataset.

[0107] The step of updating the first model based on the target supplementary dataset includes: determining whether the target supplementary dataset matches the target algorithm of the first model; if the target supplementary dataset matches the target algorithm, then updating the first model based on the target supplementary dataset; if the target supplementary dataset does not match the target algorithm, then generating a prompt message indicating that the optimization scheme is incorrect.

[0108] It should be understood that updating the first model based on the target supplementary dataset includes: inputting the target supplementary dataset into the first model, and training the first model based on the target supplementary dataset. The training process for the first model can refer to existing model training processes. For example, the end of updating the first model can be determined by monitoring changes in the loss function of the first model or by monitoring the convergence state of the first model. For instance, if the value of the loss function of the first model is found to be less than a preset loss threshold, the end of updating the first model is determined. Or, if the rate of change of the parameters of the first model within a preset time period is found to be less than a preset rate of change threshold, the end of updating the first model is determined, and the optimized second model is obtained.

[0109] In some embodiments, to improve the efficiency of updating the first model, obtaining a target supplementary dataset according to the second data supplementation strategy and updating the first model according to the target supplementary dataset includes: determining whether the target supplementary dataset matches the target algorithm of the first model; if the target supplementary dataset matches the target algorithm, updating the first model according to the target supplementary dataset; if the target supplementary dataset does not match the target algorithm, generating a prompt message indicating that the optimization scheme is incorrect.

[0110] As can be seen from the above analysis, the model optimization method provided in this application embodiment tests the accuracy of a first model pre-trained on a first dataset using a second dataset to obtain test results; based on the test results, a third dataset is extracted from the second dataset, the third dataset being the dataset corresponding to the inaccurate test results in the second dataset; the scene information of all samples in the third dataset is analyzed, and an optimization scheme is generated based on the scene information. This optimization scheme is used to update the first model to generate a second model. By analyzing the impact of the scene information of the samples on the model's recognition accuracy, generating an optimization scheme for the model based on the scene information of the samples, and then optimizing the model according to the optimization scheme, the recognition accuracy of the model is improved.

[0111] Please see Figure 8 As shown, Figure 8 This is a schematic diagram illustrating the implementation flow of a model optimization method provided in another embodiment of this application. The model optimization method provided in this embodiment can be implemented by... Figure 2 The model optimization system 20 shown is now complete. It should be noted that this embodiment exemplifies how the training and testing process of the first model is completed by an artificial intelligence platform, and the model optimization device optimizes the model based on the test results. It should be understood that the training or testing process of the first model can also be completed by the model optimization device; for details, please refer to [reference needed]. Figure 2 The descriptions in the document will not be repeated here. Exemplarily, embodiments of this application include the following steps S801 to S805.

[0112] S801, the artificial intelligence platform acquires a preset number of sample images and divides the sample images into a first dataset and a second dataset with a preset ratio.

[0113] S802, the artificial intelligence platform trains the model based on the first dataset and generates the first model.

[0114] S803, the artificial intelligence platform performs a model accuracy test on the first model based on the second dataset and obtains the test results.

[0115] S804, the model optimization device extracts a third dataset from the second dataset based on the test results, wherein the third dataset is the dataset in the second dataset for which the test results are inaccurate.

[0116] S805, the model optimization device analyzes the scene information of all samples in the third dataset, generates an optimization scheme based on the scene information, and the optimization scheme is used to update the first model to generate the second model.

[0117] It should be noted that the specific implementation principles of the above steps are different from those of... Figure 3The specific implementation principles of each step in the illustrated embodiments are similar and will not be repeated here.

[0118] As can be seen from the above analysis, the model optimization method provided in this application embodiment tests the accuracy of a first model pre-trained on a first dataset using a second dataset to obtain test results; based on the test results, a third dataset is extracted from the second dataset, the third dataset being the dataset in the second dataset corresponding to inaccurate test results; the scene information of all samples in the third dataset is analyzed, and an optimization scheme is generated based on the scene information. This optimization scheme is used to update the first model to generate a second model. By analyzing the impact of the scene information of the samples on the model's recognition accuracy, generating an optimization scheme for the model based on the scene information of the samples, and then optimizing the model according to the optimization scheme, the recognition accuracy of the model is improved.

[0119] Please see Figure 9 As shown, Figure 9 This is a schematic diagram illustrating an application scenario of the model optimization method provided in this application embodiment. In this embodiment, the model optimization method is applied to optimize a clothing detection model for construction workers at a construction site. In this application scenario, construction site images taken by at least one image acquisition device 90 installed at the construction site at different time periods and corresponding construction locations can be acquired. Since the construction site images 91 taken at different time periods and construction locations are diverse, the initial model is trained using diverse samples to achieve the adaptability of the first model to the image scene. However, since the construction site is greatly affected by the surrounding environment, such as light intensity, tools used in construction progress, and the actions of construction workers, the recognition accuracy of the first model will be affected. In this embodiment, to improve the recognition accuracy of the clothing detection model, construction images are divided into a first dataset and a second dataset with a preset ratio. The initial model is trained using the first dataset to obtain a first model. Then, the accuracy of the second model is tested using the second dataset to obtain test results. Based on the test results, a third dataset is extracted from the second dataset, which corresponds to the dataset in the second dataset where the test results are inaccurate. The scene information of all samples in the third dataset is analyzed, and an optimization scheme is generated based on the scene information. The optimization scheme is used to update the first model to generate a second model.

[0120] It should be understood that in this embodiment, the model accuracy test of the second model is performed based on the second dataset, and the test results include the results of labeling whether the target object is wearing a safety helmet. Specifically, the correct labeling results for whether the target object is wearing a safety helmet include: labeling the head of the target object wearing a safety helmet as "helmet", labeling the head of the target object not wearing a safety helmet as "head", not labeling locations that are not target objects with target object boxes, and labeling the locations of target objects with target object boxes.

[0121] Incorrect labeling results for whether a target object is wearing a safety helmet include: labeling the head of a target object wearing a safety helmet as "head", labeling the head of a target object not wearing a safety helmet as "helmet", labeling a location that is not a target object as a target object bounding box, and not labeling the location of a target object as a target object bounding box.

[0122] Further, based on the test results corresponding to all samples in the second dataset, samples corresponding to erroneous annotations regarding whether the target object is wearing a safety helmet are extracted from the second test samples to obtain the third dataset. The scene information corresponding to all samples in the third dataset is analyzed. For example, the scene information corresponding to all samples in the third dataset includes at least one of the following: image illumination quality, target object size in the image, target object completeness in the image, image data balance, target object category distinguishability in the image, target object density in the image, and target object pose information in the image.

[0123] For example, Figure 10 yes Figure 9 The diagram illustrates the distribution of scene information in the third dataset within the application scenario shown. Figure 10 As can be seen, in this embodiment, after analyzing the scene information in the third dataset, the scene information obtained includes the integrity of the target object in the image, the size of the target object in the image, and the lighting quality of the image.

[0124] For example, in the third dataset, samples containing incomplete target objects account for 25% of the total number of samples in the third dataset, samples containing target objects with unqualified sizes account for 22% of the total number of samples in the third dataset, and samples containing images with unqualified lighting quality account for 17% of the total number of samples in the third dataset.

[0125] It should be understood that after obtaining the distribution results corresponding to the scene information in the third dataset, supplementary prompt information for the corresponding image scene can be generated, either based on the corresponding proportion of samples or based on the determined distribution results. In this embodiment, based on the distribution results corresponding to the scene information in the third dataset, it can be determined that 25% of the total samples in the third dataset contain complete target objects, 22% of the total samples in the third dataset contain target objects of acceptable size, and 17% of the total samples in the third dataset have lighting quality that meets the preset lighting conditions.

[0126] After determining the additional sample data needed, the required samples for each corresponding image scene are obtained to form a new training set. The first model for that application scene is then updated based on this new training set, resulting in an optimized first model. This effectively avoids the impact of the corresponding image scene on the model's recognition accuracy, thus improving the model's recognition accuracy.

[0127] It should be noted that the above embodiments are merely illustrative of the application of the model optimization method provided in this application, and are not intended to limit the application of the model optimization method provided in this application.

[0128] Please see Figure 11 As shown, Figure 11 This is a schematic block diagram of a model optimization device provided in one embodiment of this application.

[0129] For example, the model optimization device 10 can be a server or a terminal. The server can be a cloud server, a remote server, a server cluster, etc., and the terminal device can be a user's electronic device such as a mobile phone, tablet computer, laptop computer, desktop computer, personal digital assistant, and wearable device.

[0130] The model optimization device 10 includes a processor 101 and a memory 102.

[0131] For example, processor 101 and memory 102 are connected via bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0132] Specifically, the processor 101 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0133] Specifically, the memory 102 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0134] The processor 101 is used to run a computer program stored in the memory 102, and implements the steps of the above-described model optimization method when executing the computer program.

[0135] For example, the processor 101 is configured to run a computer program stored in the memory 102, and when executing the computer program, perform the following steps:

[0136] Obtain the first model pre-trained based on the first dataset;

[0137] The accuracy of the first model was tested using the second dataset, and the test results were obtained.

[0138] Based on the test results, a third dataset is extracted from the second dataset, wherein the third dataset is the dataset in the second dataset for which the test results are inaccurate;

[0139] The scene information of all samples in the third dataset is analyzed, and an optimization scheme is generated based on the scene information. The optimization scheme is used to update the first model to generate the second model.

[0140] In some embodiments, analyzing the scene information of each sample in the third dataset and generating an optimization scheme based on the scene information includes:

[0141] Determine at least one scenario corresponding to all samples in the third dataset, wherein each scenario in the at least one scenario corresponds to a scenario information list, and the scenario information list includes at least one scenario information;

[0142] Obtain first scene information corresponding to the at least one scene, and generate the optimization scheme based on the first scene information.

[0143] In some embodiments, determining at least one scenario corresponding to all samples in the third dataset includes:

[0144] Extract the scene index values ​​of all samples in the third dataset under the preset scenario;

[0145] The scene indicator values ​​are compared with the threshold values ​​of each indicator under the preset scene;

[0146] If a sample corresponds to a scenario indicator value that meets the indicator threshold of the preset scenario, then the preset scenario is determined to be the scenario of the corresponding sample.

[0147] In some embodiments, before obtaining the first scene information corresponding to the at least one scene and generating the optimization scheme based on the first scene information, the method further includes:

[0148] Obtain the second scene information corresponding to the at least one scene;

[0149] Correspondingly, obtaining the first scene information corresponding to the at least one scene, and generating the optimization scheme based on the first scene information, includes:

[0150] Determine the association between the first scene information and the second scene information;

[0151] Based on the aforementioned correlation, the optimization scheme is determined.

[0152] In some embodiments, the association relationship is that the second scene information includes all scene information in the first scene information; determining the optimization scheme based on the association relationship includes:

[0153] If the second scene information includes all the scene information in the first scene information, then each sample in the third dataset is optimized based on the first scene information;

[0154] The target algorithm is analyzed and updated based on the optimized third dataset to form the target computation framework;

[0155] The first model is updated according to the target computation framework.

[0156] In some embodiments, the association relationship is that the second scene information includes a portion of the scene information in the first scene information; determining the optimization scheme based on the association relationship includes:

[0157] If the second scene information includes part of the scene information in the first scene information, then the image scene corresponding to the first scene information and the image scene corresponding to the second scene information are combined, and a first optimization strategy is generated based on the combined scene.

[0158] The first model is updated according to the first optimization strategy.

[0159] In some embodiments, the association relationship is that the second scene information does not include the first scene information; determining the optimization scheme based on the association relationship includes:

[0160] If the second scene information does not include the first scene information, then a second optimization strategy is generated based on the scene corresponding to the first scene information and a third optimization strategy is generated based on the scene corresponding to the second scene information.

[0161] The first model is updated according to the second optimization strategy and the third optimization strategy.

[0162] In some embodiments, updating the first model according to the second optimization strategy and the third optimization strategy includes:

[0163] The target supplementary dataset is determined based on the second optimization strategy and the third optimization strategy;

[0164] Determine whether the target supplementary dataset matches the target algorithm of the first model;

[0165] If the target supplementary dataset matches the target algorithm, then the first model is updated based on the target supplementary dataset;

[0166] If the target supplementary dataset does not match the target algorithm, a prompt message indicating that the optimization scheme is incorrect will be generated.

[0167] The specific principle and implementation method of the model optimization device provided in this embodiment are similar to the model optimization method implementation process in the previous embodiments, and will not be repeated here.

[0168] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the model optimization method provided in the above embodiments.

[0169] The computer-readable storage medium can be an internal storage unit of the model optimization device described in the foregoing embodiments, such as the hard drive or memory of the model optimization device. Alternatively, the computer-readable storage medium can be an external storage device of the model optimization device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the model optimization device.

[0170] It should be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application.

[0171] It should also be understood that the term “and / or” as used in this application and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0172] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model optimization method, characterized in that, include: Obtain the first model pre-trained based on the first dataset; The accuracy of the first model was tested using the second dataset, and the test results were obtained. Based on the test results, a third dataset is extracted from the second dataset, wherein the third dataset is the dataset in the second dataset for which the test results are inaccurate; Analyze the scene information of all samples in the third dataset, generate an optimization scheme based on the scene information, and use the optimization scheme to update the first model to generate the second model; The step of analyzing the scene information of all samples in the third dataset and generating an optimization scheme based on the scene information includes: Determine at least one scenario corresponding to all samples in the third dataset, wherein each scenario in the at least one scenario corresponds to a scenario information list, and the scenario information list includes at least one scenario information; Obtain first scene information corresponding to the at least one scene, and generate the optimization scheme based on the first scene information; Determining at least one scenario corresponding to all samples in the third dataset includes: Extract scene index values ​​for all samples in the third dataset under a preset scenario. The scene index values ​​include at least one of the following: illumination intensity value, target object density, foreground and background image color similarity, target completeness, and target size. Compare the scene indicator value with at least one indicator threshold under the preset scene; If a sample's scenario indicator value in the preset scenario meets the indicator threshold in the preset scenario, then the preset scenario is determined to be the scenario of the corresponding sample. Before obtaining the first scene information corresponding to the at least one scene, and generating the optimization scheme based on the first scene information, the method further includes: Obtain the second scene information corresponding to the at least one scene; The step of obtaining first scene information corresponding to the at least one scene and generating the optimization scheme based on the first scene information includes: Determine the association between the first scene information and the second scene information; Based on the aforementioned correlation, the optimization scheme is determined.

2. The method according to claim 1, characterized in that, The association relationship is that the second scene information includes all scene information in the first scene information; determining the optimization scheme based on the association relationship includes: If the second scene information includes all scene information in the first scene information, then all samples in the third dataset are optimized based on the second scene information; The target algorithm is analyzed and updated based on the optimized third dataset to form a target computation framework, where the target algorithm is the model architecture of the first model. The first model is updated according to the target computation framework.

3. The model optimization method according to claim 1, characterized in that, The association relationship is that the second scene information includes a portion of the scene information in the first scene information; determining the optimization scheme based on the association relationship includes: If the second scene information includes some scene information in the first scene information, then the scene corresponding to the first scene information and the scene corresponding to the second scene information are combined, and a first optimization strategy is generated based on the combined scene. The first model is updated according to the first optimization strategy.

4. The model optimization method according to claim 1, characterized in that, The association relationship is that the second scenario information does not include the first scenario information; determining the optimization scheme based on the association relationship includes: If the second scene information does not include the first scene information, then a second optimization strategy is generated based on the scene corresponding to the first scene information and a third optimization strategy is generated based on the scene corresponding to the second scene information. The first model is updated according to the second optimization strategy and the third optimization strategy.

5. The method according to claim 4, characterized in that, The step of updating the first model according to the second optimization strategy and the third optimization strategy includes: The target supplementary dataset is obtained based on the second optimization strategy and the third optimization strategy; Determine whether the target supplementary dataset matches the target algorithm of the first model; If the target supplementary dataset matches the target algorithm, then the first model is updated based on the target supplementary dataset; If the target supplementary dataset does not match the target algorithm, a prompt message indicating that the optimization scheme is incorrect will be generated.

6. A model optimization device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the model optimization method as described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the steps of the model optimization method as described in any one of claims 1 to 5.

Images