Image processing model training method, device, equipment, storage medium and computer program product

Abstract

The present application relates to the field of computer vision technology, and discloses an image processing model training method, device, equipment, storage medium and computer program product, including: constructing an initial data enhancement strategy based on the data characteristics of a preset training data set; when training a preset image processing model based on the initial data enhancement strategy and the preset training data set, monitoring the first performance indicator of the preset image processing model in real time; adjusting the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy; enhancing the data in the preset training data set based on the target data enhancement strategy, and training the preset image processing model again according to the enhanced data to obtain a first target image processing model. Since the present application adaptively adjusts the data enhancement strategy by real-time monitoring the performance indicators of the model, the model training effect and generalization ability are optimized.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an image processing model training method, apparatus, device, storage medium and computer program product. Background Art

[0002] At present, the existing technology proposes data enhancement methods based on random parameter selection and enhancement strategies based on image generation to enhance data when training image processing models, and trains the model with the enhanced data. Although these methods can increase the diversity of data to a certain extent, they lack the ability to adaptively adjust for specific tasks and data sets, which may lead to overfitting or uneven data quality. Summary of the invention

[0003] The main purpose of this application is to provide an image processing model training method, device, equipment, storage medium and computer program product, aiming to solve the technical problems that the existing image processing model training methods lack adaptive adjustment capabilities for specific tasks and data sets, which may lead to overfitting or uneven data quality.

[0004] To achieve the above objectives, the present application proposes an image processing model training method, the image processing model training method comprising:

[0005] Construct an initial data augmentation strategy based on the data characteristics of the preset training dataset;

[0006] When a preset image processing model is trained based on the initial data enhancement strategy and the preset training data set, a first performance indicator of the preset image processing model is monitored in real time;

[0007] Adjust the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy;

[0008] The data in the preset training data set is enhanced based on the target data enhancement strategy, and the preset image processing model is trained again according to the enhanced data to obtain a first target image processing model.

[0009] Optionally, the step of adjusting the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy includes:

[0010] Comparing the first performance indicator with a preset indicator threshold, and testing the performance of the target image processing model according to the comparison result to obtain a performance test result;

[0011] When the performance detection result shows that the accuracy and recall rate of the target image processing model decrease, adjusting the enhancement parameters of the initial data enhancement strategy according to the set enhancement parameter range to obtain first adjustment information;

[0012] When the performance detection result indicates that the data training speed of the target image processing model decreases, dynamically adjusting the complexity and enhancement type of the initial data enhancement strategy according to a multi-objective optimization algorithm to obtain second adjustment information;

[0013] A target data enhancement strategy is generated according to the first adjustment information and / or the second adjustment information.

[0014] Optionally, after the step of enhancing the data in the preset training data set based on the target data enhancement strategy and retraining the preset image processing model according to the enhanced data to obtain the first target image processing model, the method further includes:

[0015] Verifying the first target image processing model based on a verification data set, and monitoring a second performance indicator of the first target image processing model during the verification process in real time;

[0016] Evaluate and provide feedback on the enhancement effect and model training speed of the current target data enhancement strategy based on the second performance indicator;

[0017] A data enhancement strategy combination is generated according to the data enhancement effect and the model training speed, and the first target image processing model is trained based on the data enhancement strategy combination and the verification data set to obtain a second target image processing model.

[0018] Optionally, after the step of generating a data enhancement strategy combination according to the data enhancement effect and the model training speed, and training the first target image processing model based on the data enhancement strategy combination and the verification data set to obtain the second target image processing model, the method further includes:

[0019] Inputting the preprocessed prediction data into the second target image processing model;

[0020] Perform forward propagation calculation on the predicted data through each layer network in the second target image processing model to generate a prediction result;

[0021] Generate a specific data enhancement strategy based on the prediction result, and enhance the predicted data according to the specific data enhancement strategy to obtain specific target data;

[0022] The second target image processing model is trained based on the specific target data to obtain a specific target image processing model.

[0023] Optionally, after the step of generating a data enhancement strategy combination according to the data enhancement effect and the model training speed, and training the first target image processing model based on the data enhancement strategy combination and the verification data set to obtain the second target image processing model, the method further includes:

[0024] Access multiple public image datasets;

[0025] Performing data enhancement on the image data set according to the enhancement strategy combination to obtain an enhancement result;

[0026] The enhancement result is compared with the data in the image data set before the enhancement, and an analysis report is generated according to the comparison result.

[0027] Optionally, the step of constructing an initial data enhancement strategy based on data characteristics of a preset training data set includes:

[0028] Define corresponding parameter spaces according to different enhancement operations;

[0029] Identify the preset training data set to obtain a recognition result;

[0030] An enhancement operation is determined according to the recognition result, and an initial enhancement parameter and an enhancement parameter range are set from the corresponding parameter space.

[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes an image processing model training device, the image processing model training device comprising:

[0032] A strategy building module is used to build an initial data enhancement strategy based on the data characteristics of a preset training data set;

[0033] A performance detection module, used to monitor a first performance indicator of the preset image processing model in real time when the preset image processing model is trained based on the initial data enhancement strategy and the preset training data set;

[0034] A strategy adjustment module, used to adjust the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy;

[0035] The model training module is used to enhance the data in the preset training data set based on the target data enhancement strategy, and to train the preset image processing model again according to the enhanced data to obtain a first target image processing model.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes an image processing model training device, which includes: a memory, a processor, and an image processing model training program stored in the memory and executable on the processor, and when the image processing model training program is executed by the processor, it implements the image processing model training method described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which an image processing model training program is stored. When the image processing model training program is executed by a processor, the image processing model training method described above is implemented.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, which includes an image processing model training program, and when the image processing model training program is executed by a processor, it implements the image processing model training method described above.

[0039] The present application discloses constructing an initial data enhancement strategy based on the data characteristics of a preset training data set; when training a preset image processing model based on the initial data enhancement strategy and the preset training data set, monitoring a first performance indicator of the preset image processing model in real time; adjusting the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy; enhancing the data in the preset training data set based on the target data enhancement strategy, and training the preset image processing model again according to the enhanced data to obtain a first target image processing model. Since the present application adaptively adjusts the data enhancement strategy by real-time monitoring the performance indicators of the model, the model training effect and generalization ability are optimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 This is a flowchart of the first embodiment of the image processing model training method of the present application;

[0043] Figure 2 This is a flow chart of the second embodiment of the image processing model training method of the present application;

[0044] Figure 3 This is a flowchart of the third embodiment of the image processing model training method of the present application;

[0045] Figure 4 A specific target image processing model training graph for the image processing model training method of this application;

[0046] Figure 5 This is a flowchart of the fourth embodiment of the image processing model training method of the present application;

[0047] Figure 6 This is a schematic diagram of the module structure of the image processing model training device according to an embodiment of the present application;

[0048] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the image processing model training method in the embodiment of the present application.

[0049] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The main solution of the embodiment of the present application is: constructing an initial data enhancement strategy based on the data characteristics of a preset training data set; when training a preset image processing model based on the initial data enhancement strategy and the preset training data set, monitoring the first performance indicator of the preset image processing model in real time; adjusting the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy; enhancing the data in the preset training data set based on the target data enhancement strategy, and training the preset image processing model again according to the enhanced data to obtain a first target image processing model.

[0053] Since data enhancement is a commonly used technical means in various computer vision tasks such as image classification, target detection, and image segmentation, the generalization ability of the model is improved by artificially expanding the data set. Traditional data enhancement methods such as random rotation, scaling, cropping, flipping, etc. usually use fixed parameter settings, which are difficult to meet the specific needs of different data sets and tasks. In the training of image processing models, the prior art proposes data enhancement methods based on random parameter selection and enhancement strategies based on image generation to enhance data, and trains the model with the enhanced data. Although these methods can increase the diversity of data to a certain extent, they lack the ability to adaptively adjust for specific tasks and data sets, and may cause the model to overfit or uneven data quality.

[0054] Therefore, this application proposes a new image processing model training technology, which adaptively adjusts the data enhancement strategy by real-time monitoring of the performance indicators of model training. It can be dynamically adjusted according to actual task requirements and data set characteristics, avoiding the limitations of fixed parameter enhancement methods, thereby optimizing model training effects and generalization capabilities.

[0055] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a computer, or an electronic device capable of realizing the above functions. The following takes the image processing model training device as an example to illustrate this embodiment and the following embodiments.

[0056] Based on this, the present application embodiment provides an image processing model training method, please refer to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the image processing model training method of the present application.

[0057] In this embodiment, the image processing model training method includes:

[0058] Step S10, constructing an initial data enhancement strategy based on the data characteristics of the preset training data set.

[0059] It should be noted that the preset training data set refers to a resource widely used in the fields of machine learning and deep learning. They contain a large amount of data that has been labeled or unlabeled but can be used for unsupervised learning. These data can be used to train models to enable them to recognize patterns, perform classification, prediction and other tasks. Data characteristics refer to a series of properties and characteristics inherent in the data itself, including data diversity, accuracy and scale. Data augmentation strategy is a method in the field of machine learning and deep learning that relies on generating new data points from existing data to artificially increase the amount of data, including perturbing the data in different directions or using deep learning models to generate new data points in the latent space of the original data to artificially expand the new data set. The initial data augmentation strategy refers to a series of data transformation methods applied before or at the beginning of model training, such as rotation, scaling, cropping, adding noise, etc.

[0060] It can be understood that constructing an initial data augmentation strategy based on the data characteristics of a preset training data set can be based on the data distribution and data characteristics of the data set, or it can be based on data augmentation goals (for example, improving model generalization ability, reducing overfitting, etc.).

[0061] For ease of understanding, the following examples are given, but are not intended to limit the present application. In one example, the image processing model training platform performs data analysis on the acquired preset training data set, analyzes the number of samples in each category in the data set, understands whether there is a category imbalance problem, and evaluates the overall distribution characteristics of the data, such as the degree of clustering and sparsity of data points. In order to alleviate the problem of category imbalance, an adversarial network is used to generate samples of minority categories to perform more data enhancement on minority categories.

[0062] Step S20, when the preset image processing model is trained based on the initial data enhancement strategy and the preset training data set, a first performance indicator of the preset image processing model is monitored in real time.

[0063] It should be noted that performance indicators refer to quantitative standards for measuring the performance of models during model training and evaluation. The first performance indicator usually refers to the performance indicators that are paid attention to and monitored at the beginning of training or at a certain stage, such as accuracy, precision, recall, F1 score and loss value, etc. These indicators can intuitively reflect the performance of the model in the current training state.

[0064] Specifically, during the model training process, the first performance indicator of the model is regularly checked and recorded programmatically. This can involve adding corresponding code snippets to the training loop, using evaluation functions or directly extracting performance indicators from the training process through callback functions. Real-time monitoring can help you keep abreast of the model's training progress and performance so that you can make adjustments when necessary, such as adjusting the learning rate, modifying the model architecture, changing the optimizer, etc.

[0065] For ease of understanding, the following examples are provided, but the present application is not limited. In one example, assuming that the first performance indicator is accuracy, recall and training speed, the image processing model training device loads the original image data set, and applies initial data enhancement methods such as rotation and cropping to generate enhanced training samples, and then selects a suitable deep learning model architecture, such as a convolutional neural network, according to the task requirements. A callback function or a timed checkpoint is set in the training loop to evaluate the accuracy of the model on the training set at the end of each set time period. After starting the training, the model starts processing the training data and learns feature representation and weight updates through forward propagation and back propagation. At the end of each set time period, the callback function or checkpoint triggers the evaluation process, calculates the accuracy, recall and training speed of the model on the training data set, and prints the results to the console or saves them to a log file.

[0066] Step S30: adjusting the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy.

[0067] It should be noted that the target data enhancement strategy is a data enhancement strategy that is obtained by adjusting and optimizing the initial data enhancement strategy according to the feedback of the first performance indicator and enables the model to achieve the best performance indicator.

[0068] It should be understood that adjusting the initial data enhancement strategy according to the first performance indicator may be to increase, reduce, replace or adjust parameters of the initial data enhancement technology in the hope of improving the performance of the model. Adjusting the initial data enhancement strategy according to the first performance indicator can also be said to be an analysis of model performance indicators (such as loss function, accuracy, etc.) to adaptively adjust the enhancement parameters of the initial enhancement strategy.

[0069] It can be understood that adjusting the initial data enhancement strategy according to the first performance indicator can also be adjusting the data enhancement strategy according to the training speed of the model, by balancing the diversity of data enhancement and the training speed to ensure that the enhanced data is both rich and conducive to the effective training of the model.

[0070] Step S40, enhancing the data in the preset training data set based on the target data enhancement strategy, and training the preset image processing model again according to the enhanced data to obtain a first target image processing model.

[0071] It should be noted that the first target image processing model is the result of training the preset image processing model based on the training data set. Since the diversity and complexity of the data are improved through the data augmentation strategy, the first target processing model usually has better generalization ability than the model trained only using the initial training data.

[0072] For ease of understanding, the following examples are provided, but the present application is not limited thereto. In one example, an image processing model training device is training an image processing model for recognizing handwritten numbers (such as 0 to 9), and the preset training data set contains a large number of images of handwritten numbers. The current target data enhancement strategy is rotation and adding noise. Multiple rotated versions are generated through data enhancement, for example, rotating ±10°, ±20°, and adding different degrees of noise to obtain an enhanced data set that is more complex than the initial data set. The image processing training platform then uses the enhanced data set to train the initial image processing model. During the training process, a first target image processing model with higher generalization ability is obtained by learning how to recognize handwritten numbers at different rotation angles and noise levels.

[0073] In this embodiment, an initial data enhancement strategy is constructed based on the data characteristics of a preset training data set; when a preset image processing model is trained based on the initial data enhancement strategy and the preset training data set, a first performance indicator of the preset image processing model is monitored in real time; the initial data enhancement strategy is adjusted according to the first performance indicator to obtain a target data enhancement strategy; data in the preset training data set is enhanced based on the target data enhancement strategy, and the preset image processing model is trained again according to the enhanced data to obtain a first target image processing model. Since this embodiment adaptively adjusts the data enhancement strategy by real-time monitoring the performance indicators of the model, the model training effect and generalization ability are optimized.

[0074] Reference Figure 2 , Figure 2 This is a flow chart of the second embodiment of the image processing model training method of the present application. Based on the above-mentioned first embodiment, the second embodiment of the image processing model training method of the present application is proposed.

[0075] In the second embodiment, the step S30 includes:

[0076] Step S301, comparing the first performance indicator with a preset indicator threshold, and testing the performance of the target image processing model according to the comparison result to obtain a performance test result.

[0077] It should be noted that the preset indicator threshold is a performance requirement indicator of the model artificially set in the image processing model training device. The preset indicator threshold is usually determined based on business needs, industry standards or the results of previous experiments. For example, if the user requires the accuracy of the current image processing model to be above 80%, the preset indicator threshold can be set to 0.8.

[0078] It is understandable that in order to further quantify the performance of the current image processing model on a specific task, the first performance indicator and the preset indicator threshold need to include a variety of data. For example, in image classification tasks, commonly used performance indicators include accuracy, precision, recall rate, and F1 score.

[0079] Step S302, when the performance test result shows that the accuracy and recall rate of the target image processing model decrease, the enhancement parameters of the initial data enhancement strategy are adjusted according to the set enhancement parameter range to obtain first adjustment information.

[0080] It should be noted that the enhancement parameter range refers to the range in which the parameters of the current initial data enhancement strategy can be adjusted during the training process. For example, for rotation enhancement, the possible setting range is -30° to 30°; for brightness adjustment, the possible setting range is -50% to +50% brightness change. Accuracy is the proportion of samples correctly predicted by the model to the total number of samples in the classification task. Recall rate, also known as TPR (True Positive Rate), refers to the proportion of samples correctly predicted as positive examples by the model among all samples that are actually positive examples. It measures the sensitivity of the model to positive samples. The first adjustment information refers to the new strategy and its related performance information obtained by adjusting the enhancement parameters of the initial data enhancement strategy according to the set enhancement parameter range when the model accuracy and recall rate decrease.

[0081] Specifically, when the accuracy and recall of the target image processing model decrease, the possible reasons for the decline in model performance are analyzed, including the quality of the data set, overfitting or underfitting of the model, the effectiveness of the data enhancement strategy, etc. Based on the above analysis, the enhancement parameters in the initial data enhancement strategy are adjusted, including increasing or decreasing the rotation angle, adjusting the scaling ratio, changing the noise intensity, etc., to find an enhancement method that can increase data diversity without introducing too much noise.

[0082] Step S303, when the performance detection result is that the data training speed of the target image processing model decreases, the complexity and enhancement type of the initial data enhancement strategy are dynamically adjusted according to the multi-objective optimization algorithm to obtain second adjustment information.

[0083] It should be noted that data training speed refers to the time or number of iterations required for a model to be trained on a given data set. Data training speed may be affected by many factors, including the size of the data set, the complexity of the model, the performance of computing resources (such as CPU, GPU), and the complexity of data preprocessing (such as data enhancement). A multi-objective optimization algorithm is an algorithm that can optimize multiple objective functions at the same time. In practical problems, there are often multiple conflicting objectives, and a set of solutions needs to be found to achieve the best balance between these objectives. Multi-objective optimization algorithms can generate a set of optimal solutions that improve any one objective without deteriorating other objectives. Complexity refers to the computing resources and time required for enhancement technology. Different enhancement technologies have different complexities. Enhancement type refers to the specific enhancement technology or method used in the data enhancement strategy. For example, rotation, scaling, cropping, etc. The second adjustment information refers to the new strategy and its related performance information obtained by adjusting the initial data enhancement strategy through the multi-objective optimization algorithm when the data training speed decreases, including data diversity and data training speed.

[0084] Specifically, when the data training speed of the target image processing model decreases, evaluate the computational complexity of various enhancement technologies in the current data enhancement strategy, identify the enhancement types that have a greater impact on the training speed, and select a suitable multi-objective optimization algorithm based on the optimization goals and constraints (such as maintaining or improving model performance, increasing data training speed, etc.). At the same time, clarify the search space of the enhancement parameters (i.e., optimization space), and define the optimization objective function, which is used to evaluate the data training speed and model performance under different enhancement strategy configurations. Use the multi-objective optimization algorithm in the optimization space to search for the best data enhancement strategy configuration to obtain the optimal solution, and adjust the data enhancement strategy based on the calculated optimal solution.

[0085] For ease of understanding, the following examples are given, but the present application is not limited. In one example, the data training speed of the target image processing model under the initial data enhancement strategy (including rotation, scaling, cropping and color adjustment) is slow. After analysis, the image processing model training device finds that the computational complexity of the rotation and scaling operations is high, which has a greater impact on the training speed. The data enhancement strategy is adjusted by using NSGA-II (Non-dominated Sorting Genetic Algorithm II) as a multi-objective optimization algorithm, and the optimization space is defined. The range of rotation angles, the range of scaling ratios, the parameter range of cropping size and position, and the range of color adjustment parameters (such as brightness, contrast, etc.) are set in the space, and NSGA-II is used to search for the best data enhancement strategy configuration in the optimization space. A configuration is selected from the Pareto (non-dominated solution) optimal solution, which maintains the training speed and data diversity of the model on the test set while reducing the rotation angle range and setting a smaller scaling ratio. The model is retrained using the adjusted data enhancement strategy. The performance of the retrained model on the test set was evaluated, and it was found that the data training speed was significantly improved, while the model performance remained within an acceptable range. The adjusted enhancement strategy configuration (such as rotation angle ±5°, scaling ratio 0.9-1.1, etc.) and the corresponding performance improvement information were recorded as the second adjustment information.

[0086] Step S304: Generate a target data enhancement strategy according to the first adjustment information and / or the second adjustment information.

[0087] It should be understood that in order to enable dynamic adjustment according to actual task requirements and data set characteristics and avoid the limitations of fixed parameter enhancement methods, a target data enhancement strategy can be generated based on the first adjustment information and / or the second adjustment information.

[0088] Specifically, the first adjustment information is the adjustment of the data enhancement strategy in the preliminary analysis or preliminary optimization stage, and the second adjustment information is obtained by further adjusting the complexity and enhancement type of the data enhancement strategy through a multi-objective optimization algorithm after detecting performance degradation (such as a decrease in training speed). Analyze the impact of the first adjustment information and the second adjustment information on the performance of the model before adjustment. Determine which adjustments are effective and which may require further adjustment or optimization. After comprehensively evaluating all adjustment information and considering the interactions and synergies between different adjustments, determine the most likely to produce the best overall performance. Based on the above analysis, generate a target data enhancement strategy. The target data enhancement strategy should include the selected data enhancement techniques, their parameter settings, and their application order.

[0089] In the second embodiment, by analyzing the model performance indicators, adaptively adjusting the enhancement parameters, and obtaining a data enhancement strategy with better performance, it is possible to ensure that the enhanced data is both rich and conducive to the effective training of the model. Through the optimized data enhancement strategy, the training effect and generalization ability of the model are improved.

[0090] Reference Figure 3 , Figure 3 This is a flow chart of the third embodiment of the image processing model training method of the present application. Based on the above-mentioned second embodiment, the third embodiment of the image processing model training method of the present application is proposed.

[0091] In the third embodiment, after step S40, the method further includes:

[0092] Step S401: verify the first target image processing model based on a verification data set, and monitor in real time a second performance indicator of the first target image processing model during the verification process.

[0093] It should be noted that the validation dataset is a dataset used in machine learning and deep learning to adjust model hyperparameters, select model architecture, or verify the performance of the model on unseen data. The second performance indicator is the performance indicator that is paid attention to and monitored in the validation phase, such as accuracy, precision, recall, F1 score, and loss value. These indicators can intuitively reflect the performance of the model in the current training state.

[0094] Specifically, the trained first target image processing model is loaded into the verification environment and the verification data set is used to verify the model, and the output of the model is recorded. During the verification process, the changes of the second performance indicator are monitored in real time, and may include drawing a chart showing the changes of the indicator over time.

[0095] Step S402: evaluate and provide feedback on the enhancement effect and model training speed of the current target data enhancement strategy based on the second performance indicator.

[0096] It should be understood that the training speed of the current model can be measured by the time required for each iteration or by the number of samples processed per second (SPS).

[0097] It can be understood that the enhancement effect and model training speed of the current target data enhancement strategy can be evaluated and fed back based on the second performance indicator, and a more detailed performance analysis can be performed based on the performance analysis tool in the image processing model, including GPU utilization, memory usage, computing bottlenecks, etc.

[0098] Step S403, generating a data enhancement strategy combination according to the data enhancement effect and the model training speed, and training the first target image processing model based on the data enhancement strategy combination and the verification data set to obtain a second target image processing model.

[0099] It should be understood that the data enhancement strategy combination is a sequential combination of a series of data enhancement strategies for the current image processing model generated according to the results of model feedback during the model verification process. The second target image processing model is an image processing model optimized on the basis of the first target image processing model.

[0100] Furthermore, in order to enhance specific data by applying a specific target enhancement strategy in the model reasoning stage to improve the robustness of the model in practical applications, after step S403, the following steps are further included:

[0101] Inputting the preprocessed prediction data into the second target image processing model;

[0102] Perform forward propagation calculation on the predicted data through each layer network in the second target image processing model to generate a prediction result;

[0103] Generate a specific data enhancement strategy based on the prediction result, and enhance the predicted data according to the specific data enhancement strategy to obtain specific target data;

[0104] The second target image processing model is trained based on the specific target data to obtain a specific target image processing model.

[0105] It should be understood that real-time enhancement of input data is generally not recommended during the model inference phase because it will increase inference time and computational complexity. However, in some specific scenarios, applying specific target enhancement strategies to enhance specific data can indeed improve the robustness of the model in practical applications.

[0106] For ease of understanding, the following examples are provided, but are not intended to limit the present application. In one example, Figure 4This is a specific target image processing model training diagram for the image processing model training method of the present application. The method provided in the embodiment of the present application can provide model reasoning services for a data set on an image processing model. The processor loads the data in the data set stored in the memory into the image processing model, obtains the prediction result through model reasoning, and the operator generates a specific data enhancement strategy through the obtained prediction result and returns it to the processor. The CPU (Central Processing Unit) enhances the data set through a specific data enhancement strategy. Under normal circumstances, the processor can also change the enhancement parameters and parameter range of the current data enhancement strategy based on the requirements of training accuracy.

[0107] Of course, in order to further demonstrate the significant advantages of this solution in improving model accuracy and reducing overfitting, after step S403, the following is also included:

[0108] Access multiple public image datasets;

[0109] Performing data enhancement on the image data set according to the enhancement strategy combination to obtain an enhancement result;

[0110] The enhancement result is compared with the data in the image data set before the enhancement, and an analysis report is generated according to the comparison result.

[0111] It should be understood that the acquired image datasets may include image classification, target detection, image segmentation, etc. The image processing model training platform verifies the effectiveness of the new data enhancement method through a series of experiments, including comparative experiments with traditional data enhancement methods, and generalization ability tests on different datasets, to evaluate whether the generated enhancement strategy combination effectively increases the diversity and complexity of the data while maintaining the basic features and label information of the image, and can gradually optimize the performance of the image processing model.

[0112] In the third embodiment of the present application, according to the feedback results of the model on the validation set, an appropriate combination of enhancement strategies is selected for training to avoid the negative impact of excessive enhancement or inappropriate enhancement operations on the model.

[0113] Reference Figure 5 , Figure 5 This is a flow chart of the fourth embodiment of the image processing model training method of the present application. Based on the above-mentioned third embodiment, the fourth embodiment of the image processing model training method of the present application is proposed.

[0114] In the fourth embodiment, the step S10 includes:

[0115] Step S101, defining corresponding parameter spaces according to different enhancement operations.

[0116] It should be noted that the parameter space refers to the set of all possible values ​​of the enhancement operation parameters. The parameters define the scope and strength of the enhancement operation, thereby allowing the generation of diverse training samples.

[0117] In a specific implementation, for example, when the enhancement operation is rotation, the parameter space may be [-180°, +180°], indicating the maximum angle that the image can be rotated clockwise or counterclockwise around its center. For a scaling operation, the parameter space may be [0.5, 1.5], indicating the minimum ratio that the image can be reduced and the maximum ratio that it can be enlarged.

[0118] Step S102, identifying the preset training data set to obtain a recognition result.

[0119] It should be understood that before model training, a preliminary analysis of the rotated dataset is required to determine its characteristics, such as data distribution, category imbalance, etc.

[0120] Step S103: determining an enhancement operation according to the recognition result, and setting an initial enhancement parameter and an enhancement parameter range from the corresponding parameter space.

[0121] It should be understood that before data augmentation, a preliminary model is usually trained using unaugmented data. The performance of this model (such as accuracy, recall, etc.) will serve as a benchmark for subsequent optimization.

[0122] Specifically, analyze the recognition results of the preliminary model to find out the performance differences of the model on different types of samples. This may include: misclassified samples, confused categories, data imbalance, and model sensitivity to specific features. Based on the analysis of the recognition results, it can be determined which data enhancement operations may help improve model performance. For example, if the model is sensitive to rotation, rotation enhancement can be introduced. For each selected enhancement operation, it is necessary to set the initial enhancement parameters and parameter range. For rotation enhancement, the initial parameter can be 0 degrees (no rotation), and the parameter range can be [-15°, 15°], which means the maximum angle that the image can be rotated clockwise or counterclockwise around its center.

[0123] In the fourth embodiment of the present application, by identifying the characteristics of the data set and selecting appropriate enhancement operations, parameters and parameter ranges accordingly, it is possible to ensure that the effect of data enhancement is maximized and the optimization efficiency is improved.

[0124] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the data enhancement method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0125] This application also provides an image processing model training device, please refer to Figure 6, the image processing model training device comprises:

[0126] A strategy building module 10 is used to build an initial data enhancement strategy based on data characteristics of a preset training data set;

[0127] A performance detection module 20, configured to monitor in real time a first performance indicator of the preset image processing model when the preset image processing model is trained based on the initial data enhancement strategy and the preset training data set;

[0128] A strategy adjustment module 30, configured to adjust the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy;

[0129] The model training module 40 is used to enhance the data in the preset training data set based on the target data enhancement strategy, and to train the preset image processing model again according to the enhanced data to obtain a first target image processing model.

[0130] The image processing model training device provided by the present application adopts the image processing model training method in the above-mentioned embodiment, which can solve the technical problem that the existing image processing model training method lacks the adaptive adjustment ability for specific tasks and data sets, which may lead to overfitting or uneven data quality. Compared with the prior art, the beneficial effects of the image processing model training device provided by the present application are the same as the beneficial effects of the image processing model training method provided by the above-mentioned embodiment, and the other technical features in the image processing model training device are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0131] The present application provides an image processing model training device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing model training method in the above-mentioned embodiment one.

[0132] Reference below Figure 7, which shows a schematic diagram of the structure of an image processing model training device suitable for implementing an embodiment of the present application. The image processing model training device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The image processing model training device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0133] like Figure 7 As shown, the image processing model training device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the image processing model training device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the data enhancement device to communicate with other devices wirelessly or by wire to exchange data. Although the image processing model training device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0134] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0135] The image processing model training device provided by the present application adopts the image processing model training method in the above embodiment, which can solve the technical problem that the existing image processing model training method lacks the adaptive adjustment ability for specific tasks and data sets, which may lead to overfitting or uneven data quality. Compared with the prior art, the beneficial effects of the image processing model training device provided by the present application are the same as the beneficial effects of the image processing model training method provided by the above embodiment, and the other technical features in the image processing model training device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0136] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0138] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the image processing model training method in the above-mentioned embodiment.

[0139] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0140] The above-mentioned computer-readable storage medium may be included in the image processing model training device; or it may exist independently without being assembled into the image processing model training device.

[0141] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the image processing model training device, the image processing model training device implements the image processing model training method described above.

[0142] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0143] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0144] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0145] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned image processing model training method, and can solve the technical problems that the existing image processing model training method lacks adaptive adjustment capabilities for specific tasks and data sets, which may lead to overfitting or uneven data quality. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the image processing model training method provided by the above-mentioned embodiment, and will not be repeated here.

[0146] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned image processing model training method when executed by a processor.

[0147] The computer program product provided by the present application can solve the technical problem that the existing image processing model training method lacks the ability to adaptively adjust for specific tasks and data sets, which may lead to overfitting or uneven data quality. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the image processing model training method provided by the above embodiment, and will not be repeated here.

[0148] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for training an image processing model, characterized in that: The image processing model training method comprises: Construct an initial data augmentation strategy based on the data characteristics of the preset training dataset; When a preset image processing model is trained based on the initial data enhancement strategy and the preset training data set, a first performance indicator of the preset image processing model is monitored in real time; Adjust the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy; Based on the target data enhancement strategy, the data in the preset training data set is enhanced, and the preset image processing model is trained again according to the enhanced data to obtain a first target image processing model; The step of adjusting the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy includes: Comparing the first performance indicator with a preset indicator threshold, and testing the performance of the current image processing model according to the comparison result to obtain a performance test result; When the performance detection result shows that the accuracy and recall rate of the current image processing model decrease, adjusting the enhancement parameters of the initial data enhancement strategy according to the set enhancement parameter range to obtain first adjustment information; When the performance detection result indicates that the data training speed of the current image processing model decreases, dynamically adjusting the complexity and enhancement type of the initial data enhancement strategy according to a multi-objective optimization algorithm to obtain second adjustment information; A target data enhancement strategy is generated according to the first adjustment information and / or the second adjustment information.

2. The image processing model training method according to claim 1, characterized in that: After the step of enhancing the data in the preset training data set based on the target data enhancement strategy and retraining the preset image processing model according to the enhanced data to obtain the first target image processing model, the method further includes: Verifying the first target image processing model based on a verification data set, and monitoring a second performance indicator of the first target image processing model during the verification process in real time; Evaluate and provide feedback on the data enhancement effect and model training speed of the current target data enhancement strategy based on the second performance indicator; A data enhancement strategy combination is generated according to the data enhancement effect and the model training speed, and the first target image processing model is trained based on the data enhancement strategy combination and the verification data set to obtain a second target image processing model.

3. The image processing model training method according to claim 2, characterized in that: After the step of generating a data enhancement strategy combination according to the data enhancement effect and the model training speed, and training the first target image processing model based on the data enhancement strategy combination and the verification data set to obtain the second target image processing model, the step further includes: Inputting the preprocessed prediction data into the second target image processing model; Perform forward propagation calculation on the predicted data through each layer network in the second target image processing model to generate a prediction result; Generate a specific data enhancement strategy based on the prediction result, and enhance the predicted data according to the specific data enhancement strategy to obtain specific target data; The second target image processing model is trained based on the specific target data to obtain a specific target image processing model.

4. The image processing model training method according to claim 2, characterized in that: After the step of generating a data enhancement strategy combination according to the data enhancement effect and the model training speed, and training the first target image processing model based on the data enhancement strategy combination and the verification data set to obtain the second target image processing model, the step further includes: Access multiple public image datasets; Performing data enhancement on the image data set according to the enhancement strategy combination to obtain an enhancement result; The enhancement result is compared with the data in the image data set before the enhancement, and an analysis report is generated according to the comparison result.

5. The image processing model training method according to any one of claims 1 to 4, characterized in that: The step of constructing an initial data enhancement strategy based on the data characteristics of a preset training data set includes: Define corresponding parameter spaces according to different enhancement operations; Identify the preset training data set to obtain a recognition result; An enhancement operation is determined according to the recognition result, and an initial enhancement parameter and an enhancement parameter range are set from the corresponding parameter space.

6. An image processing model training device, characterized in that: The image processing model training device comprises: A strategy building module is used to build an initial data enhancement strategy based on the data characteristics of a preset training data set; A performance detection module, used to monitor a first performance indicator of the preset image processing model in real time when the preset image processing model is trained based on the initial data enhancement strategy and the preset training data set; A strategy adjustment module, used to adjust the initial data enhancement strategy according to the first performance indicator to obtain a target data enhancement strategy; A model training module, used to enhance the data in the preset training data set based on the target data enhancement strategy, and to train the preset image processing model again according to the enhanced data to obtain a first target image processing model; The strategy adjustment module is further used to compare the first performance indicator with a preset indicator threshold, and detect the performance of the current image processing model according to the comparison result to obtain a performance detection result; When the performance detection result shows that the accuracy and recall rate of the current image processing model decrease, adjusting the enhancement parameters of the initial data enhancement strategy according to the set enhancement parameter range to obtain first adjustment information; When the performance detection result indicates that the data training speed of the current image processing model decreases, dynamically adjusting the complexity and enhancement type of the initial data enhancement strategy according to a multi-objective optimization algorithm to obtain second adjustment information; A target data enhancement strategy is generated according to the first adjustment information and / or the second adjustment information.

7. An image processing model training device, characterized in that: The image processing model training device includes: a memory, a processor, and an image processing model training program stored in the memory and executable on the processor. When the image processing model training program is executed by the processor, the image processing model training method as described in any one of claims 1 to 5 is implemented.

8. A storage medium, characterized in that: An image processing model training program is stored on the storage medium, and when the image processing model training program is executed by the processor, the image processing model training method as described in any one of claims 1 to 5 is implemented.

9. A computer program product, characterized in that The computer program product includes an image processing model training program, which, when executed by a processor, implements the image processing model training method as described in any one of claims 1 to 5.

Images