Adaptive image data enhancement processing method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a self-adaptive image data enhancement processing method and device, electronic equipment and a medium. The method comprises the following steps: acquiring to-be-processed image data; according to a pre-trained classification model, category information corresponding to the to-be-processed image data is determined, and the classification model is obtained by training a machine learning model by using the sample image data; determining a target enhancement strategy corresponding to the target category information according to the target category information; performing enhancement processing on the to-be-processed image data according to the target enhancement strategy to obtain an enhanced image corresponding to the to-be-processed image, training a machine learning model to obtain a classification model, improving the accuracy of image recognition, continuously optimizing the enhancement strategy according to a model training result, continuously performing enhancement processing on the image, and improving the accuracy of image recognition. Especially for the problem of class imbalance, the enhancement strategy of scarce classes is dynamically adjusted, so that the accuracy of image processing is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an adaptive image data enhancement processing method, device, electronic device and medium. Background Art

[0002] In modern computer vision tasks, image classification is an important task in the field of computer vision and plays a vital role in autonomous driving, intelligent monitoring, medical diagnosis and other fields. In recent years, the performance of deep learning models in image classification tasks has been significantly improved, but model training often requires a large amount of high-quality labeled data. Faced with problems such as insufficient data and uneven data distribution, the generalization ability of the model will be limited. Data augmentation technology increases the diversity of the data set by performing various transformations on the original data set, thereby improving the generalization ability of the model. However, traditional geometric transformations (such as rotation, flipping, scaling, etc.) often lack flexibility and cannot be adaptively adjusted according to different data set characteristics and task requirements. To this end, designing an augmentation technology that can automatically and adaptively adjust according to the characteristics of the data set and the model training situation has become an important research direction. Summary of the invention

[0003] Some embodiments of the present application aim to provide an adaptive image data enhancement processing method, device, electronic device and medium. Through the technical solutions of the embodiments of the present application, image data to be processed is obtained; target category information corresponding to the image data to be processed is determined according to a pre-trained classification model, wherein the classification model is obtained by training a machine learning model with sample image data; a target enhancement strategy corresponding to the target category information is determined according to the target category information; the image data to be processed is enhanced according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed. The embodiments of the present application train the machine learning model to obtain a classification model to improve the accuracy of image recognition, and continuously optimize the enhancement strategy according to the results of model training, continuously enhance the image, and especially dynamically adjust the enhancement strategy of the scarce category for the problem of category imbalance, thereby improving the accuracy of image processing.

[0004] In a first aspect, some embodiments of the present application provide an adaptive image data enhancement processing method, including:

[0005] Obtain image data to be processed;

[0006] Determining target category information corresponding to the image data to be processed according to a pre-trained classification model, wherein the classification model is obtained by training a machine learning model using sample image data;

[0007] Determining a target enhancement strategy corresponding to the target category information according to the target category information;

[0008] The image data to be processed is enhanced according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed.

[0009] Some embodiments of the present application train a machine learning model to obtain a classification model, thereby improving the accuracy of image recognition, and then optimize the enhancement strategy according to the results of the model training, continuously enhance the image, and improve the accuracy of image processing.

[0010] Optionally, the pre-trained classification model is obtained in the following manner:

[0011] Get sample image data;

[0012] Determine, based on the sample image data, a feature vector corresponding to the sample image data;

[0013] Training the machine learning model according to the classification requirements and the feature vectors of the sample image data to obtain a trained model;

[0014] Calculating the loss function of the trained model;

[0015] When the loss function of the trained model is less than a preset threshold, the trained model corresponding to the loss function is determined as the classification model; wherein the machine learning model includes at least a convolutional neural network model.

[0016] The embodiment of the present application combines reinforcement learning and evolutionary algorithms to dynamically optimize the enhancement strategy. After the random generation and preliminary screening of the enhancement strategy, evolutionary learning is responsible for optimizing the strategy combination, and reinforcement learning dynamically adjusts the selection of enhancement methods according to model feedback (such as classification accuracy, loss, etc.), and finally finds the optimal enhancement strategy, thereby improving the overall performance of the model.

[0017] Optionally, the method further comprises:

[0018] According to the category information of the image data, multiple groups of initial enhancement strategies are randomly generated;

[0019] The initial enhancement strategy is optimized by reinforcement learning or evolutionary algorithm to obtain an enhancement strategy.

[0020] Optionally, the initial enhancement strategy includes at least one or more of a geometric transformation method, a color transformation method, a noise addition method, and a hybrid enhancement method. The generation and selection of the enhancement strategy are based on the category information and feature differences of the image. Through the initialization of the random enhancement strategy, combined with geometric transformation (such as rotation, scaling, cropping, etc.), color transformation (brightness, contrast adjustment, etc.), noise addition (such as Gaussian noise, salt and pepper noise, etc.) and hybrid enhancement methods (such as CutMix, MixUp), the system can adaptively select the optimal enhancement combination.

[0021] Some embodiments of the present application integrate multiple data enhancement methods, including geometric transformation, color transformation, noise addition, and hybrid enhancement, etc. These methods can be dynamically adjusted in combination with category information and model feedback, and combined and selected according to specific enhancement strategy requirements, thereby further improving data diversity and enhancement effects.

[0022] According to the enhanced image, the enhancement strategy is further optimized, and the parameters of the enhancement method are dynamically adjusted to adapt to changes in model training.

[0023] During the model training process, some embodiments of the present application monitor the training effect in real time and dynamically adjust the enhancement strategy. By introducing the reinforcement learning mechanism, the enhancement strategy can respond in a timely manner according to the performance changes of the model, further improving the generalization ability of the model, especially in the case of category imbalance, the weaker performing categories can be enhanced.

[0024] Optionally, the acquiring sample image data includes:

[0025] Obtaining initial image data;

[0026] Using a preset image processing tool, extracting feature information of the initial image data, the feature information at least including a color histogram, texture features, and shape features;

[0027] Performing cluster analysis on the feature information according to a preset clustering algorithm to obtain feature classification data of different types of images;

[0028] According to the feature classification data of the different types of images, the sample image data is determined and an initial enhancement strategy is generated.

[0029] Some embodiments of the present application generate an initial enhancement strategy after a comprehensive analysis of the original image dataset, combined with the classification requirements of the model. Through data analysis, the distribution and feature differences of different categories of images in the dataset can be understood, and the reinforcement learning mechanism can further optimize the enhancement strategy to adapt to the task requirements.

[0030] In a second aspect, some embodiments of the present application provide an adaptive image data enhancement processing device, including:

[0031] An acquisition module, used for acquiring image data to be processed;

[0032] A determination module, used to determine target category information corresponding to the image data to be processed according to a pre-trained classification model, wherein the classification model is obtained by training a machine learning model using sample image data;

[0033] A judgment module, used to determine a target enhancement strategy corresponding to the target category information according to the target category information;

[0034] The processing module is used to perform enhancement processing on the image data to be processed according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed.

[0035] Some embodiments of the present application randomly generate enhancement strategies, and optimize and adjust the strategies in combination with evolutionary learning and reinforcement learning modules to ensure that the enhancement strategies can adapt to the characteristics and classification requirements of different categories of images, thereby effectively improving the classification performance of the model.

[0036] Optionally, the device further comprises a training module, wherein the training module is used to:

[0037] Get sample image data;

[0038] Determining a feature vector corresponding to the sample image data according to the sample image data;

[0039] Training the machine learning model according to the classification requirements and the feature vectors of the sample image data to obtain a trained model;

[0040] Calculating the loss function of the trained model;

[0041] When the loss function of the trained model is less than a preset threshold, the trained model corresponding to the loss function is determined as the classification model; wherein the machine learning model includes at least a convolutional neural network model.

[0042] The training module is also responsible for continuously adjusting the enhancement strategy through reinforcement learning during the model training process to match it with the characteristics of the dataset and task requirements, ensuring that the enhancement operation always helps to improve the performance of the classification model.

[0043] Some embodiments of the present application use reinforcement learning and evolutionary algorithms to dynamically optimize the enhancement strategy. These algorithms continuously learn and evolve to find the optimal enhancement strategy, thereby improving model performance.

[0044] Optionally, the training module is used to:

[0045] According to the category information of the image data, multiple groups of initial enhancement strategies are randomly generated;

[0046] The initial enhancement strategy is optimized by reinforcement learning or evolutionary algorithm to obtain an enhancement strategy.

[0047] Optionally, the enhancement strategy includes at least one or more of a geometric transformation method, a color transformation method, a noise addition method and a hybrid enhancement method.

[0048] During the model training process, some embodiments of the present application monitor the training effect in real time and dynamically adjust the enhancement strategy. Through this dynamic adjustment, the enhancement strategy can respond to the performance changes of the model in a timely manner and further improve the generalization ability of the model.

[0049] Optionally, the training module is used to:

[0050] Obtaining initial image data;

[0051] Using a preset image processing tool, extracting feature information of the initial image data, the feature information at least including a color histogram, texture features, and shape features;

[0052] Performing cluster analysis on the feature information according to a preset clustering algorithm to obtain feature classification data of different types of images;

[0053] The sample image data is determined according to the feature classification data of the different types of images.

[0054] Some embodiments of this application require a comprehensive analysis of the original image data set to extract the statistical features of the image and the required features of the classification task. Through data analysis, the distribution and feature differences of different categories of images in the data set can be understood, providing a basis for the subsequent generation of enhancement strategies.

[0055] In a third aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program on the memory and executable on the processor, wherein when the processor executes the program, the adaptive image data enhancement processing method as described in any embodiment of the first aspect can be implemented.

[0056] In a fourth aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the adaptive image data enhancement processing method as described in any embodiment of the first aspect.

[0057] In a fifth aspect, some embodiments of the present application provide a computer program product, comprising a computer program, wherein the computer program, when executed by a processor, can implement the adaptive image data enhancement processing method as described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 A schematic diagram of a flow chart of an adaptive image data enhancement processing method provided in an embodiment of the present application;

[0060] Figure 2 A schematic diagram of the structure of an adaptive image data enhancement processing device provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0063] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0064] In modern computer vision tasks, image classification, as a basic and important task, is widely used in many fields such as autonomous driving, intelligent monitoring, and medical diagnosis. For example, it can be used in the field of fault diagnosis. If the image quality is not high when processing the image, it is impossible to accurately analyze and diagnose based on the image. In view of this, some embodiments of the present application provide an adaptive image data enhancement processing method, which includes: obtaining image data to be processed; determining target category information corresponding to the image data to be processed according to a pre-trained classification model, and the classification model is obtained by training a machine learning model using sample image data; determining a target enhancement strategy corresponding to the target category information according to the target category information; enhancing the image data to be processed according to the target enhancement strategy, thereby obtaining an enhanced image corresponding to the image to be processed. By training the machine learning model, a classification model is obtained to improve the accuracy of image recognition, and then the enhancement strategy is optimized according to the results of the model training, and the image is continuously enhanced to improve the accuracy of image processing.

[0065] like Figure 1 As shown, an embodiment of the present application provides an adaptive image data enhancement processing method, the method comprising:

[0066] S101, obtaining image data to be processed;

[0067] Specifically, the terminal device obtains image data to be processed, which may be a picture or a video. The image data to be processed may be real-time image data collected by a camera device, or may be obtained from a network or other devices.

[0068] S102, determining target category information corresponding to the image data to be processed according to a pre-trained classification model, where the classification model is obtained by training a machine learning model using sample image data;

[0069] The terminal device obtains a large amount of sample image data and trains a machine learning model based on the sample image data. The function of the machine learning model is to automatically learn from the data and extract useful information to make predictions or decisions. In supervised learning, the model predicts the result by identifying the association between features and outputs, such as classifying emails as spam or non-spam. In unsupervised learning, the model explores the intrinsic structure of the data, such as market segmentation through cluster analysis. Reinforcement learning models learn the best action strategy through interaction with the environment. Machine learning models are mainly divided into three categories: supervised learning models, semi-supervised learning and unsupervised learning models. In the embodiments of the present application, the type of machine learning model is not specifically limited.

[0070] In a specific implementation process, the terminal device processes the image data to be processed, obtains a feature vector corresponding to the image data to be processed, inputs the feature vector into a pre-trained classification model, and obtains category information of the image data to be processed. The category information includes at least a first category and a second category. The first category may be a category with good image quality, and the second category may be a category with poor image quality. That is to say, the image data to be processed can be classified through the classification model to determine whether the image data needs enhanced processing.

[0071] S103, determining a target enhancement strategy corresponding to the target category information according to the target category information;

[0072] Specifically, the terminal device determines an enhancement strategy corresponding to the category information according to the category information after obtaining the category information;

[0073] Combine geometric transformation, color transformation, noise addition, hybrid enhancement, etc., generate the initial enhancement strategy combination by random parameter setting, and perform data enhancement on some randomly selected categories;

[0074] Based on the initial training results, we screen out strategy combinations with better enhancement effects, and use evolutionary learning to further optimize these strategy combinations. We use reinforcement learning to monitor the performance of the model on the validation set in real time, and dynamically adjust the selection and parameters of the enhancement strategy to ensure that the enhancement strategy can adaptively respond to changes in the model's performance.

[0075] For example, if it is determined that the image data to be processed is of good quality, the corresponding enhancement strategy may at least not perform enhancement processing on the image, or may perform enhancement processing on the image using a preset processing method according to demand;

[0076] If the image data to be processed is of poor quality, it is necessary to determine the corresponding enhancement strategy according to the needs, such as one or more of a geometric transformation method, a color transformation method, a noise addition method and a mixed enhancement method.

[0077] S104 , performing enhancement processing on the image data to be processed according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed.

[0078] Some embodiments of the present application train a machine learning model to obtain a classification model, thereby improving the accuracy of image recognition, and then optimize the enhancement strategy according to the results of the model training, continuously enhance the image, and improve the accuracy of image processing.

[0079] Another embodiment of the present application further supplements the adaptive image data enhancement processing method provided in the above embodiment.

[0080] Optionally, a pre-trained classification model is obtained as follows:

[0081] Get sample image data;

[0082] According to the sample image data, determining a feature vector corresponding to the sample image data;

[0083] According to the classification requirements and the feature vectors of the sample image data, the machine learning model is trained to obtain a trained model;

[0084] Calculate the loss function of the trained model;

[0085] When the loss function of the trained model is less than a preset threshold, the trained model corresponding to the loss function is determined as a classification model; wherein the machine learning model includes at least a convolutional neural network model.

[0086] Based on the data set analysis, the embodiment of the present application provides an adaptive enhancement strategy generation algorithm, which automatically generates a suitable enhancement strategy based on the characteristics of the data set and the requirements of the classification task. The algorithm can dynamically adjust and optimize the enhancement strategy based on reinforcement learning or evolutionary algorithms to maximize model performance. Specifically, the enhancement strategy generation can include the following steps:

[0087] Initialize enhancement strategies: Based on the results of the dataset analysis, initialize a set of enhancement strategies. These strategies can include different enhancement methods and their parameter settings.

[0088] Initial Strategy Set = {(M i ,P i )}

[0089] Among them, M i represents the i-th enhancement method, P i Represents the parameter collection of this method.

[0090] Strategy evaluation: During the model training process, the effects of different enhancement strategies are evaluated in real time. The performance of the model on the validation set, such as accuracy, precision, recall and other indicators, can be used to judge the pros and cons of the enhancement strategy.

[0091] Evaluation Metric={Accuracy,Precision,Recall,F1 Score}

[0092] Strategy optimization: Based on the evaluation results, use reinforcement learning or evolutionary algorithms to optimize the enhancement strategy. Reinforcement learning can use the reward mechanism to enable the model to select the enhancement strategy with the best performance; evolutionary algorithms can continuously optimize the enhancement strategy through selection, crossover and mutation operations.

[0093] Strategy optimization: Based on the training results evaluated in real time, the reinforcement learning mechanism is used to optimize the strategy combination through the reward mechanism, so that the model gradually selects the best enhancement strategy. The evolutionary algorithm continuously adjusts and optimizes the enhancement strategy through selection, crossover and mutation operations to obtain the best performance.

[0094]

[0095] Where Q is the state-action value function, α is the learning rate, γ is the discount factor, and r is the reward.

[0096] Some embodiments of the present application use reinforcement learning and evolutionary algorithms to dynamically optimize the enhancement strategy. These algorithms continuously learn and evolve to find the optimal enhancement strategy, thereby improving model performance.

[0097] Optionally, the enhancement strategy includes at least one or more of a geometric transformation method, a color transformation method, a noise addition method and a hybrid enhancement method.

[0098] In order to further improve data diversity and enhancement effect, adaptive data enhancement technology integrates enhancement strategies, which include at least multiple data enhancement methods to form an enhancement method library.

[0099] Specifically, the enhanced method library can include the following methods:

[0100] 1. Geometric transformation: including rotation, flipping, cropping, scaling, translation, etc. These methods can change the spatial layout of the image and increase the diversity of the data.

[0101] 2. Color transformation: including color jitter, brightness adjustment, contrast adjustment, saturation adjustment, etc. These methods can change the color characteristics of the image and improve the diversity of the data.

[0102] 3. Noise addition: including Gaussian noise, salt and pepper noise, Poisson noise and other noise addition methods.

[0103] These methods can increase the noise of the image and improve the robustness of the model.

[0104] 4. Mixed enhancement: including CutMix, MixUp and other mixed enhancement methods. These methods increase the complexity of the data and improve the generalization ability of the model by mixing two images together.

[0105] The adaptive enhancement strategy generation algorithm can select and combine suitable enhancement methods from the enhancement method library to further improve data diversity and enhancement effect. For example, geometric transformation and color transformation can be applied at the same time, or noise addition can be combined with hybrid enhancement to obtain better enhancement effect.

[0106] Some embodiments of the present application integrate multiple data enhancement methods, including geometric transformation, color transformation, noise addition, and hybrid enhancement, etc. These methods can be combined and selected according to specific enhancement strategy requirements, thereby further improving data diversity and enhancement effects.

[0107] Optionally, the method further comprises:

[0108] The enhancement strategy is adjusted according to the enhanced image, that is, multiple groups of initial enhancement strategies are randomly generated according to the category information of the image data;

[0109] The initial enhancement strategy is optimized by reinforcement learning or evolutionary algorithm to obtain an enhancement strategy.

[0110] During the model training process, adaptive data enhancement technology dynamically adjusts the data enhancement strategy by monitoring the model training effect and data distribution in real time. Specifically, the dynamic enhancement process can include the following steps:

[0111] 1) Training monitoring:

[0112] During the model training process, the training effect of the model is monitored in real time, including indicators such as training loss, validation loss, training accuracy, and validation accuracy.

[0113] Training Metrics={Training Loss, Validation Loss, Training Accuracy, Validation Accuracy}

[0114] 2) Strategy adjustment: Dynamically adjust the data augmentation strategy based on the results of training monitoring. For example, when the model has a low accuracy rate on a certain category, the enhancement of images in that category can be strengthened; when the model's training loss fluctuates greatly, the enhancement method and parameters can be adjusted to reduce the risk of overfitting.

[0115] Strategy adjustment: Dynamically adjust the data enhancement strategy based on the results of training monitoring. For example, when the model has a low accuracy rate in a certain category, automatically increase the number of samples in that category, increase the data enhancement efforts, compensate for the lack of samples in that category, and ensure overall category balance.

[0116]

[0117] Among them, η is the learning rate and L is the loss function.

[0118] 3) Enhancement parameter optimization: In the dynamic enhancement process, not only the enhancement method is adjusted, but also the enhancement parameters can be optimized. For example, the range of rotation angles, cropping ratio, noise intensity and other parameters can be adjusted to obtain the best enhancement effect.

[0119]

[0120] In order to verify the effect of adaptive data augmentation technology in different image classification tasks, a large number of experiments are needed to compare the performance differences between adaptive data augmentation technology and traditional data augmentation methods.

[0121] Specifically, experimental verification can include the following steps:

[0122] 4) Experimental design: Select multiple typical image classification tasks and datasets, such as CIFAR-10, ImageNet, MNIST, and custom datasets, and design experimental plans. For each dataset, apply adaptive data augmentation techniques and traditional data augmentation methods, train the model, and record performance indicators.

[0123] 5) Performance evaluation: Compare and analyze the performance indicators of different enhancement methods, such as accuracy, precision, recall, F1 score, etc. Through statistical analysis, determine whether the adaptive data enhancement technology can significantly improve the model performance.

[0124] Performance Metrics={Accuracy,Precision,Recall,F1 Score}

[0125] 6) Result analysis: Analyze the reasons and mechanisms for performance improvement. For example, some enhancement methods may have significant effects on specific categories of images, while other enhancement methods may have a greater contribution to the overall dataset. Through result analysis, the advantages and disadvantages of adaptive data enhancement technology can be summarized.

[0126] Results analysis: In-depth analysis of the specific reasons for the performance improvement, including how adaptive data enhancement technology can improve the model's ability to identify scarce samples by dynamically adjusting the enhancement strategy for different categories of features. At the same time, it explores how to continuously optimize the enhancement strategy through evolution and reinforcement learning, so that the model can better adapt to the complex features in the data set, thereby improving classification accuracy and generalization ability.

[0127] 6) Strategy optimization: Based on the experimental results, continuously optimize the enhancement strategy generation algorithm and enhancement method library. For example, you can introduce new enhancement methods, adjust the generation rules of enhancement strategies, or improve the optimization mechanism of reinforcement learning and evolutionary algorithms.

[0128] The embodiment of the present application generates a random enhancement strategy combination that matches the category information according to the category information, and screens out the strategy with the best effect; selects, crosses and mutates the initial enhancement strategy to optimize the overall performance of the enhancement strategy; enhances the image data to be processed according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed, and dynamically adjusts the selection and parameters of the enhancement strategy according to the real-time training feedback of the model to ensure that the enhancement strategy has optimal adaptability to different categories.

[0129] During the model training process, some embodiments of the present application monitor the training effect in real time and dynamically adjust the enhancement strategy. Through this dynamic adjustment, the enhancement strategy can respond to the performance changes of the model in a timely manner and further improve the generalization ability of the model.

[0130] Optionally, obtaining sample image data includes:

[0131] Obtaining initial image data;

[0132] Using a preset image processing tool, extracting feature information of the initial image data, the feature information at least including a color histogram, texture features, and shape features;

[0133] According to the preset clustering algorithm, the feature information is clustered and analyzed to obtain feature classification data of different types of images;

[0134] The sample image data is determined based on the feature classification data of different types of images.

[0135] Through in-depth analysis of the data set, an enhancement strategy that adapts to the specific data set and task requirements is generated. This method overcomes the limitation of the "one-size-fits-all" traditional data enhancement method and enables the enhancement strategy to better match the characteristics of the data set.

[0136] In adaptive data enhancement technology, we first need to conduct a comprehensive analysis of the original image dataset to extract the statistical features of the image and the required features of the classification task. Through data analysis, we can understand the distribution and feature differences of different categories of images in the dataset, and provide a basis for the subsequent generation of enhancement strategies. Specifically, dataset analysis can include the following aspects:

[0137] Category distribution analysis: Count the number of images in each category in the data set to determine whether the data distribution is balanced. If the data distribution is uneven, additional enhancement is required for categories with less data during data enhancement. This can be represented by a histogram:

[0138]

[0139] Among them, N i is the number of images in the i-th category, N totalis the total number of images in the dataset.

[0140] Image feature analysis: Extract the color histogram, texture features, shape features, etc. of the image and analyze the feature differences of different categories of images. This helps determine which enhancement methods are most effective for different categories of images. Common feature extraction methods include:

[0141]

[0142] Texture Features(I)=LBP(I)

[0143] Shape Features(I)=HOG(I)

[0144] Task requirement analysis: According to the specific image classification task, determine the sensitivity of the classification model to different features. For example, in the face recognition task, the details of the facial features are very important, while in the natural scene classification task, the overall layout and color distribution may be more important.

[0145] When implementing data set analysis, the following specific approaches can be used:

[0146] Category distribution analysis: Write a script to traverse all images in the dataset, count the number of images in each category, and draw a category distribution histogram. The histogram can be used to visually observe the balance of data distribution.

[0147] Image feature analysis: Use image processing tools such as OpenCV and scikit-image to extract the image’s color histogram, texture features, shape features, etc. Clustering algorithms can be used to perform clustering analysis on features and observe the feature distribution of different categories of images.

[0148] Task requirement analysis: Combined with specific classification tasks, determine the sensitivity of the model to different features. For example, for face recognition tasks, you can use a facial key point detection algorithm to extract facial feature points and analyze their distribution.

[0149] When implementing enhanced policy generation, the following specific approaches can be adopted:

[0150] Initialization enhancement strategy: Define a set of initial enhancement strategies, including different enhancement methods and their parameter settings. For example, the rotation angle is set to -15 to 15 degrees, the brightness adjustment range is set to 0.8 to 1.2, etc.

[0151] Strategy evaluation: During the model training process, the effects of different enhancement strategies are evaluated in real time. The performance of the model on the validation set, such as accuracy, precision, recall and other indicators, can be used to judge the pros and cons of the enhancement strategy.

[0152] Strategy optimization: Based on the training results evaluated in real time, the reinforcement learning mechanism is used to optimize the strategy combination through the reward mechanism, so that the model gradually selects the best enhancement strategy. The evolutionary algorithm continuously adjusts and optimizes the enhancement strategy through selection, crossover and mutation operations to obtain the best performance.

[0153] In the process of implementing dynamic enhancement, the following specific methods can be used:

[0154] Training monitoring: Write a training monitoring module to record training loss, validation loss, training accuracy, validation accuracy and other indicators in real time. You can use visualization tools such as TensorBoard to monitor the training process in real time.

[0155] Strategy adjustment: Dynamically adjust the data augmentation strategy based on the results of training monitoring. For example, when the model has a low accuracy rate on a certain category, you can increase the number of augmentations for images in that category; when the model's training loss fluctuates greatly, you can adjust the augmentation method and parameters, or adjust hyperparameters such as the learning rate to reduce the risk of overfitting or underfitting.

[0156] Enhancement parameter optimization: In the dynamic enhancement process, not only the enhancement method can be adjusted, but also the enhancement parameters can be optimized. For example, a parameter optimization module can be written to automatically adjust parameters such as rotation angle, cropping ratio, noise intensity, etc. to obtain the best enhancement effect.

[0157] When implementing the diversity enhancement approach, the following specific methods can be used:

[0158] Geometric transformation: Implement geometric transformation methods such as rotation, flipping, cropping, scaling, and translation. For example, using the OpenCV library, you can implement operations such as image rotation, translation, and cropping.

[0159] Color transformation: Implement color transformation methods such as color jitter, brightness adjustment, contrast adjustment, saturation adjustment, hue adjustment, color balance, etc. For example, using the Pillow library, you can implement operations such as image color adjustment and brightness adjustment.

[0160] Noise addition: Implement noise addition methods such as Gaussian noise, salt and pepper noise, Poisson noise, speckle noise, mean shift noise, etc. For example, using the NumPy library, you can generate different types of noise and add it to the image.

[0161] Mixed enhancement: Implement mixed enhancement methods such as CutMix, MixUp, Mosaic, and Cutout. For example, write a function to mix two images in a certain ratio to generate a new enhanced image.

[0162] When implementing experimental verification and optimization, the following specific methods can be used:

[0163] Experimental design: Select multiple typical image classification tasks and datasets, such as CIFAR-10, ImageNet, MNIST, and custom datasets, and design experimental plans. For each dataset, apply adaptive data augmentation techniques and traditional data augmentation methods, train models, and record performance indicators.

[0164] Performance evaluation: Compare and analyze the performance indicators of different enhancement methods, such as accuracy, precision, recall, F1 score, etc. Through statistical analysis, determine whether the adaptive data enhancement technology can significantly improve the model performance.

[0165] Result analysis: Analyze the reasons and mechanisms for performance improvement. For example, some enhancement methods may have significant effects on specific categories of images, while other enhancement methods may contribute more to the overall dataset. Through result analysis, the advantages and disadvantages of adaptive data enhancement techniques can be summarized.

[0166] Strategy optimization: Based on the experimental results, continuously optimize the enhancement strategy generation algorithm and enhancement method library. For example, you can introduce new enhancement methods, adjust the generation rules of enhancement strategies, or improve the optimization mechanism of reinforcement learning and evolutionary algorithms.

[0167] Some embodiments of this application require a comprehensive analysis of the original image data set to extract the statistical features of the image and the required features of the classification task. Through data analysis, the distribution and feature differences of different categories of images in the data set can be understood, providing a basis for the subsequent generation of enhancement strategies.

[0168] Example 1: Experiment on an unbalanced custom dataset

[0169] Experiments are conducted on an imbalanced custom dataset to verify the effectiveness of the adaptive data augmentation technique.

[0170] Dataset Analysis:

[0171] Category distribution analysis: Count the number of images in each category in the custom dataset and draw a category distribution histogram. The results show that the dataset is imbalanced, with some categories having significantly fewer images than other categories.

[0172] Image feature analysis: Extract the color histogram, texture features, shape features, etc. of the images in the custom dataset. Through cluster analysis, observe the feature distribution of images of different categories, and the results show that the features of images of different categories have significant differences.

[0173] Task requirement analysis: Based on the image classification task of the custom dataset, determine the sensitivity of the model to different features. This helps to choose the appropriate enhancement method.

[0174] Enhanced policy generation:

[0175] Initialization enhancement strategy: Define a set of initial enhancement strategies, including rotation, flipping, cropping, scaling, etc. In view of data imbalance, set higher enhancement probability and richer enhancement methods for categories with fewer samples. The rotation angle is set to -15 to 15 degrees, the scaling ratio is set to 0.8 to 1.2, the cropping ratio is set to 0.8 to 1.0, etc.

[0176] Strategy evaluation: During the training process, the effects of different enhancement strategies are evaluated in real time. The pros and cons of different enhancement strategies can be judged by monitoring the accuracy on the validation set.

[0177] Strategy optimization: Use reinforcement learning algorithms (such as Q-learning) to reward and punish different reinforcement strategies based on training results, dynamically adjust the probability distribution of strategy selection, and optimize the reinforcement strategy.

[0178] Dynamic enhancement process:

[0179] Training monitoring: Real-time recording of training loss, validation loss, training accuracy, validation accuracy and other indicators, and visualization using TensorBoard.

[0180] Strategy adjustment: Dynamically adjust the data augmentation strategy based on the results of training monitoring. For categories with a small number of samples and low model accuracy, increase the number and intensity of augmentation for images in that category.

[0181] Enhancement parameter optimization: Use Bayesian optimization, grid search and other methods to automatically adjust parameters such as rotation angle, cropping ratio, noise intensity, etc. to obtain the best enhancement effect.

[0182] Diversified enhancement methods:

[0183] Implement methods such as geometric transformation, color transformation, noise addition, and hybrid enhancement, and form an enhancement method library.

[0184] The adaptive enhancement strategy generation algorithm selects and combines suitable enhancement methods from the method library, especially for scarce categories, and applies richer and stronger enhancement methods to further improve data diversity and enhancement effects.

[0185] Experimental verification and optimization:

[0186] Experimental design: On a custom dataset, we applied adaptive data augmentation techniques and traditional data augmentation methods, trained the ResNet model, and recorded performance indicators.

[0187] Performance evaluation: Comparative analysis of different methods on the validation set and test set in terms of accuracy, precision, recall, F1 score and other indicators. The results show that adaptive data augmentation technology significantly improves the overall accuracy of the model, especially in the case of scarce categories.

[0188] Result analysis: By analyzing the reasons and mechanisms for performance improvement, it is found that adaptive data augmentation technology can better balance the category distribution of data sets, mine data features, and improve data diversity.

[0189] Strategy optimization: Based on the experimental results, further optimize the enhancement strategy generation algorithm and enhancement method library, introduce new enhancement methods, and improve the versatility and robustness of the technology.

[0190] Example 2: Experiment on ImageNet dataset

[0191] Experiments are conducted on the ImageNet dataset to verify the versatility and robustness of the adaptive data augmentation technique.

[0192] Dataset Analysis:

[0193] Category distribution analysis: Count the number of images in each category in the ImageNet dataset and draw a category distribution histogram. The results show that the category distribution of the ImageNet dataset is uneven, and some categories have fewer images.

[0194] Image feature analysis: Extract the color histogram, texture features, shape features, etc. of the images in the ImageNet dataset. Through cluster analysis, observe the feature distribution of images of different categories, and the results show that the color and shape features of images of different categories are significantly different.

[0195] Task requirement analysis: Based on the image classification task of the ImageNet dataset, determine the model's sensitivity to color features and shape features. This helps to select appropriate enhancement methods.

[0196] Enhanced policy generation:

[0197] Initialization enhancement strategy: Define a set of initial enhancement strategies, including rotation, flipping, cropping, scaling, etc. The rotation angle is set to -10 to 10 degrees, the cropping ratio is set to 0.85 to 1.15, etc.

[0198] Strategy evaluation: During the training process, the effects of different enhancement strategies are evaluated in real time. The pros and cons of different enhancement strategies can be judged by monitoring the accuracy on the validation set.

[0199] Strategy optimization: Use evolutionary algorithms such as genetic algorithms to select, crossover and mutate different enhancement strategies based on training results to optimize the enhancement strategy.

[0200] Dynamic enhancement process:

[0201] Training monitoring: real-time recording of training loss, validation loss, training accuracy, validation accuracy and other indicators.

[0202] Strategy adjustment: Dynamically adjust the data augmentation strategy based on the results of training monitoring. For example, when the model has a low accuracy rate on a certain category, increase the number of augmentations for images in that category.

[0203] Enhancement parameter optimization: Automatically adjust parameters such as rotation angle, cropping ratio, noise intensity, etc. to obtain the best enhancement effect.

[0204] Diversified enhancement methods:

[0205] Implement methods such as geometric transformation, color transformation, noise addition, and hybrid enhancement, and form an enhancement method library.

[0206] The adaptive enhancement strategy generation algorithm selects and combines suitable enhancement methods from the method library to further improve data diversity and enhancement effects.

[0207] Experimental verification and optimization:

[0208] Experimental design: On the ImageNet dataset, adaptive data augmentation technology and traditional data augmentation methods were applied to train the ResNet-18 model and record the performance indicators.

[0209] Performance evaluation: Comparative analysis of different adaptive enhancement methods on the validation set and test set in terms of accuracy, precision, recall, F1 score and other indicators. The results show that adaptive data enhancement technology significantly improves the accuracy and generalization ability of the model.

[0210] Result analysis: By analyzing the reasons and mechanisms for performance improvement, it is found that adaptive data enhancement technology can better mine data features and improve data diversity.

[0211] Strategy optimization: Based on the experimental results, further optimize the enhancement strategy generation algorithm and enhancement method library, introduce new enhancement methods, and improve the versatility and robustness of the technology.

[0212] The embodiment of the present application provides an adaptive data enhancement technology for image classification optimization. Through data set analysis, enhancement strategy generation, dynamic enhancement process, diversified enhancement methods, and experimental verification and optimization, the adaptive data enhancement technology not only improves the intelligence and adaptability of data enhancement, but also significantly improves the generalization and robustness of the model. This technology has demonstrated excellent performance in multiple image classification tasks and has broad application prospects and commercial value. Future research can further optimize the enhancement strategy generation algorithm and introduce more novel enhancement methods to further improve the versatility and robustness of the technology.

[0213] It should be noted that each implementable method in this embodiment can be implemented separately or in any combination without conflict, and this application does not limit this.

[0214] Another embodiment of the present application provides an adaptive image data enhancement processing device, which is used to execute the adaptive image data enhancement processing method provided in the above embodiment.

[0215] like Figure 2 , which is a schematic diagram of the structure of an adaptive image data enhancement processing device provided in an embodiment of the present application. The adaptive image data enhancement processing device includes an acquisition module 201, a determination module 202, a judgment module 203 and a processing module 204, wherein:

[0216] The acquisition module 201 is used to acquire the image data to be processed;

[0217] The determination module 202 is used to determine the target category information corresponding to the image data to be processed according to a pre-trained classification model, where the classification model is obtained by training a machine learning model using sample image data;

[0218] The judgment module 203 is used to determine the target enhancement strategy corresponding to the target category information according to the target category information;

[0219] The processing module 204 is used to perform enhancement processing on the image data to be processed according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed.

[0220] Regarding the device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0221] Some embodiments of the present application train a machine learning model to obtain a classification model, thereby improving the accuracy of image recognition, and then optimize the enhancement strategy according to the results of the model training, continuously enhance the image, and improve the accuracy of image processing.

[0222] Another embodiment of the present application further supplements the adaptive image data enhancement processing device provided in the above embodiment.

[0223] Optionally, the device further comprises a training module, the training module being used for:

[0224] Get sample image data;

[0225] Determine a feature vector of the sample image data according to the sample image data;

[0226] According to the classification requirements and the feature vectors of the sample image data, the machine learning model is trained to obtain a trained model;

[0227] Calculate the loss function of the trained model;

[0228] When the loss function of the trained model is less than a preset threshold, the trained model corresponding to the loss function is determined as a classification model; wherein the machine learning model includes at least a convolutional neural network model.

[0229] The embodiment of the present application combines reinforcement learning and evolutionary algorithms to dynamically optimize the enhancement strategy. After the random generation and preliminary screening of the enhancement strategy, the evolutionary learning algorithm is responsible for optimizing the strategy combination, and the reinforcement learning dynamically adjusts the selection of the enhancement method according to the model feedback (such as classification accuracy, loss, etc.), and finally finds the optimal enhancement strategy, thereby improving the overall performance of the model.

[0230] Optionally, the enhancement strategy includes at least one or more of a geometric transformation method, a color transformation method, a noise addition method, and a hybrid enhancement method. The generation and selection of the enhancement strategy are based on the category information and feature differences of the image. Through the initialization of the random enhancement strategy, combined with geometric transformation (such as rotation, scaling, cropping, etc.), color transformation (brightness, contrast adjustment, etc.), noise addition (such as Gaussian noise, salt and pepper noise, etc.) and hybrid enhancement methods (such as CutMix, MixUp), the system can adaptively select the optimal enhancement combination.

[0231] Some embodiments of the present application integrate multiple data enhancement methods, including geometric transformation, color transformation, noise addition, and hybrid enhancement, etc. These methods can be combined and selected according to specific enhancement strategy requirements, thereby further improving data diversity and enhancement effects.

[0232] Optionally, a training module is used to:

[0233] According to the category information of the image data, multiple groups of initial enhancement strategies are randomly generated;

[0234] The initial enhancement strategy is optimized by reinforcement learning or evolutionary algorithm to obtain an enhancement strategy.

[0235] The enhancement strategy is further optimized based on the enhanced image, and the parameters of the enhancement method are dynamically adjusted to adapt to changes in model training.

[0236] During the model training process, some embodiments of the present application monitor the training effect in real time and dynamically adjust the enhancement strategy. By introducing the reinforcement learning mechanism, the enhancement strategy can respond in a timely manner according to the performance changes of the model, further improving the generalization ability of the model, especially in the case of category imbalance, the weaker performing categories can be enhanced.

[0237] Optionally, the training module is used to:

[0238] Obtaining initial image data;

[0239] Using a preset image processing tool, extracting feature information of the initial image data, the feature information at least including a color histogram, texture features, and shape features;

[0240] According to the preset clustering algorithm, the feature information is clustered and analyzed to obtain feature classification data of different types of images;

[0241] According to the feature classification data of different types of images, sample image data is determined and an initial enhancement strategy is generated.

[0242] Some embodiments of the present application generate an initial enhancement strategy after a comprehensive analysis of the original image dataset, combined with the classification requirements of the model. Through data analysis, the distribution and feature differences of different categories of images in the dataset can be understood, and the reinforcement learning mechanism can further optimize the enhancement strategy to adapt to the task requirements.

[0243] Regarding the device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0244] It should be noted that each implementable method in this embodiment may be implemented separately, or may be implemented in combination in any combination without conflict, and this application is not limited thereto.

[0245] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the operation of the method corresponding to any embodiment of the adaptive image data enhancement processing method provided in the above embodiments can be implemented.

[0246] An embodiment of the present application also provides a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations of the method corresponding to any embodiment of the adaptive image data enhancement processing method provided in the above embodiments.

[0247] like Figure 3 As shown, some embodiments of the present application provide an electronic device 300, which includes: a memory 310, a processor 320, and a computer program on the memory 310 and executable on the processor 320, wherein the processor 320 can implement a method of any embodiment included in the above-mentioned adaptive image data enhancement processing method when reading the program from the memory 310 through a bus 330 and executing the program.

[0248] Processor 320 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 320 can be a microprocessor.

[0249] The memory 310 may be used to store instructions executed by the processor 320 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 320 of the disclosed embodiment may be used to execute instructions in the memory 310 to implement the method shown above. The memory 310 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0250] The above are only embodiments of the present application and are not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0251] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0252] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. An adaptive image data enhancement processing method, characterized in that: The method comprises: Obtain image data to be processed; Determining target category information corresponding to the image data to be processed according to a pre-trained classification model, wherein the classification model is obtained by training a machine learning model using sample image data; Determining a target enhancement strategy corresponding to the target category information according to the target category information; The image data to be processed is enhanced according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed.

2. The adaptive image data enhancement processing method according to claim 1, characterized in that: The pre-trained classification model is obtained in the following way: Get sample image data; Determining a feature vector of the sample image data according to the sample image data; Training the machine learning model according to the classification requirements and the feature vectors of the sample image data to obtain a trained model; Calculating the loss function of the trained model; When the loss function of the trained model is less than a preset threshold, the corresponding trained model is determined as the classification model; wherein the machine learning model includes at least a convolutional neural network model.

3. The adaptive image data enhancement processing method according to claim 1, characterized in that: The method further comprises: According to the category information of the image data, multiple groups of initial enhancement strategies are randomly generated; The initial enhancement strategy is optimized by reinforcement learning or evolutionary algorithm to obtain an enhancement strategy.

4. The adaptive image data enhancement processing method according to claim 3, characterized in that: The initial enhancement strategy includes at least one or more of a geometric transformation method, a color transformation method, a noise addition method and a hybrid enhancement method.

5. The adaptive image data enhancement processing method according to claim 2, characterized in that: The obtaining of sample image data comprises: Obtaining initial image data; Using a preset image processing tool, extracting feature information of the initial image data, the feature information at least including a color histogram, texture features, and shape features; Performing cluster analysis on the feature information according to a preset clustering algorithm to obtain feature classification data of different types of images; The sample image data is determined according to the image feature classification data.

6. An adaptive image data enhancement processing device, characterized in that: The device comprises: An acquisition module, used for acquiring image data to be processed; A determination module, used to determine target category information corresponding to the image data to be processed according to a pre-trained classification model, wherein the classification model is obtained by training a machine learning model using sample image data; A judgment module, used to determine a target enhancement strategy corresponding to the target category information according to the target category information; The processing module is used to perform enhancement processing on the image data to be processed according to the target enhancement strategy to obtain an enhanced image corresponding to the image to be processed.

7. The adaptive image data enhancement processing device according to claim 6, characterized in that: The device also includes a training module, which is used to: Get sample image data; Determining a feature vector of the sample image data according to the sample image data; Training the machine learning model according to the classification requirements and the feature vectors of the sample image data to obtain a trained model; Calculating the loss function of the trained model; When the loss function of the trained model is less than a preset threshold, the trained model is determined as the classification model; wherein the machine learning model includes at least a convolutional neural network model.

8. The adaptive image data enhancement processing device according to claim 6, characterized in that: The enhancement strategy includes at least one or more of a geometric transformation method, a color transformation method, a noise addition method and a hybrid enhancement method.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program on the memory and executable on the processor, wherein the processor can implement the adaptive image data enhancement processing method as claimed in any one of claims 1 to 5 when executing the program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the program, when executed by a processor, can implement the adaptive image data enhancement processing method described in any one of claims 1 to 5.

Images