An image recognition model generalization capability enhancement method, system and computer readable storage medium

Abstract

The application provides a method, system and computer readable storage medium for enhancing the generalization ability of an image recognition model under different scenes, the method comprising: classifying training picture samples according to scenes; and training a feature layer corresponding to a scene category in the image recognition model according to the scene category. The technical solution of the application can enhance the generalization ability of the image recognition model.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a method, system, and computer-readable storage medium for enhancing the generalization ability of image recognition models in different scenarios. Background Technology

[0002] Image recognition models generally exhibit poor generalization ability across various scenarios. Specifically, they perform better in scenarios that appear frequently in the training set, but worse in scenarios that are less common or unseen. Currently, there are generally two main approaches to addressing the issue of applying models to different scenarios. The first approach focuses on increasing the amount of data on less frequently encountered scenarios—data augmentation—to make the model more familiar with these scenarios and improve its recognition accuracy in those scenarios. The second approach addresses the limitations of the model itself, ensuring that knowledge learned from other domains or scenarios can be transferred to the current scenario.

[0003] Both data augmentation and scenario transfer methods have significant drawbacks. Data augmentation methods collect data directly, but there are issues such as whether data from different scenarios can be collected, privacy concerns, and the cost of labeling the collected data. While many scenario transfer methods have been proposed, such as zero-shot and one-shot methods, their performance in real-world applications has been poor. This is because many real-world scenarios are significantly affected by factors such as lighting, angle, and background environment, resulting in suboptimal transfer performance. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and computer-readable storage medium for enhancing the generalization ability of image recognition models in different scenarios, thereby enhancing the generalization ability of image recognition models.

[0005] In this embodiment of the invention, a method for enhancing the generalization ability of an image recognition model is provided, comprising:

[0006] Classify training image samples according to the scene;

[0007] The feature layer in the image recognition model corresponding to the scene category is trained based on the scene category of the training image samples.

[0008] In this embodiment of the invention, the method for enhancing the generalization ability of the image recognition model further includes:

[0009] The weight factor for each scene is calculated based on the frequency of different scenes appearing in the training image samples.

[0010] In this embodiment of the invention, during the training process, the Loss value of the image recognition model is calculated based on the weight factors for different scenarios. 总 The calculation method is as follows:

[0011]

[0012] Where, p i The loss represents the weight factor for the i-th scenario. i The loss value is calculated from all training image samples representing the i-th scene.

[0013] In this embodiment of the invention, the calculation formulas for the weighting factors in different scenarios are as follows:

[0014] Where, x i This represents the number of times the i-th type of scenario occurs.

[0015] In this embodiment of the invention, the method for enhancing the generalization ability of the image recognition model includes the following steps before classifying the training image samples according to the scene:

[0016] A scene classifier is obtained by training with scene-labeled training image samples. The scene classifier is used to classify training data without scene labels.

[0017] In this embodiment of the invention, an image recognition model generalization ability enhancement system is also provided, comprising:

[0018] A sample classifier is used to classify training image samples according to the scene.

[0019] The training module is used to train the feature layer in the image recognition model corresponding to the scene category based on the scene category of the training image samples.

[0020] In this embodiment of the invention, the image recognition model generalization ability enhancement system further includes:

[0021] The weight factor calculation module is used to calculate the weight factor of each scene based on the frequency of different scenes appearing in the training image samples.

[0022] In this embodiment of the invention, the image recognition model generalization ability enhancement system further includes:

[0023] The Loss calculation module is used to calculate the Loss value of the image recognition model based on the weight factors for different scenarios. 总 The calculation method is as follows:

[0024]

[0025] Where, p i The loss represents the weight factor for the i-th scenario. i The loss value is calculated from all training image samples representing the i-th scene.

[0026] In this embodiment of the invention, the formula for calculating the weight factors for different scenarios by the weight factor calculation module is as follows:

[0027]

[0028] Where, p i Let x represent the weight factor for the i-th scenario. i This represents the number of times the i-th type of scenario occurs.

[0029] In this embodiment of the invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described method for enhancing the generalization ability of an image recognition model.

[0030] Compared with existing technologies, the image recognition model generalization enhancement method and system of the present invention, on the one hand, trains the feature layer corresponding to the scene category in the image recognition model according to the scene category of the training image samples. This ensures that even scenes with fewer occurrences are fully extracted by some feature layers, enabling the image recognition model to more effectively utilize data from scenes with fewer occurrences and improving its generalization ability in different scenes. On the other hand, during training, the loss value of the image recognition model is calculated based on the weight factors of different scenes. This takes into account the transferability and occurrence frequency between different scenes. Scenes with more occurrences have lower importance in the final loss calculation, while scenes with fewer occurrences have higher importance in the loss calculation. This allows the image recognition model to converge on scenes with fewer occurrences during training, thereby improving the overall accuracy of the image recognition model in different scenes, especially scenes with fewer occurrences. Attached Figure Description

[0031] Figure 1 This is a flowchart of the method for enhancing the generalization ability of image recognition models according to an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram illustrating the calculation of weighting factors for different scenarios in an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram illustrating the training of the feature layer in an embodiment of the present invention.

[0034] Figure 4 This is a schematic diagram of the image recognition model generalization ability enhancement system of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] The implementation of the present invention will be described in detail below with reference to specific embodiments.

[0037] like Figure 1 As shown in the figure, in this embodiment of the invention, a method for enhancing the generalization ability of an image recognition model is provided, which includes steps S1-S5, which are described below.

[0038] Step S1: Train a scene classifier using training image samples with scene labels.

[0039] In this embodiment of the invention, when training the image recognition model, it is necessary to classify the images according to the scene. Since the image recognition model requires a large number of training image samples, manual classification would consume too many resources. Therefore, a scene classifier can be trained first to automatically classify the training image samples. Specifically, training images collected based on different scenes can be collected. While manually labeling objects in different scenes, a new scene label is added to represent the corresponding scene. For example, the entire training set has k scenes, and the number of training images corresponding to each scene is N. k Zhang. The scene classifier can be implemented using ResNet18, where the input is any image and the output is a k-dimensional one-hot vector [0, ..., 0, 1, 0, ..., 0], where the scene category corresponding to the image is 1 and the other categories are 0.

[0040] Step S2: Use the sample classifier to classify the training image samples.

[0041] Once the sample classifier is obtained, it can be used to automatically classify training image samples with more sample data, thereby obtaining the category of each training image based on the scene of the training image.

[0042] Step S3: Calculate the weight factor for each scene based on the frequency of different scenes appearing in the training image samples.

[0043] In this embodiment, the formulas for calculating the weighting factors for different scenarios are as follows:

[0044] Where, p i Let x represent the weight factor for the i-th scenario. iThis represents the number of times the i-th type of scenario occurs.

[0045] Specifically, such as Figure 2 As shown, the number of sample images in Scene 1 is N1, and the formula for calculating its weight factor is:

[0046]

[0047] The number of sample images in Scenario 2 is N², and the formula for calculating its weight factor is:

[0048]

[0049] The number of sample images for scene k is NK, and the formula for calculating its weight factor is:

[0050]

[0051] It should be noted that the above calculation formula is only an exemplary calculation method. Other calculation methods can also be used as needed to calculate the weight factors of different scenes based on the frequency of different scenes in the training image samples. For example, the weight factors of different scenes can be calculated based on the proportional relationship of the frequency of different scenes in the training image samples.

[0052] Step S4: Train the feature layer in the image recognition model corresponding to the scene category based on the scene category of the training image samples.

[0053] It should be noted that image recognition models are neural network models, typically composed of multiple feature layers, each corresponding to different features in the image. To ensure that even scenes with fewer occurrences are adequately featured by some feature layers, in this embodiment of the invention, the correlation between different feature layers and different scenes can be pre-determined through manual research. A correspondence is established between scene categories with strong correlations and feature layers. Then, when training the image enhancement model, the feature layers in the image recognition model corresponding to those scenes are trained based on the scene categories of the training image samples. During each training iteration, only the feature layers corresponding to the scene type are updated based on the scene category of the training image samples, while other feature layers remain unchanged. This allows different feature layers to be trained for different scenes. (The following paragraph explains the correspondence between scene categories and feature layers.) As shown in Figure 3, for images in scene i, the correlation with the first feature layer G1 is weak, but the correlation with the k-th feature layer Gk is strong. Therefore, when using images from scene i for training, feature layer G1 is not updated; instead, feature layer Gk is updated. For images in scene j, the correlation with the first feature layer G1 is strong, but the correlation with the k-th feature layer Gk is weak. Therefore, when using images from scene i for training, feature layer GK is not updated; instead, feature layer G1 is updated. In other words, training image samples are only used to train the feature layers corresponding to their respective scene categories.

[0054] Step S5: Calculate the Loss value of the image recognition model based on the weight factors for different scenarios. 总 .

[0055] In this embodiment of the invention, the Loss value of the image recognition model is Loss. 总 The calculation method is as follows:

[0056]

[0057] Where, p i The loss represents the weight factor for the i-th scenario. i The loss value is calculated from all training image samples representing the i-th scene.

[0058] It should be noted that during the training process of the image recognition model, the loss value of the image recognition model needs to be calculated, and the various feature layers of the image recognition model are adjusted based on the loss value. In this embodiment of the invention, the loss value of the image recognition model is calculated based on the weight factors of different scenes. Considering the transferability and frequency of occurrence between different scenes, the more frequently a scene occurs, the less important it is in the final loss calculation; conversely, the less frequently occurring scenes have higher loss importance. This can better improve the convergence of the image recognition model to less frequently occurring scenes during training.

[0059] like Figure 4 As shown, corresponding to the above-described method for enhancing the generalization ability of image recognition models, this embodiment of the invention also provides a system for enhancing the generalization ability of image recognition models, which includes a sample classifier 1, a training module 2, a weight factor calculation module 3, and a loss calculation module 4. These will be described in detail below.

[0060] The sample classifier 1 is used to classify training image samples according to the scene. The sample classifier 1 can be obtained by training a ResNet18 neural network using training image samples with scene labels.

[0061] The training module 2 is used to train the feature layer in the image recognition model corresponding to the scene category based on the scene category of the training image samples.

[0062] The weight factor calculation module 3 is used to calculate the weight factor of each scene based on the frequency of different scenes appearing in the training image samples. The calculation formula for the weight factor of different scenes by the weight factor calculation module 3 is as follows:

[0063]

[0064] Where, p i Let x represent the weight factor for the i-th scenario. i This represents the number of times the i-th type of scenario occurs.

[0065] The Loss calculation module 4 is used to calculate the Loss value of the image recognition model based on the weight factors for different scenarios. 总 The calculation method is as follows:

[0066]

[0067] Where, p i The loss represents the weight factor for the i-th scenario. i The loss value is calculated from all training image samples representing the i-th scene.

[0068] In this embodiment of the invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described method for enhancing the generalization ability of an image recognition model.

[0069] In summary, the image recognition model generalization enhancement method and system of the present invention, on the one hand, trains the feature layers corresponding to the scene categories in the image recognition model according to the scene categories of the training image samples. This ensures that even scenes with fewer occurrences are fully extracted by some feature layers, enabling the image recognition model to more effectively utilize data from scenes with fewer occurrences and improving its generalization ability in different scenes. On the other hand, during training, the loss value of the image recognition model is calculated based on the weight factors of different scenes. This takes into account the transferability and occurrence frequency between different scenes. Scenes with more occurrences have lower importance in the final loss calculation, while scenes with fewer occurrences have higher importance in the loss calculation. This allows the image recognition model to converge on scenes with fewer occurrences during training, thereby improving the overall accuracy of the image recognition model in different scenes, especially scenes with fewer occurrences.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for enhancing the generalization ability of an image recognition model, characterized in that, include: Classify training image samples according to the scene; The feature layer in the image recognition model corresponding to the scene category is trained based on the scene category of the training image samples. During training, the weight factor of each scene is calculated based on the frequency of different scenes in the training image samples, and the loss value of the image recognition model is calculated based on the weight factors of different scenes.

2. The method for enhancing the generalization ability of an image recognition model as described in claim 1, characterized in that, During training, the loss value of the image recognition model is calculated based on the weight factors for different scenarios. 总 The calculation method is as follows: Loss 总 =∑ i p i Loss i , Where, p i The loss represents the weight factor for the i-th scenario. i The loss value is calculated from all training image samples representing the i-th scene.

3. The method for enhancing the generalization ability of an image recognition model as described in claim 1, characterized in that, The formulas for calculating the weighting factors for different scenarios are as follows: Where, p i Let x represent the weight factor for the i-th scenario. i This represents the number of times the i-th type of scenario occurs.

4. The method for enhancing the generalization ability of an image recognition model as described in claim 1, characterized in that, Before classifying the training image samples according to the scene, the following steps are included: A scene classifier is obtained by training with scene-labeled training image samples. The scene classifier is used to classify training data without scene labels.

5. A system for enhancing the generalization ability of an image recognition model, characterized in that, include: A sample classifier is used to classify training image samples according to the scene. The training module is used to train the feature layer in the image recognition model corresponding to the scene category based on the scene category of the training image samples. During the training process, the weight factor of each scene is calculated based on the number of times different scenes appear in the training image samples, and the loss value of the image recognition model is calculated based on the weight factor of different scenes.

6. The image recognition model generalization ability enhancement system as described in claim 5, characterized in that, The Loss calculation module is used to calculate the Loss value of the image recognition model based on the weight factors for different scenarios. 总 The calculation method is as follows: Loss 总 =∑ i p i Loss i , Where, p i The loss represents the weight factor for the i-th scenario. i The loss value is calculated from all training image samples representing the i-th scene.

7. The image recognition model generalization ability enhancement system as described in claim 5, characterized in that, The formulas for calculating the weighting factors for different scenarios are as follows: Where, p i Let x represent the weight factor for the i-th scenario. i This represents the number of times the i-th type of scenario occurs.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image recognition model generalization capability enhancement method as described in any one of claims 1 to 4.

Images