Image recognition model training method, image recognition method, and electronic device

Abstract

Embodiments of the present application disclose a training method of an image recognition model, an image recognition method and an electronic device, wherein the training method of the image recognition model comprises: obtaining an image sample group, wherein the image sample group comprises at least two sample images of different spatial domains; fusing the sample images of different spatial domains in the image sample group to obtain a mixed enhanced image, and setting the same label for the sample images and the mixed enhanced image; and iteratively training the image recognition model based on the image group corresponding to each label and comprising the sample images and the mixed enhanced image to obtain a target image recognition model. Through the above training scheme, the image recognition model can learn the internal relationship of the sample images of the same label and different spatial domains, thereby improving the image recognition accuracy of the image recognition model.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to a training method of an image recognition model, an image recognition method and an electronic device. BACKGROUND

[0002] At present, image processing technology is widely used in various application scenarios as an effective means to obtain effective information from images. In many scenarios, feature extraction of images is required to capture attention information from rich image information. With the rapid development of artificial intelligence technology, various neural network methods are applied to image recognition processing to improve image recognition effect.

[0003] However, the traditional method of using a neural network model for image recognition is only effective in a homogeneous data set, and has poor recognition ability for image groups with large differences in spatial domain, which affects the recognition accuracy of image groups with large differences in spatial domain. SUMMARY

[0004] Embodiments of the present application provide a training method of an image recognition model, an image recognition method and an electronic device to improve image recognition accuracy.

[0005] In a first aspect, embodiments of the present application provide a training method of an image recognition model, comprising:

[0006] obtaining an image sample group, wherein the image sample group includes at least two sample images of different spatial domains;

[0007] fusing different spatial domain sample images in the image sample group to obtain a mixed enhanced image, and setting the same label for the sample images and the mixed enhanced image;

[0008] iteratively training an image recognition model based on image groups corresponding to each label and including the sample images and the mixed enhanced image, to obtain a target image recognition model.

[0009] In a second aspect, embodiments of the present application also provide an image recognition method, comprising:

[0010] obtaining an image to be processed;

[0011] inputting the image to be processed into a target image recognition model to obtain an image recognition result, wherein the target image recognition model is trained according to the training method of the image recognition model provided by any embodiment of the present application.

[0012] In a third aspect, embodiments of the present application also provide a training device of an image recognition model, comprising:

[0013] a sample acquisition module, configured to acquire an image sample set, wherein the image sample set comprises at least two sample images in different spatial domains;

[0014] a label setting module, configured to fuse the sample images in different spatial domains in the image sample set to obtain a mixed enhanced image, and set the same label for the sample images and the mixed enhanced image;

[0015] a model training module, configured to sequentially perform iterative training on the image recognition model based on the image set corresponding to each label and comprising the sample images and the mixed enhanced image, to obtain a target image recognition model.

[0016] In a fourth aspect, an embodiment of the present application further provides an image recognition device, comprising:

[0017] an image acquisition module, configured to acquire a to-be-processed image, wherein the to-be-processed image is an image set in different spatial domains;

[0018] an identification result determination module, configured to input the to-be-processed image into the target image recognition model to obtain an image recognition result, wherein the target image data model comprises a feature extraction module, an information enhancement module and an identification module, the feature extraction module is configured to extract global features and multi-scale attention features from the input to-be-processed image, the information enhancement module is configured to fuse the multi-scale attention features to obtain local enhanced features, and the identification module is configured to obtain an image recognition result based on the global features and the local enhanced features.

[0019] In a fifth aspect, an embodiment of the present application further provides an electronic device, which comprises:

[0020] one or more processors;

[0021] a storage device, configured to store one or more programs,

[0022] when the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the image recognition model or the image recognition method provided by any embodiment of the present application.

[0023] In a sixth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the training method of the image recognition model or the image recognition method provided by any embodiment of the present application.

[0024] The technical solution of this invention involves acquiring at least two sample images from different spatial domains, indicating that the samples of this invention include sample images from different spatial domains. Further, the sample images from different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is assigned to both the sample image and the hybrid enhanced image. Based on the image group containing the sample image and the hybrid enhanced image corresponding to each label, the image recognition model is trained to obtain the target image recognition model. This training enables the image recognition model to learn the inherent relationship between images with the same label and from different spatial domains within the image group, thereby improving the image recognition accuracy of the image recognition model. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0026] Figure 1 This is a flowchart illustrating a training method for an image recognition model provided in Embodiment 1 of the present invention.

[0027] Figure 2 This is a flowchart illustrating a training method for an image recognition model provided in Embodiment 2 of the present invention.

[0028] Figure 3 This is a schematic diagram of the structure of an image recognition model provided in Embodiment 2 of the present invention;

[0029] Figure 4 This is a flowchart illustrating a training method for an image recognition model provided in Embodiment 3 of the present invention;

[0030] Figure 5 This is a flowchart illustrating an image recognition method provided in Embodiment 4 of the present invention;

[0031] Figure 6 This is a schematic diagram of the structure of a training device for an image recognition model provided in Embodiment 5 of the present invention;

[0032] Figure 7 This is a schematic diagram of the structure of an image recognition device provided in Embodiment Six of the present invention;

[0033] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. Detailed Implementation

[0034] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0035] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0036] Example 1

[0037] Figure 1 This is a flowchart illustrating a training method for an image recognition model provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of automatic image recognition. The method can be executed by an image recognition model training device, which can be implemented by software and / or hardware and can be configured in a terminal and / or server to implement the image recognition model training method in the embodiment of the present invention.

[0038] like Figure 1 As shown, the method in this embodiment may specifically include:

[0039] S110. Obtain an image sample group, wherein the image sample group includes at least two sample images from different spatial domains.

[0040] S120. The sample images of different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is set for the sample images and the hybrid enhanced image.

[0041] S130. Based on the image group including the sample image and the hybrid enhanced image corresponding to each label, the image recognition model is iteratively trained in sequence to obtain the target image recognition model.

[0042] In this embodiment, the image sample set refers to pre-made training samples, which may include multiple sample images, including but not limited to face images or body shape images. The image sample set includes, but is not limited to, at least two sample images from different spatial domains, where the spatial domain is also called image space. There are significant modal differences between sample images from different spatial domains. For example, the at least two sample images from different spatial domains may be thermal infrared images and visible light images.

[0043] Specifically, methods for obtaining image sample sets include, but are not limited to: acquiring image sample sets using infrared cameras and ordinary optical cameras; or retrieving pre-made image sample sets from a preset storage location.

[0044] To improve the accuracy of image recognition, this invention fuses sample images within an image sample group to obtain a hybrid enhanced image, and assigns the same label to both the original sample images and the hybrid enhanced image. The hybrid enhanced image refers to the image obtained after fusing the sample images within the image sample group. By fusing sample images within the image sample group to obtain the hybrid enhanced image, the diversity of training samples is increased. Furthermore, the hybrid enhanced image with the same label is also input into the image recognition model for training. The hybrid enhanced image, being an intermediate state among the sample images, possesses the comprehensive features of each sample image, which helps the image recognition model learn the inherent relationships between sample images with the same label but different spatial domains, thereby improving the image recognition accuracy of the image recognition model.

[0045] Based on the above embodiments, the step of fusing sample images from different spatial domains within the image sample group to obtain a hybrid enhanced image includes: weighting the pixel values ​​of corresponding pixels in the fused sample images based on the image fusion ratio to obtain a hybrid enhanced image.

[0046] Here, the image fusion ratio refers to the proportion of each sample image within the image sample group that is fused, and it can be set randomly. For example, the fusion formula of the image fusion method includes:

[0047] x=λ×x i +(1-λ)×x j

[0048] Where x represents the hybrid enhancement image, x i x represents the first image to be identified. j Let λ represent the second image to be identified; λ represents the image fusion ratio, which is a random number between [0,1] and follows a beta distribution, wherein the spatial domains of the first image to be identified and the second image to be identified are different.

[0049] In this embodiment, the image recognition model is trained based on sample images, hybrid enhanced images, and corresponding labels for each image to obtain the target image recognition model. Specifically, before training the image recognition model, multiple sample images are fused to obtain hybrid enhanced images. The same labels are assigned to the sample images and the hybrid enhanced images. The model parameters in the image recognition model are trained based on the sample images, the hybrid enhanced images, and the corresponding labels for each image. By continuously adjusting the model parameters, the model loss gradually decreases and tends to stabilize, thus obtaining the target image recognition model.

[0050] The technical solution of this embodiment acquires at least two sample images from different spatial domains, indicating that the samples of the present invention include sample images from different spatial domains. Further, the sample images from different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is set for the sample images and the hybrid enhanced image. Based on the image group including the sample images and the hybrid enhanced image corresponding to each label, the image recognition model is iteratively trained in sequence to obtain the target image recognition model. This training enables the image recognition model to learn the inherent relationship between images with the same label and different spatial domains within the image group, thereby improving the image recognition accuracy of the image recognition model.

[0051] Example 2

[0052] Figure 2 This is a flowchart of the training method for the image recognition model provided in Embodiment 2 of the present invention. Optionally, based on any optional technical solution in the embodiments of the present invention, the image recognition model includes a feature extraction module and a recognition module. The step of iteratively training the image recognition model based on image groups corresponding to each label, including the sample image and the hybrid enhanced image, includes: inputting sample images and hybrid enhanced images with the same label to the feature extraction module to be trained, respectively, to obtain image features of the sample image and the hybrid enhanced image; determining a loss function based on the image features of the sample image and the hybrid enhanced image; and updating the parameters of the feature extraction module and the recognition module in the image recognition model based on the loss function.

[0053] like Figure 2 As shown, the method in this embodiment may specifically include:

[0054] S210. Obtain an image sample group, wherein the image sample group includes at least two sample images from different spatial domains.

[0055] S220. The sample images of different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is set for the sample images and the hybrid enhanced image.

[0056] S230. Input the sample image and the hybrid enhanced image with the same label into the feature extraction module to be trained, respectively, to obtain the image features of the sample image and the hybrid enhanced image.

[0057] S240. Determine a loss function based on the image features of the sample image and the image features of the hybrid enhanced image, and update the parameters of the feature extraction module and the recognition module in the image recognition model based on the loss function to obtain the target image recognition model.

[0058] In this embodiment, the image recognition model may include, but is not limited to, a feature extraction module and a recognition module. The feature extraction module is used to extract image features from sample images or hybrid enhanced images, and the recognition module is used to determine the recognition result of the image sample group.

[0059] Specifically, the image recognition model can be pre-trained using a large number of sample images with the same label and hybrid augmented images. The sample images with the same label and the hybrid augmented images are input into the feature extraction module to be trained, respectively, to obtain image features corresponding to multiple sample images and image features of the hybrid augmented images. Each image feature can be used to determine the loss function; that is, the loss function is determined by the image features of the sample images and the hybrid augmented images. Based on the loss function, the parameters of the feature extraction module and the recognition module in the image recognition model are updated, so that the model's loss gradually decreases and tends to stabilize, thus obtaining the target image recognition model.

[0060] Based on the above embodiments, the feature extraction module includes a global feature extraction module and an attention feature extraction module. The global feature extraction module is used to extract global features of the input image, and the attention feature extraction module is used to extract multiple attention features of the input image at different scales.

[0061] The input image can include sample images and hybrid enhanced images. The global feature extraction module can include multiple globally feature extraction sub-modules connected in sequence. Each sub-module extracts features from the input feature map and outputs intermediate feature maps to the next sub-module, until the final sub-module outputs global features. The attention feature extraction module is connected to the global feature extraction sub-modules and extracts attention features from the intermediate feature maps. There can be one or more attention feature extraction modules. When there are multiple modules, multiple attention features at different scales can be obtained. By extracting attention features at multiple scales, the reliability and richness of the extracted features can be improved, thereby enhancing the accuracy of image recognition.

[0062] In some embodiments, the loss function can be determined based on the global features corresponding to the sample image and the hybrid augmented image; or, the loss function can be determined based on the global features and attention features corresponding to the sample image and the hybrid augmented image; or, the loss function can be determined based on the global features and attention features of the sample image and the global features of the hybrid augmented image. This embodiment does not limit the method of determining the loss function.

[0063] In some embodiments, the image sample group includes first sample images and second sample images from different spatial domains. For example, a loss function is determined based on the global features corresponding to the sample images and the blended enhanced images. The loss function may include:

[0064] L1 = L V +L M +L A

[0065]

[0066]

[0067]

[0068] Among them, L V L represents the loss function corresponding to the first sample image. M L represents the loss function corresponding to the second sample image. A This represents the loss function corresponding to the hybrid enhanced image; s and m are hyperparameters, and y p Let θ represent the label corresponding to the p-th image sample or the hybrid augmented image, θ represent the angle between the global feature vector and the weight vector, N represent the number of training samples input in each iteration during training, and n represent the number of categories in the training set. Through the above loss function, hybrid training of the first sample image, the second sample image, and the hybrid augmented image is achieved, improving the training effect of the image recognition model and thus enhancing image recognition accuracy.

[0069] Based on the above embodiments, the image recognition model may further include an information enhancement module, which is used to update the parameters of the image recognition model according to the similarity and difference of the attention features of the sample images.

[0070] Similarity refers to the similarity between attention features at different levels within the same spatial domain; difference refers to the difference between attention features at the same level but in different spatial domains. By calculating the similarity and difference of attention features for each sample image, the parameters of the image recognition model can be updated to improve its recognition accuracy.

[0071] Based on the above embodiments, the image sample group includes first sample images and second sample images from different spatial domains, and the first sample images and the second sample images are fused to obtain a hybrid enhanced image; correspondingly, the loss function includes:

[0072] L total =α1L1 + α2L2 + α3L3

[0073] L1 = L V +L M +L A

[0074]

[0075]

[0076] Where L1 represents the first loss function, L2 represents the second loss function, L3 represents the third loss function, and α1, α2, and α3 represent the weights of the first, second, and third loss functions, respectively; L V L represents the loss function corresponding to the first sample image. M L represents the loss function corresponding to the second sample image. A This represents the loss function corresponding to the hybrid enhanced image. This represents the i-th attention feature of the first sample image. This represents the j-th attention feature of the first sample image; This represents the i-th attention feature of the second sample image. Let j represent the j-th attention feature of the second sample image.

[0077] In this embodiment, L2 represents the second loss function corresponding to similarity, and L3 represents the third loss function corresponding to difference. Training the model parameters using these loss functions effectively captures global and attention features from different spatial domains. These global and attention features are then used to update the parameters of the image recognition model, thereby improving the model's training accuracy and enhancing its recognition performance. It should be noted that the loss function can also be determined using other distance measurement methods, such as the Hamming distance method.

[0078] In this embodiment, the i-th or j-th attention feature can be extracted by the i-th or j-th attention feature extraction module. Specifically, image features output by one or more intermediate layers in the global feature extraction module are obtained; typically, the intermediate layers can be pooling layers. Further, the image features output by the intermediate layers are input to the attention feature extraction module to obtain the corresponding attention features. It is understood that the number of image features obtained from the intermediate layers is the same as the number of attention feature extraction modules.

[0079] For example, the model architecture of an image recognition model is as follows: Figure 3 As shown, the image recognition model architecture includes an input part, a feature extraction module, and a recognition module. The input part fuses at least two sample images from different spatial domains to obtain a hybrid enhanced image, increasing the diversity of training samples. The at least two sample images from different spatial domains can be visible light images and thermal infrared images. The visible light images and thermal infrared images are fused to obtain the hybrid enhanced image, and the visible light images, thermal infrared images, and hybrid enhanced images are assigned the same label. Further, the feature extraction module includes a global feature extraction module and an attention feature extraction module. The global feature extraction module can be a feature extraction network such as IR-50, ResNet-50, or IR-152, used to extract global features from the visible light images, thermal infrared images, and hybrid enhanced images. The attention feature extraction module can be an attention feature extraction network such as Convolutional Attention Mechanism (CBAM), a transformer structure, or SENET, and can include a channel attention module and a spatial attention module, used to extract attention features from the visible light images, thermal infrared images, and hybrid enhanced images. Figure 3 The model provides three attention feature extraction modules. By inputting visible light images, thermal infrared images, or hybrid enhanced images into the image recognition model, it can obtain three attention features corresponding to the image, and then average these three attention features. The recognition module performs an inner product of the global features and the averaged attention features to achieve feature fusion. The fused features are then input into a fully connected layer, and the recognition result is output after passing through the fully connected layer.

[0080] The technical solution of this embodiment acquires at least two sample images from different spatial domains, indicating that the samples of this invention include sample images from different spatial domains. Further, the sample images from different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is assigned to both the sample image and the hybrid enhanced image. The sample image with the same label and the hybrid enhanced image are respectively input into the feature extraction module to be trained, and the image features of the sample image and the hybrid enhanced image are obtained respectively. A loss function is determined based on the image features of the sample image and the image features of the hybrid enhanced image. The parameters of the feature extraction module and the recognition module in the image recognition model are updated based on the loss function to obtain the target image recognition model. This training allows the image recognition model to learn the inherent relationship between sample images with the same label but from different spatial domains, thereby improving the image recognition accuracy of the image recognition model.

[0081] Example 3

[0082] Figure 4This is a flowchart of the training method for the image recognition model provided in Embodiment 3 of the present invention. Optionally, based on any optional technical solution in the embodiments of the present invention, the method further includes: acquiring a test image group, the test image group including a first test image and a second test image; inputting the first test image into the target image recognition model to obtain first image features; performing similarity matching between the first image features and each second image feature in the feature vector library corresponding to the second test image; determining whether the current recognition of the target image recognition model is correct based on the label corresponding to the second image feature with the highest similarity and the label corresponding to the first sample image; and determining the recognition accuracy of the target image recognition model based on the number of correctly recognized test images.

[0083] like Figure 4 As shown, the method in this embodiment may specifically include:

[0084] S310. Obtain an image sample group, wherein the image sample group includes at least two sample images from different spatial domains.

[0085] S320. The sample images of different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is set for the sample images and the hybrid enhanced image.

[0086] S330. Based on the image group including the sample image and the hybrid enhanced image corresponding to each label, the image recognition model is iteratively trained in sequence to obtain the target image recognition model.

[0087] S340. Obtain a test image group, wherein the test image group includes a first test image and a second test image.

[0088] S350. Input the first test image into the target image recognition model to obtain the first image feature.

[0089] S360. Perform similarity matching between the first image features and each second image feature in the feature vector library corresponding to the second test image.

[0090] S370. Based on the label corresponding to the second image feature with the highest similarity and the label corresponding to the first sample image, determine whether the current recognition of the target image recognition model is correct.

[0091] S380. Determine the recognition accuracy of the target image recognition model based on the number of correctly recognized test images.

[0092] In this embodiment, the test image group refers to the images used to test the target image recognition model, in order to test the recognition performance of the target image recognition model. The test image group includes, but is not limited to, a first test image and a second test image, wherein the spatial domains of the first test image and the second test image are different.

[0093] For example, one thousand pairs of test images are obtained, and the first and second test images from these pairs are sequentially input into the target image recognition model for testing. The first test image can be a thermal infrared image, and the second test image can be a visible light image. Each second test image is sequentially input into the target image recognition model to obtain multiple second image features, and the feature vector library is updated based on these features to ensure its real-time performance. Further, the first test image is input into the target image recognition model to obtain first image features, which are then matched with each second image feature in the feature vector library for similarity. If the label corresponding to the second image feature with the highest similarity is the same as the label corresponding to the first sample image, the first test image is correctly recognized. If the label corresponding to the second image feature with the highest similarity is different from the label corresponding to the first sample image, the first test image is incorrectly recognized. The recognition accuracy of the target image recognition model can be determined based on the number of correctly recognized first test images. For example, when there are one thousand first test images and nine hundred are correctly recognized, the recognition accuracy can be 90%.

[0094] The technical solution of this embodiment obtains a test image group, which includes a first test image and a second test image. The first test image is input into the target image recognition model to obtain a first image feature. Further, the first image feature is matched with the similarity of each second image feature in the feature vector library corresponding to the second test image. Then, the current recognition of the target image recognition model is determined according to the label corresponding to the second image feature with the highest similarity and the label corresponding to the first sample image. The recognition accuracy of the target image recognition model is determined according to the number of correctly recognized test images. This realizes the verification of the target image recognition model. If the recognition accuracy is low, the model can be trained or improved to ensure the recognition accuracy of the target image recognition model.

[0095] Example 4

[0096] Figure 5This is a flowchart of the image recognition method provided in Embodiment 4 of the present invention. Based on any optional technical solution in the embodiments of the present invention, this embodiment optionally includes an image recognition method comprising: acquiring an image to be processed; inputting the image to be processed into a target image recognition model to obtain an image recognition result, wherein the target image recognition model is trained by the training method of any image recognition model described in the embodiments of the present invention.

[0097] like Figure 5 As shown, the method in this embodiment may specifically include:

[0098] S410. Obtain an image sample group, wherein the image sample group includes at least two sample images from different spatial domains.

[0099] S420. Fuse the sample images from different spatial domains within the image sample group to obtain a hybrid enhanced image, and assign the same label to the sample images and the hybrid enhanced image.

[0100] S430. Based on the image group including the sample image and the hybrid enhanced image corresponding to each label, the image recognition model is iteratively trained in sequence to obtain the target image recognition model.

[0101] S440. Obtain the image to be processed, wherein the image to be processed is a group of images from different spatial domains.

[0102] S450. The image to be processed is input into the target image recognition model to obtain the image recognition result. The target image data model includes a feature extraction module, an information enhancement module, and a recognition module. The feature extraction module is used to extract global features and multi-scale attention features from the input image to be processed. The information enhancement module is used to fuse multiple attention features of different scales to obtain local enhancement features. The recognition module is used to obtain the image recognition result based on the global features and local enhancement features.

[0103] In this embodiment, when the images to be processed are a group of images from different spatial domains, the two images can be images from different spatial domains at the same time and location. The images to be processed can be acquired in real time by an acquisition device or from a preset storage location; there is no limitation on this. Further, the images to be processed are input into a target image recognition model, which outputs image recognition results. The image recognition results include, but are not limited to, the classification results of the recognized images, such as identifying label names, personal names, etc. The target image recognition model can be trained using any of the image recognition model training methods described in this embodiment of the invention.

[0104] Specifically, the training method for the image recognition model includes: acquiring an image sample group, wherein the image sample group includes at least two sample images from different spatial domains; fusing the sample images from different spatial domains within the image sample group to obtain a hybrid enhanced image, and assigning the same label to the sample images and the hybrid enhanced image; iteratively training the image recognition model based on the image group including the sample images and the hybrid enhanced image corresponding to each label, to obtain the target image recognition model. Further, after obtaining the trained target image recognition model, the image to be processed can be input into the target image recognition model. The target image data model may include, but is not limited to, a feature extraction module, an information enhancement module, and a recognition module. The feature extraction module can be used to extract global features and multi-scale attention features from the input image to be processed. The information enhancement module can be used to fuse multiple attention features at different scales to obtain local enhanced features, improving the reliability of the attention features. The recognition module is used to obtain the image recognition result based on the global features and local enhanced features, so that the obtained image recognition result is more accurate.

[0105] Based on the above embodiments, the feature extraction module includes a global feature extraction module and multiple attention feature extraction modules; wherein, the global feature extraction module includes multiple global feature extraction sub-modules connected in sequence, each global feature extraction sub-module is used to extract features from the input feature map, and output intermediate feature maps to the next global feature extraction sub-module respectively, until the terminal global feature extraction sub-module outputs global features; the attention feature extraction module is connected to the global feature extraction sub-module and is used to extract attention features from the intermediate feature maps.

[0106] In this embodiment of the invention, the intermediate feature maps corresponding to each global feature extraction submodule have different scales, and the network layers in which each global feature extraction submodule is located are different. This achieves the extraction of multi-level and multi-scale attention features, which can improve the reliability and richness of attention features, thereby improving the accuracy of image recognition.

[0107] The technical solution of this embodiment acquires the image to be processed, inputs the image to be processed into the target image recognition model, and obtains the image recognition result. This realizes the application of the target image recognition model, which can identify the category of the image to be processed online to distinguish the category of the image to be processed.

[0108] Example 5

[0109] Figure 6 This is a schematic diagram of the structure of the image recognition model training device provided in Embodiment 5 of the present invention. The image recognition model training device provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal and / or server to implement the image recognition model training method in the embodiments of the present invention. Specifically, the device may include:

[0110] The sample acquisition module 510 is used to acquire an image sample group, wherein the image sample group includes at least two sample images from different spatial domains; the label setting module 520 is used to fuse the sample images from different spatial domains within the image sample group to obtain a hybrid enhanced image, and to set the same label for the sample images and the hybrid enhanced image; the model training module 530 is used to iteratively train the image recognition model based on the image group including the sample images and the hybrid enhanced image corresponding to each label, to obtain the target image recognition model.

[0111] Based on any optional technical solution in the embodiments of the present invention, the label setting module 520 may also be used for:

[0112] Based on the image fusion ratio, the pixel values ​​of corresponding pixels in the sample images to be fused are weighted to obtain a hybrid enhanced image.

[0113] Based on any optional technical solution in the embodiments of the present invention, optionally, the image recognition model includes a feature extraction module and a recognition module; the model training module 530 includes:

[0114] The feature extraction unit is used to input sample images and hybrid enhanced images with the same label into the feature extraction module to be trained, respectively, to obtain the image features of the sample images and the hybrid enhanced images;

[0115] The loss determination unit is used to determine a loss function based on the image features of the sample image and the image features of the hybrid enhanced image, and to update the parameters of the feature extraction module and the recognition module in the image recognition model based on the loss function.

[0116] Based on any optional technical solution in the embodiments of the present invention, the attention features may optionally include attention features corresponding to a preset scale and / or a preset level, wherein the preset scale includes at least one facial region size, and the preset level includes at least one intermediate layer of a global feature extraction module.

[0117] Based on any optional technical solution in the embodiments of the present invention, the feature extraction module may optionally include a global feature extraction module and an attention feature extraction module, wherein the global feature extraction module is used to extract global features of the input image, and the attention feature extraction module is used to extract multiple attention features of the input image at different scales;

[0118] The loss determination unit can also be used for:

[0119] The loss function is determined based on the global features and attention features of the sample image and the global features of the hybrid enhanced image.

[0120] Based on any optional technical solution in the embodiments of the present invention, the image sample group may optionally include a first sample image and a second sample image in different spatial domains, and the first sample image and the second sample image are fused to obtain a hybrid enhanced image;

[0121] Accordingly, the loss function includes:

[0122] L total =α1L1 + α2L2 + α3L3

[0123] L1 = L V +L M +L A

[0124]

[0125]

[0126] Where L1 represents the first loss function, L2 represents the second loss function, L3 represents the third loss function, and α1, α2, and α3 represent the weights of the first, second, and third loss functions, respectively; L V L represents the loss function corresponding to the first sample image. M L represents the loss function corresponding to the second sample image. A This represents the loss function corresponding to the hybrid enhanced image. This represents the i-th attention feature of the first sample image. This represents the j-th attention feature of the first sample image; This represents the i-th attention feature of the second sample image. Let j represent the j-th attention feature of the second sample image.

[0127] Optionally, based on any of the optional technical solutions in the embodiments of the present invention, the device is further used for:

[0128] Acquire a test image group, which includes a first test image and a second test image;

[0129] The first test image is input into the target image recognition model to obtain the first image features;

[0130] The first image features are matched with the similarity of each second image feature in the feature vector library corresponding to the second test image.

[0131] The label corresponding to the second image feature with the highest similarity is used to determine whether the current recognition of the target image recognition model is correct, based on the label corresponding to the first sample image.

[0132] The accuracy of the target image recognition model is determined based on the number of correctly recognized test images.

[0133] The above-described apparatus can execute the training method of the image recognition model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the training method of the image recognition model.

[0134] Example 6

[0135] Figure 7 This is a schematic diagram of the image recognition device provided in Embodiment Six of the present invention. The image recognition device provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal and / or server to implement the image recognition method in the embodiments of the present invention. Specifically, the device may include:

[0136] The image acquisition module 610 is used to acquire an image to be processed, which is a group of images in different spatial domains; the recognition result determination module 620 is used to input the image to be processed into a target image recognition model to obtain an image recognition result. The target image data model includes a feature extraction module, an information enhancement module, and a recognition module. The feature extraction module is used to extract global features and multi-scale attention features from the input image to be processed. The information enhancement module is used to fuse multiple attention features of different scales to obtain local enhancement features. The recognition module is used to obtain an image recognition result based on the global features and local enhancement features.

[0137] Based on any optional technical solution in the embodiments of the present invention, optionally, the feature extraction module includes a global feature extraction module and multiple attention feature extraction modules; wherein, the global feature extraction module includes multiple global feature extraction sub-modules connected in sequence, each global feature extraction sub-module is used to extract features from the input feature map, and output intermediate feature maps to the next global feature extraction sub-module respectively, until the terminal global feature extraction sub-module outputs global features; the attention feature extraction module is connected to the global feature extraction sub-module and is used to extract attention features from the intermediate feature map.

[0138] The above-described device can execute the image recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the image recognition method.

[0139] Example 7

[0140] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. Figure 8 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 8The electronic device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0141] like Figure 8 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0142] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0143] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0144] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 8 Not shown; usually referred to as a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0145] A program / utility 36 having a set (at least one) of program modules 26 may be stored, for example, in system memory 28. Such program modules 26 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 26 typically perform the functions and / or methods described in the embodiments of the present invention.

[0146] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the electronic device 12, and / or with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 8 As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 8 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0147] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a training method for an image recognition model provided in an embodiment of the present invention.

[0148] Example 8

[0149] Embodiment 8 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a training method for an image recognition model, the method comprising:

[0150] Obtain an image sample group, wherein the image sample group includes at least two sample images from different spatial domains;

[0151] The sample images from different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is assigned to the sample images and the hybrid enhanced image.

[0152] Based on the image group corresponding to each label, including the sample image and the hybrid enhanced image, the image recognition model is iteratively trained in sequence to obtain the target image recognition model.

[0153] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0154] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0155] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0156] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A training method for an image recognition model, characterized in that, include: Obtain an image sample group, wherein the image sample group includes at least two sample images from different spatial domains; The sample images from different spatial domains within the image sample group are fused to obtain a hybrid enhanced image, and the same label is assigned to the sample images and the hybrid enhanced image. Based on the image group corresponding to each label, including the sample image and the hybrid enhanced image, the image recognition model is iteratively trained in sequence to obtain the target image recognition model; The image recognition model includes a feature extraction module and a recognition module; The image recognition model is iteratively trained sequentially based on the image groups, including the sample images and the hybrid enhanced images corresponding to each label, including: The sample image and the hybrid enhanced image with the same label are respectively input into the feature extraction module to be trained to obtain the image features of the sample image and the hybrid enhanced image respectively; A loss function is determined based on the image features of the sample image and the image features of the hybrid enhanced image, and the parameters of the feature extraction module and the recognition module in the image recognition model are updated based on the loss function; The feature extraction module includes a global feature extraction module and an attention feature extraction module. The global feature extraction module is used to extract global features of the input image, and the attention feature extraction module is used to extract multiple attention features of the input image at different scales. The determination of the loss function based on the image features of the sample image and the image features of the hybrid enhanced image includes: The loss function is determined based on the global features and attention features of the sample image and the global features of the hybrid enhanced image; The image sample group includes first sample images and second sample images from different spatial domains. The first sample images and the second sample images are fused to obtain a hybrid enhanced image. Accordingly, the loss function is: L total =α1L1+α2L2+α3L3; Where, L1=L V +L M +L A ; Where L1 represents the first loss function, L2 represents the second loss function, L3 represents the third loss function, and α1, α2, and α3 represent the weights of the first, second, and third loss functions, respectively; L V L represents the loss function corresponding to the first sample image. M L represents the loss function corresponding to the second sample image. A This represents the loss function corresponding to the hybrid enhanced image. This represents the i-th attention feature of the first sample image. This represents the j-th attention feature of the first sample image; This represents the i-th attention feature of the second sample image. Let j represent the j-th attention feature of the second sample image.

2. The method according to claim 1, characterized in that, The step of fusing sample images from different spatial domains within the image sample group to obtain a hybrid enhanced image includes: Based on the image fusion ratio, the pixel values ​​of corresponding pixels in the sample images to be fused are weighted to obtain a hybrid enhanced image.

3. The method according to claim 1, characterized in that, The method further includes: Acquire a test image group, which includes a first test image and a second test image; The first test image is input into the target image recognition model to obtain the first image features; The first image features are matched with the similarity of each second image feature in the feature vector library corresponding to the second test image. The label corresponding to the second image feature with the highest similarity is used to determine whether the current recognition of the target image recognition model is correct, based on the label corresponding to the first sample image. The accuracy of the target image recognition model is determined based on the number of correctly recognized test images.

4. A training device for an image recognition model, characterized in that, include: A sample acquisition module is used to acquire an image sample group, wherein the image sample group includes at least two sample images from different spatial domains; The label setting module is used to fuse sample images from different spatial domains within the image sample group to obtain a hybrid enhanced image, and to set the same label for the sample images and the hybrid enhanced image; The model training module is used to iteratively train the image recognition model based on the image group including the sample image and the hybrid enhanced image corresponding to each label, so as to obtain the target image recognition model. The image recognition model includes a feature extraction module and a recognition module; the model training module includes: The feature extraction unit is used to input sample images and hybrid enhanced images with the same label into the feature extraction module to be trained, respectively, to obtain the image features of the sample images and the hybrid enhanced images; The loss determination unit is used to determine a loss function based on the image features of the sample image and the image features of the hybrid enhanced image, and to update the parameters of the feature extraction module and the recognition module in the image recognition model based on the loss function; The feature extraction module includes a global feature extraction module and an attention feature extraction module. The global feature extraction module is used to extract global features of the input image, and the attention feature extraction module is used to extract multiple attention features of the input image at different scales. The loss determination unit is also used for: The loss function is determined based on the global features and attention features of the sample image and the global features of the hybrid enhanced image; The image sample group includes first sample images and second sample images from different spatial domains. The first sample images and the second sample images are fused to obtain a hybrid enhanced image. Accordingly, the loss function is: L total =α1L1+α2L2+α3L3; Where, L1=L V +L M +L A ; Where L1 represents the first loss function, L2 represents the second loss function, L3 represents the third loss function, and α1, α2, and α3 represent the weights of the first, second, and third loss functions, respectively; L V L represents the loss function corresponding to the first sample image. M L represents the loss function corresponding to the second sample image. A This represents the loss function corresponding to the hybrid enhanced image. This represents the i-th attention feature of the first sample image. This represents the j-th attention feature of the first sample image; This represents the i-th attention feature of the second sample image. Let j represent the j-th attention feature of the second sample image.

5. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the training method for the image recognition model as described in any one of claims 1-3.

6. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the training method of the image recognition model as described in any one of claims 1-3.

Images