Model training method, image recognition method, medium, device and computing device

Abstract

Embodiments of the present disclosure provide a model training method, an image recognition method, a medium, an apparatus and a computing device, relating to the technical field of artificial intelligence, and the model training method comprises: obtaining a plurality of target feature maps corresponding to a sample image; inputting the plurality of target feature maps into a first sub-model for convolution processing to obtain a classification feature map, a regression feature map and an object feature map corresponding to each target feature map; inputting the classification feature map into a second sub-model for feature decorrelation processing to obtain a target sample weight; determining a target loss value based on the target sample weight, the classification feature map, the regression feature map, the object feature map and label information of the sample image; and adjusting parameters of an image recognition model according to the target loss value to obtain a trained image recognition model. The present disclosure can greatly improve the generalization ability of the image recognition model.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of artificial intelligence technology, and more specifically, the embodiments of this disclosure relate to a model training method, an image recognition method, a medium, an apparatus, and a computing device. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Faced with the massive amount of images on the Internet, it is necessary to select and identify images containing information about uncivilized or illegal behavior, i.e., prohibited images, in order to purify the online environment and ensure that users receive safe information while enjoying the convenience brought by the Internet.

[0004] Currently, pre-trained image recognition models are typically used to identify whether an input image is prohibited. The training of these models relies on the assumption that training and test images are independently and identically distributed. However, in real-world scenarios, the distribution of test images often differs from that of training images. This necessitates iterative processing during training, involving "data mining and filtering, data labeling, model training, and model testing," to continuously approximate the distribution of the training images to the distribution of the test images, resulting in a well-trained image recognition model. However, the generalization ability of the resulting image recognition model is often poor. Summary of the Invention

[0005] This disclosure provides a model training method, an image recognition method, a medium, an apparatus, and a computing device to address the problem of poor generalization ability of image recognition models obtained through current technologies.

[0006] In a first aspect of this disclosure, a model training method is provided for training an image recognition model, the image recognition model including a first sub-model and a second sub-model, the model training method comprising:

[0007] Obtain multiple target feature maps corresponding to the sample image, and the multiple target feature maps have different sizes;

[0008] Multiple target feature maps are input into the first sub-model for convolution processing to obtain the classification feature map, regression feature map and object feature map corresponding to each target feature map output by the first sub-model;

[0009] The classification feature map is input into the second sub-model for feature decorrelation processing to obtain the target sample weights output by the second sub-model. The target sample weights are used to characterize the weight of each classification feature in the classification feature map.

[0010] The target loss value is determined based on the target sample weights, classification feature maps, regression feature maps, object feature maps, and the annotation information of the sample images.

[0011] Adjust the parameters of the image recognition model based on the target loss value to obtain the trained image recognition model.

[0012] In one possible implementation, the classification feature map is input into the second sub-model for feature decorrelation processing to obtain the target sample weights output by the second sub-model. This includes: inputting multiple classification feature maps into the second sub-model to obtain a target classification feature map, which is obtained by concatenating multiple classification feature maps by the second sub-model; extracting target classification features based on the target classification feature map and the target classification feature map obtained from the previous training of the image recognition model; performing random Fourier transform processing on the target classification features to obtain random Fourier features; and iteratively training the second sub-model based on the random Fourier features to obtain the target sample weights.

[0013] In one possible implementation, the second sub-model is iteratively trained based on random Fourier features to perform feature decorrelation processing to obtain target sample weights. This includes iteratively training the second sub-model based on random Fourier features, the first sample weights obtained from the previous training of the second sub-model, and the second sample weights obtained from the previous training of the image recognition model to obtain target sample weights. In the first training of the second sub-model, the preset initial sample weights are used as the first sample weights.

[0014] In one possible implementation, determining the target loss value based on target sample weights, classification feature maps, regression feature maps, object feature maps, and annotation information of sample images includes: performing category detection on multiple classification feature maps using a first sub-model to obtain a first detection result; determining an initial classification loss value based on the first detection result and annotation information; determining a first loss value based on the initial classification loss value and target sample weights; performing position detection on multiple regression feature maps using the first sub-model to obtain a second detection result; determining a second loss value based on the second detection result and annotation information; performing object detection on multiple object feature maps using the first sub-model to obtain a third detection result; determining a third loss value based on the third detection result and annotation information; and determining the target loss value based on the first loss value, the second loss value, and the third loss value.

[0015] In one possible implementation, determining a first loss value based on an initial classification loss value and target sample weights includes: performing a weighted summation of the initial classification loss value and target sample weights to determine the first loss value.

[0016] In one possible implementation, the initial classification loss value and the target sample weights are weighted and summed to determine the first loss value, including: determining the first loss value according to the following formula:

[0017]

[0018] in, This represents the initial classification loss value; Indicates the target sample weight; Represents a sample image; Annotation information representing sample images; express The corresponding random Fourier features; B represents the first detection result output by the first sub-model; B represents the number of sample images.

[0019] In one possible implementation, the first sub-model includes a first convolutional layer and a second and a third convolutional layer connected to the first convolutional layer, respectively. A second sub-model is connected after the second convolutional layer. Multiple target feature maps are input into the first sub-model for convolutional processing to obtain a classification feature map, a regression feature map, and an object feature map output by the first sub-model corresponding to each target feature map. This includes: inputting the target feature maps into the first convolutional layer for convolutional processing to obtain the convolutional processing result output by the first convolutional layer, where the first convolutional layer contains one convolutional layer; inputting the convolutional processing result into the second and third convolutional layers for convolutional processing to obtain the classification feature map output by the second convolutional layer and the regression feature map and object feature map output by the third convolutional layer, where the second convolutional layer contains multiple cascaded convolutional layers, and the third convolutional layer contains multiple cascaded convolutional layers.

[0020] In one possible implementation, adjusting the parameters of the image recognition model according to the target loss value to obtain a trained image recognition model includes: adjusting the parameters of the image recognition model according to the target loss value and iteratively training the image recognition model until a preset number of iterations is reached.

[0021] In one possible implementation, after reaching a preset number of iterations, the model training method further includes: performing a moving average on the initial parameters corresponding to the current iteration based on the preset number of iterations, the current number of iterations, the initial parameters corresponding to the current number of iterations, and the parameters obtained from the previous training of the image recognition model, to obtain the target parameters corresponding to the current number of iterations.

[0022] The image recognition model is iteratively trained based on the target parameters corresponding to the current iteration number to obtain the trained image recognition model.

[0023] In one possible implementation, the sample image includes a first image and a second image. The first image is an image acquired in a real scene, and the second image is an image generated using a preset generation method, which includes at least one of adding background, adding noise, and combining image elements.

[0024] In one possible implementation, before acquiring multiple target feature maps corresponding to the sample image, the model training method further includes: enhancing the sample image, wherein the enhancement process includes at least one of flipping, resizing, cropping, brightness adjustment, contrast adjustment, and adding noise.

[0025] In one possible implementation, the image recognition model further includes a backbone network model and a feature pyramid network model, with the feature pyramid network model connected to the backbone network model and a first sub-model connected to the feature pyramid network model. The process of obtaining multiple target feature maps corresponding to a sample image includes: inputting the sample image into the backbone network model for feature extraction to obtain multiple initial feature maps corresponding to the sample image, the multiple initial feature maps having different sizes; and inputting the multiple initial feature maps into the feature pyramid network model for feature fusion processing to obtain multiple target feature maps corresponding to the sample image.

[0026] Secondly, embodiments of this disclosure provide an image recognition method, including:

[0027] Obtain multiple target feature maps corresponding to the image to be identified, and the multiple target feature maps have different sizes;

[0028] Multiple target feature maps are input into the first sub-model of the image recognition model for recognition processing to obtain the image recognition result output by the image recognition model. The image recognition result is used to indicate whether the image to be recognized is a prohibited image. The image recognition model is trained using the model training method described in the first aspect of this disclosure.

[0029] In one possible implementation, obtaining multiple target feature maps corresponding to the image to be identified includes: obtaining the image to be identified; preprocessing the image to be identified to obtain a preprocessed image, the preprocessing including image normalization and / or image scaling; and obtaining multiple target feature maps based on the preprocessed image.

[0030] In one possible implementation, multiple target feature maps are obtained based on the preprocessed image, including: inputting the preprocessed image into the backbone network model of the image recognition model for feature extraction to obtain multiple initial feature maps corresponding to the preprocessed image, wherein the multiple initial feature maps have different sizes; inputting the multiple initial feature maps into the feature pyramid network model of the image recognition model for feature fusion processing to obtain multiple target feature maps.

[0031] In one possible implementation, multiple target feature maps are input into the first sub-model of the image recognition model for recognition processing to obtain the image recognition result output by the image recognition model. This includes: inputting multiple target feature maps into the first sub-model of the image recognition model for recognition processing to obtain the score of the target object contained in the image to be recognized; if the score is greater than a threshold, the image recognition result is that the image to be recognized is a prohibited image; if the score is less than or equal to the threshold, the image recognition result is that the image to be recognized is not a prohibited image.

[0032] In one possible implementation, the image recognition result also includes the location information of the target object in the image to be recognized, which is used to determine the location of the target prohibited object in the prohibited image.

[0033] Thirdly, embodiments of this disclosure provide a model training apparatus for training an image recognition model, the image recognition model including a first sub-model and a second sub-model, the model training apparatus comprising:

[0034] The acquisition module is used to acquire multiple target feature maps corresponding to the sample image, and the multiple target feature maps are of different sizes;

[0035] The first processing module is used to input multiple target feature maps into the first sub-model for convolution processing, and obtain the classification feature map, regression feature map and object feature map corresponding to each target feature map output by the first sub-model;

[0036] The second processing module is used to input the classification feature map into the second sub-model for feature decorrelation processing, and obtain the target sample weights output by the second sub-model. The target sample weights are used to characterize the weight of each classification feature in the classification feature map.

[0037] The determination module is used to determine the target loss value based on the target sample weights, classification feature maps, regression feature maps, object feature maps, and annotation information of the sample images.

[0038] The third processing module is used to adjust the parameters of the image recognition model according to the target loss value in order to obtain the trained image recognition model.

[0039] In one possible implementation, the second processing module is specifically used for: inputting multiple classification feature maps into a second sub-model to obtain a target classification feature map, wherein the target classification feature map is obtained by the second sub-model concatenating multiple classification feature maps; extracting target classification features based on the target classification feature map and the target classification feature map obtained from the previous training of the image recognition model; performing random Fourier transform processing on the target classification features to obtain random Fourier features; and iteratively training the second sub-model based on the random Fourier features to perform feature decorrelation processing to obtain target sample weights.

[0040] In one possible implementation, when the second processing module is used for iterative training of the second sub-model based on random Fourier features to perform feature decorrelation processing and obtain target sample weights, it is specifically used for: iterative training of the second sub-model based on random Fourier features, the first sample weights obtained from the previous training of the second sub-model, and the second sample weights obtained from the previous training of the image recognition model to obtain target sample weights, wherein, in the first training of the second sub-model, the preset initial sample weights are used as the first sample weights.

[0041] In one possible implementation, the determining module is specifically used for: performing category detection on multiple classification feature maps using a first sub-model to obtain a first detection result; determining an initial classification loss value based on the first detection result and annotation information; determining a first loss value based on the initial classification loss value and the target sample weight; performing position detection on multiple regression feature maps using the first sub-model to obtain a second detection result; determining a second loss value based on the second detection result and annotation information; performing object detection on multiple object feature maps using the first sub-model to obtain a third detection result; determining a third loss value based on the third detection result and annotation information; and determining a target loss value based on the first loss value, the second loss value, and the third loss value.

[0042] In one possible implementation, when the determining module determines the first loss value based on the initial classification loss value and the target sample weights, it specifically performs a weighted summation of the initial classification loss value and the target sample weights to determine the first loss value.

[0043] In one possible implementation, when determining the first loss value by performing a weighted summation of the initial classification loss value and the target sample weights, the determining module specifically performs the following steps: Determine the first loss value according to the following formula:

[0044]

[0045] in, This represents the initial classification loss value; Indicates the target sample weight; Represents a sample image; Annotation information representing sample images; express The corresponding random Fourier features; B represents the first detection result output by the first sub-model; B represents the number of sample images.

[0046] In one possible implementation, the first sub-model includes a first convolutional layer and a second and a third convolutional layer connected to the first convolutional layer, respectively. The second convolutional layer is followed by a second sub-model. The first processing module is specifically used for: inputting the target feature map into the first convolutional layer for convolutional processing to obtain the convolutional processing result output by the first convolutional layer, wherein the first convolutional layer contains one convolutional layer; inputting the convolutional processing result into the second and third convolutional layers for convolutional processing to obtain the classification feature map output by the second convolutional layer and the regression feature map and object feature map output by the third convolutional layer, wherein the second convolutional layer contains multiple concatenated convolutional layers and the third convolutional layer contains multiple concatenated convolutional layers.

[0047] In one possible implementation, the third processing module is specifically used to: adjust the parameters of the image recognition model according to the target loss value, and iteratively train the image recognition model until a preset number of iterations is reached.

[0048] In one possible implementation, the third processing module is further configured to: after reaching a preset number of iterations, perform a moving average on the initial parameters corresponding to the current iteration number based on the preset number of iterations, the current number of iterations, the initial parameters corresponding to the current number of iterations, and the parameters obtained from the previous training of the image recognition model, to obtain the target parameters corresponding to the current number of iterations; and perform iterative training on the image recognition model based on the target parameters corresponding to the current number of iterations to obtain the trained image recognition model.

[0049] In one possible implementation, the sample image includes a first image and a second image. The first image is an image acquired in a real scene, and the second image is an image generated using a preset generation method, which includes at least one of adding background, adding noise, and combining image elements.

[0050] In one possible implementation, the acquisition module is further configured to: before acquiring multiple target feature maps corresponding to the sample image, perform enhancement processing on the sample image, the enhancement processing including at least one of flipping, resizing, cropping, brightness adjustment, contrast adjustment, and adding noise.

[0051] In one possible implementation, the image recognition model further includes a backbone network model and a feature pyramid network model. The feature pyramid network model is connected to the backbone network model, and the first sub-model is connected to the feature pyramid network model. The acquisition module is specifically used to: input the sample image into the backbone network model for feature extraction to obtain multiple initial feature maps corresponding to the sample image, the multiple initial feature maps having different sizes; input the multiple initial feature maps into the feature pyramid network model for feature fusion processing to obtain multiple target feature maps corresponding to the sample image.

[0052] Fourthly, embodiments of this disclosure provide an image recognition device, comprising:

[0053] The acquisition module is used to acquire multiple target feature maps corresponding to the image to be identified. The multiple target feature maps have different sizes.

[0054] The processing module is used to input multiple target feature maps into the first sub-model of the image recognition model for recognition processing, and obtain the image recognition result output by the image recognition model. The image recognition result is used to indicate whether the image to be recognized is a prohibited image. The image recognition model is trained using the model training method as described in the first aspect of this disclosure.

[0055] In one possible implementation, the acquisition module is specifically used for: acquiring an image to be identified; preprocessing the image to be identified to obtain a preprocessed image, the preprocessing including image normalization and / or image scaling; and acquiring multiple target feature maps based on the preprocessed image.

[0056] In one possible implementation, when the acquisition module acquires multiple target feature maps based on the preprocessed image, it specifically performs the following steps: inputting the preprocessed image into the backbone network model of the image recognition model for feature extraction to obtain multiple initial feature maps corresponding to the preprocessed image, wherein the multiple initial feature maps have different sizes; inputting the multiple initial feature maps into the feature pyramid network model of the image recognition model for feature fusion processing to obtain multiple target feature maps.

[0057] In one possible implementation, the processing module is specifically used to: input multiple target feature maps into the first sub-model of the image recognition model for recognition processing, and obtain the score of the target object contained in the image to be recognized; if the score is greater than the threshold, the image recognition result is that the image to be recognized is a prohibited image; if the score is less than or equal to the threshold, the image recognition result is that the image to be recognized is not a prohibited image.

[0058] In one possible implementation, the image recognition result also includes the location information of the target object in the image to be recognized, which is used to determine the location of the target prohibited object in the prohibited image.

[0059] Fifthly, embodiments of this disclosure provide a computing device, including: a processor, and a memory communicatively connected to the processor;

[0060] The memory stores the instructions that the computer executes;

[0061] The processor executes computer execution instructions stored in memory to implement the model training method as described in the first aspect of this disclosure or the image recognition method as described in the second aspect.

[0062] In a sixth aspect, embodiments of this disclosure provide a storage medium storing computer program instructions, which, when executed, implement the model training method as described in the first aspect or the image recognition method as described in the second aspect of this disclosure.

[0063] In a seventh aspect, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the model training method as described in the first aspect or the image recognition method as described in the second aspect of this disclosure.

[0064] The model training method, image recognition method, medium, apparatus, and computing device provided in this disclosure are used to train an image recognition model. The image recognition model includes a first sub-model and a second sub-model. Multiple target feature maps of different sizes are acquired corresponding to sample images. The multiple target feature maps are input into the first sub-model for convolution processing to obtain a classification feature map, a regression feature map, and an object feature map output by the first sub-model corresponding to each target feature map. The classification feature map is input into the second sub-model for feature decorrelation processing to obtain the target sample weights output by the second sub-model. The target sample weights are used to characterize the weight of each classification feature in the classification feature map. Based on the target sample weights, classification feature maps, regression feature maps, object feature maps, and annotation information of the sample images, a target loss value is determined. The parameters of the image recognition model are adjusted according to the target loss value to obtain a trained image recognition model. Since this disclosure inputs the classification feature map corresponding to the target feature map output by the first sub-model into the second sub-model for feature decorrelation processing to obtain the target sample weight output by the second sub-model, the target sample weight makes different classification features independent of each other, removes false associations between different classification features, and thus enables the image recognition model to better focus on the essential features related to the recognition result. Therefore, it can greatly improve the generalization ability of the image recognition model. Attached Figure Description

[0065] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0066] Figure 1 A schematic diagram illustrating the distribution of training and test samples provided for a related technology;

[0067] Figure 2 A schematic diagram illustrating the distribution of training and test samples provided for another related technology;

[0068] Figure 3 A schematic diagram of target detection results provided by a related technology;

[0069] Figure 4 This is a schematic diagram illustrating an application scenario of the model training method provided in an embodiment of the present disclosure.

[0070] Figure 5 A flowchart of a model training method provided in an embodiment of this disclosure;

[0071] Figure 6 This is a schematic diagram of the structure of an image recognition model provided in an embodiment of the present disclosure;

[0072] Figure 7 A flowchart of a model training method provided in another embodiment of this disclosure;

[0073] Figure 8 A flowchart of an image recognition method provided in an embodiment of this disclosure;

[0074] Figure 9 This is a schematic diagram illustrating image recognition using an image recognition model, as provided in an embodiment of this disclosure.

[0075] Figure 10 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;

[0076] Figure 11 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of the present disclosure;

[0077] Figure 12 A schematic diagram of a storage medium provided in an embodiment of this disclosure;

[0078] Figure 13 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure.

[0079] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0080] The principles and spirit of this disclosure will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0081] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0082] According to embodiments of this disclosure, a model training method, an image recognition method, a medium, an apparatus, and a computing device are proposed.

[0083] In this article, it is important to understand the following terms:

[0084] Prohibited image recognition refers to the technology that uses computers to process, analyze, and understand images in order to select and identify images containing information such as uncivilized or illegal behavior from massive amounts of internet images. Prohibited image recognition can purify the internet and ensure that the information received by the public is safe while enjoying the convenience brought by the internet.

[0085] Supervised learning is a type of machine learning task. Assuming X represents a training sample and Y represents the label of the training sample, labeled training data (X, Y) means that each training sample includes both input and expected output. Supervised learning derives an optimal prediction function from labeled training data.

[0086] In probability and statistics theory, independent and identically distributed (IID) means that in a random process, the values ​​taken at any given time are random variables. If these random variables follow the same distribution and are independent of each other, then these random variables are independently and identically distributed. In traditional algorithms, IID is usually assumed that the training samples X and the test samples Z both follow the same distribution and are independently and identically distributed, i.e., P... tr (X, Z) = P te (X, Z), where P tr (X, Z) represents the distribution of the training samples, P te (X, Z) represents the distribution of the test samples. For example, Figure 1 A schematic diagram illustrating the distribution of training and test samples provided for a related technology, such as... Figure 1 As shown, the distribution of training and test samples obtained by the independent and identically distributed algorithm is illustrated. Both training and test samples follow the same distribution and are independent and identically distributed.

[0087] Out-of-distribution generalization (OOD generalization) refers to the task of generalizing an image recognition model to scenarios with varying distributions, i.e., generalizing the distribution of an image to a different value in the P-value range. tr (X, Z) ≠ P te In the case of (X, Z), it can be assumed that under supervised learning, there is a deviation between the distribution of the test samples and the distribution of the training samples, and this deviation is unknown during training. For example, Figure 2 A schematic diagram illustrating the distribution of training and test samples provided for another related technology, such as... Figure 2 As shown, the distribution of training and test samples obtained by the out-of-distribution generalization algorithm is illustrated. The training and test samples do not follow the same distribution.

[0088] Object detection is used to find objects of interest in images or videos, and simultaneously detect their location and size. Unlike image classification tasks, object detection not only solves the classification problem but also the localization problem, making it a multi-task problem. For example, Figure 3 A schematic diagram of target detection results provided by a related technology, such as... Figure 3 As shown, object detection can obtain information about each object (such as a dog) in the image and the location of each object (such as the coordinates of the rectangle where the dog is located).

[0089] Furthermore, the number of any elements in the accompanying drawings is for illustrative purposes only and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0090] In addition, the data involved in this disclosure may be data authorized by the user or fully authorized by all parties. The collection, dissemination and use of the data shall comply with the requirements of relevant national laws and regulations. The implementation methods / executives of this disclosure may be combined with each other.

[0091] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments. Invention Overview

[0093] The inventors have discovered that using a trained image recognition model to identify whether an input image is prohibited can ensure the safety of the information received by the public while enjoying the convenience of the internet, making it an important identification technology for purifying the internet. In related technologies, identifying whether an input image is prohibited using a trained image recognition model typically relies on the assumption that the training and test images are independently and identically distributed. However, in real-world scenarios, the distribution of test images often differs from that of training images. Therefore, iterative processing during training—including data mining and filtering, data labeling, model training, and model testing—is necessary to continuously approximate the distribution of the test images, thereby improving the recognition accuracy and generalization ability of the image recognition model, resulting in a well-trained image recognition model. However, the above-mentioned related technologies have the following problems: (1) High cost; In real scenarios, the data distribution usually changes over time, and it is very expensive to keep tracking the distribution changes of the training data. In addition, the training data required for supervised training is large, which leads to high labeling costs, further increasing the cost; (2) Small amount of prohibited image data, difficult to obtain, and small range of known domains (different domains represent different data distributions); In the massive image data of the Internet, due to the extremely low proportion of prohibited images, it is not only relatively difficult to obtain a large number of images suitable for related prohibited recognition tasks, but it is also more difficult to obtain prohibited images of different domains, which further leads to a narrow range of known domains in the training data; (3) Distribution shift is unavoidable and the shift is highly random; In real scenarios, the data distribution usually changes over time. The variation in data distribution undermines the assumption of independent and identically distributed (ISD) data. Classical supervised learning methods typically optimize by minimizing training errors, which greedily absorb all correlations found in the data for prediction. While effective in ISD settings, this approach degrades performance when data distribution changes, as correlations between data points do not remain constant across the unseen test distribution. When strong distribution shifts are involved, image recognition models that only consider training errors fail significantly, sometimes even worse than random guessing. Furthermore, the constantly changing label data distribution, unknowable during model training and highly random, leads to varying degrees of performance degradation over time. Therefore, image recognition models obtained using these techniques exhibit poor generalization ability.

[0094] To address the aforementioned problems, this disclosure provides a model training method, image recognition method, medium, apparatus, and computing device. By enhancing and transforming sample images, the sample images are enhanced to generate sample images with invisible patterns outside the original distribution, thereby improving the generalization ability of the image recognition model. During the image recognition model training process, the first and second sub-models included in the image recognition model are used to separate target features from background information, extract essential features of different categories, remove irrelevant features and spurious associations, and improve the generalization ability of the image recognition model in unknown domains. Furthermore, appropriate training strategies are employed during the image recognition model training process to further enhance its generalization ability. Therefore, an image recognition model with better generalization ability can be obtained, leading to more accurate image recognition results when the image recognition model is used for image recognition.

[0095] Application Scenarios Overview

[0096] First refer to Figure 4 Examples of application scenarios for the solutions provided in this disclosure are given. Figure 4 This is a schematic diagram illustrating an application scenario of the model training method provided in an embodiment of this disclosure, such as... Figure 4 As shown, this application scenario can include a server cluster 41 and a terminal 42. The server cluster 41 includes multiple servers 411 and a storage device 412. The terminal 42 can be a tablet computer, laptop computer, desktop computer, or smart home appliance, etc. The servers 411 are used to train the image recognition model. During training, they retrieve data from the storage device 412 and store the generated data in the storage device 412. Furthermore, during training, communication between the server 411 and the terminal 42 occurs via a wireless network or wired network.

[0097] Furthermore, the embodiments disclosed herein can be applied to image recognition scenarios. For example, during content review, it can be used to determine whether the included images are prohibited.

[0098] It should be noted that, Figure 4 This is merely a schematic diagram illustrating one application scenario provided by an embodiment of this disclosure; this embodiment does not necessarily represent an application scenario. Figure 4 The included equipment is not limited, nor is it restricted. Figure 4 The positional relationships between the devices are defined. The model training method provided in this embodiment can be applied to a server, which can be a standalone server or a service cluster, etc.

[0099] Exemplary methods

[0100] The following is combined Figure 4 Application scenarios, refer to Figure 5This document describes a model training method according to exemplary embodiments of the present disclosure. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way. Rather, the embodiments of the present disclosure can be applied to any applicable scenario.

[0101] First, the model training method will be introduced through specific examples.

[0102] Figure 5 This is a flowchart illustrating a model training method provided in one embodiment of the present disclosure, used to train an image recognition model, which includes a first sub-model and a second sub-model. Figure 5 As shown, the method in this embodiment includes:

[0103] S501. Obtain multiple target feature maps corresponding to the sample image. The multiple target feature maps have different sizes.

[0104] In this embodiment of the disclosure, the sample image is, for example, a prohibited image. Before using the sample image to train the image recognition model, to ensure a balance between speed and accuracy, the sample image can be preprocessed to obtain a preprocessed sample image. This preprocessing may include sample image normalization and / or sample image scaling, thereby obtaining multiple target feature maps corresponding to the sample image based on the preprocessed sample image. When scaling the sample image, the specific scaling size can be determined according to the actual task requirements; for example, the default scaling size is typically 320 pixels × 320 pixels.

[0105] Optionally, the sample image includes a first image and a second image. The first image is an image acquired in a real scene, and the second image is an image generated using a preset generation method, which includes at least one of adding background, adding noise, and combining image elements.

[0106] The first image is, for example, a prohibited image captured in a real-world scene. Exemplarily, as described in the aforementioned techniques, image recognition models perform well when both training and test images are independently and identically distributed. However, when the image recognition model encounters images from domains it has never seen before (i.e., not included in the training images), its recognition performance deteriorates. Considering that the recognition performance of image recognition models decreases on unfamiliar images, it is necessary to acquire as many images from different domains as possible during training to improve the generalization ability of the image recognition model. However, in practical applications, especially in prohibited scenarios, acquiring images from more domains is difficult. Therefore, from the perspective of image generation, images from different domains can be generated to expand the diversity of sample images and improve the generalization ability of the image recognition model. Specifically, for example, sample images from different domains can be generated through random backgrounds, random noise, and random combinations of image elements, i.e., generating a second image. This allows the image recognition model to pay more attention to the essential feature information of the target and avoid noise information, thereby improving the generalization ability of the image recognition model.

[0107] As one embodiment, the specific process of generating the second image is as follows: a background image library B is preset. g Target object library O obj and the unrelated object library N obj Target object library O obj It contains M target objects and an unrelated object library N. obj It contains Q irrelevant objects; assuming the number of target composite images (i.e., the second image) is denoted by num, and the background image library B is randomly selected. g One of the images B gk From the target object library O obj From a pool of M target objects, m target objects are randomly selected, and then selected from a pool of N irrelevant objects. obj Randomly select q irrelevant objects from the included Q irrelevant objects, and in B gk Randomly place m selected target objects and q irrelevant objects on the target surface, and add random noise to generate a target image. It outputs various information about the target object, including its location (coordinates) and category. Similarly, a set I of num second images can be obtained. m I m Any of the second images contained therein can be used This indicates that k ranges from 1 to num.

[0108] Optionally, before acquiring multiple target feature maps corresponding to the sample image, the model training method provided in this embodiment may further include: enhancing the sample image, wherein the enhancement processing includes at least one of flipping, resizing, cropping, brightness adjustment, contrast adjustment, and adding noise.

[0109] It is understandable that after obtaining the sample images, which include the first image and the second image, the sample images can be enhanced to generate sample images with invisible styles outside the original distribution (i.e., not included in the styles corresponding to the obtained sample images), thereby improving the generalization ability of the image recognition model. Specific enhancement functions include random flipping, random resizing of the sample images, and random cropping, while specific enhancement operations include random brightness, random contrast, and random Gaussian noise.

[0110] Optionally, the image recognition model also includes a backbone network model and a feature pyramid network model. The feature pyramid network model is connected to the backbone network model, and the first sub-model is connected to the feature pyramid network model. Obtaining multiple target feature maps corresponding to the sample image may include: inputting the sample image into the backbone network model for feature extraction to obtain multiple initial feature maps corresponding to the sample image, the multiple initial feature maps having different sizes; inputting the multiple initial feature maps into the feature pyramid network model for feature fusion processing to obtain multiple target feature maps corresponding to the sample image.

[0111] For example, Figure 6 This is a schematic diagram of the structure of an image recognition model provided in an embodiment of the present disclosure, as shown below. Figure 6As shown, the image recognition model includes a backbone network model, a feature pyramid network model, a first sub-model, and a second sub-model. The backbone network model, also known as the main network model, contains multiple cascaded convolutional layers and is primarily used for feature extraction, yielding feature maps output by each convolutional layer. The feature pyramid network model, also containing multiple cascaded convolutional layers, mainly fuses the feature maps output by different convolutional layers in the backbone network model, enhancing the network's expressive power. Feature maps of different sizes can be assigned to different convolutional layers in the feature pyramid network model for fusion processing. By fusing high-level features to low-level features, the expressive power of low-level features can be increased. Therefore, a sample image of size 320 pixels × 320 pixels can be input into the backbone network model for feature extraction, resulting in multiple initial feature maps of different sizes. These initial feature maps are then input into the feature pyramid network model for feature fusion processing, yielding multiple target feature maps corresponding to the sample image. In one example, suppose the backbone network model contains five cascaded convolutional layers, which, from bottom to top, are the first, second, third, fourth, and fifth convolutional layers; the feature pyramid network model contains three cascaded convolutional layers, which, from top to bottom, are the first, second, and third convolutional layers. Inputting a 320-pixel × 320-pixel sample image into the backbone network model for feature extraction yields the feature map output by each convolutional layer in the backbone network model. Then, the feature map output from the fifth convolutional layer of the backbone network model is input into the first convolutional layer of the feature pyramid network to obtain the corresponding target feature map (e.g., P5). The feature map output from P5 and the fourth convolutional layer of the backbone network model is fused through the second convolutional layer of the feature pyramid network model to obtain the corresponding target feature map (e.g., P4). The feature map output from P4 and the third convolutional layer of the backbone network model is fused through the third convolutional layer of the feature pyramid network to obtain the corresponding target feature map (e.g., P3). The dimensions of P5 are, for example, height × width × 1024; the dimensions of P4 are, for example, height × width × 512; and the dimensions of P3 are, for example, height × width × 256; where 1024, 512, and 256 represent the number of channels. Figure 6 The first and second sub-models in the example can be found in subsequent embodiments. By first extracting features using a backbone network model and then performing feature fusion using a feature pyramid network model, the performance of acquiring multiple target feature maps corresponding to sample images can be effectively improved, while reducing resource consumption.

[0112] S502. Input multiple target feature maps into the first sub-model for convolution processing to obtain the classification feature map, regression feature map and object feature map corresponding to each target feature map output by the first sub-model.

[0113] In this step, after obtaining multiple target feature maps corresponding to the sample image, these target feature maps can be input into the first sub-model for convolution processing to obtain the classification feature map, regression feature map, and object feature map output by the first sub-model corresponding to each target feature map. It can be understood that the classification feature map is used to detect the category of objects contained in the sample image, such as the prohibited category to which the object belongs; the regression feature map is used to detect the location of objects contained in the sample image, such as the coordinates of the object; and the object feature map is used to detect objects contained in the sample image, such as determining whether it is an object. For details on how to obtain the classification feature map, regression feature map, and object feature map output by the first sub-model corresponding to each target feature map, please refer to subsequent embodiments; they will not be elaborated here.

[0114] S503. Input the classification feature map into the second sub-model for feature decorrelation processing to obtain the target sample weights output by the second sub-model. The target sample weights are used to characterize the weight of each classification feature in the classification feature map.

[0115] Based on the causal association-based judgment method, the image recognition model can learn to find a set of sample weights that ensures any variable is independent of the others. That is, when any variable is chosen as the target variable, its distribution does not change with the values ​​of other variables. This helps the image recognition model learn the correlation between each variable and the final result during training, removing spurious correlations and allowing the model to better focus on the essential features relevant to the result. Therefore, in this step, after obtaining the classification feature map corresponding to the target feature map output by the first sub-model, the classification feature map can be input into the second sub-model for feature decorrelation processing to obtain the target sample weights output by the second sub-model. The target sample weights represent the weight of each classification feature in the classification feature map, ensuring independence between different classification features. How to obtain the target sample weights output by the second sub-model can be found in subsequent embodiments and will not be repeated here.

[0116] S504. Determine the target loss value based on the target sample weights, classification feature maps, regression feature maps, object feature maps, and annotation information of the sample images.

[0117] In this step, after obtaining the target sample weights output by the second sub-model, the target loss value can be determined based on the target sample weights, classification feature map, regression feature map, object feature map, and the annotation information of the sample image. For details on how to determine the target loss value based on the target sample weights, classification feature map, regression feature map, object feature map, and the annotation information of the sample image, please refer to subsequent embodiments; these details will not be elaborated here.

[0118] S505. Adjust the parameters of the image recognition model according to the target loss value to obtain the trained image recognition model.

[0119] In this step, after determining the target loss value, the parameters of the image recognition model can be adjusted based on the target loss value. These parameters can be understood as the global weight parameters of the image recognition model to obtain the trained image recognition model. For example, during the iterative training of the image recognition model, to further improve the generalization ability of the image recognition model, a simple moving average (SMA) training method can be used to train the image recognition model. For details on how to adjust the parameters of the image recognition model based on the target loss value to obtain the trained image recognition model, please refer to subsequent embodiments; they will not be elaborated here.

[0120] The model training method provided in this disclosure is used to train an image recognition model, which includes a first sub-model and a second sub-model. Multiple target feature maps of different sizes are obtained corresponding to a sample image. These target feature maps are then input into the first sub-model for convolution processing to obtain a classification feature map, a regression feature map, and an object feature map output by the first sub-model corresponding to each target feature map. The classification feature map is then input into the second sub-model for feature decorrelation processing to obtain the target sample weights output by the second sub-model. These target sample weights characterize the weight of each classification feature in the classification feature map. Based on the target sample weights, classification feature maps, regression feature maps, object feature maps, and the annotation information of the sample image, a target loss value is determined. The parameters of the image recognition model are adjusted according to the target loss value to obtain the trained image recognition model. Since the embodiments of this disclosure input the classification feature map corresponding to the target feature map output by the first sub-model into the second sub-model for feature decorrelation processing, and obtain the target sample weight output by the second sub-model, the target sample weight makes different classification features independent of each other, removes false associations between different classification features, and thus enables the image recognition model to better focus on the essential features related to the recognition result. Therefore, it can greatly improve the generalization ability of the image recognition model.

[0121] Figure 7A flowchart illustrating a model training method provided in another embodiment of this disclosure. Based on the above embodiments, this disclosure further describes the model training method. Figure 7 As shown, the method in this embodiment of the disclosure may include:

[0122] S701. Obtain multiple target feature maps corresponding to the sample image. The multiple target feature maps have different sizes.

[0123] For a detailed description of this step, please refer to [link / reference]. Figure 5 The relevant description of S501 in the illustrated embodiment will not be repeated here.

[0124] The first sub-model includes a first convolutional layer and a second convolutional layer and a third convolutional layer connected to the first convolutional layer, respectively. A second sub-model is then connected after the second convolutional layer. In this embodiment of the disclosure... Figure 5 Step S502 can further include the following two steps, S702 and S703:

[0125] S702. Input the target feature map into the first convolutional layer for convolution processing to obtain the convolution processing result output by the first convolutional layer.

[0126] The first convolutional layer contains one convolutional layer.

[0127] S703. Input the convolution processing results into the second and third convolutional layers respectively for convolution processing to obtain the classification feature map output by the second convolutional layer and the regression feature map and object feature map output by the third convolutional layer.

[0128] The second convolutional layer contains multiple concatenated convolutional layers, and the third convolutional layer contains multiple concatenated convolutional layers.

[0129] For example, refer to Figure 6The first sub-model includes a first convolutional layer 601 and a second convolutional layer 602 and a third convolutional layer 603 connected to the first convolutional layer, respectively. The first convolutional layer 601 contains a 1×1 convolutional layer for dimensionality reduction. The second convolutional layer 602 contains three concatenated convolutional layers, including two 3×3 convolutional layers and one 1×1 convolutional layer. The third convolutional layer 603 contains four concatenated convolutional layers, including two 3×3 convolutional layers and two 1×1 convolutional layers. Based on the example of step S501, P5, P4, and P3 can be sequentially input into the first convolutional layer 601 for convolution processing, i.e., dimensionality reduction, to obtain the convolution processing result output by the first convolutional layer 601. For example, P5, P4, and P3 can all be reduced to height × width × 256. Each convolutional processing result output from the first convolutional layer 601 is input into the second convolutional layer 602 and the third convolutional layer 603 for convolutional processing, respectively. This yields the classification feature map output by the second convolutional layer 602 and the regression feature map and object feature map output by the third convolutional layer 603, i.e., the classification feature map, regression feature map, and object feature map corresponding to P5, P4, and P3, respectively. Compared with current related technologies that obtain the classification feature map, regression feature map, and object feature map corresponding to P5, P4, and P3 through only a single 1×1 convolutional layer, the embodiments of this disclosure can achieve decoupling of the prediction branch of the image recognition model through the second and third convolutional layers.

[0130] In this embodiment of the disclosure, Figure 5 Step S503 can further include the following four steps S704 to S707:

[0131] S704. Input multiple classification feature maps into the second sub-model to obtain the target classification feature map.

[0132] The target classification feature map is obtained by the second sub-model by splicing together multiple classification feature maps.

[0133] It is understandable that the second sub-model can separate target features from background information, extract essential features of different categories, remove irrelevant features and false associations, allowing the image recognition model to focus more on the feature information of the object itself, weaken the interference of background information, and thus obtain more accurate essential features of the target. It also removes other irrelevant interference information and makes predictions only based on essential features (features that are causally related to the recognition result), thereby improving the generalization ability of the image recognition model in unknown domains. For example, refer to... Figure 6 After obtaining the classification feature maps corresponding to P5, P4 and P3 respectively, these three classification feature maps can be input into the second sub-model, which will then concatenate them to obtain the target classification feature map.

[0134] S705. Extract target classification features based on the target classification feature map and the target classification feature map obtained from the previous training image recognition model.

[0135] For example, the image recognition model can be pre-trained for a first predetermined number of iterations to enable it to learn. In each training iteration, the target sample weights output by the second sub-model are preset initial sample weights. Based on this, assuming the previous training iteration of the image recognition model was N-1, the target classification feature map obtained during the Nth (current) training iteration needs to be combined with the target classification features obtained during the N-1th (previous) training iteration. Figure 1 We then extract the target classification features.

[0136] S706. Perform random Fourier transform on the target classification features to obtain random Fourier features.

[0137] It is understandable that complex dependencies exist between features in deep networks. When the original features are low-dimensional, simply removing the linear correlations between them is insufficient to completely eliminate spurious associations between irrelevant features and labels. While kernel functions (such as linear, polynomial, and Gaussian kernels) can map the original features to a high-dimensional space, the feature map dimension is expanded to infinity after mapping, making it difficult to calculate the correlations between features. Random Fourier Features (RFF) offer better performance in approximating kernel functions and measuring feature independence. Therefore, RFF is introduced into image recognition models. For example, the target classification features are processed by a random Fourier feature extractor to obtain RFF features.

[0138] S707. Based on random Fourier features, iteratively train the second sub-model by performing feature decorrelation processing to obtain the target sample weights.

[0139] It is understandable that during each iteration of training the image recognition model, the second sub-model will undergo a second, predetermined number of iterations. In this step, after obtaining the random Fourier features, the second sub-model can be iteratively trained based on these features through feature decorrelation processing to obtain the target sample weights.

[0140] Further, optionally, the iterative training of the second sub-model based on random Fourier features to perform feature decorrelation processing to obtain target sample weights may include: iterative training of the second sub-model based on random Fourier features, the first sample weights obtained from the previous training of the second sub-model, and the second sample weights obtained from the previous training of the image recognition model to obtain target sample weights, wherein in the first training of the second sub-model, the preset initial sample weights are used as the first sample weights.

[0141] For example, based on the example of step S705, the second sample weights obtained through the second sub-model can be obtained during the (N-1)th training of the image recognition model. During the second preset number of iterations of training the second sub-model, when training the second sub-model for the first time, it is necessary to combine the second sample weights and the preset initial sample weights to perform iterative training of feature decorrelation processing on the second sub-model to obtain the sample weights output by the second sub-model during the first training. In subsequent training of the second sub-model, iterative training of feature decorrelation processing on the second sub-model can be performed based on the random Fourier features and the first sample weights obtained in the previous training of the second sub-model until the number of iterations reaches the second preset number to obtain the target sample weights.

[0142] In this embodiment of the disclosure, Figure 5 Step S504 can further include the following four steps S708 to S711:

[0143] S708. Perform category detection on multiple classification feature maps using the first sub-model to obtain the first detection result; determine the initial classification loss value based on the first detection result and the annotation information; determine the first loss value based on the initial classification loss value and the target sample weight.

[0144] For example, refer to Figure 6 After obtaining the classification feature maps corresponding to P5, P4 and P3 respectively, the first sub-model can combine these three classification feature maps to perform category detection on these three classification feature maps to obtain the first detection result (i.e. prediction result). Based on the first detection result and the annotation information, the initial classification loss value is determined, and then the first loss value is determined based on the initial classification loss value and the obtained target sample weights.

[0145] Further, optionally, a first loss value is determined based on the initial classification loss value and the target sample weights, including: performing a weighted summation of the initial classification loss value and the target sample weights to determine the first loss value.

[0146] For example, the target sample weight represents the weight of each classification feature in the classification feature map. The target sample weight can be understood as a vector, and the initial classification loss value can also be understood as a vector. The corresponding elements of the two vectors are multiplied one by one, that is, a weighted summation is performed to obtain a determined first loss value, which is the cross-entropy loss.

[0147] Optionally, a weighted summation is performed on the initial classification loss value and the target sample weights to determine the first loss value, including: determining the first loss value according to the following formula:

[0148]

[0149] in, This represents the initial classification loss value; Indicates the target sample weight; Represents a sample image; Annotation information representing sample images; express The corresponding random Fourier features; B represents the first detection result output by the first sub-model; B represents the number of sample images.

[0150] It is understandable that, when a batch of sample images are input into the model, the first loss value corresponding to the batch of sample images can be obtained based on the above formula.

[0151] S709. The first sub-model is used to detect the location of multiple regression feature maps to obtain the second detection result; based on the second detection result and the annotation information, the second loss value is determined.

[0152] For example, refer to Figure 6 After obtaining the regression feature maps corresponding to P5, P4 and P3 respectively, the first sub-model can combine these three regression feature maps to perform position detection on these three regression feature maps to obtain the second detection result. Then, based on the second detection result and the annotation information, the second loss value can be determined.

[0153] S710. Perform object detection on multiple object feature maps using the first sub-model to obtain the third detection result; determine the third loss value based on the third detection result and the annotation information.

[0154] For example, refer to Figure 6 After obtaining the object feature maps corresponding to P5, P4 and P3 respectively, the first sub-model can combine these three object feature maps to perform object detection on these three object feature maps to obtain the third detection result. Then, based on the third detection result and the annotation information, the third loss value can be determined.

[0155] It should be noted that this disclosure does not limit the order in which S708, S709 and S710 are executed.

[0156] S711. Determine the target loss value based on the first loss value, the second loss value, and the third loss value.

[0157] In this step, after obtaining the first loss value, the second loss value, and the third loss value, the first loss value, the second loss value, and the third loss value can be summed to obtain the target loss value. Based on the target loss value, the target loss value can be reduced by backpropagation and gradient descent.

[0158] In this embodiment of the disclosure, Figure 5 Step S505 may further include the following three steps, S712 to S714:

[0159] S712. Adjust the parameters of the image recognition model according to the target loss value, and iteratively train the image recognition model until the preset number of iterations is reached.

[0160] S713. Based on the preset number of iterations, the current number of iterations, the initial parameters corresponding to the current number of iterations, and the parameters obtained from the previous training of the image recognition model, perform a moving average on the initial parameters corresponding to the current number of iterations to obtain the target parameters corresponding to the current number of iterations.

[0161] S714. Iteratively train the image recognition model according to the target parameters corresponding to the current iteration number to obtain the trained image recognition model.

[0162] It is understandable that, during the iterative training of an image recognition model, in order to further improve the generalization ability of the model, a training strategy such as SMA (Short-Range Moving Average) can be used. Specifically, firstly, the parameters of the image recognition model are adjusted according to the target loss value, and the model is iteratively trained until a preset number of iterations is reached. Then, the SMA training method is used to train the model, maintaining the calculated moving average until training ends. For the t-th training iteration after reaching the preset number of iterations, the parameters of the image recognition model are as follows:

[0163]

[0164] Where t represents the current iteration number; t0 represents the iteration time when the weight parameters need to start calculating the moving average; it can also be understood as the last iteration in the preset iteration number; θ represents the weight parameters of the image recognition model trained using the SMA method at time t, i.e., the target parameters corresponding to the current iteration number; tThis represents the weight parameters of the current image recognition model at time t, i.e., the initial parameters corresponding to the current iteration number; This represents the parameters obtained during the previous training of the image recognition model.

[0165] Based on the above embodiments, since multiple image recognition models can be retained based on different checkpoints (such as the accuracy index of the validation set) during the training process of the image recognition model, multiple image recognition models can be tested in multiple domains, and the image recognition model that performs well in multiple known domains can be selected as the final prediction model, that is, the accuracy of the validation set and the variance of the validation accuracy in each domain are considered.

[0166] The model training method provided in this disclosure involves acquiring multiple target feature maps of different sizes corresponding to a sample image; inputting the target feature maps into a first convolutional layer for convolution processing to obtain the convolution processing result output by the first convolutional layer; inputting the convolution processing result into a second convolutional layer and a third convolutional layer for convolution processing to obtain a classification feature map output by the second convolutional layer and a regression feature map and an object feature map output by the third convolutional layer. Compared with current related technologies that obtain classification feature maps, regression feature maps, and object feature maps through only one convolutional layer, this disclosure can achieve decoupling of the prediction branch of the image recognition model through the second and third convolutional layers. Multiple classification feature maps are input into a second sub-model to obtain a target classification feature map; based on the target classification feature map and the target classification feature map obtained from the previous training of the image recognition model, target classification features are extracted; the target classification features are processed by random Fourier transform to obtain random Fourier features, which can map the target classification features to a high-dimensional space to calculate the correlation between different target classification features. Based on random Fourier features, the second sub-model undergoes iterative training with feature decorrelation processing to obtain target sample weights. The first sub-model then performs category detection on multiple classification feature maps to obtain a first detection result. Based on the first detection result and annotation information, an initial classification loss value is determined. Based on the initial classification loss value and target sample weights, a first loss value is determined. The first sub-model then performs position detection on multiple regression feature maps to obtain a second detection result. Based on the second detection result and annotation information, a second loss value is determined. The first sub-model then performs object detection on multiple object feature maps to obtain a third detection result. Based on the third detection result and annotation information, a third loss value is determined. Finally, based on the first, second, and third loss values, a target loss value is determined. Since target sample weights are used when determining the target loss value, this ensures that different classification features are independent of each other, removing spurious associations between different classification features. This allows the image recognition model to better focus on the essential features relevant to the recognition result, thus significantly improving the generalization ability of the image recognition model. The parameters of the image recognition model are adjusted based on the target loss value, and the model is iteratively trained until a preset number of iterations is reached. Based on the preset number of iterations, the current number of iterations, the initial parameters corresponding to the current number of iterations, and the parameters obtained from the previous training of the image recognition model, a moving average is applied to the initial parameters corresponding to the current number of iterations to obtain the target parameters. The image recognition model is then iteratively trained again using these target parameters to obtain a fully trained image recognition model. Because a moving average training strategy is used, the generalization ability of the image recognition model can be further improved.

[0167] Figure 8This is a flowchart illustrating an image recognition method according to an embodiment of the present disclosure. The method of this embodiment can be applied to a computing device, which may be a server or a server cluster, etc. Figure 8 As shown, the method in this embodiment includes:

[0168] S801. Obtain multiple target feature maps corresponding to the image to be recognized. The multiple target feature maps have different sizes.

[0169] In this embodiment of the disclosure, the image to be identified is, for example, a prohibited image. Optionally, obtaining multiple target feature maps corresponding to the image to be identified includes: obtaining the image to be identified; preprocessing the image to be identified to obtain a preprocessed image, the preprocessing including image normalization and / or image scaling; and obtaining multiple target feature maps based on the preprocessed image.

[0170] Before inputting the image to be recognized into the image recognition model, to ensure a balance between speed and accuracy, the image can be preprocessed. This preprocessing may include image normalization and / or image scaling. When scaling the image, the specific scaling size can be determined based on the actual task requirements; for example, the default scaling size is usually 320 pixels × 320 pixels. Normalizing the image ensures that it maintains consistency with the data distribution during the image recognition model training process.

[0171] Optionally, based on the preprocessed image, multiple target feature maps are obtained, including: inputting the preprocessed image into the backbone network model of the image recognition model for feature extraction to obtain multiple initial feature maps corresponding to the preprocessed image, wherein the multiple initial feature maps have different sizes; inputting the multiple initial feature maps into the feature pyramid network model of the image recognition model for feature fusion processing to obtain multiple target feature maps.

[0172] For example, refer to Figure 6 The preprocessed image can be input into the backbone network model of the image recognition model for feature extraction to obtain multiple initial feature maps corresponding to the preprocessed image. Then, the multiple initial feature maps are input into the feature pyramid network model of the image recognition model for feature fusion processing to obtain multiple target feature maps.

[0173] S802. Input multiple target feature maps into the first sub-model of the image recognition model for recognition processing to obtain the image recognition result output by the image recognition model.

[0174] The image recognition result is used to indicate whether the image to be identified is a prohibited image.

[0175] The image recognition model is trained using the model training method described in any of the above method embodiments.

[0176] For example, Figure 9 This is a schematic diagram illustrating image recognition using an image recognition model, as provided in an embodiment of this disclosure. Figure 9 As shown, for reference Figure 6 In the process of image recognition using an image recognition model, the second sub-model in the image recognition model does not need to participate in the model prediction. That is, the entire image recognition model only incurs additional overhead compared to the original image recognition model during the training process, with no additional computational overhead during the prediction phase. Therefore, Figure 9 The image recognition model shown includes a backbone network model, a feature pyramid network model, and a first sub-model, but does not include... Figure 6 The second sub-model in the image recognition model shown. Multiple target feature maps are input. Figure 9 The first sub-model of the image recognition model shown is used for recognition processing to obtain the image recognition result output by the image recognition model. The image recognition result is used to indicate whether the image to be recognized is a prohibited image.

[0177] Further, optionally, multiple target feature maps are input into the first sub-model of the image recognition model for recognition processing to obtain the image recognition result output by the image recognition model, including: inputting multiple target feature maps into the first sub-model of the image recognition model for recognition processing to obtain the score of the target object contained in the image to be recognized; if the score is greater than a threshold, the image recognition result is that the image to be recognized is a prohibited image; if the score is less than or equal to the threshold, the image recognition result is that the image to be recognized is not a prohibited image.

[0178] The threshold can be preset, and this disclosure does not limit this. After obtaining the score of the target object contained in the image to be identified through the first sub-model, the score can be judged. If the score is greater than the threshold, the image recognition result is that the image to be identified is a prohibited image; if the score is less than or equal to the threshold, the image recognition result is that the image to be identified is not a prohibited image. After obtaining the image recognition result, the image recognition result can be output and fed back to the user.

[0179] Optionally, the image recognition result may also include the location information of the target object in the image to be recognized, which is used to determine the location of the target prohibited object in the prohibited image.

[0180] For example, refer to Figure 9 Since the first sub-model can perform position detection on the regression feature map, the position information of the target object in the image to be identified can be output by the first sub-model. This position information is, for example, the coordinates of the rectangle where the target object is located.

[0181] The image recognition method provided in this disclosure acquires multiple target feature maps corresponding to an image to be recognized, with the multiple target feature maps having different sizes. These multiple target feature maps are then input into a first sub-model of an image recognition model for recognition processing, resulting in an image recognition result output by the image recognition model. This image recognition result is used to indicate whether the image to be recognized is a prohibited image. Because the image recognition model of this disclosure has good generalization ability, it can obtain image recognition results more accurately.

[0182] Exemplary device

[0183] Having introduced the medium of exemplary embodiments of this disclosure, the following references are made to... Figure 10 An image recognition apparatus according to an exemplary embodiment of this disclosure will be described. The apparatus of the exemplary embodiment of this disclosure can implement the various processes in the foregoing image recognition method embodiments and achieve the same functions and effects.

[0184] Figure 10 This is a schematic diagram of a model training apparatus provided in one embodiment of the present disclosure, used to train an image recognition model. The image recognition model includes a first sub-model and a second sub-model. Figure 10 As shown, the model training apparatus 1000 of this embodiment includes: an acquisition module 1001, a first processing module 1002, a second processing module 1003, a determination module 1004, and a third processing module 1005. Wherein:

[0185] The acquisition module 1001 is used to acquire multiple target feature maps corresponding to the sample image, and the multiple target feature maps have different sizes.

[0186] The first processing module 1002 is used to input multiple target feature maps into the first sub-model for convolution processing, and obtain the classification feature map, regression feature map and object feature map corresponding to each target feature map output by the first sub-model.

[0187] The second processing module 1003 is used to input the classification feature map into the second sub-model for feature decorrelation processing to obtain the target sample weights output by the second sub-model. The target sample weights are used to characterize the weights of each classification feature in the classification feature map.

[0188] The determination module 1004 is used to determine the target loss value based on the target sample weights, classification feature maps, regression feature maps, object feature maps, and annotation information of the sample images.

[0189] The third processing module 1005 is used to adjust the parameters of the image recognition model according to the target loss value in order to obtain the trained image recognition model.

[0190] In one possible implementation, the second processing module 1003 may be specifically used to: input multiple classification feature maps into a second sub-model to obtain a target classification feature map, wherein the target classification feature map is obtained by the second sub-model concatenating multiple classification feature maps; extract target classification features based on the target classification feature map and the target classification feature map obtained from the previous training of the image recognition model; perform random Fourier transform processing on the target classification features to obtain random Fourier features; and perform iterative training of the second sub-model based on the random Fourier features to perform feature decorrelation processing to obtain target sample weights.

[0191] In one possible implementation, when the second processing module 1003 is used for iterative training of the second sub-model based on random Fourier features to perform feature decorrelation processing and obtain target sample weights, it can specifically be used for iterative training of the second sub-model based on random Fourier features, the first sample weights obtained from the previous training of the second sub-model, and the second sample weights obtained from the previous training of the image recognition model to obtain target sample weights, wherein, in the first training of the second sub-model, the preset initial sample weights are used as the first sample weights.

[0192] In one possible implementation, the determining module 1004 may be specifically used to: perform category detection on multiple classification feature maps using a first sub-model to obtain a first detection result; determine an initial classification loss value based on the first detection result and annotation information; determine a first loss value based on the initial classification loss value and the target sample weight; perform position detection on multiple regression feature maps using the first sub-model to obtain a second detection result; determine a second loss value based on the second detection result and annotation information; perform object detection on multiple object feature maps using the first sub-model to obtain a third detection result; determine a third loss value based on the third detection result and annotation information; and determine a target loss value based on the first loss value, the second loss value, and the third loss value.

[0193] In one possible implementation, when determining the first loss value based on the initial classification loss value and the target sample weights, the determining module 1004 may specifically be used to: perform a weighted summation of the initial classification loss value and the target sample weights to determine the first loss value.

[0194] In one possible implementation, when determining the first loss value by performing a weighted summation of the initial classification loss value and the target sample weights, the determining module 1004 may specifically be used to: determine the first loss value according to the following formula:

[0195]

[0196] in, This represents the initial classification loss value; Indicates the target sample weight; Represents a sample image; Annotation information representing sample images; express The corresponding random Fourier features; B represents the first detection result output by the first sub-model; B represents the number of sample images.

[0197] In one possible implementation, the first sub-model includes a first convolutional layer and a second and a third convolutional layer connected to the first convolutional layer, respectively. The second convolutional layer is followed by the second sub-model. The first processing module 1002 can be specifically used to: input the target feature map into the first convolutional layer for convolution processing to obtain the convolution processing result output by the first convolutional layer, wherein the first convolutional layer contains one convolutional layer; input the convolution processing result into the second and third convolutional layers for convolution processing to obtain the classification feature map output by the second convolutional layer and the regression feature map and object feature map output by the third convolutional layer, wherein the second convolutional layer contains multiple concatenated convolutional layers and the third convolutional layer contains multiple concatenated convolutional layers.

[0198] In one possible implementation, the third processing module 1005 may be specifically used to: adjust the parameters of the image recognition model according to the target loss value, and iteratively train the image recognition model until a preset number of iterations is reached.

[0199] In one possible implementation, the third processing module 1005 can also be used to: after reaching a preset number of iterations, perform a moving average on the initial parameters corresponding to the current number of iterations based on the preset number of iterations, the current number of iterations, the initial parameters corresponding to the current number of iterations, and the parameters obtained from the previous training of the image recognition model, to obtain the target parameters corresponding to the current number of iterations; and perform iterative training on the image recognition model based on the target parameters corresponding to the current number of iterations to obtain the trained image recognition model.

[0200] In one possible implementation, the sample image includes a first image and a second image. The first image is an image acquired in a real scene, and the second image is an image generated using a preset generation method, which includes at least one of adding background, adding noise, and combining image elements.

[0201] In one possible implementation, the acquisition module 1001 may also be used to: before acquiring multiple target feature maps corresponding to the sample image, perform enhancement processing on the sample image, the enhancement processing including at least one of flipping, resizing, cropping, brightness adjustment, contrast adjustment, and adding noise.

[0202] In one possible implementation, the image recognition model further includes a backbone network model and a feature pyramid network model. The feature pyramid network model is connected to the backbone network model, and the first sub-model is connected to the feature pyramid network model. The acquisition module 1001 can be specifically used to: input the sample image into the backbone network model for feature extraction to obtain multiple initial feature maps corresponding to the sample image, wherein the multiple initial feature maps have different sizes; input the multiple initial feature maps into the feature pyramid network model for feature fusion processing to obtain multiple target feature maps corresponding to the sample image.

[0203] The apparatus of this disclosure embodiment can be used to execute the model training method in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0204] Figure 11 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of the present disclosure, as shown below. Figure 11 As shown, the image recognition device 1100 of this embodiment includes: an acquisition module 1101 and a processing module 1102. Wherein:

[0205] The acquisition module 1101 is used to acquire multiple target feature maps corresponding to the image to be identified, and the multiple target feature maps have different sizes.

[0206] Processing module 1102 is used to input multiple target feature maps into the first sub-model of the image recognition model for recognition processing, and obtain the image recognition result output by the image recognition model. The image recognition result is used to indicate whether the image to be recognized is a prohibited image. The image recognition model is trained using the model training method as described in any of the above method embodiments.

[0207] In one possible implementation, the acquisition module 1101 may be specifically used for: acquiring an image to be identified; preprocessing the image to be identified to obtain a preprocessed image, the preprocessing including image normalization and / or image scaling; and acquiring multiple target feature maps based on the preprocessed image.

[0208] In one possible implementation, when the acquisition module 1101 acquires multiple target feature maps based on the preprocessed image, it can specifically be used to: input the preprocessed image into the backbone network model of the image recognition model for feature extraction to obtain multiple initial feature maps corresponding to the preprocessed image, wherein the multiple initial feature maps have different sizes; input the multiple initial feature maps into the feature pyramid network model of the image recognition model for feature fusion processing to obtain multiple target feature maps.

[0209] In one possible implementation, the processing module 1102 may be specifically used to: input multiple target feature maps into the first sub-model of the image recognition model for recognition processing, and obtain the score of the target object contained in the image to be recognized; if the score is greater than the threshold, the image recognition result is that the image to be recognized is a prohibited image; if the score is less than or equal to the threshold, the image recognition result is that the image to be recognized is not a prohibited image.

[0210] In one possible implementation, the image recognition result also includes the location information of the target object in the image to be recognized, which is used to determine the location of the target prohibited object in the prohibited image.

[0211] The apparatus of this disclosure embodiment can be used to execute the image recognition method in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0212] Exemplary media

[0213] After introducing the methods of exemplary embodiments of this disclosure, the following references are made. Figure 12 The storage medium of the exemplary embodiments of this disclosure will be described.

[0214] Figure 12 This is a schematic diagram of a storage medium provided according to an embodiment of the present disclosure. (See reference...) Figure 12 As shown, the storage medium 1200 stores a program product for implementing the above-described method according to embodiments of the present disclosure. This program product may be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0215] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0216] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium.

[0217] Program code for performing the operations disclosed herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0218] Exemplary computing device

[0219] Having described the methods, media, and apparatus of exemplary embodiments of this disclosure, the following references... Figure 13 A computing device according to an exemplary embodiment of the present disclosure will be described.

[0220] Figure 13 The computing device 1300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0221] Figure 13 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure, as shown below. Figure 13 As shown, the computing device 1300 is manifested as a general-purpose computing device. Components of the computing device 1300 may include, but are not limited to: at least one processing unit 1301, at least one storage unit 1302, and a bus 1303 connecting different system components (including the processing unit 1301 and the storage unit 1302). Exemplarily, the processing unit 1301 may specifically be a processor, and the storage unit 1302 stores computer execution instructions. The processing unit 1301 executes the computer execution instructions stored in the storage unit 1302 to implement the image recognition method described above.

[0222] Bus 1303 includes a data bus, a control bus, and an address bus.

[0223] Storage unit 1302 may include readable media in the form of volatile memory, such as random access memory (RAM) 13021 and / or cache memory 13022, and may further include readable media in the form of non-volatile memory, such as read-only memory (ROM) 13023.

[0224] Storage unit 1302 may also include a program / utility 13025 having a set (at least one) of program modules 13024, such program modules 13024 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0225] The computing device 1300 can also communicate with one or more external devices 1304 (e.g., keyboard, pointing device, etc.). This communication can be performed via the input / output (I / O) interface 1305. Furthermore, the computing device 1300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via a network adapter 1306. Figure 13 As shown, network adapter 1306 communicates with other modules of computing device 1300 via bus 1303. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with computing device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0226] It should be noted that although several units / modules or sub-units / modules of the image recognition device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0227] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0228] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A model training method for training an image recognition model, the image recognition model comprising a first sub-model and a second sub-model, the model training method comprising: obtaining a plurality of target feature maps corresponding to a sample image, the plurality of target feature maps having different sizes; inputting the plurality of target feature maps into the first sub-model for convolution processing to obtain a classification feature map, a regression feature map and an object feature map corresponding to each of the target feature maps output by the first sub-model; inputting the classification feature map into the second sub-model for feature decorrelation processing to obtain a target sample weight output by the second sub-model, the target sample weight being used to represent a weight of each classification feature in the classification feature map; determining a target loss value based on the target sample weight, the classification feature map, the regression feature map, the object feature map and annotation information of the sample image; adjusting parameters of the image recognition model according to the target loss value to obtain a trained image recognition model; and the inputting the classification feature map into the second sub-model for feature decorrelation processing to obtain the target sample weight output by the second sub-model comprises: inputting a plurality of the classification feature maps into the second sub-model to obtain a target classification feature map, the target classification feature map being obtained by the second sub-model performing stitching processing on the plurality of classification feature maps; extracting a target classification feature based on the target classification feature map and a target classification feature map obtained by last training of the image recognition model; performing random Fourier transform processing on the target classification feature to obtain a random Fourier feature; and performing iterative training of the feature decorrelation processing on the second sub-model based on the random Fourier feature to obtain the target sample weight.

2. The model training method of claim 1, the performing iterative training of the feature decorrelation processing on the second sub-model based on the random Fourier feature to obtain the target sample weight comprises: performing iterative training of the feature decorrelation processing on the second sub-model based on the random Fourier feature, a first sample weight obtained by last training of the second sub-model and a second sample weight obtained by last training of the image recognition model to obtain the target sample weight, wherein, in a first training of the second sub-model, a preset initial sample weight is used as the first sample weight.

3. The model training method of claim 1 or 2, the determining a target loss value based on the target sample weight, the classification feature map, the regression feature map, the object feature map and annotation information of the sample image comprises: performing category detection on a plurality of the classification feature maps by the first sub-model to obtain a first detection result; determining an initial classification loss value according to the first detection result and the annotation information; determining a first loss value according to the initial classification loss value and the target sample weight; performing position detection on a plurality of the regression feature maps by the first sub-model to obtain a second detection result; determining a second loss value according to the second detection result and the annotation information; determining a third loss value according to the second loss value and the target sample weight; and determining the target loss value according to the first loss value, the second loss value and the third loss value. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ determine a second loss value according to the second detection result and the annotation information; perform object detection on the plurality of object feature maps through the first sub-model to obtain a third detection result; and determine a third loss value according to the third detection result and the annotation information; determine the target loss value according to the first loss value, the second loss value, and the third loss value.

4. The model training method of claim 3, wherein the determining the first loss value according to the initial classification loss value and the target sample weight comprises: performing weighted summation processing on the initial classification loss value and the target sample weight to determine the first loss value.

5. The model training method of claim 4, wherein the performing weighted summation processing on the initial classification loss value and the target sample weight to determine the first loss value comprises: determining the first loss value according to the following formula: wherein, denotes an initial classification loss value; denotes a target sample weight; denotes a sample image; denotes annotation information of the sample image; denotes corresponding random Fourier features; denotes a first detection result output by the first sub-model; denotes a number of sample images.

6. The model training method of claim 1 or 2, wherein the first sub-model comprises a first convolutional layer, and a second convolutional layer and a third convolutional layer connected to the first convolutional layer respectively, the second convolutional layer is connected to the second sub-model, and the inputting the plurality of target feature maps into the first sub-model for convolutional processing to obtain the classification feature map, the regression feature map, and the object feature map corresponding to each target feature map output by the first sub-model comprises: inputting the target feature map into the first convolutional layer for convolutional processing to obtain a convolutional processing result output by the first convolutional layer, the first convolutional layer comprises one convolutional layer; inputting the convolutional processing result into the second convolutional layer and the third convolutional layer respectively for convolutional processing to obtain the classification feature map output by the second convolutional layer and the regression feature map and the object feature map output by the third convolutional layer, the second convolutional layer comprises a plurality of convolutional layers connected in cascade, and the third convolutional layer comprises a plurality of convolutional layers connected in cascade.

7. The model training method of claim 1 or 2, wherein the adjusting the parameters of the image recognition model according to the target loss value to obtain the trained image recognition model comprises: adjusting the parameters of the image recognition model according to the target loss value, and iteratively training the image recognition model until a preset iteration number is reached.

8. The model training method of claim 7, wherein after the preset iteration number is reached, the method further comprises: performing moving average processing on the initial parameters corresponding to the current iteration number based on the preset iteration number, the current iteration number, the initial parameters corresponding to the current iteration number, and the parameters obtained by training the image recognition model last time to obtain target parameters corresponding to the current iteration number; and iteratively training the image recognition model according to the target parameters corresponding to the current iteration number to obtain the trained image recognition model. ​ 9. The model training method of claim 1 or 2, wherein the sample image comprises a first image and a second image, the first image is an image collected in a real scene, and the second image is an image generated by using a preset generation manner, and the preset generation manner comprises at least one of adding a background, adding noise, and combining image elements.

10. The model training method of claim 1 or 2, further comprising, before the obtaining of the plurality of target feature maps corresponding to the sample image: performing enhancement processing on the sample image, and the enhancement processing comprises at least one of flipping, size adjustment, cropping, brightness adjustment, contrast adjustment, and adding noise.

11. The model training method of claim 1 or 2, wherein the image recognition model further comprises a backbone network model and a feature pyramid network model, the feature pyramid network model is connected to the backbone network model, and the first sub-model is connected to the feature pyramid network model, and the obtaining of the plurality of target feature maps corresponding to the sample image comprises: inputting the sample image into the backbone network model to perform feature extraction, to obtain a plurality of initial feature maps corresponding to the sample image, and the plurality of initial feature maps have different sizes; inputting the plurality of initial feature maps into the feature pyramid network model to perform feature fusion processing, to obtain the plurality of target feature maps corresponding to the sample image.

12. An image recognition method, comprising: obtaining a plurality of target feature maps corresponding to a to-be-recognized image, and the plurality of target feature maps have different sizes; inputting the plurality of target feature maps into a first sub-model of an image recognition model to perform recognition processing, to obtain an image recognition result output by the image recognition model, and the image recognition result is used to indicate whether the to-be-recognized image is a prohibited image, and the image recognition model is trained by using the model training method in any one of claims 1 to 11.

13. The image recognition method of claim 12, wherein the obtaining of the plurality of target feature maps corresponding to the to-be-recognized image comprises: obtaining the to-be-recognized image; performing preprocessing on the to-be-recognized image to obtain a preprocessed image, and the preprocessing comprises image normalization processing and / or image scaling processing; obtaining the plurality of target feature maps according to the preprocessed image.

14. The image recognition method of claim 13, wherein the obtaining of the plurality of target feature maps according to the preprocessed image comprises: inputting the preprocessed image into a backbone network model of the image recognition model to perform feature extraction, to obtain a plurality of initial feature maps corresponding to the preprocessed image, and the plurality of initial feature maps have different sizes; inputting the plurality of initial feature maps into a feature pyramid network model of the image recognition model to perform feature fusion processing, to obtain the plurality of target feature maps.

15. The image recognition method of claim 12, wherein the inputting of the plurality of target feature maps into the first sub-model of the image recognition model to perform recognition processing, to obtain the image recognition result output by the image recognition model comprises: inputting the plurality of target feature maps into a first sub-model of an image recognition model for recognition processing to obtain a score of a target object contained in the to-be-recognized image; if the score is greater than a threshold value, obtaining the image recognition result as the to-be-recognized image being a prohibited image; if the score is less than or equal to the threshold value, obtaining the image recognition result as the to-be-recognized image not being a prohibited image.

16. The image recognition method according to any one of claims 12 to 15, wherein the image recognition result further comprises position information of the target object in the to-be-recognized image, and the position information is used to determine a position of a target prohibited object in the prohibited image.

17. A model training apparatus for training an image recognition model, the image recognition model comprising a first sub-model and a second sub-model, the model training apparatus comprising: an acquisition module configured to acquire a plurality of target feature maps corresponding to a sample image, the plurality of target feature maps having different sizes; a first processing module configured to input the plurality of target feature maps into the first sub-model for convolution processing to obtain classification feature maps, regression feature maps and object feature maps output by the first sub-model and corresponding to each of the target feature maps; a second processing module configured to input the classification feature maps into the second sub-model for feature decorrelation processing to obtain target sample weights output by the second sub-model, the target sample weights being used to represent weights of each classification feature in the classification feature maps; a determination module configured to determine a target loss value based on the target sample weights, the classification feature maps, the regression feature maps, the object feature maps and annotation information of the sample image; a third processing module configured to adjust parameters of the image recognition model according to the target loss value to obtain a trained image recognition model; the second processing module is specifically configured to: input a plurality of the classification feature maps into the second sub-model to obtain target classification feature maps, the target classification feature maps being obtained by the second sub-model performing splicing processing on the plurality of classification feature maps; extract target classification features based on the target classification feature maps and target classification feature maps obtained by last training of the image recognition model; perform random Fourier transform processing on the target classification features to obtain random Fourier features; perform iterative training of the feature decorrelation processing on the second sub-model based on the random Fourier features to obtain the target sample weights.

18. The model training apparatus according to claim 17, wherein when performing the iterative training of the feature decorrelation processing on the second sub-model based on the random Fourier features to obtain the target sample weights, the second processing module is specifically configured to: The second sub-model is iteratively trained based on the random Fourier feature, a first sample weight obtained by last training of the second sub-model, and a second sample weight obtained by last training of the image recognition model, to obtain the target sample weight, wherein in a first training of the second sub-model, use a preset initial sample weight as a first sample weight.

19. The model training apparatus according to claim 17 or 18, wherein the determination module is specifically configured to: The first sub-model is used for category detection on the plurality of classification feature maps to obtain a first detection result; and an initial classification loss value is determined according to the first detection result and the annotation information. determine a first loss value according to the initial classification loss value and the target sample weights. The first sub-model is used for position detection on the plurality of regression feature maps, to obtain a second detection result; The second loss value is determined according to the second detection result and the annotation information; The first sub-model is used for object detection on the plurality of object feature maps, to obtain a third detection result; and the third loss value is determined according to the third detection result and the annotation information. The target loss value is determined according to the first loss value, the second loss value and the third loss value.

20. The model training apparatus according to claim 19, when the determining module is used to determine a first loss value according to the initial classification loss value and the target sample weight, the determining module is specifically used to: perform weighted sum processing on the initial classification loss value and the target sample weight to determine the first loss value.

21. The model training apparatus according to claim 20, when the determining module is used to perform weighted sum processing on the initial classification loss value and the target sample weight to determine the first loss value, the determining module is specifically used to: determine the first loss value according to the following formula: wherein represents an initial classification loss value; represents a target sample weight; represents a sample image; represents annotation information of the sample image; represents corresponding random Fourier features; represents a first detection result output by the first sub-model; represents a number of sample images.

22. The model training apparatus according to claim 17 or 18, the first sub-model comprises a first convolution layer, and a second convolution layer and a third convolution layer connected to the first convolution layer respectively, the second convolution layer is connected to the second sub-model, and the first processing module is specifically used to: input the target feature map into the first convolution layer for convolution processing to obtain a convolution processing result output by the first convolution layer, the first convolution layer comprises one convolution layer; input the convolution processing result into the second convolution layer and the third convolution layer respectively for convolution processing to obtain a classification feature map output by the second convolution layer and a regression feature map and an object feature map output by the third convolution layer, the second convolution layer comprises a plurality of convolution layers connected in cascade, and the third convolution layer comprises a plurality of convolution layers connected in cascade.

23. The model training apparatus according to claim 17 or 18, the third processing module is specifically used to: adjust parameters of the image recognition model according to the target loss value, and perform iterative training on the image recognition model until a preset iteration number is reached.

24. The model training apparatus according to claim 23, the third processing module is further used to: after the preset iteration number is reached, perform moving average processing on the initial parameters corresponding to the current iteration number based on the preset iteration number, the current iteration number, the initial parameters corresponding to the current iteration number and the parameters obtained by training the image recognition model last time, to obtain target parameters corresponding to the current iteration number; and perform iterative training on the image recognition model according to the target parameters corresponding to the current iteration number, to obtain the trained image recognition model.

25. The model training apparatus according to claim 17 or 18, the sample image comprises a first image and a second image, the first image is an image collected in a real scene, and the second image is an image generated by using a preset generation manner, the preset generation manner comprises at least one of adding a background, adding noise and combining image elements. ​ 26. The model training apparatus of claim 17 or 18, and the obtaining module is further configured to: perform enhancement processing on the sample image before obtaining the plurality of target feature maps corresponding to the sample image, and the enhancement processing comprises at least one of flipping, size adjustment, cropping, brightness adjustment, contrast adjustment, and adding noise.

27. The model training apparatus of claim 17 or 18, wherein the image recognition model further comprises a backbone network model and a feature pyramid network model, the feature pyramid network model is connected to the backbone network model, and the first sub-model is connected to the feature pyramid network model, and the obtaining module is specifically configured to: input the sample image into the backbone network model to perform feature extraction, to obtain a plurality of initial feature maps corresponding to the sample image, and the plurality of initial feature maps have different sizes; input the plurality of initial feature maps into the feature pyramid network model to perform feature fusion processing, to obtain the plurality of target feature maps corresponding to the sample image.

28. An image recognition apparatus, comprising: an obtaining module configured to obtain a plurality of target feature maps corresponding to a to-be-recognized image, and the plurality of target feature maps have different sizes; a processing module configured to input the plurality of target feature maps into a first sub-model of an image recognition model to perform recognition processing, to obtain an image recognition result output by the image recognition model, and the image recognition result is used to indicate whether the to-be-recognized image is a prohibited image, and the image recognition model is trained by using the model training method in any one of claims 1 to 11.

29. The image recognition apparatus of claim 28, wherein the obtaining module is specifically configured to: obtain the to-be-recognized image; perform preprocessing on the to-be-recognized image to obtain a preprocessed image, and the preprocessing comprises image normalization processing and / or image scaling processing; obtain the plurality of target feature maps according to the preprocessed image.

30. The image recognition apparatus of claim 29, wherein when the obtaining module is configured to obtain the plurality of target feature maps according to the preprocessed image, the obtaining module is specifically configured to: input the preprocessed image into a backbone network model of the image recognition model to perform feature extraction, to obtain a plurality of initial feature maps corresponding to the preprocessed image, and the plurality of initial feature maps have different sizes; input the plurality of initial feature maps into a feature pyramid network model of the image recognition model to perform feature fusion processing, to obtain the plurality of target feature maps.

31. The image recognition apparatus of claim 28, wherein the processing module is specifically configured to: input the plurality of target feature maps into a first sub-model of an image recognition model to perform recognition processing, to obtain a score of a target object contained in the to-be-recognized image; if the score is greater than a threshold value, obtain the image recognition result as the to-be-recognized image being a prohibited image; if the score is less than or equal to the threshold value, obtain the image recognition result as the to-be-recognized image not being a prohibited image.

32. The image recognition apparatus according to any one of claims 28 to 31, wherein the image recognition result further comprises position information of a target object in the image to be recognized, and the position information is used to determine a position of a target contraband object in the contraband image.

33. A computing device comprising: a processor, and a memory connected to the processor in communication; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 16.

34. A storage medium, wherein the storage medium stores computer program instructions, and the computer program instructions, when executed, implement the method according to any one of claims 1 to 16.

35. A computer program product, comprising a computer program, and the computer program, when executed by a processor, implements the method according to any one of claims 1 to 16.

Images