Image recognition method, device, computer equipment, storage medium and product

Abstract

The embodiments of the present application disclose an image recognition method, apparatus, computer equipment, storage medium, and product, which obtain a first image sample; mask at least one image region in the first image sample to obtain a processed image sample; extract image semantic association features from each masked image region in the processed image sample using an image feature extraction model to obtain regional feature information corresponding to each masked image region; and train the image feature extraction model based on the regional labels and regional feature information to obtain a trained image feature extraction model, so that the image feature extraction model can be used to perform image recognition on the image to be recognized. This solution extracts image semantic association features based on image regions, which can reduce information redundancy brought by other image regions, and sets corresponding masked image regions for the current training stage to improve the feature extraction capability of the image feature extraction model.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an image recognition method, apparatus, computer equipment, storage medium and product. Background Art

[0002] Adversarial defense against physical attacks is a crucial component of facial recognition security systems. However, the existence of adversarial attack techniques poses a significant threat to facial recognition security systems based on AI models. Adversarial attacks are a technique that applies minimal (often imperceptible to the naked eye) perturbations to an image, causing the model to misjudge the image's content.

[0003] Existing defense methods against physical attacks fall into two main categories: empirical defense and credible defense. Empirical defense is a targeted defense based on an understanding of existing attacks. For example, pre- or post-processing can be used to modify pixel values in the attack area to disrupt the attack's effectiveness. Credible defense, on the other hand, ensures a 100% defense success rate regardless of the attacker's method of generating the physical attack, as long as certain conditions are met. Common methods include designing new network structures and limiting the model's receptive field.

[0004] Empirical defenses are powerless against adaptive attacks. When attackers fully understand the defense system (including defense models, parameters, added defense strategies, etc.), they can design corresponding adaptive attacks. Empirical defenses cannot modify adaptive attacks, resulting in poor image content recognition performance.

[0005] Credibility defense uses modeling based on convolutional neural networks (CNNs). CNNs usually only receive complete image inputs. When processing masked images, CNNs will be interfered by the large amount of redundant information generated by the masked area, which directly affects the model's recognition of image content. Summary of the Invention

[0006] Embodiments of the present application provide an image recognition method, apparatus, computer device, storage medium, and product to improve the feature extraction capability of an image feature extraction model.

[0007] An image recognition method provided in an embodiment of the present application includes:

[0008] Acquire a first image sample, where the first image sample includes at least one image region and a region label corresponding to each image region;

[0009] performing mask processing on at least one image region in the first image sample to obtain a processed image sample, wherein the processed image sample includes at least one masked image region;

[0010] Performing image semantic association feature extraction on each mask image region in the processed image sample through an image feature extraction model to obtain regional feature information corresponding to each mask image region;

[0011] The image feature extraction model is trained based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model.

[0012] Accordingly, an embodiment of the present application further provides an image recognition device, comprising:

[0013] an acquiring unit, configured to acquire a first image sample, where the first image sample includes at least one image region and a region label corresponding to each image region;

[0014] a masking unit, configured to perform masking on at least one image region in the first image sample to obtain a processed image sample, wherein the processed image sample includes at least one masked image region;

[0015] a feature extraction unit, configured to extract image semantic association features from each mask image region in the processed image sample using an image feature extraction model, and obtain region feature information corresponding to each mask image region;

[0016] A training unit is used to train the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model.

[0017] Correspondingly, an embodiment of the present application also provides a computer device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any image recognition method provided in the embodiment of the present application.

[0018] Accordingly, an embodiment of the present application further provides a computer-readable storage medium, which is used to store a computer program, and the computer program is loaded by a processor to execute any image recognition method provided in the embodiment of the present application.

[0019] Accordingly, an embodiment of the present application further provides a computer program product, including a computer program, which implements any image recognition method provided in the embodiment of the present application when executed by a processor.

[0020] The embodiment of the present application obtains a first image sample, which includes at least one image area and a region label corresponding to each image area; performs mask processing on at least one image area in the first image sample to obtain a processed image sample, which includes at least one masked image area; performs image semantic association feature extraction on each masked image area in the processed image sample through an image feature extraction model to obtain region feature information corresponding to each masked image area; trains the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model.

[0021] This scheme extracts image semantic association features based on image regions, which can reduce the information redundancy brought by other image regions. It also sets the corresponding mask image region for the current training stage, and extracts image semantic association features from the mask image region through the image feature extraction model. The mask image region is restored and reconstructed based on the non-mask image region, and the image feature extraction model is self-supervised trained to improve the feature extraction capability of the image feature extraction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a scene graph of the image recognition method provided in an embodiment of the present application;

[0024] Figure 2 is a flowchart of the image recognition method provided in an embodiment of the present application;

[0025] Figure 3 is another flow chart of the image recognition method provided in an embodiment of the present application;

[0026] Figure 4 This is a schematic diagram of generating region labels provided by an embodiment of the present application;

[0027] Figure 5 This is a schematic diagram of the model training architecture provided in the embodiment of this application

[0028] Figure 6 is a schematic diagram of an image recognition device provided in an embodiment of the present application;

[0029] Figure 7It is a schematic diagram of the structure of the terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0031] The present invention provides an image recognition method, apparatus, computer device, and computer-readable storage medium. The image recognition apparatus can be integrated into a computer device, which can be a server or a terminal.

[0032] The terminal may include a mobile phone, a wearable smart device, a tablet computer, a laptop computer, a personal computer (PC), and a vehicle-mounted computer.

[0033] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, blockchain, big data and artificial intelligence platforms.

[0034] For example, Figure 1 As shown, a computer device obtains a first image sample, which includes at least one image area and a region label corresponding to each image area; performs mask processing on at least one image area in the first image sample to obtain a processed image sample, which includes at least one masked image area; performs image semantic association feature extraction on each masked image area in the processed image sample through an image feature extraction model to obtain region feature information corresponding to each masked image area; trains the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so that image recognition can be performed on the image to be recognized through the image feature extraction model.

[0035] This scheme extracts image semantic association features based on image regions, which can reduce the information redundancy brought by other image regions. It also sets the corresponding mask image region for the current training stage, and extracts image semantic association features from the mask image region through the image feature extraction model. The mask image region is restored and reconstructed based on the non-mask image region, and the image feature extraction model is self-supervised trained to improve the feature extraction capability of the image feature extraction model.

[0036] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0037] This embodiment will be described from the perspective of an image recognition device. The image recognition device may be integrated into a computer device, which may be a server or a terminal.

[0038] An image recognition method provided in an embodiment of the present application is as follows: Figure 2 As shown, the specific process of the image recognition method can be as follows:

[0039] 101. Obtain a first image sample, where the first image sample includes at least one image region and a region label corresponding to each image region.

[0040] The first image sample may include an object to be identified and an image sample containing a region label. For example, the first image sample may include text, pedestrians, faces, and objects. The trained image feature extraction model obtained by training based on first image samples containing different objects to be identified can be applied to different scenarios, for example, text recognition, pedestrian detection, object recognition, and face recognition.

[0041] The image region may be a region obtained by dividing the first image sample into image regions, for example, the first image sample may be equally divided to obtain several image regions of the same size, or the first image sample may be randomly divided to obtain multiple image regions.

[0042] The region label may be a label corresponding to an image region, and may be used to represent the image content contained in the image region.

[0043] For example, the first image sample may be obtained from a database. Optionally, the first image sample may also be stored in a blockchain or other storage locations, which is not limited here.

[0044] The region label corresponding to each image region in the first image sample may be manually pre-marked, or may be obtained by image encoding the image region, i.e., the step of "obtaining the first image sample" may specifically include:

[0045] Get the initial image sample;

[0046] Performing image region division on the initial image sample to obtain multiple image regions;

[0047] Performing image coding processing on each image region in the initial image sample to obtain a region label corresponding to each image region in the initial image sample;

[0048] A first image sample is generated according to the multiple image regions and the region label corresponding to each image region.

[0049] The initial image sample may be an image sample containing an object to be identified.

[0050] For example, the method may specifically involve obtaining an initial image sample from a database or a blockchain, dividing the initial image sample into image regions based on preset division rules, dividing the initial image sample into multiple image regions, and performing image coding processing on each image region in the initial image sample to obtain a hidden coding feature of the initial image sample, where the hidden coding feature includes a region label corresponding to each image region.

[0051] It is understandable that an initial image sample can be obtained by performing image decoding processing based on the region label corresponding to each image region. Image encoding processing of the initial image sample can be implemented by a trained image coding model. That is, in one embodiment, the step of "performing image encoding processing on each image region in the first image sample to obtain a region label corresponding to each image region in the first image sample" can specifically include:

[0052] Perform image coding processing on each image region in the first image sample through the trained image coding extraction model to obtain a region label for each image region.

[0053] The trained image coding model may be a model for performing image coding processing on an initial image sample to obtain hidden coding features as region labels.

[0054] For example, the initial image sample may be input into a trained image coding model, and the trained image coding model may be used to perform image coding processing on the initial image sample to obtain hidden coding features corresponding to each image region of the initial image sample.

[0055] Optionally, the trained image coding model may be a decoding network in a trained variational auto-encoder (VAE) model. The trained variational coding model may be trained based on the second image sample. That is, in one embodiment, before the step of “obtaining the initial image sample,” the image recognition method provided in the embodiment of the present application may further include:

[0056] obtaining a second image sample;

[0057] performing image coding processing on the second image sample by using the image coding model to obtain image coding information of the second image sample;

[0058] Performing image restoration processing on the image coding information to obtain a restored image corresponding to the second image sample;

[0059] The image coding model is trained based on the second image sample and the restored image to obtain a trained image coding model.

[0060] The second image sample may be a sample for training an image coding model, and the second image sample may be the same as or different from the initial image sample or the first image sample.

[0061] For example, the image coding model may include an encoding network and a decoding network. The second image sample is input into the image coding model, and the second image sample is subjected to image coding processing through the image coding network to obtain image coding information of the second image sample. The image coding information is decoded through the decoding network to perform image restoration processing based on the image coding information to obtain a restored image corresponding to the second image sample. The loss function is calculated based on the difference between the second image sample and the restored image, and the network parameters of the image coding model are updated based on the loss function. When the image coding model converges, the decoding network is removed, and the decoding network is retained as the trained image coding model.

[0062] 102. Perform mask processing on at least one image region in the first image sample to obtain a processed image sample, where the processed image sample includes at least one masked image region.

[0063] The mask processing may be to cover the image area to retain a partial image area of the first image sample, that is, a local image area.

[0064] For example, the pixel value of each pixel in at least one image area in the first image sample may be replaced by a fixed value of 0, or other values, such as 225, etc. The image area after mask processing is the mask image area. After performing mask processing on at least one image area in the first image sample, a processed image sample can be obtained.

[0065] In order to improve the feature extraction capability of the image feature extraction model for local areas, multiple training stages may be provided to perform multi-stage progressive training on the image feature extraction model. Different training stages correspond to different mask image areas. Correspondingly, the image sizes of the local area images retained by the first image sample are different, thereby giving the image feature extraction model multi-scale information. This makes it easier for the image feature extraction model to learn regional feature information of the local image area based on local image areas of different image sizes, thereby achieving better image recognition effects. That is, in one embodiment, the step of "masking at least one image area in the first image sample to obtain a processed image sample" may specifically include:

[0066] Determining a region to be masked in the first image sample according to a current training stage of the image feature extraction model, wherein the current training stage is a training stage in a progressive training stage;

[0067] Masking is performed on the to-be-masked area in the first image sample according to the current training stage to obtain a processed image sample.

[0068] Among them, the image feature extraction model can be a neural network model for extracting image features. The image feature extraction network can be a neural network model based on the visual transformer architecture. The visual transformer crops the input image into a sequence of image regions for processing. Due to the continuity of the image content, there is a certain semantic correlation between each image region. Therefore, when part of the image area is covered, the covered image area can theoretically be restored and reconstructed through the uncovered image area. The restoration and reconstruction of the covered image area is an effective self-supervised training method. If the image feature extraction model can learn the local feature information of the uncovered image area, the covered image area can be restored and reconstructed.

[0069] Among them, the progressive training stage may include multiple training stages, the current training stage may be a training stage determined from the progressive training stage, the current training stage may be the training stage currently described by the image feature extraction model, and each training stage corresponds to an area to be masked.

[0070] For example, specifically, the image area that needs to be masked, i.e., the area to be masked, in the image area of the first image sample may be determined according to the current training stage, and the area to be masked of the first image sample may be masked to obtain a processed image sample.

[0071] For example, the progressive training phase includes three training phases: a first training phase, a second training phase, and a third training phase. The first training phase, the second training phase, and the third training phase correspond to different mask image regions. The first training phase can mask 40% of the image region in the first image sample, the second training phase can mask 70% of the image region in the first image sample, and the third training phase can mask 90% of the image region in the first image sample. Image feature extraction is trained sequentially based on the processed image samples obtained in the first training phase, the second training phase, and the third training phase to gradually improve the feature extraction capability of the image feature extraction model.

[0072] 103. Perform image semantic association feature extraction on each mask image region in the processed image sample through the image feature extraction model to obtain regional feature information corresponding to each mask image region.

[0073] For example, due to the content continuity of the first image sample, each image area in the first image sample has a certain semantic correlation. For example, in an image containing a face to be identified, the image area where the eyes are located and the image area where the hair is located have a certain content correlation, that is, they have semantic correlation. Therefore, based on the semantic correlation between the local image area and the mask image area to be masked, image semantic correlation features can be extracted from the mask image area to obtain regional feature information corresponding to each mask image area.

[0074] The region label can represent the image content contained in the image region. The region label can be a numerical value in a data interval. The region feature information can represent the probability that the mask image region is each numerical value in the data interval, that is, the image feature extraction model predicts the probability that the mask image region is each data in the data interval based on the local image region.

[0075] Specifically, local features can be extracted from the local image region, and the mask image region can be restored and reconstructed based on the local feature information of the local image region. Restoring and reconstructing the mask image region is an effective self-supervised training method. Since the image regions in the first image sample have semantic associations, if the image feature extraction model can accurately learn the local feature information of the local image region, the mask image region can be accurately restored and reconstructed. That is, in one embodiment, the step of "extracting image semantic association features from each mask image region in the processed image sample using the image feature extraction model to obtain regional feature information corresponding to each mask image region" can specifically include:

[0076] Performing local feature extraction on each local image region in the first image sample using an image feature extraction model to obtain local feature information corresponding to each local image region;

[0077] Based on the local feature information, local information joint processing is performed on each mask image region in the first image sample to obtain regional feature information corresponding to each mask image region.

[0078] For example, the local feature extraction model may be used to extract local features from the local image area in the first image sample to extract feature information in the local image area that can identify the content contained in the first image sample, thereby obtaining local feature information corresponding to the local image area.

[0079] Based on the content relevance between the local image area and the mask image area, the positional relationship between the local image area and the mask image area, and the local feature information corresponding to each local image area, local information is combined to obtain the regional feature information corresponding to the mask image area.

[0080] Optionally, all unit image areas contained in the local image area can be obtained through the image feature extraction model. If the unit image area can be a strip image area with a length of a, then the unit image areas can be obtained one by one from the local image area with a moving step of 1 pixel to obtain all unit image areas contained in the local image area, that is, the unit image areas can be mutually crossed. It can be understood that the unit image areas may not be mutually crossed.

[0081] Local feature extraction is performed on each unit image region to obtain sub-local feature information of each unit image region, and multiple sub-local feature information are used as local feature information of the local image region.

[0082] 104. The image feature extraction model is trained based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model.

[0083] For example, the region label corresponding to each image region in the first image sample is a discrete value in the interval [1,8192], and the region feature information of the mask image region is an 8192-dimensional feature vector, where each dimension represents the probability that the mask image region is the corresponding value in the interval [1,8192].

[0084] The loss is calculated based on the region label of the mask image region and the region feature information, and back propagation is performed to train the image feature extraction model. A variety of loss functions can be used to calculate the loss, such as the softmax loss function, the weighted cross entropy loss function, and the cross entropy loss function, which are not limited here.

[0085] The image feature extraction model extracts image semantic association features from the mask image region to obtain regional feature information that represents the predicted probability of the mask image region being each value in the interval. This is similar to the classification process, so a cross-entropy loss function can be used to calculate the loss of the image feature extraction model. That is, in one embodiment, the step of "training the image feature extraction model based on the region label and the regional feature information" can specifically include:

[0086] Calculate cross entropy loss based on regional feature information and regional labels;

[0087] Gradient backpropagation is performed based on cross entropy loss to train the image feature extraction model.

[0088] For example, the cross entropy loss L can be calculated based on the following formula of the cross entropy loss function, regional feature information and regional labels: CE Among them, M is the classification category, that is, the number of dimensions contained in the regional feature information, N is the number of mask image regions, y ic is a symbolic function. If the region label of the mask image region is c, then y ic =1, otherwise y ic =0, p ic is the predicted probability that the mask image region belongs to type c.

[0089]

[0090] Different training stages correspond to different local image regions and mask image regions. When the number of local image regions is small and the number of mask image regions is large, the regional feature information of the mask image region is predicted based on the local image region. The mask image region that is farther away from the local image region has weaker content continuity and relatively lower prediction accuracy. Updating the network parameters of the image extraction model based on the regional feature information and regional labels of such mask image regions will make it difficult for the image extraction model to converge. Therefore, corresponding target mask image regions can be set for different training stages, and the image feature extraction model can be trained based on the regional feature information and regional labels of the target mask image region. That is, in one embodiment, the step of "training the image feature extraction model based on the regional labels and regional feature information" can specifically include:

[0091] determining a target mask image region from the mask image region according to the current training stage;

[0092] The image feature extraction model is trained based on the region label corresponding to the target mask image region and the region feature information corresponding to the target mask image region.

[0093] The target mask image region may be a partial image region determined from the mask image region according to the current training stage.

[0094] For example, specifically, a mask image region adjacent to the local image region corresponding to the current training stage may be determined as the target mask image region.

[0095] The image feature extraction model is trained based on the regional feature information and regional labels corresponding to the target mask image area.

[0096] In addition to updating the network parameters of the image feature extraction model for restoration and reconstruction of the masked image region, the image feature extraction model may also be trained based on the content contained in the first image sample. That is, in one embodiment, the step of "training the image feature extraction model based on the region label and region feature information" may further include:

[0097] Determining a region prediction result for each local image region based on local region feature information corresponding to the local image region in the processed image sample;

[0098] determining an image prediction result of the first image sample according to the regional prediction result of each local region;

[0099] The image feature extraction model is trained according to the image prediction results and the image labels, and the image feature extraction model is trained based on the region labels and the region feature information to obtain a trained image feature extraction model.

[0100] For example, the image classification task can be specifically performed through a fully connected layer, and the local feature information of each local image area is classified and processed through the fully connected layer to determine the category to which each local image area belongs, and obtain the regional prediction result. The type of the first image sample can be flexibly set according to the needs of the application scenario, such as being applied to scenarios for identifying different objects, and for example, being applied to scenarios for face recognition. In one embodiment, it can be applied to scenarios for face recognition verification during payment of instant messaging software, and can also be applied to access control systems, that is, the classification categories can be different objects.

[0101] The number of times each category is predicted is determined according to the regional prediction result of each local image region, the type of the first image sample is determined by the category with the largest number of predictions, and the image prediction result of the first image sample is obtained.

[0102] The image feature extraction model is trained based on the image prediction results and image labels, and the image feature extraction model is trained based on the region labels and region feature information to obtain a trained image feature extraction model.

[0103] In one embodiment, when training the image feature extraction model based on the progressive training stage, the image feature extraction model needs to complete one training stage before entering the next training stage, and the number of local region images included in the first image sample is gradually reduced layer by layer, so that the image feature extraction model can extract more accurate local feature information. That is, in one embodiment, the step of "training the image feature extraction model based on the region label and the region feature information" includes:

[0104] In the current training phase, the image feature extraction model is trained based on the region labels and region feature information to obtain a candidate image feature extraction model;

[0105] determining an associated training phase of the current training phase from among the progressive training phases;

[0106] The associated training stage is determined as the current training stage, and the mask processing of at least one image area in the first image sample according to the current training stage is returned to obtain a processed image sample corresponding to the current training stage, until the image feature extraction model converges to obtain a trained image feature extraction model.

[0107] For example, specifically, in the current training phase, the image feature extraction model is trained based on the region labels and region feature information. When a preset training condition is met, such as when the number of training cycles meets a preset number of times, or when the loss function of the image feature extraction model converges to a preset value, a candidate image feature extraction model is obtained. At this time, an associated training phase of the current training phase is determined from the progressive training phases. The progressive training phase may include multiple training phases with a preset training order, and the associated training phase of the current training phase may be a training phase that is subsequent to the current training phase in the training order.

[0108] The associated training stage is used as the current training stage, and masking is performed on at least one image region in the first image sample according to the current training stage to obtain a processed image sample corresponding to the current training stage. When training for the training stage is completed, the image feature extraction model is trained for the next training stage until the image feature extraction model completes all training stages included in the progressive training stage or converges to an optimal solution, thereby obtaining a trained feature extraction model.

[0109] After obtaining the trained feature extraction model, image recognition can be performed based on the trained feature extraction model. That is, the image recognition method provided in the embodiment of the present application may further include:

[0110] Obtain the image to be recognized;

[0111] Divide the image to be recognized into image regions to obtain multiple image regions;

[0112] Through the trained image feature extraction model, local feature extraction is performed on multiple image regions to obtain the local feature information corresponding to each image region;

[0113] Based on the local feature information corresponding to each image region, an image recognition result of the image to be recognized is determined.

[0114] The image to be recognized may be an image that needs to be subjected to image recognition, for example, an image for face recognition, or an image for object recognition.

[0115] For example, a method may be used to obtain an image to be identified, perform image region division on the image to be identified, obtain multiple image regions (for example, using a moving step of one pixel to gradually divide image regions of a fixed width b from the image to be identified, to obtain all image regions contained in the image to be identified), perform local feature extraction on the multiple image regions, and obtain local feature information corresponding to each image region. The image regions are then classified based on the local feature information of each image region to determine a predicted classification result for each image region.

[0116] According to the predicted classification results of each image region, the category predicted by the most image regions in the image to be identified is determined as the category of the image to be identified. In the face recognition scenario, the predicted classification type can be different objects.

[0117] The image recognition method provided in the embodiment of the present application is reasonable, and the specific reasoning process is as follows:

[0118] When performing image recognition on an image, the image is first divided into regions using the trained feature extraction model, and all available strip image regions in the image are taken out (the strip regions can overlap with each other). The classifier model is then used to classify and identify each strip image region based on the local feature information of each strip image region, determine the category of each strip image region, and perform a majority vote on the classification results of all the strip image regions of the image to obtain the classification result of the image.

[0119] Assume that the image is x, the width of the strip image area is b, the starting position of the strip is p, the parameters of the model are θ, and the classifier model is f c (x,b,p,θ), where c represents the probability of the true category, represents the number of strip image regions classified as category c (where w is the width of the image), and the width of the attack image is m. Since the widths of b and m are fixed, we can calculate the maximum number of strip image regions that can be attacked as Δ = m + b - 1 (here we consider the worst case, that is, as long as the strip image region intersects with the attack image, it will be attacked, resulting in classification error). In this case, as long as the conditions of formula (1) are met, it can be considered that the image is 100% unbreakable, and the image feature extraction model has achieved credibility defense for this image. Specifically, the maximum difference in the number of strip image regions that the attack image can fill is 2Δ (attacking Δ regions of the highest class, classifying them as the second highest class). The highest class represents the category of potato with the most strip image regions. Therefore, when formula (1) is met, no matter how the attacker operates, c will always be the highest class, thus ensuring that the model can successfully defend 100%.

[0120] Formula (1):

[0121] The embodiment of the present application obtains a first image sample, which includes at least one image area and a region label corresponding to each image area; performs mask processing on at least one image area in the first image sample to obtain a processed image sample, which includes at least one masked image area; performs image semantic association feature extraction on each masked image area in the processed image sample through an image feature extraction model to obtain region feature information corresponding to each masked image area; trains the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model.

[0122] This scheme extracts image semantic association features based on image regions, which can reduce the information redundancy brought by other image regions. It also sets the corresponding mask image region for the current training stage, and extracts image semantic association features from the mask image region through the image feature extraction model. The mask image region is restored and reconstructed based on the non-mask image region, and the image feature extraction model is self-supervised trained to improve the feature extraction capability of the image feature extraction model.

[0123] Based on the above embodiments, further detailed description will be given below with examples.

[0124] This embodiment will be described from the perspective of an image recognition device. The image recognition device may be integrated into a computer device, which may be a server or a terminal.

[0125] An image recognition method provided in an embodiment of the present application is as follows: Figure 3As shown, the specific process of the image recognition method can be as follows:

[0126] 201. The server obtains an initial image sample.

[0127] For example, the server may obtain an initial image sample from a database or a blockchain.

[0128] 202. The server performs feature coding processing on each image region in the initial image sample to obtain a region label corresponding to each image region, thereby obtaining a first image sample.

[0129] For example, it can be as follows Figure 4 As shown, the server divides the initial image sample into image regions based on a preset division rule. For example, the initial image sample is divided into nine equal parts to obtain 9 image regions, or the initial image is divided into more image regions. It can be flexibly adjusted according to application needs and is not limited here. The initial image sample is divided into multiple image regions, and image coding processing is performed on each image region in the initial image sample. Image coding processing is to eliminate the redundancy of the initial image sample and achieve the image information contained in the initial image sample with less data. The hidden coding features corresponding to the initial image sample can be obtained through image coding processing, that is, Figure 4 The hidden coding features can be used to represent the image information contained in the initial image sample. The original image sample can be restored by decoding based on the hidden coding features.

[0130] The value of each position in the matrix corresponds to each image region in the initial image sample, and the value of each position in the matrix serves as the region label of each image region. The value contained in the hidden coding feature can be a discrete value between the value range (1,8192).

[0131] Optionally, the size of the hidden coding feature extracted can be preset (matrix size), and image coding and dimensionality reduction processing can be performed on the initial image sample based on the size of the hidden coding feature to obtain the hidden coding feature corresponding to the initial image sample.

[0132] A first image sample is obtained according to the initial image sample and the region label corresponding to each image region in the initial image sample.

[0133] Optionally, the server may perform image encoding processing on each image region in the initial image sample through an encoder in a trained variational auto-encoder (VAE) to obtain a region label corresponding to each image region.

[0134] Since the initial image sample can be restored based on the hidden coding features, if the local feature information extracted by the image feature extraction model is the same as the hidden coding features, then the corresponding local image area can be restored based on the local feature information. The image feature extraction model accurately extracts the local image feature information of the local image area. Therefore, the hidden coding features can be used as the region label of each image area.

[0135] 203. The server performs mask processing on the first image sample based on the current training stage to obtain a processed image sample.

[0136] For example, the progressive training stage includes three training stages, namely the first training stage, the second training stage and the third training stage. The first training stage, the second training stage and the third training stage correspond to different mask image areas, respectively. The first training stage can mask 40% of the image area in the first image sample and retain 60% of the image area in the first image sample; the second training stage can mask 70% of the image area in the first image sample and retain 30% of the image area in the first image sample; the third training stage can retain a strip area of a preset width in the first image sample and mask the other image areas.

[0137] The strip areas of preset width are obtained by cropping the image to be identified during the application of the trained image feature extraction model, so that the widths of the multiple strip areas are the same.

[0138] like Figure 5 As shown, the server determines the area to be masked according to the current training stage, performs mask processing on the area to be masked to obtain processed image samples, and determines the target mask image area according to the current training stage.

[0139] 204. The server extracts image semantic association features from each target mask image region based on the local image region in the processed image sample to obtain regional feature information corresponding to each target mask image region.

[0140] For example, the server may input the processed image sample into an image feature extraction model, and the image feature extraction model may be used to crop the processed image sample into multiple image regions. For example, the processed image sample may be cropped into multiple strip image regions of the same size according to a preset width, thereby obtaining an image region sequence for the processed image sample. The image feature extraction model may be used to perform local feature extraction on a local image region in the processed image sample to extract feature information in the local image region that can identify the content contained in the processed image sample, thereby obtaining local feature information corresponding to the local image region.

[0141] Due to the continuity of the image content of the processed image samples, there is a certain semantic correlation between each image area. Therefore, when part of the image area is covered, it is theoretically possible to restore and reconstruct the covered image area through other uncovered image areas.

[0142] The amount of feature information that the image feature extraction model needs to extract is different when determining the category of an image based on the complete image and when determining the type of an image based on the image region. Therefore, the convergence of the image feature extraction model can be accelerated by gradually increasing the size of the mask image region and reducing the size of the local image region, thereby gradually reducing the difference between the complete image and the image region.

[0143] The farther the image area is from the local image area, the weaker its semantic association with the local image area, and the more difficult it is to restore and reconstruct based on the local image area. Therefore, the corresponding restoration and reconstruction area, that is, the target mask area, can be set according to the size of the local image area.

[0144] The server determines the mask image region adjacent to the local image region corresponding to the current training stage as the target mask image region. Figure 5 As shown, the local image area in the processed image sample of the first training stage is 60% of the image area. The target mask image area can be set to all mask image areas in the processed image sample, and the mask image area is restored and reconstructed based on the local image area.

[0145] The local image area in the processed image samples of the second training stage is 60% of the image area. The target mask image area can be set to all mask image areas in the processed image samples, and the mask image area is restored and reconstructed based on the local image area.

[0146] The local image area in the processed image sample of the third training stage is a strip image area of preset width, and 20% of the mask image area adjacent to the local image area in the processed image sample can be determined as the target mask image area ( Figure 5 The target mask image area is restored and reconstructed based on the local image area.

[0147] It is understandable that the above is only an achievable embodiment, and the size of the local image area corresponding to different progressive training stages and the size of the target mask image area can be flexibly adjusted according to actual application conditions, and are not limited here.

[0148] Based on the content relevance between the local image area and the mask image area, the positional relationship between the local image area and the mask image area, and the local feature information corresponding to each local image area, image semantic association features are extracted to obtain the regional feature information corresponding to the mask image area.

[0149] Optionally, when the image feature extraction model is applied, the category of the image is predicted based on an entire image. The initial image feature extraction model can be trained using an entire image to obtain an image feature extraction model. The image feature extraction model is then fine-tuned using the processed image samples after masking to improve the image feature extraction capability of the image feature extraction model for local image areas, thereby obtaining a trained image feature extraction model.

[0150] 205. The server determines an image prediction result of the first image sample according to the local region feature information corresponding to the local image region in the processed image sample.

[0151] For example, the server may perform image classification tasks through a fully connected layer, classify the local feature information of each local image area through the fully connected layer, determine the category to which each local image area belongs, and obtain a regional prediction result. The type to which the first image sample belongs can be flexibly set according to the needs of the application scenario and applied to face recognition scenarios, that is, the classified categories can be different objects.

[0152] The number of times each category is predicted is determined according to the regional prediction result of each local image region, the type of the first image sample is determined by the category with the largest number of predictions, and the image prediction result of the first image sample is obtained.

[0153] 206. The server trains the image feature extraction model based on the region label and the region feature information, and trains the image feature extraction model based on the image prediction result and the image label.

[0154] For example, the server may calculate the loss based on the region label of the mask image region and the region feature information, and perform back propagation to train the image feature extraction model; calculate the loss based on the image prediction result and the image label, and perform back propagation to train the image feature extraction model.

[0155] It can be understood that the mask image area may include multiple sub-image areas, each sub-image area corresponds to sub-area feature information, the area label includes the corresponding labels of the multiple sub-image areas in the mask image area, and back propagation is performed based on the loss between the sub-area feature information of each sub-image area and the corresponding label.

[0156] 207. When the preset training condition is met, the server determines the associated training stage of the current training stage from the progressive training stages to obtain the target training stage.

[0157] For example, specifically, in the current training stage, the server trains the image feature extraction model based on the region label and region feature information. When the preset training conditions are met, for example, the number of training times meets the preset number of times, or when the loss function of the image feature extraction model converges to a preset value, a candidate image feature extraction model is obtained. At this time, the associated training stage of the current training stage is determined from the progressive training stage.

[0158] 208. The server uses the target training stage as the current training stage and returns to execute step 203 until the image feature extraction model converges to obtain a trained image feature extraction model.

[0159] For example, the server may specifically set the associated training stage as the current training stage and return to perform masking processing on at least one image region in the first image sample according to the current training stage to obtain a processed image sample corresponding to the current training stage. When the training of the training stage is completed, the image feature extraction model is trained in the next training stage until the image feature extraction model completes all training stages included in the progressive training stage or converges to an optimal solution, thereby obtaining a trained feature extraction model.

[0160] As can be seen from the above, the server of the embodiment of the present application obtains an initial image sample; performs feature encoding processing on each image area in the initial image sample to obtain a region label corresponding to each image area, and obtains a first image sample; based on the current training stage, performs mask processing on the first image sample to obtain a processed image sample; based on the local image area in the processed image sample, performs image semantic association feature extraction on each target mask image area to obtain region feature information corresponding to each target mask image area; determines the image prediction result of the first image sample according to the local region feature information corresponding to the local image area in the processed image sample; trains the image feature extraction model based on the region label and the region feature information, and trains the image feature extraction model according to the image prediction result and the image label to obtain a trained image feature extraction model; when the preset training conditions are met, the server determines the associated training stage of the current training stage from the progressive training stage to obtain the target training stage; takes the target training stage as the current training stage, returns to execute step 203 until the image feature extraction model converges, and obtains the trained image feature extraction model.

[0161] This scheme extracts image semantic association features based on image regions, which can reduce the information redundancy brought by other image regions. It also sets the corresponding mask image region for the current training stage, and extracts image semantic association features from the mask image region through the image feature extraction model. The mask image region is restored and reconstructed based on the non-mask image region, and the image feature extraction model is self-supervised trained to improve the feature extraction capability of the image feature extraction model.

[0162] In order to facilitate better implementation of the image recognition method provided in the embodiment of the present application, an image recognition device is also provided in one embodiment. The meanings of the terms are the same as those in the above-mentioned image recognition method, and the specific implementation details can be referred to the description in the method embodiment.

[0163] The image recognition device can be integrated into a computer device, such as Figure 6 As shown, the image recognition device may include: an acquisition unit 301, a mask unit 302, a feature extraction unit 303 and a training unit 304, specifically as follows:

[0164] (1) Acquisition unit 301: used to acquire a first image sample, where the first image sample includes at least one image region and a region label corresponding to each image region.

[0165] Optionally, the acquisition unit 301 may include a sample acquisition subunit, a region division subunit, an encoding subunit, and a sample generation subunit. Specifically:

[0166] Sample acquisition subunit: used to obtain initial image samples;

[0167] Region division subunit: used to divide the initial image sample into image regions to obtain multiple image regions;

[0168] Coding subunit: used to perform image coding processing on each image region in the initial image sample to obtain a region label corresponding to each image region in the initial image sample;

[0169] The sample generation subunit is configured to generate a first image sample according to a plurality of image regions and a region label corresponding to each image region.

[0170] The encoding subunit can also be used to:

[0171] Perform image coding processing on each image region in the first image sample through the trained image coding extraction model to obtain a region label for each image region.

[0172] Optionally, the image recognition device provided in the embodiment of the present application may further include a second sample acquisition unit, an image encoding unit, a restoration unit, and a model training unit. Specifically:

[0173] A second sample acquisition unit: configured to acquire a second image sample;

[0174] An image coding unit is configured to perform image coding processing on the second image sample through an image coding model to obtain image coding information of the second image sample;

[0175] Restoration unit: used for performing image restoration processing on the image coding information to obtain a restored image corresponding to the second image sample;

[0176] Model training unit: used to train the image coding model based on the second image sample and the restored image to obtain a trained image coding model.

[0177] (2) a masking unit 302 configured to perform masking on at least one image region in the first image sample to obtain a processed image sample, wherein the processed image sample includes at least one masked image region;

[0178] In one embodiment, the mask unit 302 may include a region determination subunit and a region mask subunit, specifically:

[0179] A region determination subunit is configured to determine a region to be masked in the first image sample according to a current training stage of the image feature extraction model, wherein the current training stage is a training stage in a progressive training stage;

[0180] The region mask subunit is configured to perform mask processing on the region to be masked in the first image sample according to the current training stage to obtain a processed image sample.

[0181] (3) Feature extraction unit 303: used to extract image semantic association features from each mask image region in the processed image sample using an image feature extraction model to obtain regional feature information corresponding to each mask image region;

[0182] In one embodiment, the step feature extraction unit 303 may include a local feature extraction subunit and a joint subunit, specifically:

[0183] A local feature extraction subunit is configured to extract local features from each local image region in the first image sample using an image feature extraction model to obtain local feature information corresponding to each local image region;

[0184] The joint subunit is used to perform local information joint processing on each mask image region in the first image sample based on the local feature information to obtain regional feature information corresponding to each mask image region.

[0185] (4) Training unit 304: used to train the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model.

[0186] In one embodiment, the training unit 304 may include a loss calculation subunit and a gradient back propagation subunit, specifically:

[0187] Loss calculation subunit: used to calculate cross entropy loss based on regional feature information and regional labels;

[0188] Gradient back propagation subunit: used to perform gradient back propagation based on cross entropy loss to train the image feature extraction model.

[0189] In one embodiment, the training unit 304 may include a target region determination subunit and a first model training subunit, specifically:

[0190] Target region determination subunit: used to determine the target mask image region from the mask image region according to the current training stage;

[0191] The first model training subunit is used to train the image feature extraction model based on the region label corresponding to the target mask image region and the region feature information corresponding to the target mask image region.

[0192] In one embodiment, the training unit 304 may include a first result prediction subunit, a second result prediction subunit, and a second model training subunit. Specifically:

[0193] A first result prediction subunit is configured to determine a region prediction result for each local image region based on local region feature information corresponding to the local image region in the processed image sample;

[0194] A second result prediction subunit: configured to determine an image prediction result of the first image sample according to the regional prediction result of each local region;

[0195] The second model training subunit is used to train the image feature extraction model based on the image prediction results and image labels, and to train the image feature extraction model based on the region labels and region feature information to obtain a trained image feature extraction model.

[0196] In one embodiment, the training unit 304 may include a third model training subunit, a stage determination subunit, and a loop subunit. Specifically:

[0197] The third model training subunit is used to train the image feature extraction model based on the region label and the region feature information in the current training phase to obtain a candidate image feature extraction model;

[0198] A stage determination subunit: used for determining the associated training stage of the current training stage from the progressive training stages;

[0199] Loop subunit: used to determine the associated training stage as the current training stage, and return to perform mask processing on at least one image area in the first image sample according to the current training stage to obtain a processed image sample corresponding to the current training stage, until the image feature extraction model converges to obtain a trained image feature extraction model.

[0200] Optionally, the image recognition device provided in the embodiment of the present application may further include an image acquisition unit, a segmentation unit, a local feature extraction unit, and a result determination unit. Specifically:

[0201] Image acquisition unit: used to acquire the image to be identified;

[0202] Division unit: used for dividing the image to be recognized into image regions to obtain multiple image regions;

[0203] Local feature extraction unit: used to extract local features from multiple image regions using the trained image feature extraction model to obtain local feature information corresponding to each image region;

[0204] Result determination unit: used to determine the image recognition result of the image to be recognized based on the local feature information corresponding to each image area.

[0205] The image recognition device of the embodiment of the present application obtains a first image sample, which includes at least one image area and a region label corresponding to each image area; performs mask processing on at least one image area in the first image sample to obtain a processed image sample, which includes at least one masked image area; performs image semantic association feature extraction on each masked image area in the processed image sample through an image feature extraction model to obtain region feature information corresponding to each masked image area; trains the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so that image recognition can be performed on the image to be recognized through the image feature extraction model.

[0206] This scheme extracts image semantic association features based on image regions, which can reduce the information redundancy brought by other image regions. It also sets the corresponding mask image region for the current training stage, and extracts image semantic association features from the mask image region through the image feature extraction model. The mask image region is restored and reconstructed based on the non-mask image region, and the image feature extraction model is self-supervised trained to improve the feature extraction capability of the image feature extraction model.

[0207] The embodiment of the present application also provides a computer device, which can be a terminal or a server. Figure 7 , which shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0208] The computer device may include one or more processing core processors 1001, one or more computer readable storage media memories 1002, a power supply 1003, an input unit 1004 and other components. Those skilled in the art will understand that Figure 7 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0209] Processor 1001 is the control center of the computer device. It connects the various components of the entire computer device using various interfaces and lines. By running or executing software programs and / or modules stored in memory 1002 and accessing data stored in memory 1002, it performs various functions of the computer device and processes data, thereby performing overall testing of the computer device. Optionally, processor 1001 may include one or more processing cores; preferably, processor 1001 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and computer programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 1001.

[0210] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0211] The computer device also includes a power supply 1003 for supplying power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 1003 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0212] The computer device may further include an input unit 1004, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0213] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1001 in the computer device will load the executable files corresponding to one or more computer program processes into the memory 1002 according to the following instructions, and the processor 1001 will run the computer programs stored in the memory 1002 to implement various functions as follows:

[0214] From the above, it can be seen that the computer device of the embodiment of the present application can obtain a first image sample, which includes at least one image area and a region label corresponding to each image area; perform mask processing on at least one image area in the first image sample to obtain a processed image sample, which includes at least one mask image area; perform image semantic association feature extraction on each mask image area in the processed image sample through an image feature extraction model to obtain region feature information corresponding to each mask image area; train the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model.

[0215] This scheme extracts image semantic association features based on image regions, which can reduce the information redundancy brought by other image regions. It also sets the corresponding mask image region for the current training stage, and extracts image semantic association features from the mask image region through the image feature extraction model. The mask image region is restored and reconstructed based on the non-mask image region, and the image feature extraction model is self-supervised trained to improve the feature extraction capability of the image feature extraction model.

[0216] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0217] This scheme extracts image semantic association features based on image regions, which can reduce the information redundancy brought by other image regions. It also sets the corresponding mask image region for the current training stage, and extracts image semantic association features from the mask image region through the image feature extraction model. The mask image region is restored and reconstructed based on the non-mask image region, and the image feature extraction model is self-supervised trained to improve the feature extraction capability of the image feature extraction model.

[0218] According to one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0219] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0220] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program can be loaded by a processor to execute any image recognition method provided in the embodiment of the present application.

[0221] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0222] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0223] Since the computer program stored in the computer-readable storage medium can execute any image recognition method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image recognition method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0224] The above is a detailed introduction to an image recognition method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image recognition method, characterized in that: include: Acquire a first image sample, where the first image sample includes at least one image region and a region label corresponding to each image region; performing mask processing on at least one image region in the first image sample to obtain a processed image sample, wherein the processed image sample includes at least one masked image region; Performing image semantic association feature extraction on each mask image region in the processed image sample through an image feature extraction model to obtain regional feature information corresponding to each mask image region; Training the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model; The image feature extraction model includes multiple training stages, and the multiple training stages are used to perform multi-stage progressive training on the image feature extraction model. Different training stages correspond to different mask image areas, so that the image sizes of the local area images retained by the first image sample in different training stages are different.

2. The method according to claim 1, characterized in that The performing mask processing on at least one image region in the first image sample to obtain a processed image sample includes: Determining a region to be masked in the first image sample according to a current training stage of the image feature extraction model, wherein the current training stage is a training stage in a progressive training stage; Masking is performed on the to-be-masked area in the first image sample according to the current training stage to obtain a processed image sample.

3. The method according to claim 2, characterized in that The training of the image feature extraction model based on the region label and the region feature information includes: In the current training phase, the image feature extraction model is trained based on the region label and the region feature information to obtain a candidate image feature extraction model; determining an associated training phase of the current training phase from the progressive training phases; The associated training stage is determined as the current training stage, and the current training stage according to the image feature extraction model is returned to be executed to determine the area to be masked in the first image sample; the area to be masked in the first image sample is masked according to the current training stage to obtain a processed image sample, until the image feature extraction model converges to obtain a trained image feature extraction model.

4. The method according to claim 3, characterized in that The training of the image feature extraction model based on the region label and the region feature information in the current training phase includes: determining a target mask image region from the mask image region according to the current training stage; The image feature extraction model is trained based on the region label corresponding to the target mask image region and the region feature information corresponding to the target mask image region.

5. The method according to claim 1, wherein The at least one image region includes a local image region, and the image semantic association feature extraction is performed on each mask image region in the processed image sample by the image feature extraction model to obtain regional feature information corresponding to each mask image region, including: Performing local feature extraction on each local image region in the first image sample using the image feature extraction model to obtain local feature information corresponding to each local image region; Based on the local feature information, local information joint processing is performed on each mask image region in the first image sample to obtain regional feature information corresponding to each mask image region.

6. The method according to claim 5, characterized in that The first image sample corresponds to an image label, and the training of the image feature extraction model based on the region label and the region feature information includes: determining a region prediction result for each of the local image regions according to local region feature information corresponding to the local image regions in the processed image samples; determining an image prediction result of the first image sample according to the region prediction result of each of the local regions; The image feature extraction model is trained according to the image prediction result and the image label, and the image feature extraction model is trained based on the region label and the region feature information to obtain the trained image feature extraction model.

7. The method according to claim 1, characterized in that The obtaining of the first image sample comprises: Get the initial image sample; Performing image region division on the initial image sample to obtain a plurality of image regions; Performing image coding processing on each image region in the initial image sample to obtain a region label corresponding to each image region in the initial image sample; A first image sample is generated according to the multiple image regions and the region label corresponding to each image region.

8. The method according to claim 7, characterized in that The performing image coding processing on each image region in the first image sample to obtain a region label corresponding to each image region in the first image sample includes: Perform image coding processing on each image region in the first image sample using the trained image coding model to obtain a region label for each image region.

9. The method according to claim 8, characterized in that Before obtaining the initial image sample, the method further includes: obtaining a second image sample; performing image coding processing on the second image sample by using an image coding model to obtain image coding information of the second image sample; performing image restoration processing on the image coding information to obtain a restored image corresponding to the second image sample; The image coding model is trained based on the second image sample and the restored image to obtain a trained image coding model.

10. The method according to claim 1, characterized in that The training of the image feature extraction model based on the region label and the region feature information includes: Calculate cross entropy loss based on regional feature information and regional labels; Gradient back propagation is performed based on the cross entropy loss to train the image feature extraction model.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: Obtain the image to be recognized; Dividing the image to be identified into image regions to obtain a plurality of image regions; Performing local feature extraction on each of the plurality of image regions using a trained image feature extraction model to obtain local feature information corresponding to each image region; An image recognition result of the image to be recognized is determined based on the local feature information corresponding to each image region.

12. An image recognition device, characterized in that: include: an acquiring unit, configured to acquire a first image sample, where the first image sample includes at least one image region and a region label corresponding to each image region; a masking unit, configured to perform masking on at least one image region in the first image sample to obtain a processed image sample, wherein the processed image sample includes at least one masked image region; a feature extraction unit, configured to extract image semantic association features from each mask image region in the processed image sample using an image feature extraction model, and obtain region feature information corresponding to each mask image region; a training unit, configured to train the image feature extraction model based on the region label and the region feature information to obtain a trained image feature extraction model, so as to perform image recognition on the image to be recognized through the image feature extraction model; The image feature extraction model includes multiple training stages, and the multiple training stages are used to perform multi-stage progressive training on the image feature extraction model. Different training stages correspond to different mask image areas, so that the image sizes of the local area images retained by the first image sample in different training stages are different.

13. A computer device, characterized in that: The method comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the image recognition method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the image recognition method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image recognition method according to any one of claims 1 to 11 is implemented.

Images