Image recognition method, electronic device, and storage medium

Abstract

The application discloses an image recognition method, an electronic device and a storage medium. The image recognition method comprises the following steps: segmenting a to-be-recognized image into a plurality of image blocks, acquiring spatial information of each image block in the to-be-recognized image, and acquiring a recognition influencing factor contained in the to-be-recognized image; extracting associated information between the spatial information of each image block and the recognition influencing factor, obtaining an influence space associated feature corresponding to each image block; adjusting a weight parameter of each image block based on the influence space associated feature, obtaining image block weight information corresponding to the to-be-recognized image; and recognizing the to-be-recognized image based on the image block weight information, and obtaining an image recognition result. The application refines the adjustment granularity of the weight parameter, improves the accuracy of the weight parameter adjustment, can make the image recognition pay more attention to the information of a useful area, and obtain a more accurate image recognition result.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image recognition method, electronic device, and storage medium. Background Technology

[0002] With the development of computer technology and artificial intelligence technology, image recognition technology has brought great convenience to people's lives. Examples include facial recognition technology and behavior recognition technology.

[0003] Taking facial recognition technology as an example, facial recognition has applications in security monitoring, smart retail, and attendance tracking. Currently, when using facial recognition technology to identify users, it usually involves recognizing the original, complete facial image. However, in real life, due to factors such as the image acquisition angle and environmental lighting during image acquisition, the quality of the image to be recognized may be poor, making effective recognition impossible. Summary of the Invention

[0004] This application provides at least one image recognition method, electronic device, and storage medium.

[0005] The first aspect of this application provides an image recognition method, comprising: segmenting an image to be recognized into several image blocks, obtaining spatial information of each image block in the image to be recognized; and obtaining recognition influencing factors contained in the image to be recognized; extracting the correlation information between the spatial information of each image block and the recognition influencing factors to obtain the influence spatial correlation features corresponding to each image block; adjusting the weight parameters of each image block based on the influence spatial correlation features to obtain image block weight information corresponding to the image to be recognized; and recognizing the image to be recognized based on the image block weight information to obtain the image recognition result.

[0006] In one embodiment, obtaining the spatial information of each image block in the image to be identified includes: obtaining the reference point coordinates of each image block relative to the image to be identified, and obtaining the size parameters of each image block; using the reference point coordinates and size parameters corresponding to each image block as the spatial information of each image block.

[0007] In one embodiment, the spatial information of each image block is extracted and the correlation information between the identified influencing factors is obtained to obtain the spatial correlation features of each image block. This includes: extracting the spatial connection features between each image block based on the spatial information of each image block; and performing full connection processing on the spatial connection features between each image block and the identified influencing factors to obtain the spatial correlation features of the influence.

[0008] In one embodiment, the spatial information of each image block includes the coordinates of a reference point and the size parameters corresponding to each image block; based on the spatial information of each image block, the spatial connection features between each image block are extracted, including: calculating the positional differences between each image block based on the coordinates of the reference point corresponding to each image block; and calculating the size differences between each image block based on the size parameters corresponding to each image block; and obtaining the spatial connection features between each image block based on the positional differences and size differences between each image block.

[0009] In one embodiment, the spatial connection features between each image patch and the identification influencing factors are processed by a fully connected layer to obtain the spatial correlation features. This includes: inputting the spatial connection features between each image patch and the identification influencing factors into a pre-trained fully connected layer to obtain the initial correlation features corresponding to each image patch output by the fully connected layer; and activating the initial correlation features using a pre-trained activation function to obtain the spatial correlation features corresponding to the image to be identified.

[0010] In one embodiment, the image to be identified is identified based on image block weight information to obtain an image recognition result, including: extracting image features from the image to be identified to obtain initial image features; weighting the initial image features based on image block weight information to obtain weighted image features; and identifying the image to be identified based on the weighted image features to obtain an image recognition result.

[0011] In one embodiment, the step of acquiring the image to be identified includes: acquiring a portrait image containing a portrait to be identified; performing face image extraction and body image extraction on the portrait to be identified in the portrait image to obtain a face image and a body image corresponding to the portrait to be identified; and using the face image and body image corresponding to the portrait to be identified as the image to be identified.

[0012] In one embodiment, the image to be recognized is identified based on image block weight information to obtain an image recognition result, including: extracting facial features from the face image based on the image block weight information corresponding to the face image to obtain weighted facial features; and extracting human features from the human image based on the image block weight information corresponding to the human image to obtain weighted human features; fusing the weighted facial features and the weighted human features to obtain the human image features corresponding to the human image to be recognized; and recognizing the human image to be recognized based on the human image features to obtain an image recognition result.

[0013] A second aspect of this application provides an image recognition apparatus, comprising: an acquisition module for segmenting an image to be recognized into several image blocks and acquiring spatial information of each image block in the image to be recognized; and acquiring recognition influencing factors contained in the image to be recognized; an information association module for extracting association information between the spatial information of each image block and the recognition influencing factors, thereby obtaining influence spatial association features corresponding to each image block; a weight adjustment module for adjusting the weight parameters of each image block based on the influence spatial association features, thereby obtaining image block weight information corresponding to the image to be recognized; and a recognition module for recognizing the image to be recognized based on the image block weight information, thereby obtaining an image recognition result.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the image recognition method described above.

[0015] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described image recognition method.

[0016] The above scheme divides the image to be recognized into several image blocks, obtains the spatial information of each image block in the image to be recognized, and obtains the recognition influencing factors contained in the image to be recognized; extracts the correlation information between the spatial information of each image block and the recognition influencing factors to obtain the influence spatial correlation features corresponding to each image block; adjusts the weight parameters of each image block based on the influence spatial correlation features to obtain the image block weight information corresponding to the image to be recognized; improves the accuracy of weight parameter adjustment by fusing the spatial information of the image to be recognized and the recognition influencing factors, and refines the granularity of weight parameter adjustment by using the image block as the basic unit for adjusting the weight parameters; then, the image to be recognized is recognized based on the image block weight information to obtain the image recognition result, which can make the image recognition focus more on the information of the useful area and obtain a more accurate image recognition result.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a schematic diagram of the implementation environment involved in this application;

[0020] Figure 2 This is a flowchart illustrating an exemplary embodiment of the image recognition method of this application;

[0021] Figure 3 This is a schematic diagram illustrating the configuration of image block segmentation granularity in an exemplary embodiment of this application;

[0022] Figure 4 This is a schematic diagram illustrating the configuration of image block segmentation granularity, as shown in another exemplary embodiment of this application;

[0023] Figure 5 This is a schematic diagram illustrating the acquisition of spatial connectivity features in an exemplary embodiment of this application;

[0024] Figure 6 This is a schematic diagram of an image patch enhancement network illustrated in an exemplary embodiment of this application;

[0025] Figure 7 This is a schematic diagram illustrating a facial recognition model as an exemplary embodiment of this application;

[0026] Figure 8 This is a block diagram illustrating an image recognition device according to an exemplary embodiment of this application;

[0027] Figure 9 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application;

[0028] Figure 10 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0029] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0030] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0031] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0032] Please see Figure 1 , Figure 1 This is a schematic diagram of an implementation environment involved in the image recognition method of this application. For example... Figure 1 As shown, the implementation environment includes a terminal 110 and a server 120, which can be directly or indirectly connected via wired or wireless communication.

[0033] Terminal 110 can be a mobile phone, computer, camera, smart home appliance, vehicle terminal, aircraft, etc., but is not limited to these. Terminal 110 can refer to one of multiple terminals; this embodiment only uses terminal 110 as an example. Those skilled in the art will understand that the number of terminals can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. In this case, the implementation environment of the image recognition method may also include other terminals. This application embodiment does not limit the number or type of terminals.

[0034] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0035] Optionally, server 120 may undertake the main image recognition work and terminal 110 may undertake the secondary image recognition work; or, server 120 may undertake the secondary image recognition work and terminal 110 may undertake the main image recognition work; or, server 120 or terminal 110 may each undertake the image recognition work independently.

[0036] For example, terminal 110 sends an image recognition instruction to server 120, which carries an image to be recognized. Server 120 segments the image to be recognized sent by terminal 110 into several image blocks, obtains the spatial information of each image block in the image to be recognized, and obtains the recognition influencing factors contained in the image to be recognized. Then, it extracts the correlation information between the spatial information of each image block and the recognition influencing factors to obtain the influence spatial correlation features corresponding to each image block. Based on the influence spatial correlation features, it adjusts the weight parameters of each image block to obtain the image block weight information corresponding to the image to be recognized. Based on the image block weight information, it performs recognition on the image to be recognized to obtain the image recognition result, and returns the image recognition result to terminal 110.

[0037] It is understood that in the specific implementation of this application, data such as images to be identified and sample images are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0038] Please see Figure 2 , Figure 2 This is a flowchart illustrating an exemplary embodiment of the image recognition method of this application. This image recognition method can be applied to… Figure 1 The implementation environment is shown, and the image recognition method proposed in this application is specifically executed by a server within that environment. The following section uses the server as the specific execution entity to provide a detailed description of the image recognition method proposed in this application's embodiments.

[0039] like Figure 2 As shown, the image recognition method includes at least steps S210 to S240, which are described in detail below:

[0040] Step S210: Divide the image to be recognized into several image blocks, obtain the spatial information of each image block in the image to be recognized; and obtain the recognition influencing factors contained in the image to be recognized.

[0041] The image to be identified can be a color image, a depth image, a grayscale image, etc. This application does not limit the type of image to be identified.

[0042] The recognition influencing factors are those factors in the image to be recognized that affect image recognition. For example, recognition influencing factors include, but are not limited to: the image quality of the image to be recognized, the pose of the object to be recognized in the image to be recognized, and whether the object to be recognized in the image to be recognized is wearing a specified type of clothing, etc.

[0043] For example, the image to be identified can be input into a pre-trained identification influencing factor detection model, which has the function of detecting identification influencing factors of the image, and the identification influencing factors contained in the image to be identified output by the identification influencing factor detection model can be obtained.

[0044] The image to be recognized is segmented according to the image block segmentation granularity to obtain several image blocks.

[0045] The image patch segmentation granularity can be a pre-set parameter or a flexibly calculated parameter. For example, the types of recognition influencing factors contained in the image to be recognized can be obtained, and the image patch segmentation granularity can be determined according to the types of recognition influencing factors, so as to flexibly configure the image patch segmentation granularity according to different recognition influencing factors. Of course, the image patch segmentation granularity can also be flexibly calculated based on other data, such as the type of image recognition task, the type of object to be recognized in the image to be recognized, etc. This application does not limit the image patch segmentation granularity in this way.

[0046] For example, one image to be identified can correspond to one image patch segmentation granularity, such as... Figure 3 As shown, assuming the image block segmentation height of the image to be recognized is h and the width is w, the image to be recognized is segmented into several h*w image blocks.

[0047] For example, one image to be recognized can correspond to multiple image block segmentation granularities. This involves first dividing the image to be recognized into multiple image regions, and then using different image block segmentation granularities for different image regions. For instance, ... Figure 4 As shown, the image to be recognized is divided into object regions containing the object to be recognized and non-object regions not containing the object to be recognized. The object regions correspond to a first image block segmentation granularity, and the non-object regions correspond to a second image block segmentation granularity. The first image block segmentation granularity is smaller than the second image block segmentation granularity, in order to save computational resources while ensuring finer-grained processing of the image regions containing the object to be recognized. It is understood that the image regions can also be divided according to the type of recognition influencing factors contained in the image to be recognized, the type of image recognition task, the type of object to be recognized in the image to be recognized, etc., and this application does not limit this.

[0048] After segmenting the image to be recognized into several image blocks, the spatial information of each image block in the image to be recognized is obtained. The spatial information includes, but is not limited to, the position information and shape information of the image block.

[0049] Step S220: Extract the spatial information of each image patch and the correlation information between the identified influencing factors to obtain the spatial correlation features of each image patch.

[0050] Among them, the influence spatial correlation feature is used to characterize the degree of influence of the identified influencing factors on each image patch.

[0051] By extracting the spatial information of each image patch and the correlation information between the identified influencing factors, the spatial correlation features of each image patch are obtained.

[0052] The number of influencing factors identified can be one or more. If there are multiple influencing factors, the spatial information of the image patch and the correlation information between each identified influencing factor are extracted separately. The correlation information of the image patch for each identified influencing factor is then fused to obtain the spatial correlation feature of the image patch.

[0053] Step S230: Adjust the weight parameters of each image block based on the influence spatial correlation features to obtain the image block weight information corresponding to the image to be identified.

[0054] The weight parameter of the image patch is used to characterize the importance of the image patch.

[0055] The weight parameters of each image patch are flexibly adjusted based on the spatial correlation characteristics. This increases the weight parameters of image patches that contribute significantly to the image recognition task and decreases the weight parameters of image patches that contribute little to the image recognition task, so that subsequent image recognition focuses more on information in the useful region.

[0056] By using image patches as the basic unit for adjusting weight parameters, the granularity of weight parameter adjustment is refined, resulting in more accurate weight information.

[0057] Step S240: Recognize the image to be recognized based on the image block weight information to obtain the image recognition result.

[0058] It is understandable that different image recognition tasks use different recognition methods, resulting in different image recognition results.

[0059] For example, if the image recognition task is an identity recognition task, the process of recognizing the image to be recognized is as follows: based on the image block weight information, the identity features of the object to be recognized in the image to be recognized are extracted and classified to obtain the identity recognition result of the object to be recognized; if the image recognition task is a scene recognition task, the process of recognizing the image to be recognized is as follows: based on the image block weight information, the scene features of the image to be recognized are extracted and classified to obtain the scene recognition result of the object to be recognized.

[0060] Next, the specific implementation methods of each step in this application will be further explained:

[0061] In some implementations, the specific steps for obtaining spatial information of the image patch in step S210 may include:

[0062] Step S211: Obtain the reference point coordinates of each image block relative to the image to be recognized, and obtain the size parameters of each image block.

[0063] The reference points include, but are not limited to, center points and corner points; this application does not limit their scope. The size parameter of the image patch is used to characterize the size of the image patch.

[0064] Taking the reference point as the center point and the image block as a rectangle as an example, the calculation process of the reference point coordinates and size parameters is explained: obtain the coordinates of the two relative corner points of the image block relative to the image to be identified, and calculate the reference point coordinates and size parameters based on the coordinates of the two relative corner points.

[0065] For example, the coordinates of the two relative corner points of image patch i are (x, y, y) i min y i min ) and (x i max y i max ), reference point coordinates (cx) i cy i The formula for calculating ) can be:

[0066]

[0067]

[0068] The size parameter of image patch i is determined by the height h of the image patch. i and width w i Composition, high h i and width w i The calculation formula can be:

[0069] w i =(x i max -x i min )+1

[0070] h i =(y i max -y i min )+1

[0071] Step S212: Use the reference point coordinates and size parameters corresponding to each image block as the spatial information of each image block.

[0072] Based on the reference point coordinates and size parameters of the image patch, the spatial information of the image patch is obtained. This information is combined with the position, shape, and size of the image patch to increase the information dimension contained in the spatial information and improve the accuracy of the spatial correlation features obtained subsequently.

[0073] The spatial information of each image block relative to the image to be identified is obtained, and the influencing factors of the image to be identified are identified to obtain the influencing factors contained in the image to be identified. Then, the correlation information between the spatial information of each image block and the influencing factors is extracted to obtain the spatial correlation features of each image block.

[0074] In some implementations, the specific steps in step S220 for extracting features affecting spatial correlation may include:

[0075] S221: Based on the spatial information of each image patch, extract the spatial connection features between each image patch.

[0076] Spatial connectivity features are used to characterize spatial association information between image patches.

[0077] For example, the spatial information of each image block includes the coordinates of the reference point and the size parameters corresponding to each image block; based on the spatial information of each image block, the spatial connection features between each image block are extracted, including: calculating the positional difference between each image block based on the coordinates of the reference point corresponding to each image block; and calculating the size difference between each image block based on the size parameters corresponding to each image block; and obtaining the spatial connection features between each image block based on the positional difference and size difference between each image block.

[0078] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the acquisition of spatial connectivity features as an exemplary embodiment of this application. Figure 5 As shown, taking image patch i and image patch j as examples, the coordinates (cx) of the reference point of image patch i are obtained. i cy i ) and dimensional parameter h i *w i Obtain the reference point coordinates (cx) of image patch j. j cy j ) and dimensional parameter h j *w j The positional difference between image patch i and image patch j is calculated, including the x-axis difference (cx). i -cx j ), and y-axis difference (cy i -cy j ); and calculate the size difference, including the height difference, between image patch i and image patch j. and width differences Then, based on the positional and size differences between image block i and image block j, the spatial connectivity features between each image block are obtained, as shown in the following formula: The spatial connectivity features between image block i and image block j can be expressed as follows:

[0079] S222: Perform full connection processing on the spatial connection features and identification influencing factors between each image patch to obtain the spatial correlation features.

[0080] Among them, fully connected processing is used to integrate spatial connectivity features with identified influencing factors.

[0081] In some implementations, the spatial connectivity features between image blocks and the identification influencing factors are processed by a fully connected layer to obtain the spatial correlation features. This includes: inputting the spatial connectivity features between image blocks and the identification influencing factors into a pre-trained fully connected layer to obtain the initial correlation features corresponding to each image block output by the fully connected layer; and activating the initial correlation features using a pre-trained activation function to obtain the spatial correlation features corresponding to the image to be identified.

[0082] like Figure 6 As shown, the image to be identified is segmented into several image blocks, the spatial connection features between each image block are obtained, and the recognition influencing factors corresponding to the image to be identified are obtained. The spatial connection features and recognition influencing factors are input into the first fully connected layer (FC1) to obtain the initial association features output by the first fully connected layer.

[0083] For example, the input of the first fully connected layer corresponding to image block i and image block j can be represented as:

[0084]

[0085] Where attr_* represents the recognition influencing factors contained in the image to be recognized, and there are a total of r recognition influencing factors.

[0086] The formula for the first fully connected layer to process the spatial connectivity features and recognition influencing factors between image block i and image block j can be expressed as follows:

[0087] G ij =FC(r) ij )

[0088] Where FC represents the network parameters of the first fully connected layer, and G ij The initial association features are the image blocks i and j output by the first fully connected layer.

[0089] Then, the formula for activating the initial associated features using a pre-trained activation function can be expressed as:

[0090]

[0091] Where ReLU is the network parameter representation of the activation function, wg The training parameters are obtained from model training. Let i be the influence spatial correlation features corresponding to image block i and image block j.

[0092] By combining the influence spatial correlation features of each image patch, the influence spatial correlation features corresponding to the image to be identified are obtained.

[0093] In some implementations, after obtaining the spatial correlation features, the weight parameters of each image patch are adjusted based on these features to obtain image patch weight information. For example, as follows... Figure 6 As shown, the weight parameters of each image patch are adjusted based on the image patch enhancement network. The image patch enhancement network includes a first fully connected layer, a second fully connected layer, and a encoder layer (Decoder). The input of the first fully connected layer is spatial connectivity features and identification influencing factors, and the output is the spatial correlation features between image patches. The input of the second fully connected layer is the image patch sequence, and the output is the fully connected features of the image patches. The input of the encoder layer is the output of the first and second fully connected layers, and the output is the image patch weight information.

[0094] The calculation steps for the image patch sequence include: the two-dimensional image representation of each image patch is h*w, where h is the height of the image patch and w is the width of the image patch; converting the two-dimensional image representation of the image patch into a one-dimensional image representation h×w=d ing If the number of image patches is N, then the serialized image patch sequence is:

[0095] The image patch sequence is input into the second fully connected layer for fully connected processing. The relevant formula is as follows:

[0096] U = ReLU(FC1(I))

[0097] Where FCl represents the network parameters of the second fully connected layer, and U represents the fully connected features of the image patch output by the second fully connected layer.

[0098] The fully connected features of image patches and the influence space correlation features are input into the encoding layer. The training parameters of the encoding layer include... The calculation formulas involved in the coding layer may include:

[0099] Q = UW q K = UW k V=UW v

[0100]

[0101] Where softmax is the normalization exponential function, λ g λ represents the spatial correlation features corresponding to the image to be identified.g ∈R N×N d k U is a scaling factor. aug This refers to image block weight information.

[0102] Furthermore, the image to be recognized is identified based on the image block weight information to obtain the image recognition result.

[0103] In some implementations, the image recognition step in step S240 may include:

[0104] Step S241: Extract image features from the image to be recognized to obtain initial image features.

[0105] The extracted image features include, but are not limited to, texture features, color features, and shape features. These features are extracted to obtain the initial image features.

[0106] Step S242: Weight the initial image features based on the image block weight information to obtain weighted image features.

[0107] The initial image features are weighted based on the image patch weight information to obtain weighted image features.

[0108] For example, image features are extracted for each image block corresponding to the image to be recognized to obtain the initial image features corresponding to each image block. Then, the initial image features corresponding to each image block are weighted according to the image block weight information to obtain the weighted image features.

[0109] Step S243: Recognize the image to be recognized based on the weighted image features to obtain the image recognition result.

[0110] The image recognition method of this application will be explained next using a facial recognition application scenario as an example:

[0111] The image acquisition device acquires a portrait image containing a person to be identified. The steps of obtaining the image to be identified based on the portrait image may include: acquiring a portrait image containing a person to be identified; performing face image extraction and body image extraction on the person to be identified in the portrait image to obtain a face image and a body image corresponding to the person to be identified; and using the face image and body image corresponding to the person to be identified as the image to be identified.

[0112] Using face images and body images as the images to be recognized, features are extracted separately for the face and body portions of the person to be recognized, improving the accuracy of face recognition. The steps for obtaining the image recognition result may include: extracting face features from the face image based on the image block weight information corresponding to the face image to obtain weighted face features; and extracting body features from the body image based on the image block weight information corresponding to the body image to obtain weighted body features; fusing the weighted face features and weighted body features to obtain the face features corresponding to the person to be recognized; and recognizing the person to be recognized based on the face features to obtain the image recognition result.

[0113] The above steps are implemented based on a pre-trained facial recognition model, such as... Figure 7 As shown, the facial recognition model includes a facial feature extraction module, a human body feature extraction module, and a feature fusion module.

[0114] The face feature extraction module is used to extract face features. The face feature extraction model consists of a first image patch enhancement network and a first feature extraction network. For the computational principle of the first image patch enhancement network, please refer to [link to relevant documentation]. Figure 6 In a corresponding embodiment, the first image block enhancement network outputs the image block weight information corresponding to the image to be identified, and inputs the image block weight information and the image block corresponding to the image to be identified into the first feature extraction network to extract face features, thereby obtaining weighted face features.

[0115] The human feature extraction module is used to extract human features. The human feature extraction model consists of a second image patch enhancement network and a second feature extraction network. For the computational principle of the second image patch enhancement network, please refer to [link to relevant documentation]. Figure 6 In a corresponding embodiment, the second image block enhancement network outputs the image block weight information corresponding to the image to be identified, and inputs the image block weight information and the image block corresponding to the image to be identified into the second feature extraction network to extract human features, thereby obtaining weighted human features.

[0116] The first feature extraction network and the second feature extraction network can be implemented based on Vision Transformer (ViT) networks, Convolutional Neural Networks (CNN) networks, etc., and this application does not limit them.

[0117] Then, the weighted facial features and weighted human features are input into the feature fusion module. The feature fusion module fuses the weighted facial features and weighted human features to obtain the facial features corresponding to the person to be identified. For example, the weighted facial features output by the facial feature extraction module are... The weighted human features output by the human feature extraction module are: The resulting human features are called Featattr ∈R rfb RFB represents the sum of the dimensions of weighted facial features and weighted human body features. For example... Figure 7 The feature fusion module in the middle concatenates weighted facial features and weighted human features, and then inputs them into the Transformer network to extract the fused features. Calculate Feat fused The image recognition result is determined by comparing the similarity between the facial features of the images of registered individuals in the preset facial feature database and the image of the registered individual with the highest similarity.

[0118] The following section describes the training process of the aforementioned facial recognition model:

[0119] In order to improve the feature representation capabilities of each module in the human face recognition model, this application first trains the face feature extraction module and the human body feature extraction module separately.

[0120] For example, a set of face samples is obtained, and each face sample icon in the set of face samples is labeled with a sample label. The sample label is used to mark the identity information corresponding to the face sample image, identify influencing factors, etc.

[0121] The face sample images from the face sample set are input into the face feature extraction module for feature extraction. The specific implementation steps of feature extraction can be found in the image recognition method described above, and will not be repeated here. Then, the similarity between the extracted face features and the face features of each registered person in the preset face feature database is calculated, and the identity information of the registered person with the highest similarity is used as the prediction result of the face sample image.

[0122] The face loss value L is obtained by calculating the loss between the prediction result and the identity information in the sample label based on the classification loss function. f Based on face loss value L f The parameters of the face feature extraction module are adjusted to obtain the adjusted face feature extraction module. The training process of the face feature extraction module is iterated until the training termination condition is met, such as when the preset number of iterations is reached, and the face loss value L is calculated. f Less than the preset loss value, etc.

[0123] For example, a human sample set is obtained, and each human sample icon in the human sample set is labeled with a sample label. The sample label is used to mark the identity information corresponding to the human sample image, identify influencing factors, etc.

[0124] Human sample images from the human sample set are input into the human feature extraction module for feature extraction. The specific implementation steps of feature extraction can be found in the image recognition method described above, and will not be elaborated upon here. The similarity between the extracted human features and the facial features of each registered individual in the preset facial feature database is calculated. The identity information of the registered individual with the highest similarity is used as the prediction result for the human sample image. Based on the classification loss function, the loss value between the prediction result and the identity information in the sample label is calculated, resulting in the first loss value L. identity .

[0125] Optionally, during the model training process of the human body feature extraction module, considering that human body features are easily affected by clothing style and color, the human body sample set is set to consist of multiple sample triples. Each sample triple contains a human body sample image, a positive sample corresponding to the human body sample image, and a negative sample corresponding to the human body sample image.

[0126] Taking human body sample image a as an example, positive samples can be obtained in the following ways:

[0127] Obtain a set P of candidate body images that belong to the same person as body sample image a but are dressed differently, and extract the human features of body sample image a as feat. a And extract the human features of each candidate body image in the candidate body image set P. Calculate feat a With Feat P The similarity between each human characteristic in the group.

[0128] Among them, feat can be a With Feat P The Euclidean distance, Manhattan distance, Chebyshev distance, Mahalanobis distance, etc., between each human feature are used as features. a With Feat P The similarity between each human feature is not limited in this application.

[0129] Then, select the candidate body graph set P that matches the feature graph. a The candidate body image with the lowest similarity is taken as the positive sample corresponding to body sample image a. The human body features of the positive sample can be represented as feat. p .

[0130] Negative samples can be obtained in the following ways:

[0131] Obtain a set N of candidate human images that do not belong to the same person as human sample image a but are dressed similarly, and extract the human features of human sample image a as feat. a And extract the human features of each candidate body image in the candidate body image set N as... Calculate feata With Feat N The similarity between each human characteristic in the group.

[0132] Among them, feat can be a With Feat N The Euclidean distance, Manhattan distance, Chebyshev distance, Mahalanobis distance, etc., between each human feature are used as features. a With Feat N The similarity between each human feature is not limited in this application.

[0133] Then, select the candidate body graph set N that matches the feat a The candidate body image with the highest similarity is taken as the negative sample corresponding to body sample image a. The human body features of the negative sample can be represented as feat. n .

[0134] During training, the human features of the human sample image a corresponding to the sample triplet loss function are used as the feature. a human characteristics of positive samples p Negative sample human characteristics feature n The formula for calculating the second loss value is as follows:

[0135]

[0136] Among them, L triplet (a) represents the loss value corresponding to the human sample image a. The margin is a constant greater than 0, which can be set empirically.

[0137] In conjunction with the above embodiments, the final human body loss value L during model training is obtained from the human body feature extraction module. b The calculation formula can be:

[0138] L b =w1*L identity +w2*L triplet

[0139] Among them, w1 and w2 are preset weight parameters.

[0140] Based on human loss value L b The parameters of the human feature extraction module are adjusted to obtain the adjusted face feature extraction module. The training process of the human feature extraction module is iterated until the training termination condition is met, such as when the preset number of iterations is reached, and the human loss value L is calculated. b Less than the preset loss value, etc.

[0141] After pre-training the face feature extraction module and the body feature extraction module, the face feature extraction module and the body feature extraction module obtained from the above training are jointly trained with the feature fusion module.

[0142] The joint training process may include: acquiring a set of human image samples, where each human image sample is labeled with a sample label representing identity information; extracting face and body sample images from the human image samples in the set; inputting the face sample images into a face feature extraction module for feature extraction, and inputting the body sample images into a body feature extraction module for feature extraction; and finally, a feature fusion module fuses the features output from the face and body feature extraction modules, along with the image recognition results, to obtain a prediction result for the human image sample. Based on a classification loss function, it calculates the loss value between the prediction result and the sample label, obtaining the human image loss value L. c Furthermore, in conjunction with the above embodiments, the face loss value L of the face feature extraction module is calculated. f Human loss value L of the human feature extraction module b .

[0143] Furthermore, based on the portrait loss value L c Face loss value L f and human body loss value L b The joint loss value L is calculated using the following formula:

[0144] L = w b *L b +w f *L f +w c *L c

[0145] Among them, w b w f w c These are the preset weight parameters.

[0146] Adjust the model parameters of the face feature extraction module, body feature extraction module, and feature fusion module based on the obtained joint loss value L, and iterate the above joint training process until the module training termination conditions are met, such as the number of iterations reaching the preset number or the joint loss value L being less than the preset loss value.

[0147] By adopting the above training process, the feature expression capabilities of the face feature extraction module and the body feature extraction module are improved, and the requirements of the face feature extraction module and the body feature extraction module for human image samples are reduced, thereby reducing the difficulty of model training convergence and improving model training efficiency.

[0148] The image recognition method provided in this application divides the image to be recognized into several image blocks, obtains the spatial information of each image block in the image to be recognized, and obtains the recognition influencing factors contained in the image to be recognized; extracts the correlation information between the spatial information of each image block and the recognition influencing factors to obtain the influence spatial correlation features corresponding to each image block; adjusts the weight parameters of each image block based on the influence spatial correlation features to obtain the image block weight information corresponding to the image to be recognized; improves the accuracy of weight parameter adjustment by fusing the spatial information of the image to be recognized and the recognition influencing factors, and refines the granularity of weight parameter adjustment by using the image block as the basic unit for adjusting the weight parameters; then, the image to be recognized is recognized based on the image block weight information to obtain the image recognition result, which can make the image recognition focus more on the information of the useful area and obtain a more accurate image recognition result.

[0149] Figure 8 This is a block diagram illustrating an image recognition device according to an exemplary embodiment of this application. Figure 8 As shown, the exemplary image recognition device 800 includes: an acquisition module 810, an information association module 820, a weight adjustment module 830, and a recognition module 840. Specifically:

[0150] The acquisition module 810 is used to segment the image to be recognized into several image blocks, acquire the spatial information of each image block in the image to be recognized, and acquire the recognition influencing factors contained in the image to be recognized.

[0151] The information association module 820 is used to extract the spatial information of each image block and the association information between the identified influencing factors, and to obtain the spatial association features of each image block.

[0152] The weight adjustment module 830 is used to adjust the weight parameters of each image block based on the influence spatial correlation features to obtain the image block weight information corresponding to the image to be identified.

[0153] The recognition module 840 is used to recognize the image to be recognized based on the image block weight information and obtain the image recognition result.

[0154] In the above-described exemplary image recognition device, by using image blocks as the basic unit for adjusting weight parameters, the granularity of weight parameter adjustment is refined. By fusing spatial information of the image to be recognized and recognition influencing factors, the accuracy of weight parameter adjustment is improved. This allows image recognition to focus more on information in useful areas and obtain more accurate image recognition results.

[0155] The functions of each module can be found in the embodiment of the image recognition method, and will not be repeated here.

[0156] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 900 includes a memory 901 and a processor 902. The processor 902 is used to execute program instructions stored in the memory 901 to implement the steps in any of the above-described image recognition method embodiments. In a specific implementation scenario, the electronic device 900 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 900 may also include mobile devices such as laptops and tablets, which are not limited here.

[0157] Specifically, processor 902 controls itself and memory 901 to implement the steps in any of the above-described image recognition method embodiments. Processor 902 can also be referred to as a Central Processing Unit (CPU). Processor 902 may be an integrated circuit chip with signal processing capabilities. Processor 902 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 902 can be implemented using integrated circuit chips.

[0158] Please see Figure 10 , Figure 10 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 1000 stores program instructions 1010 that can be executed by a processor. The program instructions 1010 are used to implement the steps in any of the above-described image recognition method embodiments.

[0159] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0160] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0162] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image recognition method, characterized in that, The method includes: The image to be identified is divided into several image blocks, and the spatial information of each image block in the image to be identified is obtained. The spatial information includes the reference point coordinates and size parameters corresponding to each image block; and the identification influencing factors contained in the image to be identified are obtained. Based on the reference point coordinates corresponding to each image patch, the positional differences between the image patches are calculated; and based on the size parameters corresponding to each image patch, the size differences between the image patches are calculated; based on the positional and size differences between the image patches, the spatial connection features between the image patches are obtained; and the spatial connection features between the image patches and the identification influencing factors are fully connected to obtain the spatial association features. The image block sequence after each image block is serialized is processed by a fully connected layer to obtain the fully connected features of the image blocks. The influence spatial correlation features and the fully connected features of the image blocks are input into the coding layer to obtain the image block weight information corresponding to the image to be identified output by the coding layer. The image to be identified is identified based on the image block weight information to obtain the image recognition result.

2. The method according to claim 1, characterized in that, The step of obtaining the spatial information of each image block in the image to be identified includes: Obtain the reference point coordinates of each image block relative to the image to be identified, and obtain the size parameters of each image block; The coordinates and size parameters of the reference point corresponding to each image block are used as the spatial information of each image block.

3. The method according to claim 1, characterized in that, The process of performing a full connection processing on the spatial connectivity features between the various image blocks and the identified influencing factors to obtain the spatial correlation features includes: The spatial connectivity features between the image patches and the identification influencing factors are input into a pre-trained fully connected layer to obtain the initial association features corresponding to each image patch output by the fully connected layer. The initial association features are activated using a pre-trained activation function to obtain the influence space association features corresponding to the image to be identified.

4. The method according to claim 1, characterized in that, The process of identifying the image to be identified based on the image block weight information to obtain the image recognition result includes: Image features are extracted from the image to be identified to obtain initial image features; The initial image features are weighted based on the image block weight information to obtain weighted image features; The image to be identified is identified based on the weighted image features to obtain the image recognition result.

5. The method according to any one of claims 1 to 4, characterized in that, The steps for acquiring the image to be identified include: Acquire a portrait image, wherein the portrait image contains a portrait of the person to be identified; The face image and body image are extracted from the portrait image to obtain the face image and body image corresponding to the portrait to be identified. The face image and body image corresponding to the image to be identified are used as the images to be identified.

6. The method according to claim 5, characterized in that, The process of identifying the image to be identified based on the image block weight information to obtain the image recognition result includes: Based on the image block weight information corresponding to the face image, face features are extracted from the face image to obtain weighted face features; and based on the image block weight information corresponding to the human body image, human body features are extracted from the human body image to obtain weighted human body features. By fusing the weighted facial features and the weighted human body features, the portrait features corresponding to the portrait to be identified are obtained; The image of the person to be identified is identified based on the aforementioned facial features, and an image recognition result is obtained.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-6.

Images