Image recognition method and apparatus, and storage medium and electronic device

Abstract

Disclosed in the present application are an image recognition method and apparatus, and a storage medium and an electronic device. The method comprises: extracting image features from an image to be recognized; performing feature processing on the image features and initialized reference key point features by means of a feature processing network, so as to obtain final key point features, wherein the feature processing network is obtained by means of performing N cycles of feature processing on each sample image on the basis of an initialized feature processing network, i.e., in an ith feature processing network for executing an ith cycle of feature processing, executing feature processing on an (i-1)th sample feature pair obtained by means of an (i-1)th cycle of feature processing, so as to obtain an ith sample feature pair, and adjusting network parameters of the ith feature processing network on the basis of feature differences between the (i-1)th sample feature pair and the ith sample feature pair until a convergence condition is reached; and using the final key point features to determine the positions of key points of an object to be recognized in the image to be recognized. The present application solves the technical problem of low image recognition efficiency.

Description

Image recognition method and device, storage medium and electronic device

[0001] The present application claims priority from the Chinese patent application No. 202410413600.4 filed on April 8, 2024, and entitled "Image recognition method and device, storage medium and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of computer, in particular to an image recognition technology. BACKGROUND

[0003] In some computing vision tasks, it is usually required to recognize an image to determine the positions of key points in a to-be-recognized object presented in the image, and then further analyze the positions of the key points in the to-be-recognized object, so as to complete the above-mentioned vision task based on the result of the analysis.

[0004] For the recognition and positioning of key points in an image, the method commonly used in the related art is to extract image features by using a deep convolutional network feature extractor designed in advance by hand, and then recognize the key points of an object presented in the image by using these image features.

[0005] However, the number of parameters involved in the deep convolutional network in the above-mentioned method provided by the related art is large and the computational complexity is high, so a large amount of time is often required for training. That is, the process of image recognition using the above-mentioned deep convolutional network will consume a large amount of time, thereby causing the technical problem of low efficiency of image recognition.

[0006] For the above-mentioned problem, no effective solution has been proposed so far. SUMMARY

[0007] The embodiments of the present application provide an image recognition method and device, a storage medium and an electronic device to at least solve the technical problem of low efficiency of image recognition.

[0008] According to an aspect of some embodiments of the present application, a method for image recognition is provided. The method is performed by an electronic device and includes: extracting an image feature from a to-be-recognized image, wherein a to-be-recognized object is displayed in the to-be-recognized image; performing feature processing on the image feature and an initialized reference key point feature by a feature processing network to obtain a final key point feature of the to-be-recognized object, wherein the final key point feature is used to indicate a feature of a key point recognized from the to-be-recognized object displayed in the to-be-recognized image, the feature processing network is based on an initialized feature processing network, and is obtained by performing N times of loop feature processing on each sample image, in an i-th feature processing network used to perform an i-th loop feature processing, an i-1-th sample feature pair obtained by performing an i-1-th loop feature processing is processed to obtain an i-th sample feature pair, and a network parameter in the i-th feature processing network is adjusted based on a feature difference between the i-1-th sample feature pair and the i-th sample feature pair until a convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes an i-th sample image feature and an i-th reference sample key point feature, the i-th sample image feature is obtained by performing i times of loop feature processing based on an original sample image feature extracted from a sample image, and the i-th reference sample key point feature is obtained by performing i times of loop feature processing based on an original reference sample key point feature obtained by initialization; and determining a position of a key point of the to-be-recognized object in the to-be-recognized image by using the final key point feature.

[0009] According to an aspect of some embodiments of the present application, a method for image recognition preprocessing is provided, which is performed by an electronic device and includes: obtaining M sample images and an initialized feature processing network, and training by the following steps: extracting an original sample image feature from a kth sample image obtained, and obtaining an original reference sample key point feature obtained by initialization, where k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer; repeatedly performing the following operations on the original sample image feature and the original reference sample key point feature until N times of loop feature processing are reached: in an ith feature processing network performing ith loop feature processing, performing feature processing on an (i-1)th sample feature pair obtained by (i-1)th loop feature processing to obtain an ith sample feature pair, where in the case where i is 1, the (i-1)th sample feature pair obtained by (i-1)th loop feature processing is used to indicate an original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature; determining a sample feature distance between the (i-1)th sample feature pair and the ith sample feature pair based on a feature difference between the (i-1)th sample feature pair and the ith sample feature pair; performing weighted summation calculation on the first i sample feature distances to obtain an ith feature training loss; in the case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not reached a threshold condition, adjusting network parameters in the ith feature processing network to obtain an (i+1)th feature processing network; performing (i+1)th loop feature processing in the (i+1)th feature processing network; in the case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing reaches the threshold condition, determining that the ith loop feature processing is the Nth loop feature processing.

[0010] According to another aspect of the embodiments of the present application, an image recognition device is also provided. The device is deployed on an electronic device and includes an extraction unit configured to extract image features from a to-be-recognized image, wherein the to-be-recognized image displays a to-be-recognized object; a feature processing unit configured to perform feature processing on the image features and an initialized reference key point feature by a feature processing network to obtain a final key point feature of the to-be-recognized object, wherein the final key point feature is used to indicate a feature of a key point recognized from the to-be-recognized object displayed in the to-be-recognized image, the feature processing network is based on an initialized feature processing network, and is obtained by performing N times of loop feature processing on each sample image, in an i-th feature processing network used to perform an i-th loop feature processing, performing feature processing on an i-1-th sample feature pair obtained by the i-1-th loop feature processing to obtain an i-th sample feature pair, and adjusting network parameters in the i-th feature processing network based on a feature difference between the i-1-th sample feature pair and the i-th sample feature pair until a convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes an i-th sample image feature and an i-th reference sample key point feature, the i-th sample image feature is obtained by performing i times of loop feature processing on an original sample image feature extracted from a sample image, and the i-th reference sample key point feature is obtained by performing i times of loop feature processing on an original reference sample key point feature obtained by initialization; and a determination unit configured to determine a position of the key point of the to-be-recognized object in the to-be-recognized image by using the final key point feature.

[0011] According to another aspect of the embodiments of the present application, a pre-processing device for image recognition is also provided. The device is deployed on an electronic device and includes: an extraction unit configured to extract an original sample image feature from a kth acquired sample image and acquire an initialized original reference sample key point feature, where k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer; a feature processing unit configured to perform feature processing on an i-1th sample feature pair obtained through i-1th loop feature processing in an ith feature processing network to obtain an ith sample feature pair, where, when i is 1, the i-1th sample feature pair obtained through i-1th loop feature processing indicates an original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature; a first determination unit configured to determine a sample feature distance between the i-1th sample feature pair and the ith sample feature pair based on a feature difference between the i-1th sample feature pair and the ith sample feature pair; a calculation unit configured to perform weighted summation calculation on the first i sample feature distances to obtain an ith feature training loss; an adjustment unit configured to, when the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not reached a threshold condition, adjust network parameters in the ith feature processing network to obtain an (i+1)th feature processing network; perform (i+1)th loop feature processing in the (i+1)th feature processing network; and a second determination unit configured to, when the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing reaches the threshold condition, determine that the ith loop feature processing is the Nth loop feature processing.

[0012] According to yet another aspect of the embodiments of the present application, a computer readable storage medium is also provided. The computer readable storage medium stores a computer program. When the computer program is executed, the above-mentioned image recognition method and the above-mentioned pre-processing method for image recognition are performed.

[0013] According to yet another aspect of the embodiments of the present application, a computer program product is provided. The computer program product includes a computer program stored in a computer readable storage medium. A processor of a computer device reads the computer program from the computer readable storage medium. The processor executes the computer program, so that the computer device performs the above-mentioned image recognition method and the above-mentioned pre-processing method for image recognition.

[0014] According to yet another aspect of the embodiments of the present application, an electronic device is also provided. The electronic device includes a memory and a processor. The memory stores a computer program. The processor is configured to execute the computer program to perform the above-mentioned image recognition method and the above-mentioned pre-processing method for image recognition.

[0015] In the embodiment of the present application, the image features are extracted from the to-be-recognized image in which the to-be-recognized object is displayed. Then, the image features and the initialized reference key point features are processed by the feature processing network to obtain the final key point features of the to-be-recognized object, wherein the final key point features are used to indicate the features of the key points recognized from the to-be-recognized object displayed in the to-be-recognized image. The feature processing network is based on the initialized feature processing network, and is obtained by performing N times of loop feature processing on each sample image. In the i-th feature processing network for performing the i-th loop feature processing, the i-1-th sample feature pair obtained by the i-1-th loop feature processing is processed to obtain the i-th sample feature pair, and the network parameters in the i-th feature processing network are adjusted based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair until the convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes the i-th sample image feature and the i-th reference sample key point feature, the i-th sample image feature is obtained by performing i times of loop feature processing on the original sample image feature extracted from the sample image, and the i-th reference sample key point feature is obtained by performing i times of loop feature processing on the original reference sample key point feature. Then, the positions of the key points in the to-be-recognized object in the to-be-recognized image are determined by using the final key point features. In other words, in the embodiment of the present application, the feature processing network trained by the multi-loop self-distillation is used to locate the positions of the key points in the to-be-recognized object included in the to-be-recognized image in the to-be-recognized image, and the feature processing network trained by the multi-loop self-distillation is a lightweight network, which can improve the network performance by using a small number of parameters and low complexity calculation, thereby reducing the time consumed by network training while ensuring the accuracy of network calculation results. Thus, the technical problem of low image recognition efficiency in the image recognition method provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the present application. In the drawings:

[0017] FIG. 1 is a schematic diagram of an application environment of an optional image recognition method according to an embodiment of the present application;

[0018] FIG. 2 is a flowchart of an optional image recognition method according to an embodiment of the present application;

[0019] FIG. 3 is a flowchart of an optional image recognition method according to an embodiment of the present application;

[0020] FIG. 4 is a flow chart of an optional image recognition method according to an embodiment of the present application;

[0021] FIG. 5 is a schematic diagram of an optional image recognition method according to an embodiment of the present application;

[0022] FIG. 6 is a flow chart of an optional image recognition method according to an embodiment of the present application;

[0023] FIG. 7 is a flow chart of an optional image recognition method according to an embodiment of the present application;

[0024] FIG. 8 is a structural schematic diagram of an optional image recognition device according to an embodiment of the present application;

[0025] FIG. 9 is a structural schematic diagram of an optional preprocessing device of an image recognition according to an embodiment of the present application;

[0026] FIG. 10 is a structural schematic diagram of an optional electronic device according to an embodiment of the present application;

[0027] FIG. 11 is a structural schematic diagram of another optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should be within the protection scope of the present application.

[0029] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] According to an aspect of the embodiments of the present application, there is provided an image recognition method. Optionally, as an optional implementation, the image recognition can be applied in the environment shown in FIG. 1. As shown in FIG. 1, the terminal device 102 includes a memory 104 for storing various data generated during the operation of the terminal device 102, a processor 106 for processing the above-mentioned data, and a display 108 for displaying the image to be recognized. The terminal device 102 can interact with the server 112 through the network 110. The server 112 is connected to the database 114, which is used to store various data. The terminal device 102 can run an application program for recognizing the image to be recognized.

[0031] Further, the above method corresponds to the specific application process in the environment shown in FIG. 1, as shown in the following steps:

[0032] S102, the terminal device 102 sends the image to be recognized to the server 112 through the network 110 after obtaining the image to be recognized;

[0033] S104, the server 112 extracts image features from the image to be recognized after receiving the image to be recognized, wherein the image to be recognized displays an object to be recognized;

[0034] S106, the server 112 performs feature processing on the image features and the initialized reference key point features by the feature processing network to obtain the final key point features of the object to be recognized, wherein the final key point features are used to indicate the features of the key points recognized from the object to be recognized displayed in the image to be recognized, the feature processing network is based on the initialized feature processing network, and the Nth cycle feature processing is performed on each sample image, in the ith feature processing network for the ith cycle feature processing, the ith-1 sample feature pair obtained by the ith-1 cycle feature processing is processed to obtain the ith sample feature pair, and the network parameters in the ith feature processing network are adjusted based on the feature difference between the ith-1 sample feature pair and the ith sample feature pair until the convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the ith sample feature pair includes the ith sample image feature and the ith reference sample key point feature, the ith sample image feature is obtained by i times of cycle feature processing based on the original sample image feature extracted from the sample image, and the ith reference sample key point feature is obtained by i times of cycle feature processing based on the original reference sample key point feature;

[0035] S108, the server 112 determines the position of the key points of the object to be recognized in the image to be recognized by using the final key point features;

[0036] S110, the server 112 identifies the pose information of the to-be-identified object in the to-be-identified image by using the positions of the key points of the to-be-identified object in the to-be-identified image.

[0037] S112, the server 112 sends the pose information to the terminal device 102 through the network 110.

[0038] S114, the terminal device 102 displays the pose information in the case of receiving the pose information.

[0039] In the embodiment of the present application, the image features are extracted from the to-be-identified image, and the to-be-identified object is displayed in the to-be-identified image. Then, the image features and the initialized reference key point features are processed by the feature processing network to obtain the final key point features of the to-be-identified object, wherein the final key point features are used to indicate the features of the key points identified from the to-be-identified object displayed in the to-be-identified image, the feature processing network is based on the initialized feature processing network, and the N times of loop feature processing are performed on each sample image, in the i-th feature processing network for performing the i-th loop feature processing, the i-1-th sample feature pair obtained by the i-1-th loop feature processing is processed to obtain the i-th sample feature pair, and the network parameters in the i-th feature processing network are adjusted based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair until the convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes the i-th sample image feature and the i-th reference sample key point feature, the i-th sample image feature is obtained by i times of loop feature processing based on the original sample image feature extracted from the sample image, and the i-th reference sample key point feature is obtained by i times of loop feature processing based on the original reference sample key point feature. Then, the positions of the key points in the to-be-identified object in the to-be-identified image are determined by using the final key point features. In other words, by using the feature processing network trained based on the multiple loop self-distillation in the embodiment of the present application, the positions of the key points of the to-be-identified object included in the to-be-identified image in the to-be-identified image are located, and the feature processing network trained based on the multiple loop self-distillation is a lightweight network, which can improve the network performance by using a smaller number of parameters and lower complexity calculation, thereby reducing the time consumed by network training while ensuring the accuracy of network calculation results. Further, the technical problem of low image recognition efficiency in the image recognition method provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0040] Optionally, in the embodiment, the terminal device can be a terminal device configured with a target client, and can include but is not limited to at least one of a mobile phone (such as an Android mobile phone, an iOS mobile phone, etc.), a notebook computer, a tablet computer, a palm computer, a MID (Mobile Internet Device), a PAD, a desktop computer, a smart television, etc. The target client can be a video client, an instant messaging client, a browser client, an education client, etc. The network can include but is not limited to a wired network and a wireless network, wherein the wired network includes a local area network, a metropolitan area network and a wide area network, and the wireless network includes Bluetooth, WIFI and other wireless communication networks. The server can be a single server, a server cluster composed of multiple servers, or a cloud server. The above is only an example, and the embodiment is not limited in this regard.

[0041] Optionally, as an optional solution, as shown in FIG. 2, the image recognition method includes:

[0042] S202, extracting image features from the to-be-recognized image, wherein the to-be-recognized object is displayed in the to-be-recognized image;

[0043] S204, performing feature processing on the image features and the initialized reference key point features by using the feature processing network to obtain final key point features of the to-be-recognized object, wherein the final key point features are used to indicate features of key points recognized from the to-be-recognized object displayed in the to-be-recognized image, the feature processing network is based on the initialized feature processing network, and the N times of loop feature processing are performed on each sample image, in the i-th feature processing network performing the i-th loop feature processing, feature processing is performed on the i-1-th sample feature pair obtained by the i-1-th loop feature processing to obtain the i-th sample feature pair, and the network parameters in the i-th feature processing network are adjusted based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair until a convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes the i-th sample image feature and the i-th reference sample key point feature, the i-th sample image feature is obtained by i times of loop feature processing based on the original sample image feature extracted from the sample image, and the i-th reference sample key point feature is obtained by i times of loop feature processing based on the original reference sample key point feature obtained by initialization;

[0044] S206, determining the position of the key point of the to-be-recognized object in the to-be-recognized image by using the final key point features.

[0045] Optionally, the image recognition method can be applied in a processing scenario of a computer vision task, such as identity recognition, pose estimation, etc., but is not limited thereto in the embodiment. Specifically, in the identity recognition scenario, the image recognition method can be used to identify the positions of the key points of the to-be-recognized object (such as points in the palm print of the to-be-recognized object, points in the fingerprint of the to-be-recognized object, points in the pupil of the to-be-recognized object, etc.) displayed in the to-be-recognized image. Then, the key points in the to-be-recognized object are used to extract the key points from the to-be-recognized image, so as to further identify the identity of the to-be-recognized object by using the key points, such as identifying the identity of the to-be-recognized object by fingerprint detection, palm print detection, pupil detection, face detection, etc., which is not limited in the embodiment. In the pose estimation scenario, the image recognition method can be used to identify the positions of the key points of the to-be-recognized object (such as points in the skeleton of the to-be-recognized object, etc.) displayed in the to-be-recognized image, and then the positions are used to determine the pose of the to-be-recognized object, which can be used for further behavior analysis (such as helping to identify the action and pose of a human body or an animal by pose estimation, so as to realize the analysis and understanding of the behavior), pose alignment and registration (such as helping to align and register other objects or human bodies in different poses, so as to realize the matching and recognition of the objects or human bodies), pose generation and reconstruction (such as generating and reconstructing a three-dimensional model by using the pose, so as to realize the three-dimensional reconstruction and modeling of the objects or human bodies), etc.

[0046] It should be noted that in the embodiment, the to-be-recognized object is an object that needs to be identified by key points, which can be but is not limited to indicating a person, an object, an animal, etc., which is not limited in the embodiment. Further, assuming that the to-be-recognized object is a person object, the key points in the to-be-recognized object can be but are not limited to indicating points in a certain part of the to-be-recognized object, such as points in the skeleton of the to-be-recognized object, points in the palm print of the to-be-recognized object, points in the fingerprint of the to-be-recognized object, points in the pupil of the to-be-recognized object, etc., which is not limited in the embodiment, and is also dependent on the scenario to which the method is applied.

[0047] The reference key point feature can be a starting point of N times of loop feature processing, and the final key point feature is obtained by loop iteration optimization of the reference key point feature. The reference key point feature serves as a starting point of N times of loop feature processing, provides a direction for N times of loop feature processing, so that it can converge to the correct key point feature, i.e., the final key point feature, as soon as possible.

[0048] Wherein, the embodiment of the application does not limit the way of initialization, for example, it can be random initialization, or initialization based on prior knowledge, initialization based on statistical model, initialization based on image features, etc.

[0049] The image features are a kind of visual features of the to-be-recognized image, which can include edges, contours, textures, etc., and can reflect the shape and structure of the to-be-recognized object. By extracting and using these features, the key points of the to-be-recognized object can be more accurately located. After obtaining the reference key point features by initialization, the reference key point features are optimized by using the visual image features, which can further improve the accuracy and robustness of the prediction, and more accurate final key point features are obtained.

[0050] Further, the above-mentioned image features can be, but are not limited to, used to indicate the image features directly extracted from the above-mentioned to-be-recognized image, or can be used to indicate the image features obtained after conversion processing of the image features extracted from the above-mentioned to-be-recognized image, which is not limited in the embodiment. Specifically, the step of extracting all image features from the above-mentioned to-be-recognized image and then converting all image features to obtain the above-mentioned image features can include, but is not limited to:

[0051] S1, using a region recognition network to recognize the to-be-recognized image to determine the candidate region where the to-be-recognized object is located in the to-be-recognized image, wherein the region recognition network can be, but is not limited to, a commonly used target detection neural network model, such as a convolutional neural network (CNN), a one-time forward propagation neural network (YOLO), or other neural network models with similar functions, which is not limited in the embodiment.

[0052] S2, cropping the to-be-recognized image to retain the candidate region where the to-be-recognized object is located in the to-be-recognized image.

[0053] S3, using a feature extraction network to extract image features corresponding to the image content displayed in the candidate region, wherein the feature extraction network can be, but is not limited to, a commonly used deep learning model, such as a generative adversarial network (GAN), a pre-trained model, an autoencoder, a classic backbone network, or other neural network models with similar functions, which is not limited in the embodiment.

[0054] S4, adjusting the feature dimension of the image feature to a preset dimension to obtain an adjusted image feature, wherein the dimension of the image feature can be adjusted by using a linear neural network model (Linear) or other neural network models with similar functions, and the embodiment is not limited in this regard.

[0055] S5, performing position encoding processing on the adjusted image feature to obtain an image visual token determined as the image feature, wherein the image feature can be positionally encoded by using a position embedding model (Position Embedding) or other neural network models with similar functions, and the embodiment is not limited in this regard. Further, after the image feature is extracted from the image to be recognized, the image feature and the reference key point feature can be input into a feature processing network. It should be noted that in the embodiment, the feature processing network can be used to indicate a self-attention neural network model, and specifically, the self-attention neural network model can include a plurality of self-attention sub-neural networks, and each self-attention sub-neural network further includes a plurality of attention neural network layers.

[0056] Optionally, in the embodiment, the feature processing network is used to perform feature processing on the image feature to obtain the final key point feature of the key point in the object to be recognized, which can include the following operations performed in the feature processing network: inputting the j-1th image feature and the j-1th reference key point feature output by the j-1th feature processing sub-network in the feature processing network into the jth feature processing sub-network in the feature processing network, wherein the feature processing network includes P feature processing sub-networks, j is greater than 1 and less than or equal to P, and P is a positive integer; performing cyclic feature processing on the j-1th image feature and the j-1th reference key point feature in the jth feature processing sub-network to obtain the jth image feature and the jth reference key point feature; in the case where j is equal to P, determining the jth reference key point feature as the final key point feature of the key point in the object to be recognized; and in the case where j is less than P, inputting the obtained jth image feature and jth reference key point feature into the j+1th feature processing sub-network to perform the j+1th cyclic feature processing.

[0057] Optionally, in the embodiment, before the image features are extracted from the image to be recognized, the method can but is not limited to include: based on the initialized feature processing network, performing N times of loop feature processing on each sample image, in the i-th feature processing network for performing the i-th loop feature processing, performing feature processing on the i-1-th sample feature pair obtained by the i-1-th loop feature processing to obtain the i-th sample feature pair, and adjusting the network parameters in the i-th feature processing network based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair until a convergence condition is reached, i is greater than or equal to 1 and less than or equal to N, N is a positive integer.

[0058] Further, the determination of the positions of the key points in the image to be recognized in the object to be recognized based on the final key point features can but is not limited to include: mapping the final key point features by using a fully connected layer connected to the feature processing network to obtain the positions of the target key points in the image to be recognized, wherein the fully connected layer can but is not limited to be a network layer obtained by separate training or joint training with the feature processing network.

[0059] As an optional implementation, assuming that the image recognition method is applied to a pose detection scenario, the method can but is not limited to be completely and exemplarily explained based on the following steps:

[0060] S1, extracting image features from an image to be recognized, and obtaining reference key point features by initialization;

[0061] S2, inputting the image features and the reference key point features into a feature processing network;

[0062] S3, performing feature processing on the image features and the reference key point features by a feature processing network to obtain final key point features corresponding to the key points in the to-be-identified object, wherein the final key point features are used to indicate the features of the key points identified from the to-be-identified object displayed in the to-be-identified image, the feature processing network is obtained based on an initialized feature processing network, and the N times of loop feature processing are performed on each sample image, in the i-th feature processing network performing the i-th loop feature processing, feature processing is performed on the i-1-th sample feature pair obtained by the i-1-th loop feature processing to obtain an i-th sample feature pair, and the network parameters in the i-th feature processing network are adjusted based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair until a convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes an i-th sample image feature and an i-th reference sample key point feature, the i-th sample image feature is obtained by i times of loop feature processing based on the original sample image feature extracted from the sample image, and the i-th reference sample key point feature is obtained by i times of loop feature processing based on the original reference sample key point feature obtained by initialization;

[0063] S4, performing mapping processing on the final key point features by using a fully connected layer connected with the feature processing network to obtain the positions of the key points in the to-be-identified object in the to-be-identified image.

[0064] S5, identifying the posture of the to-be-identified object by using the positions.

[0065] As another optional implementation, assuming that the image recognition method is applied to a palm print-based identity detection scene, the method can be completely and exemplarily explained based on the following steps shown in FIG. 3, but is not limited thereto:

[0066] S302, obtaining a to-be-identified image, wherein a palm of a to-be-identified object is displayed in the to-be-identified image.

[0067] S304, palm detection, specifically, performing recognition processing on the to-be-identified image by using a region recognition network in a palm detection module to determine a candidate region where the palm is located in the to-be-identified image, and then performing cropping processing on the to-be-identified image to retain the candidate region where the palm is located in the to-be-identified image. Thus, the image features corresponding to the image content displayed in the candidate region are determined as the image features by using the feature extraction network.

[0068] S306, hand key point detection, specifically, input the image feature and the reference key point feature obtained by initialization into a feature processing network; perform feature processing on the image feature and the reference key point feature by the feature processing network to obtain a palm key point image feature corresponding to a palm key point. Then, perform mapping processing on the palm key point image feature by a fully connected layer connected to the feature processing network to obtain a position of the palm key point in the image to be recognized.

[0069] S308, palmprint recognition, specifically, extract a palm image based on the position of the palm key point in the image to be recognized, and then compare and analyze the palm image and the base library palm image to recognize the identity of the object to be recognized.

[0070] In the embodiment of the present application, the image feature is extracted from the image to be recognized, wherein the object to be recognized is displayed in the image to be recognized. Then, the image feature and the reference key point feature obtained by initialization are processed by the feature processing network to obtain the final key point feature of the object to be recognized, wherein the final key point feature is used to indicate the feature of the key point recognized from the object to be recognized displayed in the image to be recognized. The feature processing network is obtained by performing N times of cyclic feature processing on each sample image based on the initialized feature processing network. In the i-th feature processing network for performing the i-th cyclic feature processing, the i-1-th sample feature pair obtained by the i-1-th cyclic feature processing is processed to obtain the i-th sample feature pair, and the network parameters in the i-th feature processing network are adjusted based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair until the convergence condition is reached. i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes the i-th sample image feature and the i-th reference sample key point feature, the i-th sample image feature is obtained by performing i times of cyclic feature processing on the original sample image feature extracted from the sample image, and the i-th reference sample key point feature is obtained by performing i times of cyclic feature processing on the original reference sample key point feature obtained by initialization. Then, the position of the key point in the object to be recognized in the image to be recognized is determined by using the final key point feature. In other words, by using the feature processing network trained based on multiple cycles of self-distillation, the position of the key point in the object to be recognized included in the image to be recognized is located in the image to be recognized, while ensuring the accuracy of the network calculation result, the time consumed by network training is reduced. Furthermore, the technical problem of low image recognition efficiency in the image recognition method provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0071] Optionally, as an optional solution, the image features and the initialized reference key point features are processed by a feature processing network to obtain final key point features of the to-be-identified object, including:

[0072] The following operations are performed in the feature processing network:

[0073] The j-1th image features and the j-1th reference key point features output by the j-1th feature processing subnetwork in the feature processing network are input into the jth feature processing subnetwork in the feature processing network, wherein the feature processing network includes P feature processing subnetworks, j is greater than 1 and less than or equal to P, and P is a positive integer;

[0074] In the jth feature processing subnetwork, the j-1th image features and the j-1th reference key point features are processed by a loop feature processing to obtain jth image features and jth reference key point features;

[0075] In the case where j is equal to P, the jth reference key point features are determined as the final key point features;

[0076] In the case where j is less than P, the obtained jth image features and jth reference key point features are input into the j+1th feature processing subnetwork to perform j+1th loop feature processing.

[0077] Optionally, in the embodiment, the above-mentioned feature processing network can be but is not limited to indicating a self-attention neural network model, specifically, the above-mentioned self-attention neural network model can include but is not limited to a plurality of self-attention sub-neural networks, and each self-attention sub-neural network in the above-mentioned plurality of self-attention sub-neural networks further includes a plurality of attention neural network layers.

[0078] As an optional implementation, the above-mentioned feature processing network includes P feature processing subnetworks, which are feature processing subnetwork 1, feature processing subnetwork 2, …, and feature processing subnetwork P, for example, the above-mentioned method can be but is not limited to explained and described based on the following steps as shown in FIG. 4:

[0079] The image feature and the reference key point feature are input into a feature processing subnetwork 1 in the feature processing network, and the image feature and the reference key point feature are processed by using the feature processing subnetwork 1 to obtain a first image feature and a first reference key point feature. It is determined that the feature processing subnetwork 1 is not the last feature processing subnetwork, and then the first image feature and the first reference key point feature are input into a feature processing subnetwork 2 in the feature processing network, and the first image feature and the first reference key point feature are processed by using the feature processing subnetwork 2 to obtain a second image feature and a second reference key point feature. It is determined that the feature processing subnetwork 2 is not the last feature processing subnetwork, and the second image feature and the second reference key point feature are continuously input into a next feature processing subnetwork in the feature processing network for feature processing. In this way, a P-1th image feature and a P-1th reference key point feature are obtained, and then the P-1th image feature and the P-1th reference key point feature are input into a feature processing subnetwork P in the feature processing network, and the P-1th image feature and the P-1th reference key point feature are processed by using the feature processing subnetwork P to obtain a Pth image feature and a Pth reference key point feature. It is determined that the feature processing subnetwork P is the last feature processing subnetwork, and thus the Pth reference key point feature is determined as the final key point feature.

[0080] It should be noted that the above example is only one example in the embodiment, and the above steps can also be implemented in other manners, and the embodiment does not limit the same.

[0081] In the embodiment of the present application, the plurality of feature processing subnetworks included in the feature processing network trained by using the multi-cycle self-distillation are used to process the image feature to obtain the final key point feature, so that the obtained final key point feature is more reliable. Thus, the technical effect of improving the image recognition accuracy is achieved.

[0082] Optionally, as an optional solution, in the jth feature processing subnetwork, the j-1th image feature and the j-1th reference key point feature are processed to obtain the jth image feature and the jth reference key point feature, which includes:

[0083] In the jth feature processing subnetwork, the following operations are performed:

[0084] In the first feature processing layer of the jth feature processing subnetwork, the image sub-feature output by the j-1th feature processing subnetwork is processed by linear transformation to obtain the first image sub-feature; and the reference key point sub-feature output by the j-1th feature processing subnetwork is processed by linear transformation to obtain the first reference key point sub-feature.

[0085] In the hth feature processing layer of the jth feature processing subnetwork, the (h-1)th image sub-feature and the (h-1)th reference key point sub-feature output by the (h-1)th feature processing layer of the jth feature processing subnetwork are input into the hth feature processing layer in the jth feature processing subnetwork, where h is a positive integer greater than or equal to 2;

[0086] In the hth feature processing layer, the (h-1)th image sub-feature is subjected to linear transformation processing to obtain a first processing result, and the first processing result is determined as the hth image sub-feature output by the hth feature processing layer; and the (h-1)th reference key point sub-feature is subjected to linear transformation processing to obtain a second processing result; the first processing result and the second processing result are used to obtain the hth reference key point sub-feature output by the hth feature processing layer;

[0087] In the case where the hth feature processing layer is the last feature processing layer in the jth feature processing subnetwork, the hth image sub-feature is determined as the jth image feature output by the jth feature processing subnetwork, and the hth reference key point sub-feature is determined as the jth reference key point feature output by the jth feature processing subnetwork.

[0088] Optionally, as an optional embodiment, the above feature processing network includes P feature processing subnetworks as shown in FIG. 5, which are feature processing subnetwork 1, feature processing subnetwork 2, …, and feature processing subnetwork P; each feature processing subnetwork includes Y feature processing layers, for example, feature processing subnetwork 1 includes feature processing layer 1-1, feature processing layer 1-2, …, and feature processing layer 1-Y; feature processing subnetwork 2 includes feature processing layer 2-1, feature processing layer 2-2, …, and feature processing layer 2-Y; and feature processing subnetwork P includes feature processing layer P-1, feature processing layer P-2, …, and feature processing layer P-Y, which are explained and described by the following steps:

[0089] S502, the image feature and the reference key point feature are input into the feature processing layer 1-1 in the feature processing subnetwork 1, and the feature processing layer 1-1 is used to process the image feature and the reference key point feature to obtain the image feature 1-1 and the reference key point feature 1-1;

[0090] The feature processing of the image feature and the reference key point feature by the feature processing layer 1-1 can include but is not limited to the following: performing linear transformation on the image feature to obtain a query vector, a key vector and a value vector corresponding to the image feature; and performing linear transformation on the reference key point feature to obtain a query vector, a key vector and a value vector corresponding to the reference key point feature; then, determining the image feature 1-1 by using the query vector, the key vector and the value vector corresponding to the image feature; and determining the reference key point feature 1-1 by using the query vector, the key vector and the value vector corresponding to the reference key point feature.

[0091] For example, the implementation of determining the image feature 1-1 by using the query vector, the key vector and the value vector corresponding to the image feature, and determining the reference key point feature 1-1 by using the query vector, the key vector and the value vector corresponding to the reference key point feature can include but is not limited to the following examples:

[0092] The s a1 is used to represent the image feature 1-1, and the g a1 is used to represent the reference key point feature 1-1, and the Q s ,K s ,V s are respectively used to represent the query vector, the key vector and the value vector corresponding to the image feature, and the Q g ,K g ,V g are respectively used to represent the query vector, the key vector and the value vector corresponding to the reference key point feature, and the d ks is used to represent the dimension of K s , and the d kg is used to represent the dimension of K g .

[0093] S504, input the image feature 1-1 and the reference key point feature 1-1 into the feature processing layer 1-2 in the feature processing sub-network 1, and perform feature processing on the image feature 1-1 and the reference key point feature 1-1 by using the feature processing layer 1-2 to obtain the image feature 1-2 and the reference key point feature 1-2.

[0094] The feature processing on the image feature 1-1 and the reference key point feature 1-1 by the feature processing layer 1-2 to obtain the image feature 1-2 and the reference key point feature 1-2 can but not limited to include: performing linear transformation processing on the image feature 1-1 to obtain the image feature 1-2; and performing linear transformation processing on the reference key point feature 1-1 to obtain the query vector and the key vector corresponding to the reference key point feature 1-1; then, the query vector and the key vector corresponding to the image feature 1-2 and the reference key point feature 1-1 are used to determine the reference key point feature 1-2.

[0095] For example, the implementation of determining the reference key point feature 1-2 by using the query vector and the key vector corresponding to the image feature 1-2 and the reference key point feature 1-1 can but not limited to refer to the following examples:

[0096] Wherein, the above g a2 is used to represent the reference key point feature 1-2, the above Q g1 is used to represent the query vector corresponding to the reference key point feature 1-1, and the above K g1 is used to represent the key vector corresponding to the reference key point feature 1-1, and the above s a2 is used to represent the image feature 1-2 obtained by performing linear transformation processing on the image feature 1-1, and the above d kg1 is used to represent the K g1 dimension.

[0097] By analogy, until the image feature 1-Y and the reference key point feature 1-Y output by the feature processing layer 1-Y are obtained.

[0098] S506, input the image feature 1-Y and the reference key point feature 1-Y into the feature processing layer 2-1 in the feature processing subnetwork 2, and perform feature processing on the image feature 1-Y and the reference key point feature 1-Y by using the feature processing layer 2-1 to obtain the image feature 2-1 and the reference key point feature 2-1. For specific embodiments, please refer to the examples shown in the feature processing layer 1-1, which will not be described in detail in this embodiment.

[0099] S508, input the image feature 2-1 and the reference key point feature 2-1 into the feature processing layer 2-2 in the feature processing subnetwork 2, and perform feature processing on the image feature 2-1 and the reference key point feature 2-1 by using the feature processing layer 2-2 to obtain the image feature 2-2 and the reference key point feature 2-2; wherein, for specific embodiments, please refer to the examples shown in the feature processing layer 1-2.

[0100] Similarly, until the image feature 2-Y and the reference key point feature 2-Y output by the feature processing layer 2-Y are obtained. Then, the image feature 2-Y and the reference key point feature 2-Y are input into the next feature processing sub-network, until the image feature P-Y and the reference key point feature P-Y output by the feature processing layer P-Y are obtained.

[0101] S510, determining that the feature processing layer P-Y is the last feature processing layer in the last feature processing sub-network in the feature processing network, and further determining the reference key point feature P-Y as the final key point feature.

[0102] In the embodiments of the present application, by using the multiple layers of feature processing layers included in each of the multiple feature processing sub-networks to perform feature processing on the image feature, the final key point feature of the object to be recognized is obtained, which makes the obtained final key point feature more reliable. Thus, the technical effect of improving the image recognition accuracy is achieved.

[0103] Optionally, as an optional solution, before the image feature is extracted from the image to be recognized, the method further includes:

[0104] The initialized feature processing network is trained by the following steps:

[0105] The original sample image feature is extracted from the kth sample image obtained, and the original reference sample key point feature obtained by initialization is obtained, where k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer;

[0106] The following operations are repeatedly performed on the original sample image feature and the original reference sample key point feature until the Nth cycle feature processing is reached:

[0107] In the i-th feature processing network for performing the i-th cycle feature processing, the i-1 sample feature pair obtained by the i-1 cycle feature processing is subjected to feature processing to obtain the i-th sample feature pair, where in the case that i is 1, the i-1 sample feature pair obtained by the i-1 cycle feature processing is used to indicate the original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature;

[0108] Based on the feature difference between the i-1 sample feature pair and the i-th sample feature pair, the sample feature distance between the i-1 sample feature pair and the i-th sample feature pair is determined;

[0109] The first i sample feature distances are weighted and summed to obtain the i-th feature training loss;

[0110] In a case where the feature training loss determined based on the sample feature distances obtained after the i-th loop feature processing has not reached the threshold condition, the network parameters in the i-th feature processing network are adjusted to obtain an (i+1)-th feature processing network; and the (i+1)-th loop feature processing is performed in the (i+1)-th feature processing network.

[0111] In a case where the feature training loss determined based on the sample feature distances obtained after the i-th loop feature processing has reached the threshold condition, the i-th loop feature processing is determined as the N-th loop feature processing.

[0112] Optionally, the N can be, but is not limited to, used to indicate a target threshold corresponding to a highest number of times of training the feature processing network with each sample image, and can also be used to indicate a training number when the feature processing network trained with a certain sample image reaches a convergence condition. The training number when the convergence condition is reached is less than or equal to the target threshold. For example, assuming that there are M sample images, each of which is used to train the feature processing network at most x times, and assuming that the feature processing network has not reached the convergence condition until the x-th time of training with the second sample image, the x-th loop feature processing performed in the x-th feature processing network can be determined as the N-th loop feature processing corresponding to the second sample image. Assuming that the feature processing network has reached the convergence condition (i.e., the feature training loss reaches the threshold condition) at the 2nd time of training with the third sample image, the 2nd loop feature processing performed in the 2nd feature processing network can be determined as the N-th loop feature processing corresponding to the second sample image.

[0113] It should be noted that in the embodiment, after the network parameters in the i-th feature processing network are adjusted to obtain the (i+1)-th feature processing network in a case where the feature training loss determined based on the sample feature distances obtained after the i-th loop feature processing has not reached the threshold condition, it further includes determining whether the i is equal to a target threshold. The i is a positive integer and is greater than 1.

[0114] Further, the (i+1)-th loop feature processing performed in the (i+1)-th feature processing network can include, but is not limited to, performing the (i+1)-th loop feature processing in the (i+1)-th feature processing network in a case where it is determined that the i is less than the target threshold. Optionally, after the determination of whether the i is equal to the target threshold, it further includes determining the i-th loop feature processing as the N-th loop feature processing in a case where it is determined that the i is equal to the target threshold.

[0115] Optionally, in the embodiment, in the case that the feature training loss determined based on all the sample feature distances obtained after the i-th cycle feature processing reaches the threshold condition, the i-th cycle feature processing is determined to be after the N-th cycle feature processing, and the method further comprises: determining the i-th feature processing network as the feature processing network that reaches the convergence condition.

[0116] It should be noted that, in the embodiment, the specific implementation of extracting the original sample image feature from the obtained k-th sample image can be, but is not limited to, referring to the related examples of obtaining the image feature based on the to-be-recognized image in the foregoing, and the embodiment is not limited in this regard. Optionally, the M can be, but is not limited to, used to indicate the total number of sample images.

[0117] Optionally, in the embodiment, the sample feature distance between the i-1-th sample feature pair and the i-th sample feature pair based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair comprises: performing distance calculation on the i-1-th sample image feature in the i-1-th sample feature pair and the i-th sample image feature in the i-th sample feature pair to obtain an i-1-th sample image feature distance; and performing distance calculation on the i-1-th reference sample key point feature in the i-1-th sample feature pair and the i-th reference sample key point feature in the i-th sample feature pair to obtain an i-1-th reference sample key point feature distance.

[0118] Further, the i-th feature training loss obtained by performing weighted sum calculation on the first i sample feature distances comprises: performing weighted sum calculation on the first i sample image feature distances to obtain a feature training loss, and performing weighted sum calculation on the first i reference sample key point feature distances to obtain a reference key point feature training loss; and performing weighted sum calculation on the feature training loss and the reference key point feature training loss to obtain the i-th feature training loss.

[0119] Optionally, as an optional implementation, the method can be exemplarily explained and described based on the following steps:

[0120] The original sample image feature is extracted from the obtained k-th sample image. Then, the original sample image feature and the initialized original reference sample key point feature are input into the initialized feature processing network, and the original sample image feature and the original reference sample key point feature are subjected to first cycle feature processing by using the initialized feature processing network to obtain a first sample image feature and a first reference sample key point feature.

[0121] Further, a first sample image feature distance between the first sample image feature and the original sample image feature is obtained, and a first reference sample key point feature distance between the first reference sample key point feature and the original reference sample key point feature is obtained; then, based on the first sample image feature distance and the first reference sample key point feature distance, a first sample feature distance is obtained; thus, in a case where the feature training loss determined based on the first sample feature distance (i.e., all sample feature distances obtained after the previous cycle feature processing) reaches the threshold condition, the first cycle feature processing is determined as the Nth cycle feature processing.

[0122] In a case where the feature training loss determined based on the first sample feature distance does not reach the threshold condition, the network parameters in the initialized feature processing network are adjusted to obtain a second feature processing network. Then, the first sample image feature and the first reference sample key point feature are input into the second feature processing network, and the first sample image feature and the first reference sample key point feature are subjected to cycle feature processing by using the second feature processing network to obtain a second sample image feature and a second reference sample key point feature. Further, a second sample image feature distance between the first sample image feature and the second sample image feature is obtained, and a second reference sample key point feature distance between the first reference sample key point feature and the second reference sample key point feature is obtained; then, the first sample image feature distance and the second sample image feature distance are weighted and summed to obtain a first summation result, and the first reference sample key point feature distance and the second reference sample key point feature distance are weighted and summed to obtain a second summation result; then, the first summation result and the second summation result are weighted and summed to obtain a second sample feature distance.

[0123] Thus, in a case where the feature training loss determined based on the second sample feature distance reaches the threshold condition, the second cycle feature processing is determined as the Nth cycle feature processing. In a case where the feature training loss determined based on the second sample feature distance does not reach the threshold condition, the network parameters in the second feature processing network are adjusted to obtain a third feature processing network. Then, the second sample image feature and the second reference sample key point feature are input into the third feature processing network, and the above steps performed in the second feature processing network are referred to, and the feature processing network is continuously trained and adjusted until the feature processing network reaches a convergence condition (i.e., the training loss reaches the threshold condition), or until the number of cycle feature processing reaches a target threshold, which will not be described herein.

[0124] In the embodiment of the present application, the initialized feature processing network is trained in a way based on multiple cycle self-distillation to obtain a usable feature processing network. While ensuring the accuracy of the network calculation result, the time length consumed by the network training is reduced. In addition, in the process of the cyclic iteration training of the feature processing network itself, new sample data is continuously generated (i.e., the sample image feature obtained after each cycle of feature processing and the reference sample key point feature can be used as a new sample data), thereby greatly reducing the loss of resources in the training process. Thus, the technical problem of low image recognition efficiency in the image recognition manner provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0125] Optionally, as an optional solution, determining the sample feature distance between the i-1th sample feature pair and the ith sample feature pair based on the feature difference between the i-1th sample feature pair and the ith sample feature pair includes:

[0126] performing distance calculation on the i-1th sample image feature in the i-1th sample feature pair and the ith sample image feature in the ith sample feature pair to obtain an i-1th sample image feature distance.

[0127] Optionally, in the embodiment, the implementation of the above steps can refer to the following examples, but is not limited to the following examples:

[0128] wherein the above is used to represent the i-1th sample image feature distance, and the above VT i-1 is used to represent the i-1th sample image feature, and the above VT i is used to represent the ith sample image feature.

[0129] performing distance calculation on the i-1th reference sample key point feature in the i-1th sample feature pair and the ith reference sample key point feature in the ith sample feature pair to obtain an i-1th reference sample key point feature distance.

[0130] Optionally, in the embodiment, the implementation of the above steps can refer to the following examples, but is not limited to the following examples:

[0131] wherein the above is used to represent the i-1th reference sample key point feature distance, and the above KT i-1 is used to represent the i-1th reference sample key point feature, and the above KT i is used to represent the ith reference sample key point feature.

[0132] It should be noted that the above examples indicate an optional example of the above steps, and other manners can also be used to implement the above steps in this embodiment, and this is not limited in this embodiment.

[0133] In the embodiment of the application, the distance of the sample image features output by each adjacent two feature processing networks is obtained, the distance of the reference sample key point features output by each adjacent two feature processing networks is obtained, and then the distance of the sample image features and the distance of the reference sample key point features are used to determine the feature training loss value for training the feature processing network. The purpose of self-distillation cycle training of the feature processing network is achieved, thereby reducing the time length consumed by network training. The technical problem of low image recognition efficiency in the image recognition manner provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0134] Optionally, as an optional solution, the first i sample feature distances are weighted and summed to obtain the i-th feature training loss, which includes:

[0135] The first i sample image feature distances are weighted and summed to obtain the sample image feature training loss, and the first i reference sample key point feature distances are weighted and summed to obtain the reference key point feature training loss.

[0136] The sample image feature training loss and the reference key point feature training loss are weighted and summed to obtain the i-th feature training loss.

[0137] Optionally, in this embodiment, the implementation of the above steps can be but not limited to the following examples:

[0138] VT x-1 VT is used to indicate the (x-1)-th sample image feature, and KT x KT is used to indicate the x-th sample image feature, and KT x-1 KT is used to indicate the (x-1)-th reference sample key point feature, and KT x KT is used to indicate the x-th reference sample key point feature, and is used to indicate the sample image feature training loss, and is used to indicate the reference key point feature training loss, and is used to indicate the i-th feature training loss, and α2 and α1 are respectively used to indicate the respective weights of the sample image feature training loss and the reference key point feature training loss.

[0139] It should be noted that the above examples indicate an optional example of the above steps, and other manners can also be used to implement the above steps in this embodiment, and this embodiment does not make any limitation on this.

[0140] In the embodiment of the present application, the distance of the sample image features output by each adjacent two feature processing networks is obtained, and the distance of the reference sample key point features output by each adjacent two feature processing networks is obtained, and then the distance of the sample image features and the distance of the reference sample key point features are used to determine the feature training loss value for training the feature processing network. The purpose of self-distillation cycle training the feature processing network is achieved, thereby reducing the time length consumed by network training. Furthermore, the technical problem of low image recognition efficiency existing in the image recognition manner provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0141] Optionally, as an optional solution, after determining the sample feature distance between the i-1 sample feature pair and the i sample feature pair based on the feature difference between the i-1 sample feature pair and the i sample feature pair, the method further includes:

[0142] inputting the i reference sample key point feature in the i sample feature pair into the i fully connected layer to obtain the i key point prediction position;

[0143] comparing the i key point prediction position with the key point label position in the k sample image to obtain the i key point prediction difference distance;

[0144] performing weighted summation calculation on the first i key point prediction difference distances to obtain the i key point prediction training loss;

[0145] performing weighted summation calculation on the i feature training loss and the i key point prediction training loss to obtain the i target loss;

[0146] in the case where the i target loss has not reached the threshold condition, adjusting the network parameters in the i feature processing network and the i fully connected layer to obtain the i+1 feature processing network and the i+1 fully connected layer; performing the i+1 cycle feature processing in the i+1 feature processing network, and obtaining the i+1 key point prediction position in the i+1 fully connected layer;

[0147] in the case where the i target loss reaches the threshold condition, determining that the i cycle feature processing is the N cycle feature processing.

[0148] Optionally, in this embodiment, the implementation of comparing the i key point prediction position with the key point label position in the k sample image to obtain the i key point prediction difference distance can be but is not limited to referring to the following examples:

[0149] wherein pred wherein pred i wherein GT denotes the key point label position.

[0150] Further, in the embodiment, the implementation of the weighted sum calculation of the first i key point prediction difference distances to obtain the i-th key point prediction training loss can be but not limited to referring to the following examples:

[0151] wherein pred x wherein GT denotes the key point label position, and pred wherein pred

[0152] Further, in the embodiment, the implementation of the weighted sum calculation of the i-th feature training loss and the i-th key point prediction training loss to obtain the i-th target loss can be but not limited to referring to the following examples:

[0153] wherein pred wherein pred wherein pred wherein pred wherein pred

[0154] It should be noted that the above examples indicate one optional example of the above steps, and other manners can also be used to implement the above steps in the embodiment, and the embodiment does not make any limitation in this regard.

[0155] In the embodiment, the initialized feature processing network and the fully connected layer connected with the feature processing network are jointly trained based on the multi-cycle self-distillation manner to obtain the available feature processing network and the fully connected layer. The network calculation result accuracy is guaranteed, and the time length consumed by network training is reduced. In addition, new sample data is continuously generated in the process of the self-cycle iteration training of the feature processing network (i.e., the sample image feature and the reference sample key point feature obtained after each cycle of feature processing can be used as a new sample data), thereby greatly reducing the resource consumption in the training process. Thus, the technical problem of low image recognition efficiency in the image recognition manner provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0156] Optionally, as an optional solution, the pre-processing method of image recognition includes:

[0157] Obtaining M sample images and an initialized feature processing network, and training through the following steps:

[0158] Extracting an original sample image feature from the obtained kth sample image, and obtaining an initialized original reference sample key point feature, wherein k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer;

[0159] Repeating the following operations on the original sample image feature and the original reference sample key point feature until N times of loop feature processing are reached:

[0160] In the ith feature processing network for performing the ith loop feature processing, performing feature processing on the i-1 sample feature pair obtained by the i-1 loop feature processing to obtain the ith sample feature pair, wherein when i is 1, the i-1 sample feature pair obtained by the i-1 loop feature processing is used to indicate the original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature;

[0161] Based on the feature difference between the i-1 sample feature pair and the ith sample feature pair, determining the sample feature distance between the i-1 sample feature pair and the ith sample feature pair;

[0162] Weighted sum calculation is performed on the first i sample feature distances to obtain the ith feature training loss; in the case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not reached the threshold condition, adjusting the network parameters in the ith feature processing network to obtain the i+1 feature processing network;

[0163] Performing the i+1 loop feature processing in the i+1 feature processing network;

[0164] In the case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing reaches the threshold condition, the ith loop feature processing is determined as the Nth loop feature processing.

[0165] Optionally, the N can be, but is not limited to, used to indicate a target threshold corresponding to a maximum number of times of training the feature processing network by each sample image, and can also be used to indicate a training number when the feature processing network is trained by a certain sample image to reach a convergence condition. Wherein, the training number when the convergence condition is reached is less than or equal to the target threshold. For example, assuming that there are M sample images, each of which is used for training at most x times, and assuming that the feature processing network has not reached the convergence condition until the xth training by the second sample image, the xth loop feature processing performed in the xth feature processing network can be determined as the Nth loop feature processing corresponding to the second sample image. Assuming that the feature processing network has reached the convergence condition (i.e., the feature training loss reaches the threshold condition) when training to the 2nd time by the third sample image, the 2nd loop feature processing performed in the 2nd feature processing network can be determined as the Nth loop feature processing corresponding to the second sample image.

[0166] It should be noted that in the present embodiment, after the feature training loss determined based on all sample feature distances obtained after the i-1th loop feature processing has not reached the threshold condition, the network parameters in the ith feature processing network are adjusted to obtain the (i+1)th feature processing network, and then the following steps are further included: determining whether the i is equal to the target threshold. Wherein, the i is a positive integer greater than 1.

[0167] Further, the i+1th loop feature processing performed in the i+1th feature processing network can include, but is not limited to, the following: in the case where it is determined that the i is less than the target threshold, the i+1th loop feature processing is performed in the i+1th feature processing network. Optionally, after the determination of whether the i is equal to the target threshold, the following steps are further included: in the case where it is determined that the i is equal to the target threshold, the ith loop feature processing is determined as the Nth loop feature processing.

[0168] Optionally, in the present embodiment, after the feature training loss determined based on all sample feature distances obtained after the i-1th loop feature processing reaches the threshold condition, the method further includes: determining the ith loop feature processing as the Nth loop feature processing.

[0169] It should be noted that in the present embodiment, the specific implementation of extracting the original sample image feature from the kth acquired sample image can refer to, but is not limited to, the related examples of acquiring image features based on the to-be-recognized image in the foregoing, and is not limited in the present embodiment. Optionally, the M can be, but is not limited to, used to indicate the total number of sample images.

[0170] Optionally, in the embodiment, the determining, based on the feature difference between the i-1th sample feature pair and the i sample feature pair, of a sample feature distance between the i-1th sample feature pair and the i sample feature pair comprises: performing distance calculation on the i-1th sample image feature in the i-1th sample feature pair and the i sample image feature in the i sample feature pair to obtain an i-1th sample image feature distance; and performing distance calculation on the i-1th reference sample key point feature in the i-1th sample feature pair and the i reference sample key point feature in the i sample feature pair to obtain an i-1th reference sample key point feature distance.

[0171] Further, the obtaining, by performing weighted summation calculation on the first i sample feature distances, of an ith feature training loss comprises: performing weighted summation calculation on the first i sample image feature distances to obtain a sample image feature training loss, and performing weighted summation calculation on the first i reference sample key point feature distances to obtain a reference key point feature training loss; and performing weighted summation calculation on the sample image feature training loss and the reference key point feature training loss to obtain the i th feature training loss.

[0172] DETAILED DESCRIPTION, reference is made to the related embodiments in the image recognition method, which will not be described herein again.

[0173] In the embodiments of the present application, the initialized feature processing network is trained in a manner based on multiple loop self-distillation to obtain a usable feature processing network. While ensuring the accuracy of the network calculation result, the time length consumed by the network training is reduced. In addition, in the process of the loop iteration training of the feature processing network itself, new sample data is also constantly generated (i.e., the sample image feature and the reference sample key point feature obtained after each loop feature processing can be used as a new sample data), thereby greatly reducing the resource consumption in the training process. Thus, the technical problem of low image recognition efficiency existing in the image recognition manner provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0174] Optionally, as an optional solution, in the i th feature processing network performing the i th loop feature processing, after performing the feature processing on the i-1th sample feature pair obtained by the i-1th loop feature processing to obtain the i sample feature pair, the method further comprises:

[0175] inputting the i reference sample key point feature in the i sample feature pair into the i fully connected layer to obtain an i key point prediction position;

[0176] comparing the i key point prediction position with the key point label position in the k sample image to obtain an i key point prediction difference distance;

[0177] The i-th key point prediction training loss is calculated by weighted sum of the i-th key point prediction difference distance;

[0178] The i-th target loss is calculated by weighted sum of the i-th feature training loss and the i-th key point prediction training loss;

[0179] In the case where the i-th target loss has not reached the threshold condition, the network parameters in the i-th feature processing network and the i-th fully connected layer are adjusted to obtain the i+1-th feature processing network and the i+1-th fully connected layer; the i+1-th loop feature processing is performed in the i+1-th feature processing network, and the i+1-th key point prediction position is obtained in the i+1-th fully connected layer;

[0180] In the case where the i-th target loss reaches the threshold condition, the i-th loop feature processing is determined as the N-th loop feature processing.

[0181] DETAILED DESCRIPTION, reference is made to the related embodiments in the image recognition method, which will not be described in detail in this embodiment.

[0182] In the embodiments of the present application, the initialized feature processing network and the fully connected layer connected with the feature processing network are jointly trained based on the multi-loop self-distillation manner to obtain the available feature processing network and the fully connected layer. While ensuring the accuracy of the network calculation result, the time length consumed by the network training is reduced. In addition, in the process of loop iteration training of the feature processing network itself, new sample data is also constantly generated (i.e. the sample image features and reference sample key point features obtained after each loop feature processing can be used as a new sample data), thereby greatly reducing the resource consumption in the training process. Thus, the technical problem of low image recognition efficiency in the image recognition method provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0183] Optionally, as an optional implementation, assuming that the above image recognition method is applied to a pose estimation scene, the above method can be completely exemplarily explained and described based on the following steps as shown in FIG. 6, but is not limited thereto:

[0184] S602, obtaining an image to be recognized.

[0185] S604, extracting a candidate region, specifically, using a region recognition network to determine a candidate region where an object to be recognized is located in the image to be recognized, and cutting out the candidate region from the image to be recognized.

[0186] S606, extracting image features, specifically, using a Backbone network to extract image features corresponding to the image content displayed by the candidate region.

[0187] S608, image feature division, specifically, dividing the image feature into a plurality of sub-image features.

[0188] S610, adjusting feature dimension, specifically, adjusting the feature dimension of the divided image feature by using a Linear network to obtain an adjusted image feature.

[0189] S612, feature position encoding, specifically, using a Position Embedding network to perform position encoding on the adjusted image feature to obtain W*H visual tokens VT (used to indicate the image feature mentioned above).

[0190] S614, self-attention processing, specifically, inputting the additional K key point tokens KT (used to indicate the reference key point feature mentioned above) into a self-attention layer model (used to indicate the feature processing network mentioned above) to obtain K key point tokens KT1 processed by using the self-attention mechanism and W*H visual tokens VT1, and determining the K key point tokens KT1 processed by using the self-attention mechanism as the final key point feature. Wherein, the above-mentioned feature processing network is based on an initialized feature processing network, and performs N times of cyclic feature processing on each sample image, in the i-th feature processing network for performing the i-th cyclic feature processing, performing feature processing on the i-1-th sample feature pair obtained by the i-1-th cyclic feature processing to obtain the i-th sample feature pair, and adjusting the network parameters in the i-th feature processing network based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair until the convergence condition is reached, i is greater than or equal to 1 and less than or equal to N, N is a positive integer.

[0191] S616, obtaining the position of the key point of the to-be-recognized object, specifically, inputting the K key point tokens KT processed by using the self-attention mechanism into a fully connected layer prediction head (used to indicate the fully connected layer connected with the feature processing network mentioned above) to obtain the position of the key point of the to-be-recognized object in the to-be-recognized image.

[0192] S618, determining the pose of the to-be-recognized object, specifically, using the position of the key point of the to-be-recognized object in the to-be-recognized image to determine the motion pose of the to-be-recognized object in the to-be-recognized image.

[0193] In the embodiments of the present application, by using the feature processing network trained based on multiple cyclic self-distillation, the position of the key point of the to-be-recognized object included in the to-be-recognized image in the to-be-recognized image is located, while ensuring the accuracy of the network calculation result, the time length consumed by network training is reduced. Further, the technical problem of low image recognition efficiency existing in the image recognition manner provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0194] Optionally, as an optional embodiment, the pre-processing method of the above image recognition can be fully exemplarily explained based on the following steps as shown in FIG. 7, but is not limited thereto:

[0195] S702 is performed to obtain a sample image. Then S704-S712 are performed to extract a candidate region, specifically, a region recognition network is used to determine a candidate region in which the to-be-recognized object in the sample image is located in the to-be-recognized image, and the candidate region is cropped from the sample image. Image features are extracted, specifically, a Backbone network is used to extract image features corresponding to the image content displayed by the candidate region. The image features are divided, specifically, the image features are divided into a plurality of sub-image features. The feature dimension is adjusted, specifically, a Linear network is used to adjust the feature dimension of the divided image features to obtain adjusted image features. The feature position is encoded, specifically, a Position Embedding network is used to encode the position of the adjusted image features to obtain W*H visual tokens VT (used to indicate the sample image features in the foregoing).

[0196] Then S714-1 is performed, N times of self-attention layer model processing, specifically, K key point tokens KT (used to indicate the reference sample key point features in the foregoing) initialized additionally are input into a self-attention layer model (used to indicate the feature processing network in the foregoing) and processed N times by the self-attention layer model. For the i-th processing, the output of the self-attention layer model is new visual tokens VT i and key point tokens KT i , the input is the visual tokens VT i-1 and key point tokens KT i-1 of the last cycle. And S714-2 is performed, N times of sample position prediction, specifically, for each output key point token KT i ,i∈{1,2…N}, it is sent into the same fully connected layer prediction head to obtain pose estimation output pred i ,i∈{1,2…N}. Each pose estimation output contains K key point coordinates in the pose

[0197] Further, S716 is performed to calculate a training loss, specifically, a sum of minimized MSE distances between each training prediction pose estimation output and a target pose is obtained a total loss between each adjacent two key point tokens KT is obtained a total loss between each adjacent two visual tokens VT is obtained is used The training loss value is generated. Then, in a case where the training loss value reaches a threshold value, it is determined that the self-attention layer model and the fully connected layer prediction head reach a convergence condition. Otherwise, the self-attention layer model and the fully connected layer prediction head will continue to be trained according to the above steps.

[0198] In the embodiment of the present application, the initialized feature processing network is trained in a manner based on multiple cycle self-distillation to obtain a usable feature processing network. While ensuring the accuracy of the network calculation result, the time length consumed by the network training is reduced. In addition, in the process of the cyclic iteration training of the feature processing network itself, new sample data is also constantly generated (i.e., the sample image feature obtained after each cycle of feature processing and the reference sample key point feature can be used as a new sample data), thereby greatly reducing the loss of resources in the training process. Thus, the technical problem of low image recognition efficiency existing in the image recognition manner provided by the related art is solved, and the technical effect of improving the image recognition efficiency is achieved.

[0199] Table 1 Comparison of results of the embodiment and the related art on the validation set of the public dataset MSCOCO [8]

[0200] Table 1 shows the effects of the embodiment and other related technologies in the field of pose estimation on the validation set of the public dataset (MSCOCO), and the evaluation indicators are parameter quantity, GFLOPs and mAP. Among them, the parameter quantity and GFLOPs represent the speed of the model, and the smaller the parameter quantity and GFLOPs, the faster the model speed; the mAP represents the accuracy of the model prediction, and the higher the mAP, the more accurate the model prediction. Here, ResNet-50 and Stemnet are small backbone networks, and ResNet-152 and HRNet are large backbone networks. The larger the W coefficient of HRNet, the deeper and wider the network, and the larger the model. Compared with Stemnet, ResNet-50 is larger, and the size of HRNet-W32 and ResNet-152 does not differ much. That is, the effect of the embodiment far exceeds that of other related technologies.

[0201] It should be noted that, for each of the foregoing method embodiments, in order to simply describe, each is described as a series of action combinations, but those skilled in the art should know that the present application is not limited by the order of the described actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0202] According to another aspect of the embodiments of the present application, an image recognition device for implementing the above image recognition method is further provided. As shown in FIG. 8, the device comprises:

[0203] The extraction unit 802 is configured to extract image features from the image to be recognized, wherein the image to be recognized displays an object to be recognized.

[0204] The feature processing unit 804 is configured to perform feature processing on the image features and the initialized reference key point features by using the feature processing network to obtain final key point features of the object to be recognized, wherein the final key point features are used to indicate features of key points recognized from the object to be recognized displayed in the image to be recognized, the feature processing network is based on the initialized feature processing network, and the N times of loop feature processing is performed on each sample image, in the i th feature processing network performing the i th loop feature processing, feature processing is performed on the i-1 sample feature pair obtained by the i-1 times of loop feature processing to obtain the i th sample feature pair, and the network parameters in the i th feature processing network are adjusted based on the feature difference between the i-1 sample feature pair and the i th sample feature pair until a convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i th sample feature pair comprises the i th sample image feature and the i th reference sample key point feature, the i th sample image feature is obtained by i times of loop feature processing based on the original sample image feature extracted from the sample image, and the i th reference sample key point feature is obtained by i times of loop feature processing based on the original reference sample key point feature.

[0205] The determination unit 806 is configured to determine the positions of the key points of the object to be recognized in the image to be recognized by using the final key point features.

[0206] Optionally, in the present embodiment, the feature processing unit comprises: an input module configured to input the j-1 image features and the j-1 reference key point features output by the j-1 th feature processing sub-network in the feature processing network into the j th feature processing sub-network in the feature processing network, wherein the feature processing network comprises P feature processing sub-networks, j is greater than 1 and less than or equal to P, and P is a positive integer; a feature processing module configured to perform loop feature processing on the j-1 image features and the j-1 reference key point features in the j th feature processing sub-network to obtain j image features and j reference key point features; and a determination module configured to, in the case that j is equal to P, determine the j reference key point features as the final key point features, and in the case that j is less than P, input the obtained j image features and j reference key point features into the j+1 th feature processing sub-network in the feature processing network to perform the j+1 th loop feature processing.

[0207] Optionally, in the embodiment, the feature processing module is further configured to perform the following operations in the jth feature processing subnetwork: performing linear transformation processing on the image sub-feature output by the (j-1)th feature processing subnetwork to obtain a first image sub-feature in a first feature processing layer of the jth feature processing subnetwork; performing linear transformation processing on the reference key point sub-feature output by the (j-1)th feature processing subnetwork to obtain a first reference key point sub-feature; inputting the (h-1)th image sub-feature and the (h-1)th reference key point sub-feature output by the (h-1)th feature processing layer of the jth feature processing subnetwork into the hth feature processing layer in the jth feature processing subnetwork, where h is a positive integer greater than or equal to 2; performing linear transformation processing on the (h-1)th image sub-feature to obtain a first processing result in the hth feature processing layer, and determining the first processing result as the hth image sub-feature output by the hth feature processing layer; performing linear transformation processing on the (h-1)th reference key point sub-feature to obtain a second processing result; obtaining the hth reference key point sub-feature output by the hth feature processing layer by using the first processing result and the second processing result; and in the case that the hth feature processing layer is the last feature processing layer in the jth feature processing subnetwork, determining the hth image sub-feature as the jth image feature output by the jth feature processing subnetwork, and determining the hth reference key point sub-feature as the jth reference key point feature output by the jth feature processing subnetwork.

[0208] Optionally, in the embodiment, the device further comprises: a first extraction unit, configured to extract the original sample image feature from the acquired kth sample image, and acquire the initialized original reference sample key point feature, wherein k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer; a first feature processing unit, configured to perform feature processing on the (i-1)th sample feature pair obtained through the (i-1)th loop feature processing in the ith feature processing network to obtain the ith sample feature pair, wherein when i is 1, the (i-1)th sample feature pair obtained through the (i-1)th loop feature processing indicates the original sample feature pair, and the original sample feature pair comprises the original sample image feature and the original reference sample key point feature; a first determination unit, configured to determine the sample feature distance between the (i-1)th sample feature pair and the ith sample feature pair based on the feature difference between the (i-1)th sample feature pair and the ith sample feature pair; a calculation unit, configured to perform weighted summation calculation on the first i sample feature distances to obtain the ith feature training loss; an adjustment unit, configured to adjust the network parameters in the ith feature processing network to obtain the (i+1)th feature processing network in a case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not reached a threshold condition; perform the (i+1)th loop feature processing in the (i+1)th feature processing network; and a second determination unit, configured to determine the ith loop feature processing as the Nth loop feature processing in a case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing reaches the threshold condition.

[0209] Optionally, in the embodiment, the first determination unit comprises: a first calculation module, configured to perform distance calculation on the (i-1)th sample image feature in the (i-1)th sample feature pair and the ith sample image feature in the ith sample feature pair to obtain the (i-1)th sample image feature distance; and a second calculation module, configured to perform distance calculation on the (i-1)th reference sample key point feature in the (i-1)th sample feature pair and the ith reference sample key point feature in the ith sample feature pair to obtain the (i-1)th reference sample key point feature distance.

[0210] Optionally, in the embodiment, the first determination unit further comprises: a third calculation module, configured to perform weighted summation calculation on the first i sample image feature distances to obtain a sample image feature training loss, and perform weighted summation calculation on the first i reference sample key point feature distances to obtain a reference key point feature training loss; and a fourth calculation module, configured to perform weighted summation calculation on the feature training loss and the reference key point feature training loss to obtain the ith feature training loss.

[0211] Optionally, in the embodiment, the device further comprises: a first input unit, configured to input the i-th reference sample key point feature in the i-th sample feature pair into the i-th full connection layer to obtain an i-th key point prediction position; a comparison unit, configured to compare the i-th key point prediction position with the key point label position in the k-th sample image to obtain an i-th key point prediction difference distance; a first calculation unit, configured to perform weighted summation calculation on the first i key point prediction difference distances to obtain an i-th key point prediction training loss; a second calculation unit, configured to perform weighted summation calculation on the i-th feature training loss and the i-th key point prediction training loss to obtain an i-th target loss; a first adjustment unit, configured to, in a case where the i-th target loss has not reached a threshold condition, adjust network parameters in the i-th feature processing network and the i-th full connection layer to obtain an i+1-th feature processing network and an i+1-th full connection layer; perform i+1-th loop feature processing in the i+1-th feature processing network, and obtain an i+1-th key point prediction position in the i+1-th full connection layer; and a third determination unit, configured to, in a case where the i-th target loss reaches the threshold condition, determine that the i-th loop feature processing is the N-th loop feature processing.

[0212] The specific embodiments can refer to the examples shown in the image recognition method described above, and the embodiments will not be described here again.

[0213] According to another aspect of the embodiments of the present application, a pre-processing device for implementing the image recognition pre-processing method described above is also provided. As shown in FIG. 9, the device comprises:

[0214] The extraction unit 902 is configured to extract the original sample image feature from the acquired k-th sample image, and acquire the original reference sample key point feature obtained by initialization, wherein k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer;

[0215] The feature processing unit 904 is configured to perform feature processing on the i-1-th sample feature pair obtained by the i-1-th loop feature processing in the i-th feature processing network performing the i-th loop feature processing to obtain an i-th sample feature pair, wherein in a case where i is 1, the i-1-th sample feature pair obtained by the i-1-th loop feature processing is used to indicate the original sample feature pair, and the original sample feature pair comprises the original sample image feature and the original reference sample key point feature;

[0216] The first determination unit 906 is configured to determine a sample feature distance between the i-1-th sample feature pair and the i-th sample feature pair based on a feature difference between the i-1-th sample feature pair and the i-th sample feature pair;

[0217] The calculation unit 908 is configured to perform weighted summation calculation on the first i sample feature distances to obtain an i-th feature training loss.

[0218] The adjusting unit 910 is configured to adjust network parameters in the i th feature processing network to obtain an (i+1) th feature processing network in a case where the feature training loss determined based on the sample feature distances obtained after the i th loop feature processing has not reached a threshold condition, and perform (i+1) th loop feature processing in the (i+1) th feature processing network.

[0219] The second determining unit 912 is configured to determine that the i th loop feature processing is the N th loop feature processing in a case where the feature training loss determined based on the sample feature distances obtained after the i th loop feature processing reaches the threshold condition.

[0220] Optionally, in the embodiment, the device further includes an input unit configured to input the i th reference sample key point feature in the i th sample feature pair into the i th full connection layer to obtain an i th key point predicted position; a comparison unit configured to compare the i th key point predicted position with a key point marked position in the k th sample image to obtain an i th key point predicted difference distance; a first calculation unit configured to perform weighted summation calculation on the i key point predicted difference distances to obtain an i th key point predicted training loss; a second calculation unit configured to perform weighted summation calculation on the i th feature training loss and the i th key point predicted training loss to obtain an i th target loss; a first adjusting unit configured to adjust network parameters in the i th feature processing network and the i th full connection layer to obtain an (i+1) th feature processing network and an (i+1) th full connection layer in a case where the i th target loss has not reached a threshold condition, perform (i+1) th loop feature processing in the (i+1) th feature processing network, and obtain an (i+1) th key point predicted position in the (i+1) th full connection layer; and a third determining unit configured to determine that the i th loop feature processing is the N th loop feature processing in a case where the i th target loss reaches the threshold condition.

[0221] The specific embodiments can refer to the examples shown in the pre-processing method of the image recognition described above, and the embodiment will not be described here.

[0222] According to another aspect of the embodiment of the application, an electronic device for implementing the image recognition method is also provided, which can be a terminal device or a server shown in FIG. 1. The embodiment takes the electronic device as a terminal as an example. As shown in FIG. 10, the electronic device includes a memory 1002 and a processor 1004, the memory 1002 stores a computer program, and the processor 1004 is configured to execute the steps in any of the method embodiments by the computer program.

[0223] Optionally, in the embodiment, the electronic device can be located in at least one network device of a plurality of network devices of a computer network.

[0224] Optionally, in the embodiment, the processor can be configured to execute the steps of any of the method embodiments described above by means of a computer program.

[0225] Optionally, those skilled in the art can understand that the structure shown in FIG. 10 is only schematic, and the electronic device can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. FIG. 10 does not limit the structure of the electronic device. For example, the electronic device can further include more or less components (such as a network interface, etc.) than those shown in FIG. 10, or have a different configuration from that shown in FIG. 10.

[0226] The memory 1002 can be used to store software programs and modules, such as program instructions / modules corresponding to the image recognition method and device in the embodiments of the present application. The processor 1004 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002, that is, implements the image recognition method described above. The memory 1002 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 1002 can further include a memory remotely arranged with respect to the processor 1004, which can be connected to the terminal through a network. Examples of the network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. Specifically, the memory 1002 can be used to store a to-be-recognized image, but is not limited to this. As an example, as shown in FIG. 10, the memory 1002 can include but is not limited to the extraction unit 802, the feature processing unit 804, and the determination unit 806 in the image recognition device. In addition, other module units in the image recognition device can also be included, but are not limited to this, and will not be described herein.

[0227] Optionally, the transmission device 1006 is configured to receive or send data via a network. Specific examples of the network can include a wired network and a wireless network. In one example, the transmission device 1006 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers through a network cable, so as to communicate with the Internet or a local area network. In one example, the transmission device 1006 is a radio frequency (Radio Frequency, RF) module, which is configured to communicate with the Internet in a wireless manner.

[0228] Further, the electronic device described above further includes a connection bus 1008 for connecting each module component in the electronic device.

[0229] According to still another aspect of the embodiments of the present application, an electronic device for implementing the pre-processing method of the image recognition described above is further provided, which can be the terminal device or the server shown in FIG. 1. The present embodiment takes the electronic device as the server as an example for illustration. As shown in FIG. 11, the electronic device includes a memory 1102 and a processor 1104, the memory 1102 stores a computer program, and the processor 1104 is configured to execute the steps in any of the method embodiments described above by the computer program.

[0230] Optionally, in the present embodiment, the electronic device described above can be located in at least one network device of a plurality of network devices of a computer network.

[0231] Optionally, in the present embodiment, the processor described above can be configured to execute the following steps by the computer program:

[0232] Obtain M sample images and an initialized feature processing network, and train by the following steps:

[0233] Extract the original sample image feature from the kth sample image obtained, and obtain the original reference sample key point feature obtained by random initialization, wherein k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer;

[0234] Repeat the following operations on the original sample image feature and the original reference sample key point feature until the Nth loop feature processing is reached:

[0235] In the ith feature processing network for performing the ith loop feature processing, perform feature processing on the i-1th sample feature pair obtained by the i-1th loop feature processing to obtain the ith sample feature pair, wherein in the case where i is 1, the i-1th sample feature pair obtained by the i-1th loop feature processing is used to indicate the original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature;

[0236] Based on the feature difference between the i-1th sample feature pair and the ith sample feature pair, determine the sample feature distance between the i-1th sample feature pair and the ith sample feature pair;

[0237] Perform weighted summation calculation on the first i sample feature distances to obtain the ith feature training loss; in the case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not yet reached the threshold condition, adjust the network parameters in the ith feature processing network to obtain the i+1th feature processing network;

[0238] performing the i+1th loop feature processing in the i+1th feature processing network;

[0239] In a case where the feature training loss determined based on the feature distances of all the samples obtained after the i-th loop feature processing reaches a threshold condition, the i-th loop feature processing is determined as the N-th loop feature processing.

[0240] Optionally, those skilled in the art can understand that the structure shown in FIG. 11 is only schematic, and the electronic device can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. FIG. 11 does not limit the structure of the above-mentioned electronic device. For example, the electronic device can further include more or fewer components (such as a network interface, etc.) than those shown in FIG. 11, or have a different configuration from that shown in FIG. 11.

[0241] The memory 1102 can be used to store software programs and modules, such as program instructions / modules corresponding to the image recognition preprocessing method and device in the embodiments of the present application. The processor 1104 executes various function applications and data processing by running the software programs and modules stored in the memory 1102, that is, implements the image recognition preprocessing method described above. The memory 1102 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some examples, the memory 1102 can further include a memory remotely arranged with respect to the processor 1104, which can be connected to the terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The memory 1102 can specifically but not limited to be used to store sample images. As an example, as shown in FIG. 11, the memory 1102 can include but is not limited to the extraction unit 902, the feature processing unit 904, the first determination unit 906, the calculation unit 908, the adjustment unit 910, and the second determination unit 912 in the image recognition preprocessing device described above. In addition, other module units in the image recognition preprocessing device described above can also be included, but not limited to, which will not be described in detail in this example.

[0242] Optionally, the transmission device 1106 is configured to receive or send data via a network. Examples of the network include a wired network and a wireless network. In an example, the transmission device 1106 includes a network interface controller (NIC) which can be connected to other network devices and routers through a network cable to communicate with the Internet or a local area network. In an example, the transmission device 1106 is a radio frequency (RF) module which is configured to communicate with the Internet in a wireless manner.

[0243] In addition, the electronic device further includes a display 1108 configured to display a sample image, and a connection bus 1110 configured to connect various module components in the electronic device.

[0244] In other embodiments, the terminal device or the server can be a node in a distributed system, and the distributed system can be a blockchain system formed by the plurality of nodes connected through network communication. The nodes can form a point-to-point network, and any computing device such as a server, a terminal, or other electronic device can become a node in the blockchain system by joining the point-to-point network.

[0245] According to an aspect of the present application, a computer program product is provided, which includes computer programs / instructions containing program codes for executing the above method. In such embodiments, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a detachable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0246] According to an aspect of the present application, a computer readable storage medium is provided, and a processor of a computer device reads the computer program from the computer readable storage medium. The processor executes the computer program, so that the computer device executes the above method.

[0247] Optionally, in the present embodiment, the computer readable storage medium can be configured to store a computer program for executing the steps in the above method embodiments.

[0248] Optionally, in the present embodiment, the computer readable storage medium can be further configured to store a computer program for executing the steps in the following image recognition preprocessing method:

[0249] M sample images and an initialized feature processing network are obtained, and the following steps are used for training:

[0250] extract a raw sample image feature from the obtained kth sample image, and obtain an initialized raw reference sample key point feature, wherein k is a positive integer greater than or equal to 1 and less than or equal to M, M is a positive integer;

[0251] perform the following operations repeatedly on the raw sample image feature and the raw reference sample key point feature until the Nth loop feature processing is reached:

[0252] In the ith feature processing network for performing the ith loop feature processing, perform feature processing on the (i-1)th sample feature pair obtained by the (i-1)th loop feature processing to obtain the ith sample feature pair, wherein in the case where i is 1, the (i-1)th sample feature pair obtained by the (i-1)th loop feature processing is used to indicate the raw sample feature pair, and the raw sample feature pair includes the raw sample image feature and the raw reference sample key point feature;

[0253] Based on the feature difference between the (i-1)th sample feature pair and the ith sample feature pair, determine the sample feature distance between the (i-1)th sample feature pair and the ith sample feature pair;

[0254] perform weighted sum calculation on the first i sample feature distances to obtain the ith feature training loss; in the case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not reached the threshold condition, adjust the network parameters in the ith feature processing network to obtain the (i+1)th feature processing network; perform the (i+1)th loop feature processing in the (i+1)th feature processing network;

[0255] In the case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing reaches the threshold condition, determine that the ith loop feature processing is the Nth loop feature processing.

[0256] Optionally, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as processing circuitry or memory) or a combination thereof. Similarly, one processor (or multiple processors or memory) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0257] Optionally, in the embodiment, all or part of the steps in the various methods of the above embodiments can be understood by those skilled in the art that the terminal device related hardware can be instructed by a program, and the program can be stored in a computer readable storage medium, and the storage medium can include a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0258] The integrated units in the above embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the above computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0259] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0260] In the several embodiments provided by the present application, it should be understood that the disclosed client can be implemented in other ways. Of course, the above device embodiment is only illustrative, and for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0261] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment according to actual needs.

[0262] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0263] The above merely preferred embodiments of the present application, it should be noted that for those of ordinary skill in the art, without departing from the principles of the present application, can also make a number of improvements and refinements, these improvements and refinements should also be considered as the scope of protection of the present application.

Claims

1. An image recognition method, the method being performed by an electronic device, comprising: extracting an image feature from a to-be-recognized image, wherein a to-be-recognized object is displayed in the to-be-recognized image; performing feature processing on the image feature and an initialized reference key point feature by a feature processing network to obtain a final key point feature of the to-be-recognized object, wherein the final key point feature is used to indicate a feature of a key point recognized from the to-be-recognized object displayed in the to-be-recognized image, the feature processing network is based on an initialized feature processing network, and a sample image is subjected to N times of loop feature processing, in an i-th feature processing network performing i-th loop feature processing, an i-1-th sample feature pair obtained by i-1-th loop feature processing is subjected to feature processing to obtain an i-th sample feature pair, and a network parameter in the i-th feature processing network is adjusted based on a feature difference between the i-1-th sample feature pair and the i-th sample feature pair until a convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair comprises an i-th sample image feature and an i-th reference sample key point feature, the i-th sample image feature is obtained by i times of loop feature processing based on an original sample image feature extracted from the sample image, and the i-th reference sample key point feature is obtained by i times of loop feature processing based on an initialized original reference sample key point feature; determining a position of a key point of the to-be-recognized object in the to-be-recognized image by using the final key point feature.

2. The method of claim 1, wherein the performing feature processing on the image feature and the initialized reference key point feature by the feature processing network to obtain the final key point feature of the to-be-recognized object comprises: performing the following operations in the feature processing network: inputting an i-1-th image feature and an i-1-th reference key point feature output by an i-1-th feature processing subnetwork in the feature processing network into an i-th feature processing subnetwork in the feature processing network, wherein the feature processing network comprises P feature processing subnetworks, i is greater than 1 and less than or equal to P, and P is a positive integer; performing feature processing on the i-1-th image feature and the i-1-th reference key point feature in the i-th feature processing subnetwork to obtain an i-th image feature and an i-th reference key point feature; in a case where i is equal to P, determining the i-th reference key point feature as the final key point feature; in a case where i is less than P, inputting the obtained i-th image feature and the i-th reference key point feature into an i+1-th feature processing subnetwork in the feature processing network to perform i+1-th loop feature processing.

3. The method of claim 2, wherein the performing feature processing on the i-1-th image feature and the i-1-th reference key point feature in the i-th feature processing subnetwork to obtain the i-th image feature and the i-th reference key point feature comprises: performing the following operations in the jth feature processing subnetwork: performing linear transformation processing on the image sub-feature output by the j-1th feature processing subnetwork in the 1st feature processing layer of the jth feature processing subnetwork to obtain a 1st image sub-feature; and performing linear transformation processing on the reference key point sub-feature output by the j-1th feature processing subnetwork to obtain a 1st reference key point sub-feature; inputting the (h-1)th image sub-feature and the (h-1)th reference key point sub-feature output by the (h-1)th feature processing layer of the jth feature processing subnetwork to the hth feature processing layer in the jth feature processing subnetwork, where h is a positive integer greater than or equal to 2; performing linear transformation processing on the (h-1)th image sub-feature in the hth feature processing layer to obtain a first processing result, and determining the first processing result as the hth image sub-feature output by the hth feature processing layer; and performing linear transformation processing on the (h-1)th reference key point sub-feature to obtain a second processing result; and using the first processing result and the second processing result to obtain the hth reference key point sub-feature output by the hth feature processing layer; in a case where the hth feature processing layer is the last feature processing layer in the jth feature processing subnetwork, determining the hth image sub-feature as the jth image feature output by the jth feature processing subnetwork, and determining the hth reference key point sub-feature as the jth reference key point feature output by the jth feature processing subnetwork.

4. The method of any one of claims 1-3, before the extracting the image feature from the to-be-recognized image, further comprising: training the initialized feature processing network by the following steps: extracting an original sample image feature from the obtained kth sample image, and obtaining an original reference sample key point feature obtained by initialization, where k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer; repeatedly performing the following operations on the original sample image feature and the original reference sample key point feature until N times of loop feature processing are reached: performing feature processing on an (i-1)th sample feature pair obtained by (i-1)th loop feature processing in an i th feature processing network to obtain an i th sample feature pair, where in a case where i is 1, the (i-1)th sample feature pair obtained by the (i-1)th loop feature processing indicates an original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature; determining a sample feature distance between the (i-1)th sample feature pair and the i th sample feature pair based on a feature difference between the (i-1)th sample feature pair and the i th sample feature pair; performing weighted summation calculation on the first i sample feature distances to obtain an i th feature training loss; and performing the following operations in the jth feature processing subnetwork: In a case where the feature training loss determined based on the feature distances of all samples obtained after the i-th loop feature processing does not reach the threshold condition, adjusting the network parameters in the i-th feature processing network to obtain an i+1-th feature processing network; performing i+1-th loop feature processing in the i+1-th feature processing network; In a case where the feature training loss determined based on the feature distances of all samples obtained after the i-th loop feature processing reaches the threshold condition, determining that the i-th loop feature processing is the N-th loop feature processing.

5. The method of claim 4, wherein the determining the sample feature distance between the i-1-th sample feature pair and the i-th sample feature pair based on the feature difference between the i-1-th sample feature pair and the i-th sample feature pair comprises: performing distance calculation on the i-1-th sample image feature in the i-1-th sample feature pair and the i-th sample image feature in the i-th sample feature pair to obtain an i-1-th sample image feature distance; performing distance calculation on the i-1-th reference sample key point feature in the i-1-th sample feature pair and the i-th reference sample key point feature in the i-th sample feature pair to obtain an i-1-th reference sample key point feature distance, wherein the sample feature distance between the i-1-th sample feature pair and the i-th sample feature pair comprises the i-1-th sample image feature distance and the i-1-th reference sample key point feature distance.

6. The method of claim 5, wherein the performing weighted sum calculation on the i sample feature distances to obtain an i-th feature training loss comprises: performing weighted sum calculation on the i sample image feature distances to obtain a sample image feature training loss, and performing weighted sum calculation on the i reference sample key point feature distances to obtain a reference key point feature training loss; performing weighted sum calculation on the sample image feature training loss and the reference key point feature training loss to obtain the i-th feature training loss.

7. The method of any one of claims 4-6, after the performing feature processing on the i-1-th sample feature pair obtained by the i-1-th loop feature processing in the i-th feature processing network to obtain an i-th sample feature pair, further comprising: inputting the i-th reference sample key point feature in the i-th sample feature pair into an i-th fully connected layer to obtain an i-th key point prediction position; performing comparison between the i-th key point prediction position and the key point label position in the i-th sample image to obtain an i-th key point prediction difference distance; performing weighted sum calculation on the i key point prediction difference distances to obtain an i-th key point prediction training loss; performing weighted sum calculation on the i-th feature training loss and the i-th key point prediction training loss to obtain an i-th target loss. in the case that the ith target loss has not reached a threshold condition, adjusting network parameters in the ith feature processing network and the ith fully connected layer to obtain an ith+1 feature processing network and an ith+1 fully connected layer; performing ith+1 loop feature processing in the ith+1 feature processing network and obtaining an ith+1 key point prediction position in the ith+1 fully connected layer; in the case that the ith target loss reaches the threshold condition, determining that the ith loop feature processing is the Nth loop feature processing.

8. A pre-processing method of image recognition, the method being executed by an electronic device and comprising: obtaining M sample images and an initialized feature processing network, and training by the following steps: extracting an original sample image feature from a kth obtained sample image, and obtaining an original reference sample key point feature obtained by initialization, wherein k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer; repeating the following operations on the original sample image feature and the original reference sample key point feature until N loop feature processing is reached: performing feature processing on an ith-1 sample feature pair obtained by ith-1 loop feature processing in an ith feature processing network of ith loop feature processing to obtain an ith sample feature pair, wherein in the case that i is 1, the ith-1 sample feature pair obtained by the ith-1 loop feature processing is used to indicate the original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature; determining a sample feature distance between the ith-1 sample feature pair and the ith sample feature pair based on feature difference between the ith-1 sample feature pair and the ith sample feature pair; performing weighted summation calculation on the first i sample feature distances to obtain an ith feature training loss; in the case that the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not reached a threshold condition, adjusting network parameters in the ith feature processing network to obtain an ith+1 feature processing network; performing ith+1 loop feature processing in the ith+1 feature processing network; in the case that the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing reaches the threshold condition, determining that the ith loop feature processing is the Nth loop feature processing.

9. An image recognition device, the device being deployed on an electronic device and comprising: an extraction unit configured to extract an image feature from a to-be-recognized image, wherein a to-be-recognized object is displayed in the to-be-recognized image; a feature processing unit, configured to perform feature processing on the image feature and the initialized reference key point feature by a feature processing network to obtain a final key point feature of the to-be-identified object, wherein the final key point feature is used to indicate a feature of a key point identified from the to-be-identified object displayed in the to-be-identified image, the feature processing network is based on an initialized feature processing network, and the feature processing network is obtained by performing N times of loop feature processing on each sample image, in an i-th feature processing network used to perform i-th loop feature processing, feature processing is performed on an (i-1)-th sample feature pair obtained by (i-1)-th loop feature processing to obtain an i-th sample feature pair, and network parameters in the i-th feature processing network are adjusted based on a feature difference between the (i-1)-th sample feature pair and the i-th sample feature pair until a convergence condition is reached, i is a positive integer greater than or equal to 1 and less than or equal to N, N is a positive integer, the i-th sample feature pair includes an i-th sample image feature and an i-th reference sample key point feature, the i-th sample image feature is obtained by performing i times of loop feature processing on an original sample image feature extracted from the sample image, and the i-th reference sample key point feature is obtained by performing i times of loop feature processing on an initialized original reference sample key point feature; a determination unit, configured to determine, by using the final key point feature, a position of the key point of the to-be-identified object in the to-be-identified image.

10. A pre-processing apparatus for image recognition, the apparatus being deployed on an electronic device and comprising: an extraction unit, configured to extract an original sample image feature from a k-th sample image obtained by the obtaining unit, and obtain an original reference sample key point feature obtained by the initialization unit, wherein k is a positive integer greater than or equal to 1 and less than or equal to M, and M is a positive integer; a feature processing unit, configured to perform feature processing on an (i-1)-th sample feature pair obtained by (i-1)-th loop feature processing in an i-th feature processing network used to perform i-th loop feature processing to obtain an i-th sample feature pair, wherein when i is 1, the (i-1)-th sample feature pair obtained by the (i-1)-th loop feature processing is used to indicate the original sample feature pair, and the original sample feature pair includes the original sample image feature and the original reference sample key point feature; a first determination unit, configured to determine a sample feature distance between the (i-1)-th sample feature pair and the i-th sample feature pair based on a feature difference between the (i-1)-th sample feature pair and the i-th sample feature pair; a calculation unit, configured to perform weighted summation calculation on the first i sample feature distances to obtain an i-th feature training loss; an adjustment unit, configured to adjust network parameters in the i-th feature processing network to obtain an (i+1)-th feature processing network in a case where the feature training loss determined based on all sample feature distances obtained after the first i loop feature processing has not reached a threshold condition, and perform (i+1)-th loop feature processing in the (i+1)-th feature processing network. The second determining unit is configured to determine that the i-th loop feature processing is the N-th loop feature processing when a feature training loss determined based on all sample feature distances obtained after the i-th loop feature processing reaches the threshold condition.

11. A computer readable storage medium comprising a stored computer program, wherein, The computer program, when executed by a processor, performs the method of any one of claims 1 to 7 or 8.

12. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any one of claims 1 to 7 or 8.

13. An electronic device comprising a memory and a processor, the memory having stored therein a computer program, the processor being arranged to perform the method of any one of claims 1 to 7 or 8 by means of the computer program.

Images