Image recognition method, storage medium, and apparatus

Abstract

This application discloses an image recognition method, storage medium, and device, which can be applied to the fields of artificial intelligence, intelligent transportation, and assisted driving. The method includes: acquiring an image to be recognized, including a target object; extracting planar features from the image to be recognized to obtain planar feature information of the target object; converting the image to be recognized into a depth image; extracting depth features from the depth image to obtain depth feature information of the target object; fusing the planar feature information and the depth feature information to obtain fused feature information of the target object; and identifying the attributes of the target object based on the fused feature information to obtain attribute information of the target object. This application can, to a certain extent, prevent attacks on image feature information and improve the reliability and accuracy of image recognition.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to an image recognition method, storage medium and device. Background Technology

[0002] Image recognition technology refers to the technology of extracting, analyzing, and matching features from images. The recognition result depends entirely on the image data information, and the stability and security of the image data information itself also determine the credibility of the recognition result.

[0003] Currently, when image recognition technology extracts features from image data, it can only extract a single feature from the image data itself. This means that if this single feature is attacked, the image recognition result will be incorrect. Therefore, how to ensure the stability and security of image data and improve the credibility and accuracy of image recognition is an urgent problem to be solved in the field of image recognition. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of this application is to provide an image recognition method, storage medium and device, which can prevent the feature information of an image from being attacked to a certain extent and improve the reliability and accuracy of image recognition.

[0005] One embodiment of this application provides an image recognition method, including:

[0006] The process involves acquiring an image to be identified that includes the target object, extracting planar features from the image to be identified, and obtaining the planar feature information of the target object.

[0007] The image to be identified is converted into a depth image, and depth features are extracted from the depth image to obtain the depth feature information of the target object.

[0008] By fusing planar and depth feature information, the fused feature information of the target object is obtained.

[0009] The attribute information of the target object is obtained by identifying the attribute information of the target object based on the fusion feature information of the target object.

[0010] This process involves converting the image to be identified into a depth image, extracting depth features from the depth image, and obtaining the depth feature information of the target object, including:

[0011] Interpolate the image to be recognized to obtain a standard-sized image to be recognized;

[0012] The target depth image conversion model is invoked to convert the standard-sized image to be recognized, thereby obtaining an initial depth image;

[0013] Generate a depth image from the initial depth image;

[0014] The depth feature extraction model is invoked to extract depth features from the depth image, thereby obtaining the depth feature information of the target object.

[0015] This involves calling a target depth image conversion model to convert a standard-sized image to be identified, resulting in an initial depth image, including:

[0016] The encoder in the target depth image conversion model is used to obtain the distance information between each pixel in the standard-sized image to be recognized and the reference horizontal plane.

[0017] The decoder in the target depth image conversion model predicts the distance information between each pixel in the standard-sized image to be identified and the reference horizontal plane, and outputs the depth information of each pixel in the standard-sized image to be identified.

[0018] An initial depth image is generated based on the depth information of each pixel.

[0019] The process of generating a depth image from an initial depth image includes:

[0020] Obtain the normal component of the initial depth image in the first direction, and obtain the normal component of the initial depth image in the second direction;

[0021] Based on the normal components in the first direction and the normal components in the second direction, calculate the normal components of the initial depth image in the third direction.

[0022] The depth image is obtained by combining the normal components in the first direction, the second direction, and the third direction.

[0023] This involves calling a depth feature extraction model to extract depth features from the depth image, obtaining the depth feature information of the target object, including:

[0024] The depth feature extraction model is invoked to perform vector transformation on the depth image, resulting in the depth image vector corresponding to the depth image.

[0025] By using the residual convolutional layer in the depth feature extraction model, the depth image vector is convolved to obtain the convolutional image vector. Based on the difference between the depth image vector and the convolutional image vector, the depth residual feature vector corresponding to the depth image is obtained.

[0026] By using a fully connected layer in the deep feature extraction model, the depth residual feature vector is linearly transformed to obtain the depth feature information of the target object.

[0027] The methods also include:

[0028] Obtain the initial depth image transformation model, sample images, and the corresponding initial depth image labels for the sample images;

[0029] An initial depth image conversion model is used to perform depth recognition on the sample image to obtain the predicted initial depth image corresponding to the sample image.

[0030] The first loss function is determined based on the pixel difference between the initial depth image label and the predicted initial depth image;

[0031] The second loss function is determined based on the difference between the normal vectors of pixels in the initial depth image label and the normal vectors of pixels in the predicted initial depth image;

[0032] Based on the first loss function and the second loss function, the initial depth image conversion model is iteratively trained to obtain the target depth image conversion model.

[0033] The process of acquiring the image to be identified, which includes the target object, includes:

[0034] Obtain the original image including the target object;

[0035] Extract key feature points of the target object from the original image;

[0036] The original image is corrected based on the key feature points of the target object to obtain an image to be identified that includes the target object.

[0037] The process involves correcting the original image based on the key feature points of the target object to obtain an image to be identified that includes the target object, including:

[0038] Obtain key features of the object template;

[0039] Determine the differences between the key feature points of the object template and the key feature points of the target object;

[0040] The original image is adjusted based on the difference information to obtain an image to be identified that includes the target object.

[0041] One embodiment of this application provides an image recognition device, including:

[0042] The planar feature extraction module is used to acquire an image to be identified, including the target object, and to extract planar features from the image to be identified to obtain the planar feature information of the target object.

[0043] The depth feature extraction module is used to convert the image to be identified into a depth image, extract depth features from the depth image, and obtain the depth feature information of the target object.

[0044] The fusion module is used to fuse planar feature information and depth feature information to obtain the fused feature information of the target object;

[0045] The attribute recognition module is used to identify the attributes of the target object based on the fusion feature information of the target object, and obtain the attribute information of the target object.

[0046] The deep feature extraction module includes:

[0047] The interpolation processing unit is used to perform interpolation processing on the image to be recognized to obtain a standard-sized image to be recognized;

[0048] The image conversion unit is used to call the target depth image conversion model to convert the standard-sized image to be recognized and obtain the initial depth image;

[0049] A generation unit is used to generate a depth image based on an initial depth image;

[0050] The depth feature extraction unit is used to call the depth feature extraction model to extract depth features from the depth image and obtain the depth feature information of the target object.

[0051] Specifically, the image conversion unit is used for:

[0052] The encoder in the target depth image conversion model is used to obtain the distance information between each pixel in the standard-sized image to be recognized and the reference horizontal plane.

[0053] The decoder in the target depth image conversion model predicts the distance information between each pixel in the standard-sized image to be identified and the reference horizontal plane, and outputs the depth information of each pixel in the standard-sized image to be identified.

[0054] An initial depth image is generated based on the depth information of each pixel.

[0055] Specifically, the generating unit is used for:

[0056] Obtain the normal component of the initial depth image in the first direction, and obtain the normal component of the initial depth image in the second direction;

[0057] Based on the normal components in the first direction and the normal components in the second direction, calculate the normal components of the initial depth image in the third direction.

[0058] The depth image is obtained by combining the normal components in the first direction, the second direction, and the third direction.

[0059] Specifically, the deep feature extraction unit is used for:

[0060] The depth feature extraction model is invoked to perform vector transformation on the depth image, resulting in the depth image vector corresponding to the depth image.

[0061] By using the residual convolutional layer in the depth feature extraction model, the depth image vector is convolved to obtain the convolutional image vector. Based on the difference between the depth image vector and the convolutional image vector, the depth residual feature vector corresponding to the depth image is obtained.

[0062] By using a fully connected layer in the deep feature extraction model, the depth residual feature vector is linearly transformed to obtain the depth feature information of the target object.

[0063] The deep feature extraction module also includes:

[0064] The first acquisition unit is used to acquire the initial depth image conversion model, the sample image, and the initial depth image label corresponding to the sample image.

[0065] The depth recognition unit is used to perform depth recognition on the sample image using an initial depth image conversion model to obtain the predicted initial depth image corresponding to the sample image.

[0066] The first determining unit is used to determine a first loss function based on the pixel difference between the initial depth image label and the predicted initial depth image;

[0067] The second determining unit is used to determine the second loss function based on the difference between the normal vector of the pixel in the initial depth image label and the normal vector of the pixel in the predicted initial depth image;

[0068] The iterative training unit is used to iteratively train the initial depth image conversion model according to the first loss function and the second loss function to obtain the target depth image conversion model.

[0069] The planar feature extraction module includes:

[0070] The second acquisition unit is used to acquire the original image including the target object;

[0071] The extraction unit is used to extract key feature points of the target object from the original image;

[0072] The correction processing unit is used to correct the original image based on the key feature points of the target object to obtain an image to be identified that includes the target object.

[0073] Specifically, the correction processing unit is used for:

[0074] Obtain key features of the object template;

[0075] Determine the differences between the key feature points of the object template and the key feature points of the target object;

[0076] The original image is adjusted based on the difference information to obtain an image to be identified that includes the target object.

[0077] One embodiment of this application provides a computer device, including: a processor and a memory;

[0078] The processor is connected to a memory, which stores a computer program. When the computer program is executed by the processor, it causes the computer device to perform the method provided in the embodiments of this application.

[0079] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method provided in this application.

[0080] One embodiment of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in this application embodiment.

[0081] In this embodiment, an image to be identified, including the target object, is acquired. Planar features are extracted from the image to obtain planar feature information of the target object, which reflects the two-dimensional feature information of the target object. The image to be identified is converted into a depth image, and depth features are extracted from the depth image to obtain depth feature information of the target object. This depth feature information reflects the geometry of the visible surface of the target object, i.e., it contains the three-dimensional feature information of the target object. The planar feature information and the depth feature information are fused to obtain fused feature information of the target object. Attribute identification is performed on the target object based on the fused feature information to obtain the attribute information of the target object. Multi-dimensional fused feature information is difficult to attack; that is, both planar and depth feature information are difficult to attack simultaneously. When planar feature information is attacked, attribute identification can still be performed based on depth feature information, and vice versa. Therefore, by using multi-dimensional fused feature information to identify the attributes of the target object, the feature information of the image can be prevented from being attacked to a certain extent, achieving a better identification defense effect and improving the reliability and accuracy of image recognition. Attached Figure Description

[0082] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0083] Figure 1 This is a schematic diagram of the architecture of an image recognition system provided in an embodiment of this application;

[0084] Figure 2 This is a schematic diagram illustrating an application scenario of image recognition provided in an embodiment of this application;

[0085] Figure 3 This is a schematic flowchart of an image recognition method provided in an embodiment of this application;

[0086] Figure 4 This is a schematic diagram of a method for obtaining fused feature information provided in an embodiment of this application;

[0087] Figure 5 This is a schematic diagram illustrating attribute recognition based on fused feature information provided in an embodiment of this application;

[0088] Figure 6 This is a schematic flowchart of an image recognition method provided in an embodiment of this application;

[0089] Figure 7 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of this application;

[0090] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0091] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0092] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0093] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning. In this application, a color feature extraction model can be trained by learning to extract planar features from an image to be identified, including the target object. This model extracts the planar feature information of the target object, which can refer to two-dimensional feature information such as color and shape features. Furthermore, a target depth image conversion model can be trained by learning to convert normally captured color images into depth images. This model is then used to convert the image to be identified into a depth image. Alternatively, features can be extracted from depth images through learning, and a depth feature extraction model can be trained. This model can then be used to extract depth features from depth images to obtain the depth feature information of the target object.

[0094] See Figure 1 , Figure 1 This is a schematic diagram of the structure of an image recognition system provided in an embodiment of this application. For example... Figure 1As shown, the image recognition system may include a server 10 and a user terminal cluster. The user terminal cluster may include one or more user terminals; the number of user terminals is not limited here. Figure 1 As shown, it can specifically include user terminal 100a, user terminal 100b, user terminal 100c, ..., user terminal 100n. Figure 1 As shown, user terminals 100a, 100b, 100c, ..., 100n can each connect to the server 10 via a network, so that each user terminal can interact with the server 10 through the network connection.

[0095] Each user terminal in this user terminal cluster can include: in-vehicle terminals, smartphones, tablets, laptops, desktop computers, wearable devices, smart home devices, head-mounted devices, and other intelligent terminals with image recognition capabilities. It should be understood that, for example... Figure 1 Each user terminal in the user terminal cluster shown can have the target application (i.e., the application client) installed. When the application client runs on each user terminal, it can interact with the aforementioned... Figure 1 Data interaction occurs between the servers 10 shown.

[0096] Among them, such as Figure 1 As shown, the server 10 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0097] For ease of understanding, the embodiments of this application may be described in detail below. Figure 1 From the plurality of user terminals shown, one user terminal is selected as the target user terminal. The target user terminal may include: a smartphone, tablet computer, laptop computer, desktop computer, smart TV, or other smart terminal with image recognition capabilities. For example, for ease of understanding, embodiments of this application may... Figure 1The user terminal 100a shown serves as the target user terminal. User terminal 100a can acquire an image to be identified, including a target object. This image can be a color image. After acquiring the image containing the target object, user terminal 100a can send it to server 10. Server 10 can perform planar feature extraction on the image to obtain planar feature information of the target object. This planar feature information can refer to the two-dimensional feature information of the target object in the image to be identified, such as color feature information (e.g., RGB image information), contour feature information, shape feature information, etc. Server 10 can convert the image to be identified into a depth image and perform depth feature extraction on the depth image to obtain depth feature information of the target object. This depth feature information can be the three-dimensional feature information of the target object in the image to be identified. This three-dimensional feature information can refer to the correlation and positional relationships between various parts of the target object in three-dimensional space. Server 10 can fuse the planar feature information and depth feature information of the target object to obtain fused feature information of the target object. Server 10 can send the fused feature information of the target object to user terminal 100a. User terminal 100a can then perform attribute identification on the target object based on the fused feature information to obtain the target object's attribute information. For example, user terminal 100a can search a legitimate object database for sample feature information that matches the fused feature information of the target object. This legitimate object database includes sample feature information from multiple sample objects. If a sample feature information matching the fused feature information of the target object exists in the legitimate object database, then the target object's attribute information is determined to be legitimate, i.e., the target object is legitimate. If no sample feature information matching the fused feature information of the target object exists in the legitimate object database, then the target object's attribute information is determined to be illegitimate, i.e., the target object is illegitimate. Since identification based solely on the planar feature information of the target object is vulnerable to attack, resulting in low reliability and accuracy, this solution combines the planar and depth feature information of the target object to perform attribute identification using multi-dimensional feature information. This achieves a better identification and defense effect, improving the reliability and accuracy of the identification. The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0098] For example, in a smart transportation scenario, the image to be identified can be an image captured by an in-vehicle terminal on the road in which the vehicle is traveling. The target object in the image to be identified can be a vehicle, a face, etc. By analyzing the image to be identified, attribute information such as the distance between the vehicle or pedestrian in the road and the vehicle can be identified, which is helpful to assist the vehicle in safe driving.

[0099] For better understanding, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating an image recognition application scenario provided in an embodiment of this application. Wherein, as... Figure 2 The server 20d shown can be the server 10 mentioned above, such as... Figure 2 The target user terminal 20b shown can be the one described above. Figure 1 Any user terminal in the user terminal cluster shown, for example, target user terminal 20a can be the aforementioned user terminal 100a. Figure 2As shown, the image to be recognized can be a facial image. When target user 20a wants to conduct business A, business A needs to verify the identity information of target user 20a to determine whether target user 20a is a legitimate user. Only when target user 20a is a legitimate user can business A be conducted. Business A can obtain the facial capture and recognition permission of target user 20a. Through the camera component of the target user terminal 20b held by target user 20a, it captures the face of target user 20a, obtaining a facial image 20c containing the face of target user 20a. After obtaining the facial image 20c of target user 20a, target user terminal 20b can send the facial image 20c to server 20d. After receiving the facial image 20c sent by target user terminal 20b, server 20d can extract features from the facial image 20c through the color feature extraction model 20e to obtain the planar feature information of target user 20a. This planar feature information can refer to two-dimensional feature information such as the facial color feature information and facial shape feature information of target user 20a. Server 10 can convert face image 20c into an initial depth image using target depth image conversion model 20g, and generate depth image 20h based on the initial depth image. Depth image 20h includes distance information between each pixel in the face image and a reference horizontal plane, which can be any horizontal plane between the camera component of target user terminal 20b and the face of target user 20a. Server 10 can extract depth features from depth image 20h using depth feature extraction model 20i to obtain depth feature information of target user 20a. This depth feature information includes three-dimensional feature information such as the height and contour features of target user 20a's face, i.e., the depth information used to display the face depth of target user 20a. Server 10 can fuse the planar feature information 20f and depth feature information 20j of target user 20a to obtain fused feature information 20k of target user 20a, which server 10 can return to target user terminal 20b. The target user terminal 20b can output interface 20l to indicate that the target user 20a is currently undergoing attribute verification. The target user terminal 20b can verify the attributes of the target user 20a based on the fused feature information 20k to determine whether the target user 20a is a legitimate user. Specifically, the target user terminal 20b can search in a legitimate user database for sample feature information that matches the fused feature information 20k of the target user 20a. This legitimate user database includes sample feature information from multiple legitimate users. If a sample feature information matching the fused feature information of the target user 20a exists in the legitimate user database, then the target user 20a is determined to be a legitimate user. The target user terminal can then output interface 20m to indicate that the verification has passed, and the target user 20a can proceed to business A to process the business.If no sample feature information matching the fused feature information of target user 20a is found in the legitimate user database, then target user 20a can be determined to be an illegitimate user, and a "Verification Failed" interface will be output to indicate that target user 20a cannot process business A. Alternatively, the server can perform attribute verification on target user 20a based on its fused feature information to obtain target user 20a's attribute information. In this way, by identifying the legitimacy of target user 20a's identity using both planar and deep feature information, a good identification and defense effect can be achieved, improving the credibility and accuracy of the identification. This avoids the vulnerability of single planar feature information to attacks, which can lead to low credibility and accuracy in the identification results.

[0100] Please see Figure 3 , Figure 3 This is a flowchart illustrating an image recognition method provided in an embodiment of this application. The image recognition method can be executed by a computer device, which can be a server (as described above). Figure 1 Server 10 in the middle), or user terminal (as described above) Figure 1 This application does not limit the scope to any user terminal in the user terminal cluster. Figure 3 As shown, the image recognition method may include steps S101-S104.

[0101] S101, acquire the image to be identified, including the target object, extract planar features from the image to be identified, and obtain the planar feature information of the target object.

[0102] Specifically, to ensure the security and privacy of business processes, some processes require user attribute identification to obtain user attribute information. Business processing can only proceed if the user's attribute information meets preset conditions, such as the user being a legitimate user or the user's attribute information being valid. Specifically, computer equipment can acquire an image containing the target object to be identified, perform image recognition on the image, extract the fused feature information of the target object, and then use this fused feature information to perform attribute identification on the target object, obtaining the target object's attribute information, which may refer to the legitimacy of the target object's identity. In this way, attribute identification of the target object through fused feature information can achieve a good identification and defense effect, improving the credibility and accuracy of the identification. Specifically, computer equipment can acquire an image containing the target object to be identified; this image may be a color image, meaning it includes the target object's color features, shape features, and other identity features. The target object can include, but is not limited to, human faces, plants, goods, pedestrians, various animals, and various scenes. When the target object is the face of the target user, the processing of the image to be recognized in this application is authorized by the target user in the image to be recognized. The computer device can perform planar feature extraction on the image to be recognized to obtain the planar feature information of the target object. This planar feature information can refer to the two-dimensional feature information of the target object, such as color feature information, shape feature information, texture feature information, and pose feature information, etc.

[0103] Optionally, the computer device can invoke a color face feature extraction model. The image to be identified, containing the target object, is input into the color feature extraction model, which extracts features from the region containing the target object in the image to obtain planar feature information of the target object. The color face feature extraction model can be a convolutional neural network structure, designed and combined with residual convolutional layers, pooling layers, and fully connected layers, outputting planar feature information with a dimension of 512. Specifically, the computer device can perform vector transformation on the image to be identified, obtaining the corresponding image vector. This image vector is then input into the residual convolutional layer of the color feature extraction model, where a convolution operation is performed to obtain the convolutional image vector of the image to be identified. The number of residual convolutional layers can be one or more, depending on specific needs. The convolution operation in the residual convolutional layer refers to using a convolution kernel to perform matrix multiplication and summation on each pixel in the image to be identified. Each residual convolutional layer corresponds to one or more convolutional kernels (also called filters or receptive fields). These kernels can be weighted matrices and are typically square grid structures (e.g., 3x3 matrices or pixel regions). The number of channels in each kernel is determined by the number of channels in the input data, and the number of channels in the output data of each layer is determined by the number of kernels in that residual convolutional layer. The image height H in the output data is... out and image width W out It is determined by the size of the input data, the size of the convolution kernel, the stride, and the padding, i.e., H. out =(H in -H Kernel +2*padding) / stride+1, W out =(W in -W Kernel +2*padding) / stride+1. H in H Kernel W represents the height of the image in the input data and the height of the convolution kernel, respectively; in W Kernel These represent the width of the image in the input data and the width of the convolutional kernel, respectively. Residual convolutional layers, designed to reduce computation and extract more feature information, employ a network structure that is small at both ends and large in the middle. Specifically, they use convolutional kernels with fewer channels at the beginning, larger kernels in the middle to increase the channel count, and finally smaller kernels at the end to reduce the channel count. Color feature extraction models utilize these residual convolutional layers to extract features from the image being processed. This reduces computation, allowing for the extraction of planar feature information from the image with less computation, thus improving feature extraction efficiency.

[0104] Specifically, the computer device can obtain the difference between the image vector of the image to be recognized and the convolutional image vector to obtain the residual feature vector of the image to be recognized. The computer device can then input the residual feature vector of the image to be recognized into the pooling layer of the color face feature extraction model to standardize the residual feature vector, obtaining a standardized residual feature vector. After feature extraction in the residual convolutional layer, the output residual feature vector is passed to the pooling layer for feature selection and information filtering. The pooling layer contains a pre-defined pooling function, which replaces the result of a single point in the residual feature vector with the feature map statistics of its neighboring regions. The selection of the pooling region in the pooling layer is the same as the scanning of the feature map in the convolutional kernel, controlled by the pooling size, stride, and padding. Simultaneously, through pooling, the dimensionality of the residual feature vector output by the residual convolutional layer can be reduced, improving the results and avoiding overfitting. The color feature extraction model can be trained using a large amount of image data containing the target object. The initial color feature extraction model can be constrained using the ArcFace loss function to ensure convergence, thus obtaining the final color feature extraction model. This model is used to extract features from images containing the target object, obtaining the planar feature information of the target object. When the image to be identified is a face image, the loss function used during training of the color feature extraction model can be the ArcFace loss function. The ArcFace loss function is a face recognition algorithm that reduces the distance between similar classes and increases the distance between dissimilar classes.

[0105] Optionally, the computer device can pre-train a color feature extraction model. When it is necessary to extract features from an image containing a target object to obtain planar feature information of the target object, the color feature extraction model can be directly called for feature extraction. Specifically, the computer device can pre-train the color feature extraction model in the following ways: the computer device can acquire an initial color feature extraction model, a sample color image containing the target object, and planar feature information labels of the target object. The initial color feature extraction model is used to extract features from the sample color image to obtain predicted planar feature information of the target object. The computer device can determine the loss value of the initial color feature extraction model based on the difference between the planar feature information labels of the target object and the predicted planar feature information, and adjust the parameters of the initial color feature extraction model based on this loss value. When the parameter-adjusted initial color feature extraction model meets the convergence condition, it is determined as the color feature extraction model and used to extract features from the image containing the target object to obtain planar feature information of the target object. The convergence condition can refer to the initial color feature extraction model reaching a target number of training iterations, or the loss value of the initial color feature extraction model being less than or equal to a target threshold.

[0106] S102, convert the image to be recognized into a depth image, extract depth features from the depth image, and obtain the depth feature information of the target object.

[0107] Specifically, computer equipment can convert the image to be recognized into a depth image, also known as a range image. This image uses pixel values ​​to represent the distance (depth) between the image acquisition device (such as a camera component) and various points in the scene (i.e., the image to be recognized, including the target object). It directly reflects the geometry of the visible surface of the target object. Computer equipment can extract depth features from the depth image to obtain the depth feature information of the target object. Because the distance between different parts of the target object and the camera component varies when the target object is photographed, meaning that different parts of the target object may not be on the same horizontal plane and are uneven, computer equipment can convert the image to be recognized into a depth image and extract depth features from it to obtain the depth feature information of the target object. This allows for the acquisition of depth feature information of the target object that differs from planar feature information. By obtaining multi-dimensional feature information of the target object and performing attribute recognition based on this multi-dimensional feature information, a good defensive effect can be achieved, improving the accuracy of recognition and preventing attacks on the target object's feature information that could lead to inaccurate or unreliable recognition results.

[0108] Optionally, after acquiring an image of the target object, the computer device can calculate the distance information between each pixel in the image and the camera component based on the lens imaging formula.

[0109] Optionally, the computer device converts the image to be recognized into a depth image and extracts depth features from the depth image to obtain the depth feature information of the target object. Specific methods may include: interpolating the image to be recognized to obtain a standard-sized image; calling a target depth image conversion model to convert the standard-sized image to obtain an initial depth image; generating a depth image based on the initial depth image; and calling a depth feature extraction model to extract depth features from the depth image to obtain the depth feature information of the target object.

[0110] Specifically, the computer device can perform interpolation processing on the image to be recognized, adjusting its size to a standard size. This facilitates subsequent image conversion within the target depth image conversion model. The computer device can invoke the target depth image conversion model, which converts the image captured by the camera component into a depth image. This conversion process transforms the standard-sized image to obtain an initial depth image. The target depth image conversion model utilizes the depth relationships between pixels in the image to be recognized and the contextual relationships within the image to reconstruct the initial depth image. This initial depth image includes the normal components of the target object in the X and Y directions. Therefore, a depth image can be generated based on this initial depth image, containing the normal components of the target object in the X, Y, and Z directions. After obtaining the depth image corresponding to the image to be identified, the computer device can call the depth feature extraction model to extract the depth features from the depth image and obtain the depth feature information of the target object. The depth feature information of the target object can refer to the feature information of the target object in three-dimensional space (such as the positional and directional relationship information of the various parts of the target object in three-dimensional space).

[0111] Optionally, the computer device invokes a target depth image conversion model to convert a standard-sized image to be recognized, obtaining an initial depth image. Specific methods may include: using the encoder in the target depth image conversion model to obtain distance information between each pixel in the standard-sized image to be recognized and a reference horizontal plane; using the decoder in the target depth image conversion model to predict the distance information between each pixel in the standard-sized image to be recognized and the reference horizontal plane, outputting the depth information of each pixel in the standard-sized image to be recognized; and generating an initial depth image based on the depth information of each pixel.

[0112] Specifically, the computer device can obtain the distance information between each pixel in the standard-sized image to be recognized and a reference horizontal plane through the encoder in the target depth image conversion model. This reference horizontal plane can be the horizontal plane where the camera of the camera assembly is located, or it can be any horizontal plane between the horizontal plane where the camera assembly is located and the horizontal plane where the target object is located, or any other horizontal plane, depending on specific needs. This embodiment does not impose any limitations on this. When the camera assembly captures the target object, the pixel feature information corresponding to pixels at different locations is different. Therefore, the computer device can obtain the pixel feature information of each pixel and, based on this pixel feature information, obtain the distance information between each pixel in the standard-sized image to be recognized and the reference horizontal plane. Since the various parts of the target object are uneven, the distance information between each pixel in the area where the target object is located in the standard-sized image to be recognized and the reference horizontal plane is different. The encoder in the target depth image conversion model can also encode the distance information between each pixel in the area where the target object is located in the standard-sized image to be recognized and the reference horizontal plane. Computer equipment can predict the distance information between each pixel in a standard-sized image to be recognized and a reference horizontal plane using the decoder in the target depth image conversion model. It then outputs the depth information of each pixel in the standard-sized image, meaning different distances correspond to different depths. The computer equipment can perform 3D modeling based on the distance information of each pixel, determine the depth information of each pixel in the image to be recognized, and combine the depth information of each pixel to obtain an initial depth image. The target depth image conversion model is an encoder-decoder network structure, mainly composed of an encoder and a decoder, with an overall U-shaped structure. It includes residual convolutional layers and sub-pixel convolutional layers. The residual convolutional layers are used for feature extraction from the image to be processed, reducing computation and improving the efficiency of feature extraction. The sub-pixel convolutional layers are used to improve image resolution, achieving super-resolution. The encoder-decoder network structure is mainly divided into three parts: downsampling, upsampling, and skip connections. This structure can be analyzed by dividing it into two parts: the left side is the compression (encoding) process, i.e., the encoder, which reduces the image size and extracts some superficial features through convolution and downsampling. The right side represents the decoding process, or Decoder, which uses convolution and upsampling to acquire deeper features. The middle section then combines the features obtained during the encoding and decoding stages, refining the image by combining deep and shallow features, and outputting the combined feature information.

[0113] Optionally, the specific method by which the computer device generates a depth image based on an initial depth image may include: obtaining the normal component of the initial depth image in a first direction, obtaining the normal component of the initial depth image in a second direction, calculating the normal component of the initial depth image in a third direction based on the normal components in the first and second directions, and combining the normal components in the first, second, and third directions to obtain the depth image.

[0114] Specifically, after obtaining the initial depth image, the computer device can acquire the normal component of the initial depth image in a first direction, which can be the X direction, and the normal component in a second direction, which can be the Y direction. For example, the computer device can calculate the first-order differential images in the X and Y directions of the initial depth image, and based on these first-order differential images in the X and Y directions, calculate the normal component of the initial depth image in a third direction, which can be the Z direction. After acquiring the first-order differential images in the X and Y directions of the initial depth image, the computer device can calculate the normal component of the initial depth image in the X direction... 2 +Y 2 +Z 2 =1, calculate the normal component in the Z direction. After the computer device obtains the normal components in the X, Y, and Z directions of the initial depth image, it can combine the normal components in these three directions to obtain a depth image. This depth image includes the normal components of the image to be identified in the three directions, that is, it makes the target object in the image to be identified three-dimensional, which facilitates the subsequent extraction of depth features from the depth image of the image to be identified, and obtains the depth feature information of the target object.

[0115] Optionally, the computer device may invoke a depth feature extraction model to extract depth features from a depth image to obtain the depth feature information of the target object. Specific methods may include: invoking the depth feature extraction model to perform vector transformation on the depth image, obtaining a corresponding depth image vector; performing convolution operations on the depth image vector through a residual convolutional layer in the depth feature extraction model, obtaining a convolutional image vector; and obtaining a depth residual feature vector based on the difference between the depth image vector and the convolutional image vector. Finally, performing a linear transformation on the depth residual feature vector through a fully connected layer in the depth feature extraction model to obtain the depth feature information of the target object.

[0116] Specifically, the computer device can invoke a deep feature extraction model to extract deep features from a depth image. This deep feature extraction model can be a convolutional neural network structure, composed of residual convolutional layers and fully connected layers, outputting a depth feature vector with a dimension of 512. The computer device can use the deep feature extraction model to perform vector transformation on the depth image, obtaining a corresponding depth image vector. It can then use the residual convolutional layer in the deep feature extraction model to perform convolution processing on the depth image vector, obtaining a convolutional image vector. This convolution processing refers to matrix multiplication, used for feature extraction. Based on the difference between the depth image vector and the convolutional image vector, a depth residual feature vector corresponding to the depth image is obtained. The computer device can then use the pooling layer in the deep feature extraction model to pool this depth residual feature vector, thereby standardizing it to obtain a standardized depth residual feature vector. The pooling layer in the deep feature extraction model also pools the depth residual feature vector output by the residual convolutional layer. The role of pooling is similar to that in the aforementioned color feature extraction model, and will not be repeated here. Computer equipment obtains the depth residual feature vector corresponding to the depth image. This residual feature vector can then be linearly transformed (i.e., linearly combined) through a fully connected layer in the depth feature extraction model to obtain the depth feature information of the target object. The depth feature extraction model can be trained on a large amount of sample image data, and its convergence can be constrained using the arcface loss function to ensure it meets the convergence conditions.

[0117] Optionally, the computer device can pre-train a target depth image conversion model. The computer device can acquire an initial depth image conversion model, sample images, and corresponding initial depth image labels. The initial depth image conversion model is used to perform depth recognition on the sample images to obtain predicted initial depth images. A first loss function is determined based on the pixel difference between the initial depth image labels and the predicted initial depth images. A second loss function is determined based on the difference between the normal vectors of pixels in the initial depth image labels and the normal vectors of pixels in the predicted initial depth images. The initial depth image conversion model is iteratively trained using the first and second loss functions to obtain the target depth image conversion model.

[0118] Specifically, the computer device can acquire an initial depth image conversion model, a sample image, and the corresponding initial depth image label for that sample image. The sample image can be a color image. The computer device can use the initial depth image conversion model to perform depth recognition on the sample image, obtaining a predicted initial depth image corresponding to the sample image. The computer device can obtain the pixel difference between pixels in the initial depth image label and pixels in the predicted initial depth image to determine the first loss function. For example, the loss function MSE (mean-square error) or MAE (mean absolute error) can be determined based on the pixel difference between the initial depth image label and the predicted initial depth image. MSE refers to the expected value of the squared difference between the parameter estimate and the parameter value, a method used to measure "mean error." The smaller the MSE value, the better the model's accuracy. MAE refers to the average distance between the model's predicted value f(x) and the true value y, a method used to measure "model error." The computer device can obtain the difference between the normal vector of pixels in the initial depth image label and the normal vector of pixels in the predicted initial depth image to determine the second loss function.

[0119] The specific formula for the second loss function can be shown in formula (1) below:

[0120]

[0121] In formula (1), W refers to the width of the initial depth image label and the predicted initial depth image, H refers to the height of the initial depth image label and the predicted initial depth image, No refers to the true normal vector, No refers to the predicted normal vector, No and Nr represent the three-channel normal images calculated by Do (i.e., the initial depth image label) and Dr (i.e., the predicted initial depth image), respectively. The three channels store the component magnitudes of the normal vector in the x, y, and z directions, respectively. No(i,j) represents the normal vector (x(i,j), y(i,j), z(i,j)) of the Do(i,j) coordinate point, and Nr(i,j) represents the normal vector (x(i,j), y(i,j), z(i,j)) of the Dr(i,j) coordinate point. The similarity between No and Nr is calculated by the average cosine distance of the normal vector of each pixel.

[0122] Specifically, after obtaining the first and second loss functions, the computer device can determine the model adjustment parameters corresponding to the initial depth image conversion model based on these functions. The initial model parameters in the initial depth image conversion model are then updated to the model adjustment parameters, thereby adjusting the model parameters in the initial depth image conversion model. The computer device can continue training the parameter-adjusted initial depth image conversion model and check whether it meets the convergence condition. If the parameter-adjusted initial depth image conversion model meets the convergence condition, it is determined as the target depth image conversion model. If the parameter-adjusted initial depth image conversion model does not meet the convergence condition, parameter adjustment and training continue until the convergence condition is met. The convergence condition can refer to the initial depth image conversion model reaching the target number of training iterations, or the loss value of the initial depth image conversion model being less than or equal to a target threshold.

[0123] S103, fuse planar feature information and depth feature information to obtain fused feature information of the target object.

[0124] Specifically, computer equipment can fuse planar and depth feature information of a target object to obtain fused feature information of the target object. In this way, multi-dimensional feature information of the target object can be obtained. Based on this multi-dimensional feature information, attribute recognition of the target object can be performed, which can improve the credibility and accuracy of the recognition. At the same time, it can also achieve a good recognition defense effect, avoiding the situation where the single feature information is easily attacked when using a single feature information to identify the attribute of the target object, resulting in inaccurate recognition results.

[0125] like Figure 4 As shown, Figure 4 This is a schematic diagram of a method for obtaining fused feature information provided in an embodiment of this application, as shown below. Figure 4 As shown, the computer device can interpolate the image 40a to be identified, which includes the target object, to obtain a standard-sized image 40b to be identified. The computer device can then input the standard-sized image 40b into the target depth image conversion model 40c to perform image conversion on the standard-sized image 40b, obtaining an initial depth image 40d. For example... Figure 4As shown, the target depth image conversion model has a U-shaped structure, including downsampling, upsampling, and skip connections. This encoder-decoder network structure can be analyzed by dividing it into two parts: the left side is the compression (encoding) process, i.e., the Encoder, which reduces the image size and extracts some superficial features through convolution and downsampling. The right side is the decoding process, i.e., the Decoder, which obtains deeper features through convolution and upsampling. Then, the middle part combines the feature information obtained in the encoding stage with that obtained in the decoding stage, i.e., combining deep and shallow features to refine the image and output the combined feature information. After obtaining the initial depth image 40d, the computer device can acquire the normal components in the X, Y, and Z directions of the initial depth image. Combining these normal components in these three directions yields the depth image 40e. The computer device can interpolate the depth image 40e to obtain a standard-sized depth image 40f, and input this standard-sized depth image 40f into the depth feature extraction model 40g, inputting the depth feature information of the target object. The specific details of feature extraction performed by the depth feature extraction model can be found in the description of step S102 above, and will not be repeated here. The computer device can also input the image to be identified containing the target object into the color feature extraction model 40h, outputting the planar feature information of the target object. The specific details of feature extraction performed by the color feature extraction model 40h can be found in the description of step S101 above, and will not be repeated here.

[0126] This solution utilizes multimodal features (i.e., planar feature information of the target object in the image to be identified and depth feature information of the target object in the depth image) and a dual-branch network (a target depth image transformation model and a depth feature extraction model for acquiring depth feature information of the target object, and a color feature extraction model for acquiring planar feature information of the target object). This approach can effectively defend against adversarial attacks while ensuring the accuracy of image recognition, improving the stability and security of image data, and enhancing the credibility of image recognition.

[0127] S104, perform attribute recognition on the target object based on the fusion feature information of the target object to obtain the attribute information of the target object.

[0128] Specifically, computer equipment can identify the attributes of a target object based on its fused feature information, thus obtaining the target object's attribute information. This attribute information can indicate whether the target object is a legitimate object. For example, the computer equipment can search a feature library for sample feature information that matches the fused feature information of the target object. The feature library includes multiple legitimate sample feature information. If a sample feature information matching the fused feature information of the target object exists in the feature library, then the target object's attribute information can be determined to be legitimate; if no such sample feature information exists, then the target object's attribute information can be determined to be illegitimate.

[0129] like Figure 5 As shown, Figure 5 This is a schematic diagram illustrating attribute recognition based on fused feature information provided in an embodiment of this application, such as... Figure 5 As shown, the image to be identified can be a face image. When performing identity verification on an image including a target object, if the target object's feature information is not maliciously attacked, the feature information extracted by the computer device is as shown in 50a, and the identity of the target object can be accurately identified. When the target object's feature information is maliciously attacked, if the computer device only performs identity verification based on the target object's single planar feature information, the feature information extracted by the computer device is as shown in 50b. The target object's feature information has been altered, and when performing identity verification based on this altered feature information, an identification error will occur. However, in this embodiment, as shown in 50c, the computer device can extract both planar and depth feature information of the target object. A malicious attack can only target either the planar or depth feature information; the possibility of both being attacked simultaneously is small. Therefore, performing identity verification based on the fused feature information obtained by fusing planar and depth feature information can achieve a good defensive effect and improve the reliability and accuracy of the identification. As can be seen from Figure 50c, even after the fused feature information of the target object is attacked, it can still basically maintain its original features. That is, after fusing the planar feature information and depth feature information of the target object, it can be satisfactorily attacked. Therefore, by introducing planar feature information and depth feature information, the embodiments of this application increase the difficulty of attacking and tampering with the image data of the image to be identified compared with single planar feature information, thus achieving a good defense effect and improving the reliability of image recognition.

[0130] In this embodiment, an image to be identified, including the target object, is acquired. Planar features are extracted from the image to obtain planar feature information of the target object, which reflects its two-dimensional features. The image to be identified is then converted into a depth image, and depth features are extracted from the depth image to obtain depth feature information of the target object. This depth feature information reflects the geometry of the visible surface of the target object, i.e., it contains its three-dimensional features. The planar and depth feature information are fused to obtain fused feature information of the target object. Attribute identification is then performed on the target object based on this fused feature information to obtain its attribute information. The multi-dimensional fused feature information is difficult to attack; that is, both planar and depth feature information are difficult to attack simultaneously. Even when planar feature information is attacked, attribute identification can still be performed based on depth feature information, and vice versa. Therefore, by using multi-dimensional fused feature information to identify the attributes of the target object, attacks on image feature information can be prevented to a certain extent, achieving a good identification defense effect and improving the reliability and accuracy of the identification. In addition, this application also extracts planar feature information of the target object through a color feature extraction model and extracts depth feature information of the target object through a depth feature extraction model, which can improve the efficiency of feature extraction.

[0131] Please see Figure 6 , Figure 6 This is a flowchart illustrating an image recognition method provided in an embodiment of this application. The image recognition method can be executed by a computer device, which can be a server (as described above). Figure 1 Server 10 in the middle), or user terminal (as described above) Figure 1 This application does not limit the scope to any user terminal in the user terminal cluster. Figure 6 As shown, the image recognition method may include steps S201-S207.

[0132] S201, Obtain the original image including the target object.

[0133] Specifically, the computer device can acquire an original image of the target object, which can be obtained by a camera component capturing the target object or by the computer device responding to a user's upload operation.

[0134] S202, Extract key feature points of the target object from the original image.

[0135] Specifically, because the shape, size, and orientation of the target object are inconsistent in the original images captured by the camera component each time it takes a picture, it is necessary to correct the target object in the original image to ensure that it is in a standard shape, size, and orientation, facilitating subsequent feature extraction. Specifically, the computer device can extract key feature points of the target object from the original image. These key feature points can refer to boundary key points and contour key points of the target object in the original image. Different target objects may have different key feature points, which can be pre-set for different target objects. The computer device can acquire the location information of the key feature points corresponding to different target objects and establish a correspondence between the object identifier of different target objects and the location information of their corresponding key features, storing this correspondence in a key point library. Thus, the computer device can determine the location information of the key feature points of the target object from the key point library based on the object identifier of the target object, and extract the key feature points of the target object from the original image based on this location information.

[0136] S203, Correct the original image based on the key feature points of the target object to obtain an image to be identified that includes the target object.

[0137] Specifically, computer equipment can perform correction processing on the original image based on the key feature points of the target object, resulting in an image to be identified that includes the target object. Specifically, the computer equipment can perform correction processing on the target object in the original image based on the positional relationships (such as directional and distance relationships) between the key feature points of the target object, resulting in an image to be identified that includes the target object.

[0138] Optionally, the computer device may perform correction processing on the original image based on the key feature points of the target object to obtain a recognition image including the target object. Specific methods may include: acquiring key point features of an object template; determining the difference information between the key point features of the object template and the key feature points of the target object; and adjusting the original image based on the difference information to obtain the recognition image including the target object.

[0139] Specifically, the computer device acquires the object template keypoint features of the target object. These object template keypoint features can refer to an object template keypoint matrix, which can be pre-set based on the target object's placement, shape, and size. For example, if the target object's placement, shape, and size in image A are of a standard style, then the object keypoints of the target object in image A can be extracted and determined as the object template keypoint features. The computer device can determine the difference information between the object template keypoint features and the key feature points of the target object, and adjust the placement, shape, and size of the target object in the original image based on this difference information to obtain an image to be recognized that includes the target object. Furthermore, the computer device can perform affine transformations on the original image of the target object based on the object template keypoint matrix to correct the original image and obtain an image to be recognized that includes the target object. Affine transformations include translation, scaling, flipping, rotation, and shearing.

[0140] S204, perform planar feature extraction on the image to be recognized to obtain the planar feature information of the target object.

[0141] S205, convert the image to be recognized into a depth image, extract depth features from the depth image, and obtain the depth feature information of the target object.

[0142] S206, fuse planar feature information and depth feature information to obtain fused feature information of the target object.

[0143] S207, Based on the fusion feature information of the target object, attribute identification is performed on the target object to obtain the attribute information of the target object.

[0144] Specifically, the details of steps S204-S207 can be found above. Figure 3 The specific details of steps S101-S104 are not repeated here in the embodiments of this application.

[0145] Specifically, the image to be identified in this application embodiment can refer to a face image. This application embodiment can be applied to facial recognition payment scenarios. It should be noted that when the target object is the face of the target user, the processing of the image to be identified in this application is authorized by the target user in the image to be identified. The computer device can acquire the face image of the target user through a user terminal (such as a mobile phone, computer, or tablet). The face image of the target user can be a color image, and the key facial points in the face image of the target user can be detected. The computer device can perform alignment processing on the face image of the target user to obtain a face image of standard image size. It can also perform quality assessment on the face image of standard image size to determine whether the quality of the face image of standard image size is qualified. If the quality of the face image of standard image size is qualified, planar feature extraction is performed on the face image of standard image size to obtain the planar feature information of the target user. The computer device can perform image conversion on the face image of standard image size to obtain a depth image corresponding to the target user, and perform depth feature extraction on the depth image corresponding to the target user to obtain the depth feature information of the target user. The computing device can fuse the planar and depth feature information of a target user to obtain fused feature information. Based on this fused feature information, it retrieves facial features from a user database and outputs the recognition result. If a sample feature information matching the target user's fused feature information exists in the user database, the target user is determined to be a valid user, and subsequent payment is processed. If no matching sample feature information exists in the user database, the target user is determined to be an invalid user, and a facial recognition payment failure interface is displayed. Alternatively, the target user's fused feature information can be stored in the user database for later retrieval.

[0146] In this embodiment, key feature points of the target object are extracted from the original image containing the target object, and the original image is corrected based on these key feature points to obtain an image to be identified containing the target object. This correction process eliminates some interference factors, thereby improving the accuracy of image recognition. Planar feature extraction is performed on the image to be identified to obtain planar feature information of the target object, which reflects its two-dimensional feature information. The image to be identified is converted into a depth image, and depth feature extraction is performed on the depth image to obtain depth feature information of the target object, which reflects the geometry of the visible surface of the target object, i.e., it contains its three-dimensional feature information. The planar feature information and depth feature information are fused to obtain fused feature information of the target object. Attribute recognition is then performed on the target object based on this fused feature information to obtain its attribute information. The multi-dimensional fused feature information is difficult to attack; that is, both planar and depth feature information are difficult to attack simultaneously. Even if the planar feature information is attacked, attribute recognition can still be performed based on the depth feature information, and vice versa. It is evident that by fusing multi-dimensional feature information to identify the attributes of target objects, the attack on image feature information can be prevented to a certain extent, achieving a good identification defense effect and improving the credibility and accuracy of identification. Furthermore, this application also extracts planar feature information of the target object through a color feature extraction model and depth feature information through a depth feature extraction model, thereby improving the efficiency of feature extraction.

[0147] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of this application. The aforementioned image recognition device can be a computer program (including program code) running on a computer device; for example, the image recognition device is application software. The image recognition device can be used to execute corresponding steps in the image recognition method provided in the embodiments of this application. Figure 7 As shown, the image recognition device may include: a planar feature extraction module 11, a depth feature extraction module 12, a fusion module 13, and an attribute recognition module 14.

[0148] Planar feature extraction module 11 is used to acquire an image to be identified including the target object, perform planar feature extraction on the image to be identified, and obtain planar feature information of the target object;

[0149] The depth feature extraction module 12 is used to convert the image to be recognized into a depth image, extract depth features from the depth image, and obtain the depth feature information of the target object.

[0150] The fusion module 13 is used to fuse planar feature information and depth feature information to obtain fused feature information of the target object;

[0151] The attribute recognition module 14 is used to identify the attributes of the target object based on the fusion feature information of the target object, and obtain the attribute information of the target object.

[0152] The deep feature extraction module 12 includes:

[0153] The interpolation processing unit 1201 is used to perform interpolation processing on the image to be recognized to obtain a standard-sized image to be recognized.

[0154] The image conversion unit 1202 is used to call the target depth image conversion model to perform image conversion on the standard-sized image to be recognized to obtain an initial depth image;

[0155] The generation unit 1203 is used to generate a depth image based on the initial depth image;

[0156] The depth feature extraction unit 1204 is used to call the depth feature extraction model to extract depth features from the depth image and obtain the depth feature information of the target object.

[0157] Specifically, the image conversion unit 1202 is used for:

[0158] The encoder in the target depth image conversion model is used to obtain the distance information between each pixel in the standard-sized image to be recognized and the reference horizontal plane.

[0159] The decoder in the target depth image conversion model predicts the distance information between each pixel in the standard-sized image to be identified and the reference horizontal plane, and outputs the depth information of each pixel in the standard-sized image to be identified.

[0160] An initial depth image is generated based on the depth information of each pixel.

[0161] Specifically, the generating unit 1203 is used for:

[0162] Obtain the normal component of the initial depth image in the first direction, and obtain the normal component of the initial depth image in the second direction;

[0163] Based on the normal components in the first direction and the normal components in the second direction, calculate the normal components of the initial depth image in the third direction.

[0164] The depth image is obtained by combining the normal components in the first direction, the second direction, and the third direction.

[0165] Specifically, the deep feature extraction unit 1204 is used for:

[0166] The depth feature extraction model is invoked to perform vector transformation on the depth image, resulting in the depth image vector corresponding to the depth image.

[0167] By using the residual convolutional layer in the depth feature extraction model, the depth image vector is convolved to obtain the convolutional image vector. Based on the difference between the depth image vector and the convolutional image vector, the depth residual feature vector corresponding to the depth image is obtained.

[0168] By using a fully connected layer in the deep feature extraction model, the depth residual feature vector is linearly transformed to obtain the depth feature information of the target object.

[0169] The deep feature extraction module 12 also includes:

[0170] The first acquisition unit 1205 is used to acquire the initial depth image conversion model, the sample image, and the initial depth image label corresponding to the sample image.

[0171] The depth recognition unit 1206 is used to perform depth recognition on the sample image using an initial depth image conversion model to obtain the predicted initial depth image corresponding to the sample image.

[0172] The first determining unit 1207 is used to determine a first loss function based on the pixel difference between the initial depth image label and the predicted initial depth image;

[0173] The second determining unit 1208 is used to determine a second loss function based on the difference between the normal vector of the pixel in the initial depth image label and the normal vector of the pixel in the predicted initial depth image;

[0174] The iterative training unit 1209 is used to iteratively train the initial depth image conversion model according to the first loss function and the second loss function to obtain the target depth image conversion model.

[0175] The planar feature extraction module 11 includes:

[0176] The second acquisition unit 1101 is used to acquire the original image including the target object;

[0177] Extraction unit 1102 is used to extract key feature points of the target object from the original image;

[0178] The correction processing unit 1103 is used to perform correction processing on the original image based on the key feature points of the target object to obtain an image to be identified that includes the target object.

[0179] Specifically, the correction processing unit 1103 is used for:

[0180] Obtain key features of the object template;

[0181] Determine the differences between the key feature points of the object template and the key feature points of the target object;

[0182] The original image is adjusted based on the difference information to obtain an image to be identified that includes the target object.

[0183] According to one embodiment of this application, Figure 3 The steps involved in the image processing method shown can be derived from... Figure 7 The image processing apparatus shown is executed by various modules. For example, Figure 3 Step S101 shown can be performed by Figure 7 The planar feature extraction module 11 in the middle is used to perform the operation. Figure 3 Step S102 shown can be performed by Figure 7 The deep feature extraction module 12 in the middle is used to perform the operation. Figure 3 Step S103 shown can be performed by Figure 7 The fusion module 13 in the middle is used to execute it. Figure 3 Step S104 shown can be performed by Figure 7 The attribute recognition module 14 in the middle is used to perform, etc.

[0184] According to one embodiment of this application, Figure 7 The various modules in the image processing device shown can be individually or entirely combined into one or more units, or some of these units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical functional division; in practical applications, the function of one module can be implemented by multiple units, or the functions of multiple modules can be implemented by one unit. In other embodiments of this application, the testing device may also include other units; in practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0185] In this embodiment, key feature points of the target object are extracted from the original image containing the target object, and the original image is corrected based on these key feature points to obtain an image to be identified containing the target object. This correction process eliminates some interference factors, thereby improving the accuracy of image recognition. Planar feature extraction is performed on the image to be identified to obtain planar feature information of the target object, which reflects its two-dimensional feature information. The image to be identified is converted into a depth image, and depth feature extraction is performed on the depth image to obtain depth feature information of the target object, which reflects the geometry of the visible surface of the target object, i.e., it contains its three-dimensional feature information. The planar feature information and depth feature information are fused to obtain fused feature information of the target object. Attribute recognition is then performed on the target object based on this fused feature information to obtain its attribute information. The multi-dimensional fused feature information is difficult to attack; that is, both planar and depth feature information are difficult to attack simultaneously. Even if the planar feature information is attacked, attribute recognition can still be performed based on the depth feature information, and vice versa. It is evident that by fusing multi-dimensional feature information to identify the attributes of target objects, the attack on image feature information can be prevented to a certain extent, achieving a good identification defense effect and improving the credibility and accuracy of identification. Furthermore, this application also extracts planar feature information of the target object through a color feature extraction model and depth feature information through a depth feature extraction model, thereby improving the efficiency of feature extraction.

[0186] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 8 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a target user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The target user interface 1003 may include a display screen and a keyboard; optionally, the target user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 8 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a target user interface module, and a device control application.

[0187] exist Figure 8 In the computer device 1000 shown, the network interface 1004 provides network communication functionality; the target user interface 1003 is mainly used to provide an input interface for the target user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0188] The process involves acquiring an image to be identified that includes the target object, extracting planar features from the image to be identified, and obtaining the planar feature information of the target object.

[0189] The image to be identified is converted into a depth image, and depth features are extracted from the depth image to obtain the depth feature information of the target object.

[0190] By fusing planar and depth feature information, the fused feature information of the target object is obtained.

[0191] The attribute information of the target object is obtained by identifying the attribute information of the target object based on the fusion feature information of the target object.

[0192] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 or Figure 6 The description of the image recognition method in the corresponding embodiments can also be performed as described above. Figure 7 The description of the image recognition device in the corresponding embodiments will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0193] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned image recognition device. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figure 3 and Figure 6The description of the image recognition method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed and executed on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network. These multiple computing devices distributed across multiple locations and interconnected via a communication network can constitute a blockchain system.

[0194] Furthermore, it should be noted that this application also provides a computer program product or computer program, which may include computer instructions, which may be stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, causing the computer device to perform the aforementioned actions. Figure 3 and Figure 6 The description of the image recognition method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer program products or computer program embodiments related to this application, please refer to the description of the method embodiments of this application.

[0195] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0196] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0197] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0198] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0199] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An image recognition method, characterized in that, include: Obtain the initial depth image conversion model, sample images, and the initial depth image labels corresponding to the sample images; The initial depth image conversion model is used to perform depth recognition on the sample image to obtain the predicted initial depth image corresponding to the sample image. A first loss function is determined based on the pixel difference between the initial depth image label and the predicted initial depth image; The second loss function is determined based on the difference between the normal vectors of pixels in the initial depth image label and the normal vectors of pixels in the predicted initial depth image; Based on the first loss function and the second loss function, the initial depth image conversion model is iteratively trained to obtain the target depth image conversion model; A target image containing the target object is acquired. The target image is then vector-transformed to obtain an image vector. This image vector is input into a residual convolutional layer in a color feature extraction model. A convolution operation is performed on the image vector to obtain a convolutional image vector. The difference between the image vector and the convolutional image vector is obtained, resulting in a residual feature vector. This residual feature vector is then input into a pooling layer in a color face feature extraction model. The residual feature vector is standardized to obtain a standardized residual feature vector. Based on this residual feature vector, the planar feature information of the target object is determined. The standardized residual feature vector is obtained by replacing the result of a single point in the residual feature vector with the feature map statistics of its vector region. The target depth image conversion model is invoked to convert the image to be identified into a depth image, and depth features are extracted from the depth image to obtain the depth feature information of the target object. The planar feature information and the depth feature information are fused to obtain the fused feature information of the target object; The attribute information of the target object is obtained by performing attribute identification on the target object based on the fusion feature information of the target object.

2. The method according to claim 1, characterized in that, The step of calling the target depth image conversion model to convert the image to be identified into a depth image, extracting depth features from the depth image to obtain the depth feature information of the target object includes: The image to be identified is interpolated to obtain a standard-sized image to be identified; The target depth image conversion model is invoked to perform image conversion on the standard-sized image to be identified, thereby obtaining an initial depth image; Generate a depth image based on the initial depth image; The depth feature extraction model is invoked to extract depth features from the depth image, thereby obtaining the depth feature information of the target object.

3. The method according to claim 2, characterized in that, The step of calling the target depth image conversion model to perform image conversion on the standard-sized image to be recognized to obtain an initial depth image includes: The distance information between each pixel in the standard-sized image to be identified and the reference horizontal plane is obtained through the encoder in the target depth image conversion model. The decoder in the target depth image conversion model predicts the distance information between each pixel in the standard-sized image to be identified and the reference horizontal plane, and outputs the depth information of each pixel in the standard-sized image to be identified. An initial depth image is generated based on the depth information of each pixel.

4. The method according to claim 2, characterized in that, The step of generating a depth image based on the initial depth image includes: Obtain the normal component of the initial depth image in the first direction, and obtain the normal component of the initial depth image in the second direction; Based on the normal components in the first direction and the normal components in the second direction, the normal components of the initial depth image in the third direction are calculated. The normal components in the first direction, the normal components in the second direction, and the normal components in the third direction are combined to obtain a depth image.

5. The method according to claim 2, characterized in that, The process of calling a depth feature extraction model to extract depth features from the depth image to obtain the depth feature information of the target object includes: The depth feature extraction model is invoked to perform vector transformation on the depth image, resulting in the depth image vector corresponding to the depth image. The depth image vector is convolved by the residual convolutional layer in the depth feature extraction model to obtain a convolutional image vector. Based on the difference between the depth image vector and the convolutional image vector, the depth residual feature vector corresponding to the depth image is obtained. The depth feature information of the target object is obtained by linearly transforming the depth residual feature vector through the fully connected layer in the depth feature extraction model.

6. The method according to claim 1, characterized in that, The acquisition of the image to be identified, including the target object, includes: Obtain the original image including the target object; Extract key feature points of the target object from the original image; The original image is corrected based on the key feature points of the target object to obtain an image to be identified that includes the target object.

7. The method according to claim 6, characterized in that, The step of correcting the original image based on the key feature points of the target object to obtain a to-be-identified image including the target object includes: Obtain key features of the object template; Determine the differences between the key feature points of the object template and the key feature points of the target object; The original image is adjusted based on the difference information to obtain an image to be identified that includes the target object.

8. A computer device, characterized in that, include: Processor and memory; The processor is connected to a memory, wherein the memory is used to store a computer program, and the processor is used to invoke the computer program to cause the computer device to perform the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-7.

Images