Object attribute recognition method, device, medium and electronic device

Abstract

The present application provides an object attribute recognition method, device, medium and electronic device, which relate to the field of image processing technology. The method obtains a target image according to an image to be processed containing a target object; inputs the target image into a trained attribute recognition model to obtain an object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the attribute feature extraction branch includes a preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features. The method can effectively improve the efficiency of object attribute recognition.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an object attribute recognition method, device, medium and electronic device. Background Art

[0002] With the development of social economy, image recognition is becoming more and more common, and object attribute recognition associated with image recognition has become particularly important.

[0003] In the process of image recognition, the requirements for object attribute recognition are getting higher and higher. The process of object attribute recognition often involves identifying multiple attributes, and object attribute recognition is becoming more and more complex. For example, in traffic scenes, using video surveillance technology to assist in identifying whether non-motor vehicles have violated traffic regulations can improve law enforcement efficiency.

[0004] At present, the mainstream technology for non-motor vehicle attribute recognition is to use deep learning-based image classification methods to identify non-motor vehicle attributes, thereby determining whether the non-motor vehicle violates traffic rules. Due to the large number of attributes related to non-motor vehicles and cyclists, the method of one attribute corresponding to one model is not friendly to training overhead and inference time. Object attribute recognition usually takes a long time. Therefore, the object attribute recognition method in the related technology is not only time-consuming and labor-intensive, but also inefficient. Summary of the invention

[0005] To solve the existing technical problems, the embodiments of the present application provide an object attribute recognition method, device, medium and electronic device, which can improve the efficiency of object attribute recognition.

[0006] To achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for identifying object attributes, the method comprising:

[0008] Obtaining a target image according to an image to be processed containing a target object; the target image contains the target object;

[0009] The target image is input into a trained attribute recognition model to obtain an object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network sub-model is a network model based on the Transformer architecture; the attribute feature extraction branch includes the preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain target object graph features based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph features; the decoder module is used to obtain the target attribute features based on the target object graph features and the attribute feature information; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features.

[0010] The object attribute recognition method provided by the embodiment of the present application obtains a target image based on a to-be-processed image containing a target object; the target image contains the target object; the target image is input into a trained attribute recognition model to obtain an object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network sub-model is a network model based on the Transformer architecture; the attribute feature extraction branch includes the preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain target object graph features based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph features; the decoder module is used to obtain the target attribute features based on the target object graph features and the attribute feature information; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features. This method extracts target object graph features through a deep learning-based attribute recognition model, and then obtains target attribute features through a preset number of attribute query networks and decoder modules included in the attribute recognition model, and finally determines the object attribute recognition result of the target object. A preset number of multiple attribute predictions can be integrated into a unified attribute recognition model, which can save training overhead and reasoning time. At the same time, the deep feature extraction network sub-model can explore the correlation between different attributes, thereby improving the accuracy of object attribute recognition, which can effectively improve the efficiency of object attribute recognition.

[0011] In an optional embodiment, obtaining the target image according to the image to be processed containing the target object includes:

[0012] Performing object detection on the image to be processed to identify the target object contained in the image to be processed;

[0013] The image to be processed is cropped based on the target object to obtain the target image.

[0014] In this embodiment, the target image is obtained based on the image to be processed containing the target object, including: performing object detection on the image to be processed to identify the target object contained in the image to be processed; and performing image cropping on the image to be processed based on the target object to obtain the target image. This method can automatically perform object detection and image cropping based on the image to be processed containing the target object to obtain the target image containing the target object, and can quickly generate the target image without manual operation, further saving the labor cost of object attribute recognition, and can effectively improve the efficiency of object attribute recognition.

[0015] In an optional embodiment, the target attribute feature is obtained by the following steps:

[0016] Performing feature extraction processing on the acquired target image to obtain the target object graph features;

[0017] Performing attribute extraction processing on the target object graph features to obtain the attribute feature information;

[0018] The target attribute feature is obtained according to the target object graph feature and the attribute feature information.

[0019] In this embodiment, the target attribute feature is obtained by the following steps: feature extraction processing is performed on the acquired target image to obtain the target object graph feature; attribute extraction processing is performed on the target object graph feature to obtain the attribute feature information; the target attribute feature is obtained based on the target object graph feature and the attribute feature information. This method can determine the target object graph feature and the attribute feature information, and then obtain the target attribute feature based on the target object graph feature and the attribute feature information, providing a mechanism for mining the association relationship between different attributes, which can improve the accuracy of object attribute recognition and effectively improve the efficiency of object attribute recognition.

[0020] In an optional embodiment, the image feature extraction branch includes a backbone network and an encoder module; the step of performing feature extraction processing on the acquired target image to obtain target object graph features includes:

[0021] Extracting image features from the target image through the backbone network to obtain feature map information;

[0022] The encoder module performs object feature extraction on the feature map information to obtain the target object map features.

[0023] In this embodiment, the image feature extraction branch includes a backbone network and an encoder module; the backbone network is used to extract image features from the target image to obtain feature map information; the encoder module is used to extract object features from the feature map information to obtain the target object map features. This method can automatically extract object features from the feature map information based on self-attention through the encoder module to obtain the target object map features, which can reduce the inference time of the object attribute recognition process, improve the accuracy of object attribute recognition, and effectively improve the efficiency of object attribute recognition.

[0024] In an optional embodiment, the attribute query network is obtained by improving the target detection framework DETR so that the object query of the DETR is modified into an attribute query corresponding to the attribute.

[0025] In this embodiment, the attribute query network is obtained by improving the target detection framework DETR so that the object query of the DETR is modified into an attribute query corresponding to the attribute. The method, by improving the target detection framework DETR, modifies the object query of the original DETR into an attribute query corresponding to the attribute, thereby being able to extract image features through the decoder module, obtain attribute features through the decoder module and the attribute query, and finally perform attribute classification through each attribute classification head of the attribute prediction sub-model, thereby improving the accuracy of object attribute recognition and effectively improving the efficiency of object attribute recognition.

[0026] In an optional embodiment, the decoder includes a multi-head self-attention layer and a multi-head cross-attention layer; the target attribute feature is obtained according to the target object graph feature and the attribute feature information, including:

[0027] The attribute feature information is self-attention enhanced by the multi-head self-attention layer to obtain inter-attribute self-attention features;

[0028] The target attribute features are obtained by performing cross-attention feature extraction on the target object graph features and the self-attention features between the attributes through the multi-head cross-attention layer.

[0029] In this embodiment, the decoder includes a multi-head self-attention layer and a multi-head cross-attention layer; the process of obtaining the target attribute feature according to the target object graph feature and the attribute feature information includes: performing self-attention enhancement on the attribute feature information through the multi-head self-attention layer to obtain the self-attention feature between attributes; performing cross-attention feature extraction on the target object graph feature and the self-attention feature between attributes through the multi-head cross-attention layer to obtain the target attribute feature. This method performs self-attention enhancement on the attribute feature information through the multi-head self-attention layer to obtain the self-attention feature between attributes, and performs cross-attention feature extraction on the target object graph feature and the self-attention feature between attributes through the multi-head cross-attention layer to obtain the target attribute feature. This method can mine the correlation between different attributes, improve the accuracy of object attribute recognition, and effectively improve the efficiency of object attribute recognition.

[0030] In an optional embodiment, determining the object attribute recognition result of the target object based on the target attribute feature includes:

[0031] The target attribute feature is input into the attribute prediction sub-model to obtain the object attribute recognition result of the target object; the attribute prediction sub-model includes a feed-forward neural network FFN and a classifier softmax.

[0032] In this embodiment, the target attribute feature is input into the attribute prediction sub-model to obtain the object attribute recognition result of the target object; the attribute prediction sub-model includes a feedforward neural network FFN and a classifier softmax. In this method, the feedforward neural network FFN and the classifier softmax included in the attribute prediction sub-model can constitute the classification head of each attribute, and the attribute classification can be performed through the classification head of each attribute to obtain the object attribute recognition result of the target object, which can reduce the reasoning time of the object attribute recognition process, improve the accuracy of object attribute recognition, and effectively improve the efficiency of object attribute recognition.

[0033] In an optional embodiment, the attribute recognition model is trained by the following method:

[0034] Obtaining training sample data, wherein the training sample data includes object annotated images with the preset number of attribute information labels;

[0035] Based on the training sample data, a batch of training images is selected; the batch of training images includes a part of the object annotated image;

[0036] Inputting the batch training images into the attribute recognition model to be trained for training, and determining the attribute recognition loss value of the target loss function of the attribute recognition model to be trained; the target loss function is used to determine the difference between the object attribute recognition result determined by the target attribute feature extracted based on the object annotated image in the batch training images and the attribute information label of the object annotated image in the batch training images;

[0037] Determine whether the attribute recognition loss value converges to a preset target value. If the judgment result is no, adjust the parameters of the attribute recognition model to be trained according to the determined attribute recognition loss value and continue training until the judgment result is yes, thereby obtaining a trained attribute recognition model.

[0038] The method of this embodiment obtains training sample data, wherein the training sample data includes object annotated images with the preset number of attribute information labels; based on the training sample data, batch training images are selected; the batch training images include a part of the object annotated images; the batch training images are input into the attribute recognition model to be trained, and the attribute recognition loss value of the target loss function of the attribute recognition model to be trained is determined; the target loss function is used to determine the difference between the object attribute recognition result determined by the target attribute feature extracted based on the object annotated images in the batch training images and the attribute information labels of the object annotated images in the batch training images; it is determined whether the attribute recognition loss value converges to a preset target value, and if the determination result is no, the parameters of the attribute recognition model to be trained are adjusted according to the determined attribute recognition loss value and the training is continued until the determination result is yes, thereby obtaining a trained attribute recognition model. The method, the training sample data includes object annotated images with the preset number of attribute information labels, provides a training mechanism for an attribute recognition model, realizes the integration of a preset number of multiple attribute predictions into a unified attribute recognition model, and utilizes an attribute prediction sub-model to realize the determination of the object attribute recognition result of the target object based on the target attribute characteristics, which can save training overhead and reasoning time. At the same time, the deep feature extraction network sub-model included in the trained attribute recognition model can mine the correlation between different attributes, which can improve the accuracy of object attribute recognition and effectively improve the efficiency of object attribute recognition.

[0039] In a second aspect, an embodiment of the present application further provides an object attribute recognition device, the device comprising:

[0040] An image processing module, used to obtain a target image according to an image to be processed containing the target object; the target image contains the target object;

[0041] An attribute determination module is used to input the target image into a trained attribute recognition model to obtain an object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network sub-model is a network model based on the Transformer architecture; the attribute feature extraction branch includes the preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain target object graph features based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph features; the decoder module is used to obtain the target attribute features based on the target object graph features and the attribute feature information; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features.

[0042] In an optional embodiment, the image processing module is specifically used to:

[0043] Performing object detection on the image to be processed to identify the target object contained in the image to be processed;

[0044] The image to be processed is cropped based on the target object to obtain the target image.

[0045] In an optional embodiment, the attribute determination module is specifically used to:

[0046] The target attribute characteristics are obtained by the following steps:

[0047] Performing feature extraction processing on the acquired target image to obtain the target object graph features;

[0048] Performing attribute extraction processing on the target object graph features to obtain the attribute feature information;

[0049] The target attribute feature is obtained according to the target object graph feature and the attribute feature information.

[0050] In an optional embodiment, the image feature extraction branch includes a backbone network and an encoder module; the attribute determination module is specifically used to:

[0051] Extracting image features from the target image through the backbone network to obtain feature map information;

[0052] The encoder module performs object feature extraction on the feature map information to obtain the target object map features.

[0053] In an optional embodiment, the attribute query network is obtained by improving the target detection framework DETR so that the object query of the DETR is modified into an attribute query corresponding to the attribute.

[0054] In an optional embodiment, the decoder includes a multi-head self-attention layer and a multi-head cross-attention layer; the attribute determination module is specifically used to:

[0055] The attribute feature information is self-attention enhanced by the multi-head self-attention layer to obtain inter-attribute self-attention features;

[0056] The target attribute features are obtained by performing cross-attention feature extraction on the target object graph features and the self-attention features between the attributes through the multi-head cross-attention layer.

[0057] In an optional embodiment, the attribute determination module is specifically used to: input the target attribute feature into the attribute prediction sub-model to obtain the object attribute recognition result of the target object; the attribute prediction sub-model includes a feedforward neural network FFN and a classifier softmax.

[0058] In an optional embodiment, the device further includes a recognition model training module; the recognition model training module is used to train the attribute recognition model by the following method:

[0059] Obtaining training sample data, wherein the training sample data includes object annotated images with the preset number of attribute information labels;

[0060] Based on the training sample data, a batch of training images is selected; the batch of training images includes a part of the object annotated image;

[0061] Inputting the batch training images into the attribute recognition model to be trained for training, and determining the attribute recognition loss value of the target loss function of the attribute recognition model to be trained; the target loss function is used to determine the difference between the object attribute recognition result determined by the target attribute feature extracted based on the object annotated image in the batch training images and the attribute information label of the object annotated image in the batch training images;

[0062] Determine whether the attribute recognition loss value converges to a preset target value. If the judgment result is no, adjust the parameters of the attribute recognition model to be trained according to the determined attribute recognition loss value and continue training until the judgment result is yes, thereby obtaining a trained attribute recognition model.

[0063] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the object attribute recognition method of the first aspect is implemented.

[0064] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor implements the object attribute recognition method of the first aspect.

[0065] The technical effects brought about by any one of the implementation methods in the second to fourth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0067] Figure 1 A flowchart of an object attribute recognition method provided in an embodiment of the present application;

[0068] Figure 2 A schematic diagram of a process of obtaining a target image in a method for identifying object attributes provided in an embodiment of the present application;

[0069] Figure 3 A schematic diagram of a flow chart of obtaining an object attribute recognition result of a target object in an object attribute recognition method provided in an embodiment of the present application;

[0070] Figure 4 A schematic diagram of a process for obtaining target object graph features in another object attribute recognition method provided in an embodiment of the present application;

[0071] Figure 5 A schematic diagram of a flow chart of obtaining target attribute features of another object attribute recognition method provided in an embodiment of the present application;

[0072] Figure 6 A schematic diagram of a flow chart of obtaining an attribute recognition model by training another object attribute recognition method provided in an embodiment of the present application;

[0073] Figure 7 A schematic diagram of the structure of an object attribute recognition device provided in an embodiment of the present application;

[0074] Figure 8 A schematic diagram of the structure of another object attribute recognition device provided in an embodiment of the present application;

[0075] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0077] It should be noted that the terms "including" and "having" and their variations involved in the documents of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0078] The following are explanations of some of the terms that appear in the text:

[0079] (1) Transformer: Transformer is a deep learning model architecture used for natural language processing (NLP) and other sequence-to-sequence tasks. The Transformer architecture introduces a self-attention mechanism, which performs well when processing sequence data. The Transformer architecture mainly consists of four parts: input part, encoder, decoder, and output part.

[0080] (2) DETR (Detection Transformer): DETR is an end-to-end object detection network based on Transformer. The overall framework of DETR is as follows: first, the image features are extracted through CNN; then they are sent to the Transformer encoder-decoder. The structure of the encoder-decoder is basically the same as that of the Transformer, but there are modifications in the input and output parts; finally, the category and bbox predictions are obtained, and the network is optimized by calculating the loss through binary matching.

[0081] (3) Object queries (OQ): In the target detection framework of some embodiments of the present application, object queries are the number of predefined object queries, for example, the code can default to 100. The meaning of object queries in the target detection framework is that according to the features encoded by the Encoder, the Decoder converts 100 queries into 100 objects, that is, finally predicts the categories and bbox positions of these 100 objects. In some embodiments of the present application, object queries can also be called target queries.

[0082] With the rapid development of data storage, server equipment deployment is becoming more and more common. Object attribute identification associated with server equipment deployment has become particularly important.

[0083] Nowadays, the deployment environment of server equipment is becoming more and more complex. Equipment may be affected by the power supply network, power supply load, or high and low temperature, vibration and other environmental factors, and abnormal power failures often occur. Once an abnormal power failure occurs, it often causes abnormalities in the file system. When the file system is abnormal, the operating system cannot be mounted normally when powered on, resulting in the operating system being unable to start normally.

[0084] Since server equipment is usually deployed remotely, the operation and maintenance time is usually long. Now once the server stops service, it will face great losses. The longer the service is stopped, the greater the loss will be. Usually, in the scenario where the file system is damaged due to abnormal power failure, the operation and maintenance personnel need to enter the computer room and use the system boot disk to manually repair the file system. This object attribute recognition method is not only time-consuming and labor-intensive, but also has low efficiency in object attribute recognition.

[0085] To solve the existing technical problems, an embodiment of the present application provides an object attribute recognition method, wherein a target image is obtained according to an image to be processed containing a target object; the target image contains the target object; the target image is input into a trained attribute recognition model to obtain an object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network sub-model is a network model based on a Transformer architecture; the attribute feature extraction branch includes a preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain target object graph features based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph features; the decoder module is used to obtain target attribute features based on the target object graph features and the attribute feature information; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features. This method extracts target object graph features through a deep learning-based attribute recognition model, and then obtains target attribute features through a preset number of attribute query networks and decoder modules included in the attribute recognition model, and finally determines the object attribute recognition result of the target object. A preset number of multiple attribute predictions can be integrated into a unified attribute recognition model, which can save training overhead and reasoning time. At the same time, the deep feature extraction network sub-model can explore the correlation between different attributes, thereby improving the accuracy of object attribute recognition and effectively improving the efficiency of object attribute recognition.

[0086] The technical solution provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0087] The present application embodiment provides a method for identifying object attributes, such as Figure 1 As shown, the following steps are included:

[0088] Step S101, obtaining a target image according to an image to be processed containing a target object; the target image contains the target object.

[0089] In some embodiments of the present application, the target object is a non-motor vehicle; the image to be processed is a video surveillance image; and the target image is a non-motor vehicle image.

[0090] The following embodiments of the present application are all illustrated by taking the target object as a non-motor vehicle; the image to be processed as a video surveillance image; and the target image as a non-motor vehicle image as an example.

[0091] In specific implementation, the image to be processed can be a video surveillance image with a resolution of 1920*1080 commonly used for output video surveillance. The non-motor vehicle in the video surveillance image is identified by the target detection algorithm, and the non-motor vehicle image is cropped out according to the detection frame coordinates as the input of the subsequent object attribute recognition model.

[0092] In an optional embodiment, in step S101, the process of obtaining the target image according to the image to be processed containing the target object is as follows: Figure 2 As shown, this is achieved through the following steps:

[0093] Step S201 : performing object detection on the image to be processed to identify the target object contained in the image to be processed.

[0094] Step S202: cropping the image to be processed based on the target object to obtain a target image.

[0095] Step S102, input the target image into the trained attribute recognition model to obtain the object attribute recognition result of the target object; the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features.

[0096] Among them, the deep feature extraction network sub-model is a network model based on the Transformer architecture; the attribute feature extraction branch includes a preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain the target object graph features based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph features; the decoder module is used to obtain the target attribute features based on the target object graph features and the attribute feature information.

[0097] In an optional embodiment, in step S102, the target image is input into the trained attribute recognition model to obtain the object attribute recognition result of the target object, such as Figure 3 As shown, the target attribute features are obtained through the following steps:

[0098] Step S301, performing feature extraction processing on the acquired target image to obtain target object graph features.

[0099] In an optional embodiment, the image feature extraction branch includes a backbone network and an encoder module; S301, a process of performing feature extraction processing on the acquired target image to obtain the target object graph feature, such as Figure 4 As shown, the following steps are included:

[0100] Step S401, extracting image features from a target image through a backbone network to obtain feature map information.

[0101] In some embodiments, the backbone network is a Backbone network; the Backbone network is a general backbone network for image recognition tasks.

[0102] In some embodiments, the Backbone network can be ResNet or ViT (Vision Transformer).

[0103] In some embodiments, during specific implementation, non-motor vehicle features are extracted through Backbone and encoder. Backbone is a general backbone network for image recognition tasks, which can be ResNet or ViT. The preprocessed and enhanced image is input to Backbone to obtain a feature map. The feature dimension changes from B*C*H0*W0 to B*L*H*W. After reshaping, a feature map of B*HW*L is obtained, which is output to the encoder; where B represents the batch size; for the feature dimension B*C*H0*W0, C represents the number of image channels of the input image of Backbone, and the number of image channels of RGB images is generally 3, H0 represents the image height of the input image of Backbone, and W0 represents the image width of the input image of Backbone; for the feature dimension B*L*H*W, L represents the number of image channels of the feature map obtained by the image input to Backbone, H represents the image height of the feature map obtained by the image input to Backbone, and W represents the image width of the feature map obtained by the image input to Backbone; for the feature dimension B*HW*L, HW represents the feature map height of the reshaped feature map, and L represents the feature map width of the reshaped feature map.

[0104] Step S402: extract object features from feature graph information through an encoder module to obtain target object graph features.

[0105] In the embodiments of the present application, the encoder module may be referred to as an encoder for short; the decoder module may be referred to as a decoder for short.

[0106] In some embodiments, during implementation, the encoder uses the original DETR encoder, which includes 6 encoder layers, each of which includes a self-attention layer and a FFN (feedforward neural network) layer. The encoder takes the output of the Backbone network as input and outputs the extracted target object graph features, and the feature dimension of the target object graph features is B*HW*L, where B represents the batch size, HW represents the feature map height, and L represents the feature map width.

[0107] In some embodiments of the present application, the object features obtained by the encoder may be non-motor vehicle features.

[0108] Step S302: perform attribute extraction processing on the target object graph features to obtain attribute feature information.

[0109] In an optional embodiment, the attribute query network is obtained by improving the target detection framework DETR so that the object query of DETR is modified into an attribute query corresponding to the attribute.

[0110] Specifically, the embodiment of the present application adopts a Transformer-based deep neural network to perform object attribute recognition. Specifically, the target detection framework DETR is improved, and the original DETR's Object Query is modified to an attribute query (Attribute Query) corresponding to the attribute.

[0111] In some embodiments of the present application, the attribute query may be implemented by changing the object of the target query from a target to an attribute.

[0112] Step S303, obtaining target attribute features according to the target object graph features and attribute feature information.

[0113] In an optional embodiment, the decoder includes a multi-head self-attention layer and a multi-head cross-attention layer; step S303, according to the target object graph feature and attribute feature information, obtain the target attribute feature, such as Figure 5 As shown, this is achieved through the following steps:

[0114] Step S501, perform self-attention enhancement on attribute feature information through a multi-head self-attention layer to obtain inter-attribute self-attention features.

[0115] Step S502, cross-attention feature extraction is performed on the target object graph features and the self-attention features between attributes through a multi-head cross-attention layer to obtain the target attribute features.

[0116] In some embodiments of the present application, the image features are extracted by the encoder to obtain the target object graph features; the attribute features are obtained by the decoder and attribute query to obtain the target attribute features; finally, the attributes are classified by each attribute classification head to obtain the object attribute recognition result of the target object. The decoder adopts the decoder of the improved DETR, and takes the target object graph features output by the encoder and the attribute feature information obtained by the attribute query as input, wherein the feature dimension of the target object graph features output by the encoder is B*HW*L; the feature dimension of the attribute feature information obtained by the attribute query is B*N*L, wherein N represents the number of attributes, B represents the batch size, and L represents the feature map width. The decoder consists of 6 decoder layers, each of which contains a self-attention layer, a cross-attention layer, and an FFN (feedforward neural network) layer. The decoder outputs attribute features with a dimension of B*N*L, wherein N represents the number of attributes, B represents the batch size, and L represents the feature map width.

[0117] In some embodiments of the present application, the feature map width is a hyperparameter, which is set based on experience when designing the network structure and can be set to 256 or 512.

[0118] In some embodiments of the present application, the process of obtaining attribute features through a decoder and attribute query is specifically implemented through the decoder of DETR. The decoder is provided with 6 layers, each of which mainly includes Multi-Head Self-Attention and Multi-Head Cross-Attention. The Self-Attention layer encodes the attribute query as Q, K, and V to calculate self-attention to obtain the association relationship information between different attributes. The Cross-Attention layer encodes the attribute query as Q, and encodes the output of the encoder (i.e., image features) as K and V, and calculates cross-attention between the two to use attribute queries to obtain feature information related to the attributes from the image features.

[0119] In an optional embodiment, in step S102, the process of inputting the target image into the trained attribute recognition model to obtain the object attribute recognition result of the target object also includes image preprocessing. Image preprocessing is to preprocess the input target image; image preprocessing includes normalization, resizing, etc. Among them, normalization refers to the normalization method commonly used in image recognition tasks, generally including scaling the pixel value from 0-255 to between 0-1, and then performing a normalization operation of subtracting the mean and dividing the variance; resizing includes scaling the image to a preset resolution, and the width and height are generally 224*224.

[0120] In an optional embodiment, determining the object attribute recognition result of the target object based on the target attribute feature includes:

[0121] The target attribute features are input into the attribute prediction sub-model to obtain the object attribute recognition result of the target object; the attribute prediction sub-model includes a feed-forward neural network FFN and a classifier softmax.

[0122] In some embodiments of the present application, the object attribute recognition result of the target object is obtained by predicting the attribute category through FFN and softmax.

[0123] In the process of predicting attribute categories through FFN and softmax, the input of FFN is the attribute feature, and the output is the predicted value of the attribute. One attribute corresponds to one FFN. Taking the attribute of the number of non-motor vehicle riders as an example, assuming that this attribute has three values, 0, 1, and 2, then FFN will output 3 values. These 3 values ​​are converted into probabilities after softmax, and the attribute value corresponding to the largest probability is the predicted result output by the model.

[0124] In an optional embodiment, the object attribute recognition method, such as Figure 6 As shown, the attribute recognition model is also obtained by training through the following steps:

[0125] Step S601: obtaining training sample data, where the training sample data includes object annotated images with a preset number of attribute information labels.

[0126] Step S602: selecting a batch of training images based on the training sample data; the batch of training images includes a portion of the object annotated image.

[0127] Step S603, input the batch training images into the attribute recognition model to be trained for training, and determine the attribute recognition loss value of the target loss function of the attribute recognition model to be trained; the target loss function is used to determine the difference between the object attribute recognition result determined by the target attribute feature extracted based on the object annotated image in the batch training images and the attribute information label of the object annotated image in the batch training images.

[0128] Step S604, determine whether the attribute recognition loss value converges to a preset target value. If the judgment result is no, adjust the parameters of the attribute recognition model to be trained according to the determined attribute recognition loss value and continue training until the judgment result is yes, thereby obtaining a trained attribute recognition model.

[0129] In some embodiments of the present application, the process of training the attribute recognition model also includes data enhancement of the input non-motor vehicle image, including translation, rotation, etc.

[0130] The object attribute recognition method provided by the embodiment of the present application obtains a target image based on a to-be-processed image containing a target object; the target image contains the target object; the target image is input into a trained attribute recognition model to obtain an object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network sub-model is a network model based on the Transformer architecture; the attribute feature extraction branch includes a preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain target object graph features based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph features; the decoder module is used to obtain target attribute features based on the target object graph features and the attribute feature information; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features. This method extracts target object graph features through a deep learning-based attribute recognition model, and then obtains target attribute features through a preset number of attribute query networks and decoder modules included in the attribute recognition model, and finally determines the object attribute recognition result of the target object. A preset number of multiple attribute predictions can be integrated into a unified attribute recognition model, which can save training overhead and reasoning time. At the same time, the deep feature extraction network sub-model can explore the correlation between different attributes, thereby improving the accuracy of object attribute recognition, which can effectively improve the efficiency of object attribute recognition.

[0131] and Figure 1 The object attribute recognition method shown is based on the same inventive concept, and an object attribute recognition device is also provided in the embodiment of the present application. Since the device is a device corresponding to the object attribute recognition method of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the above method, and the repeated parts will not be repeated.

[0132] Figure 7 A schematic diagram of the structure of an object attribute recognition device provided in an embodiment of the present application is shown. Figure 7 As shown, the object attribute recognition device includes an image processing module 701 and an attribute determination module 702 .

[0133] The image processing module 701 is used to obtain a target image according to an image to be processed containing a target object; the target image contains the target object;

[0134] The attribute determination module 702 is used to input the target image into the trained attribute recognition model to obtain the object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network sub-model is a network model based on the Transformer architecture; the attribute feature extraction branch includes a preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain the target object graph features based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph features; the decoder module is used to obtain the target attribute features based on the target object graph features and the attribute feature information; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute features.

[0135] In an optional embodiment, the image processing module 701 is specifically configured to:

[0136] Perform object detection on the image to be processed to identify the target object contained in the image to be processed;

[0137] The image to be processed is cropped based on the target object to obtain a target image.

[0138] In an optional embodiment, the attribute determination module 702 is specifically configured to:

[0139] The target attribute features are obtained through the following steps:

[0140] Perform feature extraction processing on the acquired target image to obtain target object graph features;

[0141] Perform attribute extraction processing on the target object graph features to obtain attribute feature information;

[0142] According to the target object graph features and attribute feature information, the target attribute features are obtained.

[0143] In an optional embodiment, the image feature extraction branch includes a backbone network and an encoder module; an attribute determination module 702, specifically configured to:

[0144] Image features are extracted from the target image through the backbone network to obtain feature map information;

[0145] The encoder module is used to extract object features from the feature map information to obtain the target object map features.

[0146] In an optional embodiment, the attribute query network is obtained by improving the target detection framework DETR so that the object query of DETR is modified into an attribute query corresponding to the attribute.

[0147] In an optional embodiment, the decoder includes a multi-head self-attention layer and a multi-head cross-attention layer; an attribute determination module 702, specifically configured to:

[0148] The attribute feature information is enhanced through the multi-head self-attention layer to obtain the self-attention features between attributes;

[0149] The target object graph features and the self-attention features between attributes are cross-attention features extracted through a multi-head cross-attention layer to obtain the target attribute features.

[0150] In an optional embodiment, the attribute determination module 702 is specifically used to: input the target attribute feature into the attribute prediction sub-model to obtain the object attribute recognition result of the target object; the attribute prediction sub-model includes a feed-forward neural network FFN and a classifier softmax.

[0151] In an optional embodiment, if Figure 8 As shown, the device also includes a recognition model training module 801; the recognition model training module 801 is used to train and obtain an attribute recognition model by the following method:

[0152] Obtaining training sample data, the training sample data comprising object annotated images with a preset number of attribute information labels;

[0153] Based on the training sample data, a batch of training images is selected; the batch of training images includes a portion of the object annotated images;

[0154] Inputting the batch training images into the attribute recognition model to be trained for training, and determining the attribute recognition loss value of the target loss function of the attribute recognition model to be trained; the target loss function is used to determine the difference between the object attribute recognition result determined by the target attribute feature extracted based on the object annotated image in the batch training images and the attribute information label of the object annotated image in the batch training images;

[0155] Determine whether the attribute recognition loss value converges to the preset target value. If the judgment result is no, adjust the parameters of the attribute recognition model to be trained according to the determined attribute recognition loss value and continue training until the judgment result is yes, thereby obtaining a trained attribute recognition model.

[0156] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. The electronic device can be used for object attribute recognition. In one embodiment, the electronic device can be a server, or a terminal device or other electronic device. In this embodiment, the structure of the electronic device can be as follows: Fig. 9 As shown, it includes a memory 901 , a communication module 903 and one or more processors 902 .

[0157] The memory 901 is used to store computer programs executed by the processor 902. The memory 901 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0158] The memory 901 may be a volatile memory, such as a random-access memory (RAM); the memory 901 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or the memory 901 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 901 may be a combination of the above memories.

[0159] The processor 902 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 902 is configured to implement the above object attribute recognition method when calling the computer program stored in the memory 901 .

[0160] The communication module 903 is used to communicate with terminal devices and other servers.

[0161] The specific connection medium between the memory 901, the communication module 903 and the processor 902 is not limited in the embodiment of the present application. Fig. 9 In the embodiment, the memory 901 and the processor 902 are connected via a bus 904. The bus 904 is connected to the processor 902 via a bus 904. Fig. 9 The connections between other components are shown in bold lines, and are not intended to be limiting. Bus 904 can be divided into address bus, data bus, control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0162] According to one aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computer device executes the object attribute recognition method in the above-mentioned embodiment. The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0163] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for identifying object attributes, characterized in that: The method comprises: Obtaining a target image according to an image to be processed containing a target object; the target image contains the target object; Input the target image into the trained attribute recognition model to obtain the object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network sub-model and an attribute prediction sub-model; the deep feature extraction network sub-model is used to extract a preset number of target attribute features; the deep feature extraction network sub-model includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network sub-model is a network model based on the Transformer architecture; the attribute feature extraction branch includes the preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain the target object graph feature based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph feature; the decoder module is used to obtain the target attribute feature based on the target object graph feature and the attribute feature information; the attribute prediction sub-model is used to determine the object attribute recognition result of the target object based on the target attribute feature; the attribute query network is obtained by improving the target detection framework DETR so that the object query of the DETR is modified to the attribute query corresponding to the attribute; the attribute query is realized by changing the object of the target query from the target to the attribute; The target attribute features are obtained by the following steps: performing feature extraction processing on the acquired target image to obtain the target object graph features; performing attribute extraction processing on the target object graph features to obtain the attribute feature information; obtaining the target attribute features according to the target object graph features and the attribute feature information; the decoder includes a multi-head self-attention layer and a multi-head cross-attention layer; obtaining the target attribute features according to the target object graph features and the attribute feature information includes: performing self-attention enhancement on the attribute feature information through the multi-head self-attention layer to obtain inter-attribute self-attention features; performing cross-attention feature extraction on the target object graph features and the inter-attribute self-attention features through the multi-head cross-attention layer to obtain the target attribute features; the object attribute recognition result of the target object is obtained by performing attribute classification through each attribute classification head of the attribute prediction sub-model.

2. The method according to claim 1, characterized in that: The step of obtaining a target image according to the image to be processed containing the target object comprises: Performing object detection on the image to be processed to identify the target object contained in the image to be processed; The image to be processed is cropped based on the target object to obtain the target image.

3. The method according to claim 1, characterized in that The image feature extraction branch includes a backbone network and an encoder module; the feature extraction process is performed on the acquired target image to obtain the target object graph feature, including: Extracting image features from the target image through the backbone network to obtain feature map information; The encoder module performs object feature extraction on the feature map information to obtain the target object map features.

4. The method according to claim 1, characterized in that The determining the object attribute recognition result of the target object based on the target attribute feature includes: The target attribute feature is input into the attribute prediction sub-model to obtain the object attribute recognition result of the target object; the attribute prediction sub-model includes a feed-forward neural network FFN and a classifier softmax.

5. The method according to any one of claims 1 to 4, characterized in that: The attribute recognition model is trained by the following method: Obtaining training sample data, wherein the training sample data includes object annotated images with the preset number of attribute information labels; Based on the training sample data, a batch of training images is selected; the batch of training images includes a part of the object annotated image; Inputting the batch training images into the attribute recognition model to be trained for training, and determining the attribute recognition loss value of the target loss function of the attribute recognition model to be trained; the target loss function is used to determine the difference between the object attribute recognition result determined by the target attribute feature extracted based on the object annotated image in the batch training images and the attribute information label of the object annotated image in the batch training images; Determine whether the attribute recognition loss value converges to a preset target value. If the judgment result is no, adjust the parameters of the attribute recognition model to be trained according to the determined attribute recognition loss value and continue training until the judgment result is yes, thereby obtaining a trained attribute recognition model.

6. An object attribute recognition device, characterized in that: The device comprises: An image processing module, used to obtain a target image according to an image to be processed containing the target object; the target image contains the target object; An attribute determination module is used to input the target image into a trained attribute recognition model to obtain an object attribute recognition result of the target object; wherein the attribute recognition model includes a deep feature extraction network submodel and an attribute prediction submodel; the deep feature extraction network submodel is used to extract a preset number of target attribute features; the deep feature extraction network submodel includes an image feature extraction branch and an attribute feature extraction branch; the deep feature extraction network submodel is a network model based on a Transformer architecture; the attribute feature extraction branch includes a preset number of attribute query networks and a decoder module connected in series with the preset number of attribute query networks; the image feature extraction branch is used to obtain a target object graph feature based on the target image; the attribute query network is used to obtain attribute feature information based on the target object graph feature; the decoder module is used to obtain the target attribute feature based on the target object graph feature and the attribute feature information; the attribute prediction submodel is used to determine the object attribute recognition result of the target object based on the target attribute feature; the attribute query network is obtained by improving the target detection framework DETR so that the object query of the DETR is modified to an attribute query corresponding to an attribute; the attribute query is realized by changing the object of the target query from a target to an attribute; The target attribute features are obtained by the following steps: performing feature extraction processing on the acquired target image to obtain the target object graph features; performing attribute extraction processing on the target object graph features to obtain the attribute feature information; obtaining the target attribute features according to the target object graph features and the attribute feature information; the decoder includes a multi-head self-attention layer and a multi-head cross-attention layer; obtaining the target attribute features according to the target object graph features and the attribute feature information includes: performing self-attention enhancement on the attribute feature information through the multi-head self-attention layer to obtain inter-attribute self-attention features; performing cross-attention feature extraction on the target object graph features and the inter-attribute self-attention features through the multi-head cross-attention layer to obtain the target attribute features; the object attribute recognition result of the target object is obtained by performing attribute classification through each attribute classification head of the attribute prediction sub-model.

7. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5 is implemented.

Images