Target object recognition method, device and electronic system

Abstract

The present invention provides a method, device and electronic system for identifying a target object, and relates to the field of image processing technology. The method comprises: obtaining a set of detection frames of the same target object in a sequence of video frames to be identified and an image quality score of each detection frame in the set of detection frames; selecting a first detection frame and a second detection frame from the set of detection frames based on the image quality score of each detection frame; determining a second retrieval feature vector of the second detection frame based on a first retrieval feature vector of the first detection frame, a first tracking feature vector of the first detection frame and a second tracking feature vector of the second detection frame; fusing the first retrieval feature vector and the second retrieval feature vector to obtain an optimal feature vector of the same target object; determining the recognition result of the same target object based on the optimal feature vector and a pre-stored target object base library. The present invention alleviates the problem of poor generalization ability of features extracted by a CNN network, and improves the retrieval accuracy of target objects.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a target object recognition method, device and electronic system. Background Art

[0002] Pedestrian Re-identification (ReID) technology can be applied to video structuring. This technology is widely used in security, intelligent transportation and other fields. The existing pedestrian ReID technology usually uses the following method to search for pedestrians: the detection model is used to detect the location of pedestrians, and then the image of the pedestrian detection area is sent to the CNN network (i.e., convolutional neural network) to extract the feature vector of the pedestrian to be searched, and the feature vector of the pedestrian to be searched is compared with the feature vector of the pedestrian image in the search library, and other vectors similar to the pedestrian vector to be searched are selected and sorted according to the similarity to obtain the pedestrian search results.

[0003] Since the same pedestrian picture has different shooting angles, lighting conditions, driving postures, etc., these uncertain factors increase the difficulty of CNN network to extract image features, making the features extracted by CNN network lack generalization ability, which in turn affects the subsequent retrieval ranking results and the retrieval accuracy is low. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a target object recognition method, device and electronic system to alleviate the problem of poor generalization ability of features extracted by CNN network and improve the retrieval accuracy of target objects.

[0005] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a target object recognition method, which is applied to an electronic device, including: obtaining a detection frame set of the same target object in a video frame sequence to be recognized and an image quality score of each detection frame in the detection frame set; selecting a first detection frame and a second detection frame from the detection frame set based on the image quality score of each detection frame; determining a second retrieval feature vector of the second detection frame based on a first retrieval feature vector of the first detection frame, a first tracking feature vector of the first detection frame, and a second tracking feature vector of the second detection frame; wherein the dimension of the first retrieval feature vector is greater than the first tracking feature vector; the dimension of the second retrieval feature vector is greater than the second tracking feature vector; fusing the first retrieval feature vector and the second retrieval feature vector to obtain an optimal feature vector for the same target object; and determining a recognition result of the same target object based on the optimal feature vector and a pre-stored target object database.

[0007] In combination with the first aspect, an embodiment of the present invention provides a first possible implementation of the first aspect, wherein the step of obtaining a set of detection frames of the same target object in a video frame sequence to be identified and an image quality score of each detection frame in the detection frame set includes: performing target object detection on the video frames in the video frame sequence to be identified to obtain detection frames containing the target objects; performing image quality scoring and tracking feature vector extraction on each detection frame to obtain a tracking feature vector and an image quality score for each detection frame; and determining a set of detection frames of the same target object based on the tracking feature vector of each detection frame.

[0008] In combination with the first aspect, an embodiment of the present invention provides a second possible implementation of the first aspect, wherein the steps of performing image quality scoring and tracking feature vector extraction for each detection frame include: inputting each detection frame into a quality scoring model and a first convolutional neural network model respectively to obtain an image quality score and a tracking feature vector for each detection frame; wherein the quality scoring model is used to perform image quality scoring on the input detection frame; and the first convolutional neural network model is used to extract tracking feature vectors for the input detection frame.

[0009] In combination with the first aspect, an embodiment of the present invention provides a third possible implementation of the first aspect, wherein the step of selecting a first detection frame and a second detection frame from a detection frame set based on the image quality score of each detection frame includes: sorting the detection frames in the detection frame set based on the size of the image quality score of each detection frame; selecting the first detection frame and the second detection frame based on the sorting, wherein the image quality score of the first detection frame and the image quality score of the second detection frame are both greater than the image quality scores of unselected detection frames.

[0010] In combination with the first aspect, an embodiment of the present invention provides a fourth possible implementation of the first aspect, wherein, after the step of selecting a first detection box and a second detection box from a detection box set, the method further includes: inputting the first detection box into a second convolutional neural network model to obtain a first retrieval feature vector of the first detection box; wherein the second convolutional neural network model is used to extract a retrieval feature vector for the input detection box.

[0011] In combination with the first aspect, an embodiment of the present invention provides a fifth possible implementation of the first aspect, wherein the step of determining the second retrieval feature vector of the second detection frame based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame includes: inputting the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame into a pre-trained feature upgrading network model to obtain the second retrieval feature vector of the second detection frame; wherein the feature upgrading network model is obtained by training a convolutional neural network model based on image samples annotated with object labels.

[0012] In combination with the first aspect, an embodiment of the present invention provides a sixth possible implementation of the first aspect, wherein the training process of the feature upgrading network model includes: inputting a set of image samples marked with object labels into a third convolutional neural network model and a fourth convolutional neural network model respectively, and obtaining a tracking feature vector and a retrieval feature vector corresponding to each object label; wherein the third convolutional neural network model is used to extract a tracking feature vector for the input image sample, and the fourth convolutional neural network model is used to extract a retrieval feature vector for the input image sample; selecting multiple image sample pairs of two objects based on the object labels, wherein each image sample pair contains two different objects, and for each image sample pair, performing the following training operations: inputting the retrieval feature vector, the tracking feature vector of the first image sample and the tracking feature vector of the second image sample in each image sample pair into a preset convolutional neural network model to obtain an output vector; using the retrieval feature vector of the second image sample as a standard vector, and calculating the loss value between the output vector of the convolutional neural network model and the standard vector; adjusting the parameters of the convolutional neural network model based on the loss value, and continuing to perform the training operation until the loss value converges to a preset threshold, and using the trained convolutional neural network model as the feature upgrading network model.

[0013] In combination with the first aspect, an embodiment of the present invention provides a seventh possible implementation of the first aspect, wherein the step of fusing the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector of the same target object includes: performing mean fusion on the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector of the same target object.

[0014] In a second aspect, an embodiment of the present invention provides a target object recognition device, which is applied to an electronic device, and includes: an acquisition module, used to obtain a detection frame set of the same target object in a video frame sequence to be identified and an image quality score of each detection frame in the detection frame set; a detection frame selection module, used to select a first detection frame and a second detection frame from the detection frame set based on the image quality score of each detection frame; a vector determination module, used to determine a second retrieval feature vector of the second detection frame based on a first retrieval feature vector of the first detection frame, a first tracking feature vector of the first detection frame, and a second tracking feature vector of the second detection frame; wherein the dimension of the first retrieval feature vector is greater than the first tracking feature vector; the dimension of the second retrieval feature vector is greater than the second tracking feature vector; a feature fusion module, used to fuse the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector of the same target object; and an identification module, used to determine the recognition result of the same target object based on the optimal feature vector and a pre-stored target object database.

[0015] In a third aspect, an embodiment of the present invention provides an electronic system, comprising: an image acquisition device, a processing device, and a storage device. The image acquisition device is used to obtain a video frame sequence to be identified; the storage device stores a computer program, and the computer program executes the target object identification method described in any of the above embodiments when the processed device is running.

[0016] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processing device, the steps of the target object recognition method described in any of the aforementioned embodiments are executed.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] Embodiments of the present invention provide a method, device and electronic system for identifying a target object. First, a detection frame set of the same target object in a video frame sequence to be identified and an image quality score of each detection frame in the detection frame set are obtained, and a first detection frame and a second detection frame are selected from the detection frame set based on the image quality score of each detection frame, and a second retrieval feature vector of the second detection frame is determined based on a first retrieval feature vector of the first detection frame, a first tracking feature vector of the first detection frame and a second tracking feature vector of the second detection frame (the dimension of the first retrieval feature vector is larger than that of the first tracking feature vector, and the dimension of the second retrieval feature vector is larger than that of the second tracking feature vector), and then the first retrieval feature vector and the second retrieval feature vector are fused to obtain an optimal feature vector of the target object, and then the recognition result of the same target object is determined based on the optimal feature vector and a pre-stored target object database. In the above recognition method, two detection frames can be selected based on the image quality score, and the retrieval feature vector of the other detection frame can be determined based on the low-dimensional tracking feature vectors of the two detection frames and the high-dimensional retrieval feature vector of one of the detection frames, without directly obtaining the retrieval feature vectors for both detection frames, thereby reducing the process of calculating the high-dimensional retrieval feature vector. The optimal feature vector is obtained by fusing the two high-dimensional retrieval feature vectors, which is more robust and has more representation capabilities. The result of recognition based on the optimal feature vector is more accurate than the recognition result obtained by only using one retrieval feature vector.

[0019] Other features and advantages of the embodiments of the present invention will be described in the following description, or some features and advantages can be inferred or unambiguously determined from the description, or can be learned by implementing the above-mentioned technology of the present disclosure.

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of the structure of an electronic system provided in Embodiment 1 of the present invention;

[0023] Figure 2 A flowchart of a target object recognition method provided in Embodiment 2 of the present invention;

[0024] Figure 3 A flowchart of a target object recognition method provided in Embodiment 3 of the present invention;

[0025] Figure 4 A flowchart of a target object recognition method provided in Embodiment 4 of the present invention;

[0026] Figure 5 A schematic diagram of the overall architecture of a network model provided in Embodiment 4 of the present invention;

[0027] Figure 6 A schematic diagram of a target object recognition method provided in Embodiment 4 of the present invention;

[0028] Figure 7 A schematic diagram of a training process of a feature upgrading network model provided in Embodiment 4 of the present invention;

[0029] Figure 8 A schematic diagram of the structure of a target object recognition device provided in Embodiment 5 of the present invention;

[0030] Fig. 9 A schematic diagram of the structure of another target object recognition device provided in Embodiment 5 of the present invention;

[0031] Fig.10 A schematic diagram of the structure of another target object recognition device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The existing pedestrian ReID technology currently lacks generalization ability in the features extracted by the CNN network during the pedestrian retrieval process, which in turn affects the subsequent retrieval ranking results and has a low retrieval accuracy. Based on this, a target object recognition method, device and system provided in an embodiment of the present invention can be applied to the ReID technology of pedestrians or other objects to improve the retrieval accuracy of the target object.

[0034] To facilitate understanding of this embodiment, a target object recognition method disclosed in an embodiment of the present invention is first introduced in detail.

[0035] Embodiment 1

[0036] First, refer to Figure 1The electronic system 100 is a schematic diagram of the structure of the electronic system 100. The electronic system can be used to implement a target object recognition method and device according to an embodiment of the present invention.

[0037] like Figure 1 The electronic system 100 includes one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and one or more image acquisition devices 110. These components are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1 The components and structure of the electronic system 100 shown are merely exemplary and not limiting. The electronic system may also have other components and structures as required.

[0038] The processing device 102 can be a server, an intelligent terminal, or a device including a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. It can process data of other components in the electronic system 100 and can also control other components in the electronic system 100 to perform target object statistics functions.

[0039] The storage device 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processing device 102 may run the program instructions to implement the client functions and / or other desired functions in the embodiments of the present invention (implemented by the processing device) described below. Various applications and various data, such as various data used and / or generated by the application, may also be stored in the computer-readable storage medium.

[0040] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

[0041] The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

[0042] The image acquisition device 110 can acquire images or video frame sequences to be detected, and store the acquired images or video frame sequences in the storage device 104 for use by other components.

[0043] Exemplarily, for implementing a target object recognition method according to an embodiment of the present invention, the devices and components in the electronic system can be integrated or dispersed, such as integrating the processing device 102, the storage device 104, the input device 106, and the output device 108 into one body, and setting the image acquisition device 110 at a specified position where an image can be acquired. When the components in the above electronic system are integrated, the electronic system can be implemented as a smart terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle-mounted terminal, etc.

[0044] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0045] Embodiment 2

[0046] Reference Figure 2 The flowchart of a target object recognition method shown in FIG. 1 is applicable to the above electronic system and mainly includes the following steps S202 to S210:

[0047] Step S202: obtaining a set of detection frames of the same target object in the video frame sequence to be identified and an image quality score of each detection frame in the set of detection frames.

[0048] The above video frame sequence can be a video received from an image acquisition device, or a video obtained from a network or a third party. The target object can be a pedestrian or a specified part of the human body (such as a face), or a vehicle or a specified part of a vehicle (such as a license plate), etc. The target object can be flexibly set according to actual application needs, and the embodiment of the present invention does not limit this.

[0049] The detection frame set of each target object is a set of regions containing the target object in the images corresponding to each video frame in the video frame sequence, and these regions are represented in the form of detection frames. Taking the detection frame as a rectangular frame as an example, the detection frame can be represented by the coordinates of the upper left corner and the lower right corner of the rectangular frame.

[0050] The image quality score of each detection frame is used to characterize the image quality of the detection frame. If the image quality is high, the image quality score of the detection frame is high. On the contrary, if the image quality is low, the image quality score of the detection frame is low. The image quality is usually determined by factors such as different shooting angles, lighting conditions, and the posture of the target object. The image quality score of each detection frame can be determined by a pre-trained quality scoring model.

[0051] Step S204: selecting a first detection frame and a second detection frame from the detection frame set based on the image quality score of each detection frame.

[0052] Among them, the above-mentioned first detection frame and second detection frame can be the two detection frames with the highest image quality scores in the detection frame set, or they can be two detection frames randomly selected from multiple detection frames in the detection frame set whose image quality scores are greater than a pre-set quality threshold.

[0053] Step S206, determining a second retrieval feature vector of the second detection frame based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame, wherein the dimension of the first retrieval feature vector is greater than the dimension of the first tracking feature vector; and the dimension of the second retrieval feature vector is greater than the dimension of the second tracking feature vector.

[0054] The first retrieval feature vector of the first detection frame can be obtained through a target detection network model, and the first tracking feature vector of the first detection frame and the second tracking feature vector of the second detection frame can be obtained through a target tracking network model. The target detection network model and the target tracking network model can be pre-trained neural network models.

[0055] The dimension of the above-mentioned retrieval feature vector is larger than that of the tracking feature vector. Generally speaking, the feature vector (feature) used for tracking (mot) is called the tracking feature vector. The tracking feature vector can be understood as a simplified version of the retrieval feature vector (searchfeature). Because tracking (mot) has high real-time requirements, the performance of the obtained tracking feature vector is relatively low, so the calculation speed will be faster and meet the real-time requirements.

[0056] Step S208: fusing the first search feature vector and the second search feature vector to obtain an optimal feature vector for the same target object.

[0057] The above fusion can select a suitable fusion mode based on actual needs, for example, selecting average fusion or weighted fusion of feature vectors.

[0058] Step S210, determining the recognition result of the same target object based on the optimal feature vector and the pre-stored target object database.

[0059] In the above recognition method, two detection frames can be selected based on the image quality score, and the retrieval feature vector of the other detection frame can be determined based on the low-dimensional tracking feature vectors of the two detection frames and the high-dimensional retrieval feature vector of one of the detection frames, without directly obtaining the retrieval feature vectors for both detection frames, thereby reducing the process of calculating the high-dimensional retrieval feature vectors, and the optimal feature vector is obtained by fusing the two high-dimensional retrieval feature vectors, which is more robust and has more representation capabilities. The result of recognition based on the optimal feature vector is more accurate than the recognition result obtained by only using one retrieval feature vector.

[0060] Embodiment 3

[0061] On the basis of the above-mentioned second embodiment, the embodiment of the present invention further provides a method for identifying a target object. In order to improve the recognition efficiency of the target object, the method of this embodiment optimizes the above-mentioned step S202 (i.e., obtaining a set of detection frames of the same target object in the video frame sequence to be identified and the image quality score of each detection frame in the set of detection frames). The step specifically includes: (1) performing target object detection on the video frames in the video frame sequence to be identified to obtain a detection frame containing the target object; (2) performing image quality scoring and tracking feature vector extraction on each detection frame to obtain a tracking feature vector and image quality score for each detection frame; (3) determining a set of detection frames of the same target object based on the tracking feature vector of each detection frame. Through this processing method, multiple detection frames of the same target object can be combined into a detection frame set, providing an accurate data basis for the subsequent recognition of the same target object, thereby ensuring the accuracy of target object recognition.

[0062] See also Figure 3 The flowchart of the target object recognition method shown in FIG. 1 mainly includes the following steps:

[0063] Step S302 : performing target object detection on the video frames in the video frame sequence to be identified, and obtaining a detection frame containing the target object.

[0064] In the process of detecting the target object, the deep learning target method can be used to locate the position information of the pedestrian and obtain the position coordinates of the behavior. Taking the target object as a pedestrian as an example, a video of the pedestrian can be obtained by using a camera or other video acquisition equipment. At this time, the video containing the pedestrian is the video frame sequence to be identified, and the pedestrian in the video is the target object to be identified. Through pedestrian detection, the position coordinates of the area where the pedestrian is located (i.e., the detection frame) included in each video frame can be obtained.

[0065] In step S304, each detection frame is input into a quality scoring model (used to perform image quality scoring on the input detection frame) and a first convolutional neural network model (used to extract tracking feature vectors on the input detection frame) to obtain an image quality score and a tracking feature vector for each detection frame.

[0066] When scoring the image quality of each detection frame, the scoring basis usually includes the image integrity, contrast, brightness, clarity, etc. When the target object is a pedestrian, the image quality score of each detection frame can be obtained by constructing a quality scoring model. The quality scoring model can be trained with different data such as pedestrian clothing, skin, gender, etc., and then each detection frame is input into the quality scoring model, and the model outputs the image quality score corresponding to each detection frame.

[0067] Step S306: determining a set of detection frames for the same target object based on the tracking feature vector of each detection frame.

[0068] Step S308: Select a first detection frame and a second detection frame from the detection frame set based on the image quality score of each detection frame.

[0069] In this embodiment, two detection frames with relatively high image quality scores may be selected from a set of detection frames of the same target object based on the image quality scores as the first detection frame and the second detection frame, respectively.

[0070] Step S310, input the first detection frame into the second convolutional neural network model to obtain a first retrieval feature vector of the first detection frame; wherein the second convolutional neural network model is used to extract the retrieval feature vector of the input detection frame.

[0071] Step S312, determining a second retrieval feature vector of the second detection frame based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame, wherein the dimension of the first retrieval feature vector is greater than the dimension of the first tracking feature vector; and the dimension of the second retrieval feature vector is greater than the dimension of the second tracking feature vector.

[0072] Step S314: fuse the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector for the same target object.

[0073] The above fusion can select a suitable fusion mode based on actual needs, for example, selecting mean fusion or weighted fusion of feature vectors.

[0074] Step S316, determining the recognition result of the same target object based on the optimal feature vector and the pre-stored target object database.

[0075] In the above recognition method, the quality scoring model is combined with the first convolutional neural network model to perform image quality scoring and low-dimensional tracking feature vector extraction on the detection frame. Two detection frames with relatively high image quality scores can be selected from the detection frame set of the same target object based on the image quality score. The retrieval feature vector of the other detection frame is determined based on the low-dimensional tracking feature vectors of the two selected detection frames and the high-dimensional retrieval feature vector of one of the detection frames. The optimal feature vector is obtained by fusing the two high-dimensional retrieval feature vectors. The optimal feature vector has higher robustness and representation ability. The recognition result based on the optimal feature vector is more accurate than the recognition result obtained by only using one retrieval feature vector.

[0076] In addition, since the accuracy of the final pedestrian recognition depends on the quality of the selected detection frame, once the quality of the selected detection frame is poor, the accuracy of the final recognition result will be greatly reduced. Compared with the method of selecting only one detection frame, the above recognition method does not need to calculate high-dimensional retrieval feature vectors in the detection frame selection stage. It can greatly reduce the amount of model calculation while ensuring the quality of the selected detection frame in the detection frame selection stage, thereby improving the work efficiency of the detection frame selection stage. Moreover, the two detection frames selected in the above recognition method will contribute to the final recognition result, reducing the impact of a single detection frame on the recognition result and improving the risk resistance of the recognition process.

[0077] Embodiment 4

[0078] On the basis of the above-mentioned second embodiment, the embodiment of the present invention further provides a method for identifying a target object. The method of this embodiment mainly optimizes the above-mentioned step S204 (i.e., selecting a first detection frame and a second detection frame from a set of detection frames based on the image quality score of each detection frame). Figure 4 The flowchart of the target object recognition method shown in FIG. 1 mainly includes the following steps:

[0079] Step S402 : performing target object detection on the video frames in the video frame sequence to be identified, and obtaining a detection frame containing the target object.

[0080] In step S404, each detection frame is input into a quality scoring model (used to perform image quality scoring on the input detection frame) and a first convolutional neural network model (used to extract tracking feature vectors on the input detection frame) to obtain an image quality score and a tracking feature vector for each detection frame.

[0081] Step S406: determining a set of detection frames for the same target object based on the tracking feature vector of each detection frame.

[0082] Step S408, sorting the detection frames in the detection frame set based on the image quality score of each detection frame, and selecting the first detection frame and the second detection frame based on the sorting, wherein the image quality score of the first detection frame and the image quality score of the second detection frame are both greater than the image quality score of the unselected detection frame.

[0083] In this embodiment, the sorting based on the image quality score may be sorting from large to small image quality scores, or may be sorting from small to large image quality scores. If the sorting is from large to small image quality scores, the two detection frames with the highest sorting are selected as the first detection frame and the second detection frame, respectively. If the sorting is from small to large image quality scores, the two detection frames with the lowest sorting are selected as the first detection frame and the second detection frame, respectively.

[0084] Step S410, input the first detection frame into the second convolutional neural network model to obtain a first retrieval feature vector of the first detection frame; wherein the second convolutional neural network model is used to extract the retrieval feature vector of the input detection frame.

[0085] Step S412, determining a second retrieval feature vector of the second detection frame based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame, wherein the dimension of the first retrieval feature vector is greater than the dimension of the first tracking feature vector; and the dimension of the second retrieval feature vector is greater than the dimension of the second tracking feature vector.

[0086] Step S414: performing mean fusion on the first search feature vector and the second search feature vector to obtain an optimal feature vector for the same target object.

[0087] The above fusion can select a suitable fusion method based on actual needs. In addition to mean fusion, it can also be weighted fusion.

[0088] Step S416, determining the recognition result of the same target object based on the optimal feature vector and the pre-stored target object database.

[0089] In the above recognition method, the quality scoring model is combined with the first convolutional neural network model to perform image quality scoring and low-dimensional tracking feature vector extraction on the detection frame. Based on the image quality score, two detection frames with the highest image quality scores can be selected from the detection frame set of the same target object. Since the accuracy of the final target object recognition depends on the quality of the selected detection frame, once the quality of the selected detection frame is poor, the accuracy of the final recognition result will be greatly reduced. Compared with the method of selecting only one detection frame, the method of selecting the detection frame with the best image quality can effectively ensure that the quality of the detection frame used for feature recognition is the best. Based on the low-dimensional tracking feature vectors of the two selected detection frames and the high-dimensional retrieval feature vector of one of the detection frames, the retrieval feature vector of the other detection frame is determined. The optimal feature vector is obtained by fusing the two high-dimensional retrieval feature vectors, which is more robust and has more representation capabilities. The result of recognition based on the optimal feature vector is more accurate than the recognition result obtained by only one retrieval feature vector. In addition, the above recognition method does not need to calculate high-dimensional retrieval feature vectors in the detection frame selection stage. It can greatly reduce the model calculation amount while ensuring the quality of the selected detection frame in the detection frame selection stage, thereby improving the work efficiency of the detection frame selection stage. Moreover, the two detection frames selected in the above recognition method will contribute to the final recognition result, reducing the influence of a single detection frame on the recognition result and improving the risk resistance of the recognition process.

[0090] In order to further improve the accuracy of the target object recognition results, the method of this embodiment specifically includes the following steps for the above step S206 (determining the second retrieval feature vector of the second detection frame based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame. Wherein, the dimension of the first retrieval feature vector is greater than the first tracking feature vector; the dimension of the second retrieval feature vector is greater than the second tracking feature vector): inputting the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame into a pre-trained feature upgrade network model to obtain the second retrieval feature vector of the second detection frame. Wherein, the feature upgrade network model is obtained by training a convolutional neural network model based on image samples annotated with object labels. By adopting the feature upgrade network model to determine the second retrieval feature vector of the second detection frame, it can be ensured that the determined second retrieval feature vector of the second detection frame is more accurate and more efficient.

[0091] See also Figure 5 The overall architecture diagram of the network model shown in FIG. 1 is a schematic diagram of the overall architecture of the network model, wherein the network model is used to perform the above-mentioned target object recognition method, and includes: a target object detection model, a quality scoring model, a first convolutional neural network model, a tracking model, a graph inference model, a second convolutional neural network model, a feature upgrade network model, a fusion model and a target object matching model. Figure 6 A schematic diagram of a target object recognition method is shown in FIG. Figure 5 The method is implemented based on the model of , which can be applied to pedestrian re-identification scenarios. The execution of the method includes the following steps:

[0092] (1) Input the set of video frames to be identified into the target object detection model to obtain each target object detection frame. Each target object detection frame can be obtained by cutting out the image.

[0093] (2) Inputting each target object detection frame into the first convolutional neural network model and the quality scoring model respectively, obtaining the tracking feature vector of each target object detection frame output by the first convolutional neural network model and the image quality score of each target object detection frame output by the quality scoring model.

[0094] (3) The tracking feature vectors of the target object detection frames output by the first convolutional neural network model are input into the tracking model. The tracking model determines the detection frames belonging to the same target object based on the tracking feature vectors of the detection frames of the target objects, and assigns the same object identifier to the detection frames of the same target object, so that the detection frames of different objects can be distinguished from each other.

[0095] (4) The detection frame and image quality score of the same target object are input into the inference model. The inference model selects two target object detection frames based on the image quality score of the pedestrian detection frame of the same target object. Here, the two target object detection frames with the highest image quality scores are selected as the first detection frame and the second detection frame, respectively.

[0096] Of course, the above process can also be used to detect only one target object. The first convolutional neural network model will obtain a set of detection frames of the target object, and the quality scoring model will obtain the image quality score of each detection frame in the set. Then, the detection frames in the above detection frame set are sorted from large to small according to the image quality score, such as Figure 6 As shown, in this embodiment, the target object detection frame set has a total of 8 detection frames, and the scores of each detection frame are 0.4, 0.3, 0.6, 0.9, 0.5, 0.2, 0.8, and 0.1 respectively. After sorting in descending order of quality score, the detection frames with higher scores of 0.9 and 0.8 are the first detection frame and the second detection frame. Figure 6 In the figure, they are represented by grid and diagonal line filling respectively.

[0097] (5) Inputting the first detection frame into the second convolutional neural network model to obtain a first retrieval feature vector of the first detection frame; wherein the second convolutional neural network model is used to extract the retrieval feature vector of the input detection frame.

[0098] (6) Inputting the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame into the feature upgrading network model to obtain the second retrieval feature vector of the second detection frame. The dimension of the first retrieval feature vector is greater than that of the first tracking feature vector; and the dimension of the second retrieval feature vector is greater than that of the second tracking feature vector.

[0099] (7) The first retrieval feature vector and the second retrieval feature vector are input into a fusion model, and the fusion model is used to fuse the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector of the same target object.

[0100] (8) The optimal feature vector of the same target object is input into the target object matching model, and the target object matching model performs a matching operation on the optimal feature vector based on the target object base library to obtain the recognition result of the same target object (this process Figure 6 not shown).

[0101] In summary, the target object recognition method provided by the present embodiment is based on the image quality score of each detection frame, and the detection frames in the detection frame set are sorted in descending order of the image quality score, and the first detection frame and the second detection frame whose image quality scores are sorted as first and second are selected from the detection frame set, and the second retrieval feature vector of the second detection frame is determined based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame, and the first retrieval feature vector and the second retrieval feature vector are fused to obtain the optimal feature vector of the same target object, and the recognition result of the same target object is determined based on the optimal feature vector and the target object base library. When performing target object recognition, the embodiment of the present invention performs feature fusion on the image features of two high-quality detection frames, so that the image features of the target object finally extracted are more robust and representative. While integrating the image features of multiple detection frames of the same target object, only the high-dimensional retrieval feature vector of one detection frame needs to be calculated, and the amount of calculation is small. Compared with the feature extraction method that only extracts the image features of a single detection frame of the same target object, the present invention alleviates the problem of poor generalization ability of the existing CNN feature extraction, and takes into account the improvement of the recognition efficiency of the target object and the accuracy of the recognition result.

[0102] The training process of the above feature upgrade network model can refer to the following steps a1 to a2:

[0103] Step a1: Input the set of image samples marked with object labels into the third convolutional neural network model (used for extracting tracking feature vectors for the input image samples) and the fourth convolutional neural network model (used for extracting retrieval feature vectors for the input image samples) respectively to obtain the tracking feature vector and retrieval feature vector corresponding to each object label.

[0104] The object label (or object identifier) ​​is a unique label used to characterize the target object. Different target objects are correspondingly provided with different object labels, and the object label of the same target object is the same.

[0105] Step a2, based on the object label, select multiple image sample pairs of two objects, each image sample pair contains two different objects, and for each image sample pair, perform the following training operations: (1) input the retrieval feature vector, tracking feature vector of the first image sample and the tracking feature vector of the second image sample in each image sample pair into a preset convolutional neural network model to obtain an output vector; (2) use the retrieval feature vector of the second image sample as the standard vector, and calculate the loss value between the output vector of the convolutional neural network model and the standard vector; (3) adjust the parameters of the convolutional neural network model based on the loss value, continue to perform the training operation, until the loss value converges to a preset threshold, and use the trained convolutional neural network model as the feature upgrade network model.

[0106] The following is an example of a pedestrian re-identification application scenario. The pedestrian in the pedestrian re-identification scenario is the target object to be identified. Figure 7 A schematic diagram of the training process of a feature upgrade network model is shown. For ease of description, in the figure, model A represents the third convolutional neural network model, model B represents the fourth convolutional neural network model, and model C represents the convolutional neural network model to be trained. The training process specifically includes the following steps:

[0107] (1) Obtain a sample set of pictures (or images) labeled with pedestrian labels, pair two pictures of pedestrians with different identities based on the pedestrian labels, input the paired pictures into model A and model B respectively, and extract tracking feature vectors and retrieval feature vectors respectively.

[0108] Figure 7 In the P*K image sample set, P means there are P different pedestrians, and K means each pedestrian has K different images. If the total number of images of the pedestrian is greater than K, K different images are taken. If it is less than K, repeated sampling is performed to take K images. The value of K is generally 4, which can be adjusted according to actual needs.

[0109] Among them, two image samples of different pedestrians are randomly selected from the image sample set. For the convenience of description, these two image samples are respectively recorded as k1 and k2. k1 and k2 are input into model A. Model A extracts low-dimensional tracking feature vectors from the input image samples to obtain tracking feature vectors of k1 and k2. k1 and k2 are input into model B. Model B extracts high-dimensional retrieval feature vectors from the input image samples to obtain retrieval feature vectors of k1 and k2.

[0110] (2) Input the tracking feature vectors of k1 and k2 and the retrieval feature vector of k1 into the convolutional neural network model to be trained to obtain an output vector C;

[0111] (3) Taking the retrieval feature vector of k2 as the standard vector, calculate the loss value between the output vector C of the convolutional neural network model and the standard vector, where the loss value can be calculated based on the following functions: l2 loss (l2 loss function), kdloss (kd loss function), triplet loss (triplet loss function), cosface function or other existing loss functions.

[0112] (4) Based on the above loss value, the parameters of the convolutional neural network model are adjusted, and the training operation is continued until the loss value converges to a preset threshold, and the trained convolutional neural network model is used as the feature upgrade network model.

[0113] The above k1 and k2 are randomly selected. In the training stage, it is assumed that the input image is Zhang San, so feature C is the feature of Zhang San. During the training process, Li Si's retrieval feature vector, Li Si's tracking feature vector and Zhang San's tracking feature vector can be used as input to obtain Zhang San's upgraded feature vector, that is, Zhang San's feature C. The training process makes Zhang San's feature C and Zhang San's retrieval feature vector as similar as possible. The above loss value can be calculated by the l2_loss function of Zhang San's feature C and Zhang San's retrieval feature vector.

[0114] In order to improve the training effect, this embodiment introduces a shuffle operation, which performs a shuffle operation on the images based on the tracking feature vectors and retrieval feature vectors of each image output by models A and B. For example, the image order of the same person is 1234 before, and after the shuffle operation, the image order of the person becomes 4321;

[0115] Specifically, the shuffle function can be called to rearrange all vectors in the same model in a random order to prevent jitter and overfitting of the model during training and ensure the generalization ability of the model. Specifically, the shuffle function is called to synchronously shuffle the feature vectors output by model A and model B. For the convenience of description, the feature vector obtained after shuffling the image samples of the first person is recorded as feature A, and the feature vector obtained after shuffling the image samples of the second person is recorded as feature B.

[0116] Based on the above feature A and feature B, feature vectors of two different objects are randomly selected for subsequent training operations. The specific training process is the same as described above and will not be repeated here.

[0117] Embodiment 5

[0118] For the target object recognition method provided in the second embodiment, the present invention provides a target object recognition device, see Figure 8 The schematic diagram of the structure of a target object recognition device shown in FIG. 1 includes the following modules:

[0119] The acquisition module 82 is used to obtain a set of detection frames of the same target object in the video frame sequence to be identified and an image quality score of each detection frame in the set of detection frames.

[0120] The detection frame selection module 84 is used to select a first detection frame and a second detection frame from the detection frame set based on the image quality score of each detection frame.

[0121] The vector determination module 86 is used to determine the second retrieval feature vector of the second detection frame based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame. The dimension of the first retrieval feature vector is greater than that of the first tracking feature vector, and the dimension of the second retrieval feature vector is greater than that of the second tracking feature vector.

[0122] The feature fusion module 88 is used to fuse the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector for the same target object.

[0123] The recognition module 90 is used to determine the recognition result of the same target object based on the optimal feature vector and the pre-stored target object database.

[0124] The target object recognition device provided in this embodiment can select two detection frames based on the image quality score, and determine the retrieval feature vector of the other detection frame based on the low-dimensional tracking feature vectors of the two detection frames and the high-dimensional retrieval feature vector of one of the detection frames, without directly obtaining the retrieval feature vectors for both detection frames, thereby reducing the process of calculating the high-dimensional retrieval feature vector, and obtaining the optimal feature vector by fusing the two high-dimensional retrieval feature vectors, which is more robust and has more representation capabilities, and the result of recognition based on the optimal feature vector is more accurate than the recognition result obtained by only using one retrieval feature vector.

[0125] The acquisition module 82 is also used for: performing target object detection on the video frames in the video frame sequence to be identified, and obtaining a detection frame containing the target object; performing image quality scoring and tracking feature vector extraction on each of the detection frames, and obtaining a tracking feature vector and image quality score for each of the detection frames; and determining a set of detection frames for the same target object based on the tracking feature vector of each of the detection frames.

[0126] The acquisition module 82 is also used to: input each of the detection frames into the quality scoring model and the first convolutional neural network model respectively, to obtain the image quality score and tracking feature vector of each of the detection frames; wherein the quality scoring model is used to perform image quality scoring on the input detection frames; and the first convolutional neural network model is used to extract tracking feature vectors for the input detection frames.

[0127] The detection frame selection module 84 is also used to: sort the detection frames in the detection frame set based on the image quality score of each detection frame; select a first detection frame and a second detection frame based on the sorting, wherein the image quality score of the first detection frame and the image quality score of the second detection frame are both greater than the image quality score of the unselected detection frame.

[0128] In the above Figure 8Based on this, this embodiment also provides another target object recognition device, see Fig. 9 The above-mentioned device also includes: a vector extraction module 92, which is used to input the first detection frame into the second convolutional neural network model to obtain a first retrieval feature vector of the first detection frame; wherein the second convolutional neural network model is used to extract the retrieval feature vector of the input detection frame.

[0129] The vector determination module 86 is also used to: input the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame into a pre-trained feature upgrade network model to obtain the second retrieval feature vector of the second detection frame; wherein the feature upgrade network model is obtained by training a convolutional neural network model based on image samples marked with object labels.

[0130] See also Fig. 9 The above-mentioned device also includes a model training module 94, which is used to train the feature upgrade network model, and its training process includes: inputting the image sample set marked with the object label into the third convolutional neural network model and the fourth convolutional neural network model respectively, and obtaining the tracking feature vector and the retrieval feature vector corresponding to each of the object labels; wherein the third convolutional neural network model is used to extract the tracking feature vector of the input image sample, and the fourth convolutional neural network model is used to extract the retrieval feature vector of the input image sample; based on the object label, multiple image sample pairs of two objects are selected, wherein each image sample pair contains two different objects, and for each of the image sample pairs, the following training operation is performed: the retrieval feature vector, the tracking feature vector of the first image sample and the tracking feature vector of the second image sample in each of the image sample pairs are input into the preset convolutional neural network model to obtain an output vector; using the retrieval feature vector of the second image sample as the standard vector, calculating the loss value between the output vector of the convolutional neural network model and the standard vector; adjusting the parameters of the convolutional neural network model based on the loss value, and continuing to perform the training operation until the loss value converges to a preset threshold, and the trained convolutional neural network model is used as the feature upgrade network model.

[0131] The feature fusion module 88 is further used to perform mean fusion on the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector of the same target object.

[0132] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0133] Embodiment 6

[0134] See also Fig.10 The present invention provides a structural diagram of a target object recognition device. The embodiment of the present invention also provides another target object recognition device 200, including: a processor 10, a memory 11, a bus 12 and a communication interface 13, the processor 10, the communication interface 13 and the memory 11 are connected through the bus 12; the processor 10 is used to execute an executable module stored in the memory 11, such as a computer program.

[0135] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 13 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0136] The bus 12 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Fig.10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0137] Among them, the memory 11 is used to store programs, and the processor 10 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any of the embodiments of the above-mentioned embodiments of the present invention can be applied to the processor 10 or implemented by the processor 10.

[0138] The processor 10 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 10. The above processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 11, and the processor 10 reads the information in the memory 11 and completes the steps of the above method in combination with its hardware.

[0139] Unless otherwise specifically stated, the relative steps, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be interpreted as indicating or suggesting relative importance.

[0140] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] A computer program product of a target object identification method, device, and electronic system provided in an embodiment of the present invention includes a computer-readable storage medium storing a non-volatile program code executable by a processor. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0142] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In addition, each functional unit in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0143] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0144] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A target object recognition method, characterized in that: The method is applied to an electronic device, comprising: Obtain a set of detection frames of the same target object in a video frame sequence to be identified and an image quality score of each detection frame in the set of detection frames; Selecting a first detection frame and a second detection frame from the set of detection frames based on the image quality score of each of the detection frames; Determine a second retrieval feature vector of the second detection frame based on a first retrieval feature vector of the first detection frame, a first tracking feature vector of the first detection frame, and a second tracking feature vector of the second detection frame; wherein the dimension of the first retrieval feature vector is greater than that of the first tracking feature vector; and the dimension of the second retrieval feature vector is greater than that of the second tracking feature vector; fusing the first retrieval feature vector and the second retrieval feature vector to obtain an optimal feature vector for the same target object; The recognition result of the same target object is determined based on the optimal feature vector and a pre-stored target object database.

2. The method according to claim 1, characterized in that: The step of obtaining a set of detection frames of the same target object in a video frame sequence to be identified and an image quality score of each detection frame in the set of detection frames comprises: Performing target object detection on the video frames in the video frame sequence to be identified to obtain a detection frame containing the target object; Performing image quality scoring and tracking feature vector extraction on each of the detection frames to obtain a tracking feature vector and an image quality score for each of the detection frames; A set of detection frames for the same target object is determined based on the tracking feature vector of each detection frame.

3. The method according to claim 2, characterized in that The steps of scoring the image quality and extracting the tracking feature vector for each detection frame include: Each of the detection frames is input into the quality scoring model and the first convolutional neural network model respectively to obtain an image quality score and a tracking feature vector of each detection frame; wherein the quality scoring model is used to perform image quality scoring on the input detection frame; and the first convolutional neural network model is used to extract tracking feature vectors for the input detection frame.

4. The method according to claim 1, characterized in that: The step of selecting a first detection frame and a second detection frame from the detection frame set based on the image quality score of each detection frame comprises: sorting the detection frames in the detection frame set based on the image quality score of each detection frame; A first detection frame and a second detection frame are selected based on the ranking, wherein the image quality score of the first detection frame and the image quality score of the second detection frame are both greater than the image quality score of the unselected detection frame.

5. The method according to claim 1, characterized in that After the step of selecting the first detection box and the second detection box from the detection box set, the method further includes: The first detection frame is input into a second convolutional neural network model to obtain a first retrieval feature vector of the first detection frame; wherein the second convolutional neural network model is used to extract a retrieval feature vector for the input detection frame.

6. The method according to claim 1, characterized in that The step of determining a second retrieval feature vector of the second detection frame based on the first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame comprises: The first retrieval feature vector of the first detection frame, the first tracking feature vector of the first detection frame, and the second tracking feature vector of the second detection frame are input into a pre-trained feature upgrading network model to obtain a second retrieval feature vector of the second detection frame; wherein the feature upgrading network model is obtained by training a convolutional neural network model based on image samples marked with object labels.

7. The method according to claim 6, characterized in that The training process of the feature upgrading network model includes: Inputting the image sample sets marked with object labels into the third convolutional neural network model and the fourth convolutional neural network model respectively, to obtain the tracking feature vector and the retrieval feature vector corresponding to each of the object labels; wherein the third convolutional neural network model is used to extract the tracking feature vector of the input image samples, and the fourth convolutional neural network model is used to extract the retrieval feature vector of the input image samples; A plurality of image sample pairs of two objects are selected based on the object labels, wherein each image sample pair includes two different objects, and for each of the image sample pairs, the following training operations are performed: Inputting the retrieval feature vector, the tracking feature vector of the first image sample and the tracking feature vector of the second image sample in each of the image sample pairs into a preset convolutional neural network model to obtain an output vector; Taking the retrieved feature vector of the second image sample as a standard vector, calculating a loss value between an output vector of the convolutional neural network model and the standard vector; The parameters of the convolutional neural network model are adjusted based on the loss value, and the training operation is continued until the loss value converges to a preset threshold, and the trained convolutional neural network model is used as a feature upgrade network model.

8. The method according to claim 1, characterized in that The step of fusing the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector of the same target object comprises: The first retrieval feature vector and the second retrieval feature vector are mean-fused to obtain the optimal feature vector of the same target object.

9. A target object recognition device, characterized in that: The device is applied to electronic equipment, including: An acquisition module is used to acquire a set of detection frames of the same target object in a video frame sequence to be identified and an image quality score of each detection frame in the set of detection frames; a detection frame selection module, configured to select a first detection frame and a second detection frame from the detection frame set based on an image quality score of each detection frame; a vector determination module, configured to determine a second retrieval feature vector of the second detection frame based on a first retrieval feature vector of the first detection frame, a first tracking feature vector of the first detection frame, and a second tracking feature vector of the second detection frame; wherein the dimension of the first retrieval feature vector is greater than that of the first tracking feature vector; and the dimension of the second retrieval feature vector is greater than that of the second tracking feature vector; A feature fusion module, used for fusing the first retrieval feature vector and the second retrieval feature vector to obtain the optimal feature vector of the same target object; The recognition module is used to determine the recognition result of the same target object based on the optimal feature vector and a pre-stored target object database.

10. An electronic system, characterized in that: The electronic system includes: an image acquisition device, a processing device and a storage device; The image acquisition device is used to obtain a video frame sequence to be identified; The storage device stores a computer program, which, when executed by the processing device, executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 8 are executed.

Images