Target object identification method and device, electronic device and storage medium

Abstract

The invention relates to a target object recognition method and device, an electronic device and a storage medium, and the method comprises the steps: obtaining a historical video image containing a historical target object, determining a first segmentation mask corresponding to the historical target object, and further determining the features of the historical target object, determining a mask feature queue according to historical target object features and a first transformation network layer, obtaining a to-be-recognized video image containing a to-be-recognized target object, determining a second segmentation mask corresponding to the to-be-recognized target object, further determining to-be-recognized target object features, and obtaining a to-be-recognized video image containing the to-be-recognized target object; and determining a high-quality mask according to the mask feature queue, the to-be-recognized target object feature and the second transformation network layer, and determining a time sequence gait feature according to the high-quality mask and the gait recognition network so as to determine a key target object. The problem that the extracted gait features are incomplete when gait recognition is carried out on the target object in the prior art is solved, and the accuracy of target object recognition is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a target object recognition method, device, electronic device and storage medium. Background Art

[0002] With the acceleration of urbanization, the urban population density continues to increase, and urban management and public safety issues are becoming increasingly prominent, requiring further empowerment of the security monitoring industry. Among them, how to quickly and accurately identify and find key pedestrians in complex street environments has become an important topic in the field of urban management and public safety. Traditional monitoring methods mainly rely on human tracking, face recognition and other methods, but these methods have many limitations in practical applications.

[0003] For example, common face recognition, due to the limited viewing angle and clarity of surveillance cameras, especially in long-distance and low-light environments, the accuracy of face recognition is greatly reduced, and face recognition is difficult to perform in non-cooperative scenarios, and it is difficult to identify pedestrians when they are unconscious or uncooperative. Secondly, the target tracking method that relies solely on motion trajectory and appearance features is difficult to maintain effective recognition of specific key people in complex scenes with many obstructions or multiple pedestrians.

[0004] Therefore, gait, as a unique and difficult-to-disguise biometric feature, still has a high degree of recognition at long distances, low resolution and low light conditions, and has gradually become a hot topic of research. In recent years, with the rapid development of computer vision and deep learning technology, pedestrian recognition technology based on gait recognition has made significant progress. Gait is a biometric feature that presents an individual's unique walking pattern. Compared with other biometric methods such as face, fingerprint and iris, gait is difficult to disguise and can be easily captured at a certain distance in a non-invasive manner. Based on the above advantages, gait recognition is highly feasible in security applications. However, most of the existing gait recognition methods are based on an ideal laboratory environment and are difficult to directly apply to complex street scenes with serious interference from occluders and multiple pedestrians. In particular, in actual street monitoring scenes, pedestrians' body postures and contours are easily obscured (such as being partially obscured by other pedestrians, trees, vehicles, etc.), or some parts are integrated with the background, resulting in the extracted pedestrian mask being often incomplete. The lack of mask results in the inability to form a complete gait sequence of pedestrians in continuous moments, which further affects the gait recognition model's extraction of gait temporal features, resulting in a significant decrease in the recognition effect.

[0005] Currently, no effective solution has been proposed for the problem of incomplete gait features extracted when performing gait recognition on a target object in related technologies. Summary of the invention

[0006] The embodiments of the present application provide a target object recognition method, device, electronic device and storage medium to at least solve the problem of incomplete gait features extracted when performing gait recognition on a target object in the related art.

[0007] In a first aspect, an embodiment of the present application provides a method for identifying a target object.

[0008] In some embodiments, the target object recognition method comprises the following steps: Acquire a historical video image containing a historical target object, determine a first segmentation mask corresponding to the historical target object, and determine features of the historical target object according to the historical video image and the first segmentation mask; Determine a mask feature queue corresponding to the historical target object according to the historical target object feature and the first transformation network layer; Acquire a video image to be identified that includes a target object to be identified, determine a second segmentation mask corresponding to the target object to be identified, and determine features of the target object to be identified based on the video image to be identified and the second segmentation mask, wherein the object identifier of the historical target object is consistent with the object identifier of the target object to be identified; Determine a high-quality mask corresponding to the target object to be identified according to the mask feature queue, the feature of the target object to be identified and the second transformation network layer; According to the high-quality mask and the gait recognition network, the temporal gait features corresponding to the target object to be identified are determined, and the key target object is determined according to the temporal gait features.

[0009] In some embodiments, the first transformation network layer includes a mask key transformation network layer and a mask value transformation network layer, and determining the mask feature queue corresponding to the historical target object according to the historical target object feature and the first transformation network layer includes: Inputting the historical target object features into the mask key transformation network layer to determine mask key features; Inputting the historical target object features into the mask value transformation network layer to determine mask value features; A mask feature queue corresponding to the historical target object is determined according to the mask key feature and the mask value feature.

[0010] In some embodiments, the second transformation network layer includes a query key transformation network layer and a query value transformation network layer, and determining the high-quality mask corresponding to the target object to be identified according to the mask feature queue, the feature of the target object to be identified and the second transformation network layer includes: Inputting the target object feature to be identified into the query key transformation network layer to determine the query key feature; Inputting the target object feature to be identified into the query value transformation network layer to determine the query value feature; The high-quality mask is determined according to the mask key feature, the mask value feature, the query key feature, and the query value feature.

[0011] In some embodiments, determining the high-quality mask according to the mask key feature, the mask value feature, the query key feature, and the query value feature comprises: Determining an attention weight matrix according to the query key feature and the mask key feature; The high-quality mask is determined according to the attention weight matrix, the query value feature and the mask value feature.

[0012] In some embodiments, the method further comprises: Acquire a video image to be tracked, and if the target object contained in the video image to be tracked is less than the target object to be identified, determine the target object missing in the video image to be tracked as a lost target object, acquire a video image to be restored containing the target object to be restored, and determine the gait feature corresponding to the target object to be restored as the gait feature to be restored; Determine that the gait feature corresponding to the disappeared target object is the disappeared target gait feature, and when the similarity between the gait feature to be restored and the gait feature of the disappeared target is greater than or equal to a gait feature similarity threshold, determine the object identifier of the target object to be restored to be consistent with the object identifier of the disappeared target object.

[0013] In some embodiments, a plurality of target objects to be identified are included, and after determining the temporal gait features corresponding to the target objects to be identified according to the high-quality mask and the gait recognition network, and determining the key target objects according to the temporal gait features, the method further includes: Obtaining and determining an object reasoning duration corresponding to the key target object, and determining a non-key target object included in the plurality of target objects to be identified when the object reasoning duration is greater than a reasoning duration threshold; The target objects other than the non-critical target objects in the video image to be identified are determined as current target objects to be identified.

[0014] In some embodiments, after determining the temporal gait features corresponding to the target object to be identified according to the high-quality mask and the gait recognition network, and determining the key target object according to the temporal gait features, the method further includes: Acquire a key target gait feature in a target database, and determine that the target object to be identified is a suspected key target object when the similarity between the time-series gait feature and the key target gait feature is greater than a feature similarity threshold; The high-quality mask corresponding to the suspected key target object is added to the mask feature queue to determine the current mask feature queue.

[0015] In a second aspect, an embodiment of the present application provides a target object recognition device.

[0016] In some embodiments, the target object identification device includes a historical feature determination module, a mask queue determination module, a to-be-identified feature determination module, a high-quality mask determination module, and a key target determination module: The historical feature determination module is used to obtain a historical video image containing a historical target object, determine a first segmentation mask corresponding to the historical target object, and determine the historical target object feature based on the historical video image and the first segmentation mask; The mask queue determination module is used to determine the mask feature queue corresponding to the historical target object according to the historical target object characteristics and the first transformation network layer; The module for determining features to be identified is used to obtain a video image to be identified that contains a target object to be identified, determine a second segmentation mask corresponding to the target object to be identified, and determine features of the target object to be identified based on the video image to be identified and the second segmentation mask, wherein the object identifier of the historical target object is consistent with the object identifier of the target object to be identified; The high-quality mask determination module is used to determine the high-quality mask corresponding to the target object to be identified according to the mask feature queue, the feature of the target object to be identified and the second transformation network layer; The key target determination module is used to determine the temporal gait features corresponding to the target object to be identified based on the high-quality mask and the gait recognition network, and determine the key target object based on the temporal gait features.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the target object recognition method as described in the first aspect above is implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the target object recognition method as described in the first aspect above.

[0019] Compared with the related art, the target object recognition method, device, electronic device and storage medium provided in the embodiments of the present application obtain a historical video image containing a historical target object, determine a first segmentation mask corresponding to the historical target object, and determine the historical target object features according to the historical video image and the first segmentation mask; determine a mask feature queue corresponding to the historical target object according to the historical target object features and the first transformation network layer; obtain a video image to be identified containing a target object to be identified, determine a second segmentation mask corresponding to the target object to be identified, and determine the features of the target object to be identified according to the video image to be identified and the second segmentation mask, wherein the object identification of the historical target object is consistent with the object identification of the target object to be identified; determine a high-quality mask corresponding to the target object to be identified according to the mask feature queue, the features of the target object to be identified and the second transformation network layer; determine the temporal gait features corresponding to the target object to be identified according to the high-quality mask and the gait recognition network, and determine the key target object according to the temporal gait features, thereby solving the problem of incomplete gait features extracted when gait recognition is performed on the target object in the related art, and improving the accuracy of target object recognition.

[0020] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a hardware structure block diagram of a terminal according to a target object identification method of an embodiment of the present application; Figure 2 is a flow chart of a target object identification method according to an embodiment of the present application; Figure 3 is a flow chart of another target object recognition method according to an embodiment of the present application; Figure 4 It is a structural block diagram of a target object recognition device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.

[0023] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0024] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application means greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.

[0025] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of a target object recognition method according to an embodiment of the present invention. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the target object recognition method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0028] This embodiment provides a target object recognition method. Figure 2 is a flow chart of a target object recognition method according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps: Step S201 , obtaining a historical video image containing a historical target object, determining a first segmentation mask corresponding to the historical target object, and determining features of the historical target object according to the historical video image and the first segmentation mask.

[0029] The video image in the embodiment of the present application may be video data containing street scenes, which is collected by surveillance cameras distributed in various blocks and transmitted to the cloud server in real time through the network for processing. Specifically, the algorithm in the detection and segmentation network is used to extract features from each collected video image and then output the detection result, where the pedestrian ID detected in the video image is recorded as , the corresponding detection box is recorded as , the segmented mask result is recorded as .

[0030] Furthermore, the tracking algorithm is used to maintain the ID of the detection frame of the above sequence of pictures based on information such as distance and feature similarity, and a historical video image queue is constructed for each ID to store the sequence of historical video images corresponding to the ID, and the pedestrians therein are determined as historical target objects. The historical video image queue corresponding to the historical target object with ID n is recorded as , determine the first segmentation mask corresponding to the historical target object, and the first segmentation mask queue corresponding to the historical target object with ID n is recorded as . Where m is the length of the historical video image queue, which can be selected as 5. In addition, a detection frame queue can be constructed for each ID to store the most recent detection frame sequence. The detection frame queue of the historical target object with ID n is recorded as .

[0031] The historical video images and the first segmentation mask in the historical video image queue corresponding to each historical target object are respectively input into the feature encoding network of the pedestrian mask recovery module, and after splicing and scaling, the historical target object features corresponding to the historical target object can be obtained.

[0032] Step S202: determining a mask feature queue corresponding to the historical target object according to the historical target object features and the first transformation network layer.

[0033] The feature encoding network of the pedestrian mask recovery module includes a first transformation network layer, into which the historical target object features are input, thereby determining the mask feature queue corresponding to the historical target object, wherein the length of the mask feature queue is consistent with the historical video image queue.

[0034] Step S203, obtaining a video image to be identified that contains a target object to be identified, determining a second segmentation mask corresponding to the target object to be identified, and determining features of the target object to be identified based on the video image to be identified and the second segmentation mask, wherein the object identifier of the historical target object is consistent with the object identifier of the target object to be identified.

[0035] A video image to be identified containing a target object to be identified is obtained, wherein the object identifier of the target object to be identified is consistent with the object identifier of the historical target object, a second segmentation mask corresponding to the target object to be identified is determined, and the video image to be identified and the second segmentation mask are input into a feature decoding network of a pedestrian mask recovery module, and after splicing and scaling, the target object features corresponding to the target object to be identified can be obtained.

[0036] Step S204: determining a high-quality mask corresponding to the target object to be identified according to the mask feature queue, the features of the target object to be identified and the second transformation network layer.

[0037] The feature decoding network of the pedestrian mask recovery module includes a second transformation network layer, which inputs the mask feature queue and the features of the target object to be identified into the second transformation network layer to determine a high-quality mask corresponding to the target object to be identified.

[0038] Step S205, determining the temporal gait features corresponding to the target object to be identified based on the high-quality mask and the gait recognition network, and determining the key target object based on the temporal gait features.

[0039] A high-quality mask sequence is formed based on the high-quality masks corresponding to the multiple frames of video images containing the target object to be identified. When the length of the high-quality mask sequence of the target object to be identified reaches the number of walking frames required for gait recognition, the high-quality mask sequence is input into the gait recognition network to determine the temporal gait features corresponding to the target object to be identified.

[0040] The time-series gait features are matched and compared with the gait feature library of key people, and the similarity corresponding to the time-series gait features is calculated using the Euclidean distance. For the time-series gait features whose similarity reaches the preset gait similarity threshold, the corresponding target object to be identified is judged to be the key target object.

[0041] Through the above steps, the embodiment of the present application obtains a historical video image containing a historical target object, determines a first segmentation mask corresponding to the historical target object, and determines the characteristics of the historical target object based on the historical video image and the first segmentation mask, thereby obtaining a temporal feature based on the characteristics of the historical target object, and determines a mask feature queue containing temporal features corresponding to the historical target object according to the characteristics of the historical target object and the first transformation network layer, and the mask feature queue can perform missing complementation of a local missing mask of the video image to be detected, and further obtains a video image to be identified containing a target object to be identified, determines a second segmentation mask corresponding to the target object to be identified, and determines the characteristics of the target object to be identified according to the video image to be identified and the second segmentation mask, and determines the characteristics of the target object to be identified according to the mask feature. The feature queue, the features of the target object to be identified and the second transformation network layer are used to determine a more complete high-quality mask corresponding to the target object to be identified, so as to generate a more complete target object contour foreground. According to the high-quality mask and gait recognition network, the restored high-quality pedestrian walking mask sequence is input into the gait recognition network to extract gait features, so as to determine the temporal gait features corresponding to the target object to be identified, and the key target object is determined according to the temporal gait features, that is, it is matched and compared with the target database containing the key target gait features to find the key target object, thereby solving the problem of incomplete gait features extracted when performing gait recognition on the target object in the related technology, and significantly improving the accuracy and robustness of gait recognition and target object recognition.

[0042] In some embodiments, the first transformation network layer includes a mask key transformation network layer and a mask value transformation network layer, and step S202 includes: Step S2021, input the historical target object features into the mask key transformation network layer to determine the mask key features.

[0043] Step S2022: input the historical target object features into the mask value transformation network layer to determine the mask value features.

[0044] In the embodiment of the present application, the first transformation network layer includes a mask key transformation network layer and a mask value transformation network layer. Specifically, the mask key transformation network layer can be a four-layer transformation network layer, which includes layer normalization (LM), self-attention mechanism (Self-Attention), layer normalization and multilayer perceptron (MLP) in sequence; the mask value transformation network layer can be a four-layer transformation network layer with a network structure similar to that of the mask key transformation network layer, but its parameter weights are different from those of the mask key transformation network layer.

[0045] The historical target object features are input into the mask key transformation network layer and the mask value transformation network layer respectively to extract deep features, and the feature residual is transferred after the self-attention mechanism and the multi-layer perceptron to determine the mask key features. and mask value features .

[0046] Step S2023, determining the mask feature queue corresponding to the historical target object according to the mask key feature and the mask value feature.

[0047] Will It is stored in the mask feature queue of the pedestrian in its corresponding tracking, that is, the mask feature queue corresponding to the historical target object is determined.

[0048] Through the above steps, the embodiment of the present application provides a specific method for determining the mask feature queue based on the mask key transformation network layer and the mask value transformation network layer, which has high feasibility and high mask extraction accuracy.

[0049] In some embodiments, the second transformation network layer includes a query key transformation network layer and a query value transformation network layer, and step S204 includes: Step S2041: Input the features of the target object to be identified into the query key transformation network layer to determine the query key features.

[0050] Step S2042: input the features of the target object to be identified into the query value transformation network layer to determine the query value features.

[0051] In the embodiment of the present application, the second transformation network layer includes a query key transformation network layer and a query value transformation network layer. Specifically, the query key transformation network layer can be a four-layer transformation network layer, which includes layer normalization (LM), self-attention mechanism (Self-Attention), layer normalization and multilayer perceptron (MLP) in sequence; the query value transformation network layer can be a four-layer transformation network layer with a network structure similar to that of the query key transformation network layer, but its parameter weights are different from those of the query key transformation network layer.

[0052] The features of the target object to be identified are input into the query key transformation network layer and the query value transformation network layer respectively to extract deep features and thus determine the query key features. and mask value features .

[0053] Step S2043, determining a high-quality mask according to the mask key feature, the mask value feature, the query key feature and the query value feature.

[0054] Furthermore, a high-quality mask corresponding to the target object to be identified is determined based on the mask key features, the mask value features, the query key features, and the query value features.

[0055] Through the above steps, the embodiment of the present application provides a specific method for determining query key features and query value features based on the query key transformation network layer and the query value transformation network layer, and further determines the high-quality mask corresponding to the target object to be identified, thereby improving the accuracy of the obtained high-quality mask and having high feasibility.

[0056] In some embodiments, step S2043 includes: Step S2143, determine the attention weight matrix based on the query key features and the mask key features.

[0057] Specifically, in the embodiment of the present application, the query key feature and mask value features The dot product operation is performed on the transpose of , and the obtained score matrix is ​​input into the normalized exponential function to calculate the similarity between the query key feature and the first segmentation mask, and the attention weight matrix is ​​obtained to determine where to fuse the value features from the mask feature queue, where Ck is the feature dimension number of the query key feature: Step S2243, determine a high-quality mask based on the attention weight matrix, query value features and mask value features.

[0058] In this embodiment of the present application, the attention weight matrix is ​​combined with The result of the weighted summation obtained by multiplication represents the association information between the second segmentation mask and the mask in the mask feature queue: The associated information is added to the current query value feature by splicing to determine the feature to be repaired. The feature to be repaired is input into the segmentation head network in the aforementioned detection and segmentation network to obtain a high-quality mask after repair. .

[0059] Through the above steps, the embodiment of the present application introduces an attention mechanism to extract and fuse features, thereby obtaining a high-quality mask to implement subsequent gait and target recognition, further improving the accuracy of target recognition.

[0060] This embodiment also provides a target object recognition method. Figure 3 is a flow chart of another target object recognition method according to an embodiment of the present application. Figure 3 As shown, the process further includes the following steps based on steps S201 to S205: Step S301, obtaining a video image to be tracked, and in the case that the target object contained in the video image to be tracked is less than the target object to be identified, determining the target object missing in the video image to be tracked as a disappeared target object, obtaining a video image to be restored containing the target object to be restored, and determining the gait feature corresponding to the target object to be restored as the gait feature to be restored.

[0061] In an embodiment of the present application, when the tracking of the target object in the video image is interrupted due to occlusion or multiple targets interlaced, the historical gait features are called and used as a supplement to the feature set required for tracking to maintain the target object ID. The video image to be tracked is obtained, and the video image to be tracked may be an image obtained after the video image to be identified. The number of target objects contained in the video image to be tracked is less than the number of target objects to be identified in the video image to be identified, that is, some target objects are not identified. At this time, the target objects that are missing in the video image to be tracked due to not being identified are determined as disappeared target objects. Specifically, the disappeared target objects may be moved to a queue of pedestrians that have recently disappeared. The ID set of the disappeared target objects may be recorded as Further, continue to acquire video images after the video image to be tracked, and when a new target object appears in the acquired image compared with the previous frame, determine the acquired frame image as the video image to be restored, determine the newly appeared target object as the target object to be restored, and determine the gait feature corresponding to the target object to be restored as the gait feature to be restored.

[0062] Step S302, determining that the gait feature corresponding to the disappeared target object is the disappeared target gait feature, and when the similarity between the gait feature to be restored and the gait feature of the disappeared target is greater than or equal to the gait feature similarity threshold, determining the object identifier of the target object to be restored to be consistent with the object identifier of the disappeared target object.

[0063] The gait feature corresponding to the disappeared target object is determined to be the gait feature of the disappeared target. When a new target object to be restored appears, the gait feature to be restored corresponding to it is compared with the gait feature of the most recent disappeared target in the recently disappeared pedestrian queue, and the target object ID whose similarity reaches the gait feature similarity threshold is restored, that is, the object identifier of the newly appeared target object to be restored is determined to be consistent with the object identifier of the previously disappeared target object. In addition, every time a new target object to be restored appears and is restored and matched with the disappeared target object, if there is a disappeared target object in the recently disappeared pedestrian queue that fails to match the gait feature to be restored, the number of unmatched times of the disappeared target object is recorded. When the number of unmatched times is greater than the preset unmatched threshold, the disappeared target object is deleted from the recently disappeared pedestrian queue and no next match is performed.

[0064] Through the above steps, when the tracking of the target object in the picture is interrupted due to occlusion or multiple pedestrians intersecting, the embodiment of the present application calls the historical gait features and uses the gait features as a supplement to the feature set required for tracking to maintain the target object ID. When a new target object appears, its gait features are compared with the gait features in the recently disappeared pedestrian queue, and the target object ID with a similarity reaching a threshold is restored, thereby alleviating the problem of tracking failure caused by occlusion and intersecting, and improving the completeness of the target object information.

[0065] In some embodiments, the target object recognition method includes a plurality of target objects to be recognized, and after step S205, includes: Step S206, obtaining the object reasoning duration corresponding to the key target object, and when the object reasoning duration is greater than the reasoning duration threshold, determining the non-key target object included in the multiple target objects to be identified.

[0066] In the embodiment of the present application, the time required from acquiring the video image to determining the key target object can be determined as the object reasoning time. If the object reasoning time is greater than the preset reasoning time threshold, the temporal gait features of the target object to be identified are sorted by similarity with the gait features in the database, and the target object to be identified is determined as a non-key target object in order of similarity from low to high, until the object reasoning time is no greater than the preset reasoning time threshold. Specifically, when the object reasoning time is greater than the preset reasoning time threshold, the non-key target object is added to the non-key person queue, and the target object ID set in the non-key person queue is recorded as ; When the object reasoning time is not greater than the preset reasoning time threshold, non-critical target objects are popped up in the order of gait similarity from large to small.

[0067] Step S207: determining the target objects other than the non-key target objects in the video image to be identified as the current target objects to be identified.

[0068] The target objects other than the non-key target objects in the video image to be identified are determined as the current target objects to be identified. In this way, when the video image to be identified containing the target objects to be identified is obtained again, the target objects to be identified will not include the non-key target objects, that is, the non-key target objects will no longer be used for mask recovery and gait recognition of occluded pedestrians, so that the recognition frequency can be reduced to reduce the object inference time of target object recognition.

[0069] Through the above steps, when there are too many target objects in the picture or the transmission speed is unstable, resulting in a delay in gait recognition, the embodiment of the present application reduces the frequency of identifying non-critical target objects. During the delay, the target objects in the non-critical human queue are only detected and tracked without segmentation and gait recognition, thereby reducing the queue length and recognition frequency of gait recognition of the target objects, improving the recognition efficiency in complex street scenes, and saving resources.

[0070] In some embodiments, after step S205, the method further includes: Step S208, obtaining the key target gait feature in the target database, and determining that the target object to be identified is a suspected key target object when the similarity between the time series gait feature and the key target gait feature is greater than a feature similarity threshold.

[0071] Specifically, the key target gait features in the target database are obtained, and the target object whose similarity between the temporal gait features and the key target gait features is greater than the feature similarity threshold is determined as a suspected key target object, and its ID is added to the suspected key person set, recorded as ,When the similarity between the temporal gait features of the suspected key target object and the key target gait features is less than the feature similarity threshold, it is deleted from the set.

[0072] Step S209: adding the high-quality mask corresponding to the suspected key target object to the mask feature queue to determine the current mask feature queue.

[0073] In an embodiment of the present application, the high-quality mask corresponding to the suspected key target object is added to the mask feature queue to form a new mask feature queue, and it is determined as the current mask feature queue, and re-input into the mask recovery module to obtain the corresponding high-quality mask. Specifically, the target object in the suspected key target object set can be identified by a sliding window sequence method (half of the images are repeated between adjacent mask sequences input to the gait recognition network), that is, when the similarity between the temporal gait features output by the gait recognition and the key target gait features is higher than the threshold, the second half of the mask image sequence is extracted from the high-quality mask image queue of the suspected key target object input to the gait recognition network, and the mask feature queue of the suspected key target object ID is added again to overclock it for recognition, thereby speeding up the frequency of gait recognition of the suspected key target object.

[0074] In addition, in an embodiment of the present application, multiple key target objects can be determined, and the similarities between the multiple key target objects and the key target gait features in the target database can be sorted, and the information of the first n (n can be pre-set as needed) target objects is continuously maintained for human inspection to see whether they are key persons; when a target object is continuously identified as a key target object for a number of times greater than a preset threshold, a notification "The key target object has been found" is issued.

[0075] Through the above steps, when there is a target object in the picture whose similarity with the key target features in the database is higher than a threshold, the embodiment of the present application over-frequency identifies the suspected key target object, and uses a sliding window sequence recognition method (there is a certain sequence overlap between windows) to over-frequency identify the target objects in the possible key person queue, thereby adjusting the target object recognition frequency, improving the recognition speed of key target objects in complex street scenes, and improving efficiency.

[0076] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0077] The present embodiment also provides a target object recognition device, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module", "unit", "subunit" and the like can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0078] Figure 4 is a structural block diagram of a target object recognition device according to an embodiment of the present application, such as Figure 4 As shown, the device includes a historical feature determination module 10, a mask queue determination module 20, a to-be-recognized feature determination module 30, a high-quality mask determination module 40 and a key target determination module 50: A historical feature determination module 10 is used to obtain a historical video image containing a historical target object, determine a first segmentation mask corresponding to the historical target object, and determine the historical target object feature based on the historical video image and the first segmentation mask; The mask queue determination module 20 is used to determine the mask feature queue corresponding to the historical target object according to the historical target object characteristics and the first transformation network layer; The module 30 for determining the features to be identified is used to obtain a video image to be identified that contains a target object to be identified, determine a second segmentation mask corresponding to the target object to be identified, and determine the features of the target object to be identified based on the video image to be identified and the second segmentation mask, wherein the object identifier of the historical target object is consistent with the object identifier of the target object to be identified; A high-quality mask determination module 40 is used to determine a high-quality mask corresponding to the target object to be identified according to the mask feature queue, the features of the target object to be identified and the second transformation network layer; The key target determination module 50 is used to determine the temporal gait features corresponding to the target object to be identified based on the high-quality mask and the gait recognition network, and determine the key target object based on the temporal gait features.

[0079] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0080] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0081] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0082] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program: Acquire a historical video image containing a historical target object, determine a first segmentation mask corresponding to the historical target object, and determine features of the historical target object based on the historical video image and the first segmentation mask; Determine a mask feature queue corresponding to the historical target object according to the historical target object feature and the first transformation network layer; Acquire a video image to be identified that includes a target object to be identified, determine a second segmentation mask corresponding to the target object to be identified, and determine features of the target object to be identified based on the video image to be identified and the second segmentation mask, wherein the object identifier of the historical target object is consistent with the object identifier of the target object to be identified; Determine a high-quality mask corresponding to the target object to be identified according to the mask feature queue, the features of the target object to be identified and the second transformation network layer; According to the high-quality mask and gait recognition network, the temporal gait features corresponding to the target object to be identified are determined, and the key target object is determined according to the temporal gait features.

[0083] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0084] In addition, in combination with the target object recognition method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any target object recognition method in the above embodiment is implemented.

[0085] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0087] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A target object recognition method, characterized in that: The following steps are involved: Acquire a historical video image containing a historical target object, determine a first segmentation mask corresponding to the historical target object, and determine features of the historical target object according to the historical video image and the first segmentation mask; Determine a mask feature queue corresponding to the historical target object according to the historical target object feature and the first transformation network layer; Acquire a video image to be identified that includes a target object to be identified, determine a second segmentation mask corresponding to the target object to be identified, and determine features of the target object to be identified based on the video image to be identified and the second segmentation mask, wherein the object identifier of the historical target object is consistent with the object identifier of the target object to be identified; Determine a high-quality mask corresponding to the target object to be identified according to the mask feature queue, the feature of the target object to be identified and the second transformation network layer; According to the high-quality mask and the gait recognition network, the temporal gait features corresponding to the target object to be identified are determined, and the key target object is determined according to the temporal gait features.

2. The target object recognition method according to claim 1, characterized in that: The first transformation network layer includes a mask key transformation network layer and a mask value transformation network layer, and the determining, according to the historical target object feature and the first transformation network layer, of the mask feature queue corresponding to the historical target object includes: Inputting the historical target object features into the mask key transformation network layer to determine mask key features; Inputting the historical target object features into the mask value transformation network layer to determine mask value features; A mask feature queue corresponding to the historical target object is determined according to the mask key feature and the mask value feature.

3. The target object recognition method according to claim 2, characterized in that: The second transformation network layer includes a query key transformation network layer and a query value transformation network layer, and the determining of a high-quality mask corresponding to the target object to be identified according to the mask feature queue, the feature of the target object to be identified and the second transformation network layer includes: Inputting the target object feature to be identified into the query key transformation network layer to determine the query key feature; Inputting the target object feature to be identified into the query value transformation network layer to determine the query value feature; The high-quality mask is determined according to the mask key feature, the mask value feature, the query key feature, and the query value feature.

4. The target object recognition method according to claim 3, characterized in that: The determining the high-quality mask according to the mask key feature, the mask value feature, the query key feature, and the query value feature comprises: Determining an attention weight matrix according to the query key feature and the mask key feature; The high-quality mask is determined according to the attention weight matrix, the query value feature and the mask value feature.

5. The target object recognition method according to any one of claims 1 to 4, characterized in that: The method further comprises: Acquire a video image to be tracked, and if the target object contained in the video image to be tracked is less than the target object to be identified, determine the target object missing in the video image to be tracked as a lost target object, acquire a video image to be restored containing the target object to be restored, and determine the gait feature corresponding to the target object to be restored as the gait feature to be restored; Determine that the gait feature corresponding to the disappeared target object is the disappeared target gait feature, and when the similarity between the gait feature to be restored and the gait feature of the disappeared target is greater than or equal to a gait feature similarity threshold, determine the object identifier of the target object to be restored to be consistent with the object identifier of the disappeared target object.

6. The target object recognition method according to claim 5, characterized in that: The method comprises a plurality of target objects to be identified, and after determining the time-series gait features corresponding to the target objects to be identified according to the high-quality mask and the gait recognition network, and determining the key target objects according to the time-series gait features, the method further comprises: Obtaining and determining an object reasoning duration corresponding to the key target object, and determining a non-key target object included in the plurality of target objects to be identified when the object reasoning duration is greater than a reasoning duration threshold; The target objects other than the non-critical target objects in the video image to be identified are determined as current target objects to be identified.

7. The target object recognition method according to claim 6, characterized in that: After determining the temporal gait features corresponding to the target object to be identified according to the high-quality mask and the gait recognition network, and determining the key target object according to the temporal gait features, the method further includes: Acquire a key target gait feature in a target database, and determine that the target object to be identified is a suspected key target object when the similarity between the time-series gait feature and the key target gait feature is greater than a feature similarity threshold; The high-quality mask corresponding to the suspected key target object is added to the mask feature queue to determine the current mask feature queue.

8. A target object recognition device, characterized in that: It includes historical feature determination module, mask queue determination module, to-be-identified feature determination module, high-quality mask determination module and key target determination module: The historical feature determination module is used to obtain a historical video image containing a historical target object, determine a first segmentation mask corresponding to the historical target object, and determine the historical target object feature based on the historical video image and the first segmentation mask; The mask queue determination module is used to determine the mask feature queue corresponding to the historical target object according to the historical target object characteristics and the first transformation network layer; The module for determining features to be identified is used to obtain a video image to be identified that contains a target object to be identified, determine a second segmentation mask corresponding to the target object to be identified, and determine features of the target object to be identified based on the video image to be identified and the second segmentation mask, wherein the object identifier of the historical target object is consistent with the object identifier of the target object to be identified; The high-quality mask determination module is used to determine the high-quality mask corresponding to the target object to be identified according to the mask feature queue, the feature of the target object to be identified and the second transformation network layer; The key target determination module is used to determine the temporal gait features corresponding to the target object to be identified based on the high-quality mask and the gait recognition network, and determine the key target object based on the temporal gait features.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the target object recognition method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the target object recognition method according to any one of claims 1 to 7 when running.

Images