Method, device, equipment and storage medium for generating affiliation

Abstract

The present invention relates to the field of computer vision technology, and discloses a method, device, equipment and storage medium for generating an affiliation, which is used to improve the stability of affiliation recognition. The affiliation generation method comprises: determining a first affiliation set between target pedestrian information and target accessory information in the target information according to target information observed in the current frame; obtaining a historical affiliation set, and performing missed detection recognition and affiliation correction on the first affiliation set according to the historical affiliation set to obtain a second affiliation set; performing consistency verification on the second affiliation set according to the historical target trajectory, and generating a third affiliation set according to the verification result.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device, equipment and storage medium for generating an affiliation relationship. Background Art

[0002] Object tracking is a key research topic in computer vision. When tracking pedestrians, it is common for pedestrians to carry accessories, such as pedestrians dragging suitcases, cleaners holding brooms, etc.

[0003] However, in actual applications, since the attachments are easily blocked by pedestrians and the bounding boxes are easily filtered by thresholds, the recognition of the attachment relationship is unstable and prone to repeated jumps. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for generating an affiliation relationship, which are used to improve the stability of affiliation relationship identification.

[0005] A first aspect of the present invention provides a method for generating an affiliation relationship, comprising:

[0006] Determine, according to the target information observed in the current frame, a first set of affiliation relationships between target pedestrian information and target accessory information in the target information;

[0007] Acquire a historical affiliation set, and perform missed detection identification and affiliation correction on the first affiliation set according to the historical affiliation set to obtain a second affiliation set;

[0008] According to the historical target trajectory, the second set of affiliations is verified for consistency, and a third set of affiliations is generated according to the verification result.

[0009] Optionally, determining, according to the target information observed in the current frame, a first set of affiliation relationships between the target pedestrian information and the target accessory information in the target information includes:

[0010] Obtaining target information observed in the current frame, and classifying the target information into target types, to obtain target pedestrian information and target accessory information in the target information;

[0011] A first affiliation relationship set between the target pedestrian information and the target accessory information is determined according to a preset search distance threshold.

[0012] Optionally, determining a first set of affiliation relationships between the target pedestrian information and the target accessory information according to a preset search distance threshold includes:

[0013] Determining whether there is pedestrian information in the target accessory information in the target information within a preset search distance threshold range;

[0014] If the target accessory information in the target information has pedestrian information within the preset search distance threshold range, a first affiliation relationship set between the corresponding target accessory information and the nearest target pedestrian information is determined.

[0015] Optionally, the acquiring of a historical affiliation set, and performing missed detection identification and affiliation correction on the first affiliation set according to the historical affiliation set to obtain a second affiliation set includes:

[0016] Get the historical pedestrian information in the historical affiliation collection;

[0017] Determine whether the historical pedestrian information has corresponding accessory information in the first accessory relationship set;

[0018] If so, performing affiliation correction on the first affiliation set according to the distance between the historical pedestrian information and the corresponding affiliation information in the first affiliation set to obtain a second affiliation set;

[0019] If not, then performing missed detection identification on the first affiliation set according to the historical attachment information corresponding to the historical pedestrian information in the historical affiliation set to obtain a second affiliation set.

[0020] Optionally, performing affiliation correction on the first affiliation set according to the distance between the historical pedestrian information and the corresponding affiliation information in the first affiliation set to obtain the second affiliation set includes:

[0021] Determining whether the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is greater than a preset accessory relationship distance threshold;

[0022] If so, the affiliation between the historical pedestrian information and the corresponding attachment information in the first affiliation set is cleared to obtain a second affiliation set.

[0023] Optionally, performing missed detection identification on the first affiliation set according to the historical attachment information corresponding to the historical pedestrian information in the historical affiliation set to obtain a second affiliation set includes:

[0024] Obtaining historical attachment information corresponding to the historical pedestrian information in the historical attachment relationship set;

[0025] Determining whether the historical attachment information has corresponding pedestrian information in the first attachment relationship set;

[0026] If so, pedestrian information corresponding to the historical attachment information in the first attachment relationship set is identified according to the historical target trajectory to obtain a second attachment relationship set.

[0027] Optionally, the identifying pedestrian information corresponding to the historical attachment information in the first attachment relationship set according to the historical target trajectory to obtain the second attachment relationship set includes:

[0028] Calculating an average distance between the historical pedestrian information and the corresponding historical accessory information according to the accessory trajectory corresponding to the historical accessory information and the pedestrian trajectory corresponding to the historical pedestrian information in the historical target trajectory;

[0029] If the average distance is less than a preset average distance threshold, pedestrian information corresponding to the historical attachment information in the first attachment relationship set is determined to obtain a second attachment relationship set.

[0030] Optionally, according to the historical target trajectory, the second set of affiliations is verified for consistency, and a third set of affiliations is generated according to the verification result, including:

[0031] Performing historical target trajectory matching on the pedestrian information and accessory information in the second accessory relationship set to obtain a target pedestrian trajectory and a target accessory trajectory;

[0032] Verifying the consistency of the target pedestrian trajectory and the target accessory trajectory with the historical target trajectory to obtain a verification result;

[0033] The second affiliation set is corrected according to the verification result to obtain a third affiliation set.

[0034] A second aspect of the present invention provides a device for generating a dependency relationship, comprising:

[0035] A determination module, configured to determine, based on target information observed in a current frame, a first set of affiliation relationships between target pedestrian information and target accessory information in the target information;

[0036] a correction module, configured to obtain a historical affiliation set, and perform missed detection identification and affiliation correction on the first affiliation set according to the historical affiliation set to obtain a second affiliation set;

[0037] A verification module is used to verify the consistency of the second set of affiliations according to the historical target trajectory, and generate a third set of affiliations according to the verification result.

[0038] Optionally, the determining module includes:

[0039] A classification unit, used to obtain target information observed in the current frame, and classify the target information into target types to obtain target pedestrian information and target accessory information in the target information;

[0040] A determination unit is used to determine a first set of affiliation relationships between the target pedestrian information and the target accessory information according to a preset search distance threshold.

[0041] Optionally, the determining unit is specifically configured to:

[0042] Determining whether there is pedestrian information in the target accessory information in the target information within a preset search distance threshold range;

[0043] If the target accessory information in the target information has pedestrian information within the preset search distance threshold range, a first affiliation relationship set between the corresponding target accessory information and the nearest target pedestrian information is determined.

[0044] Optionally, the correction module includes:

[0045] The acquisition submodule is used to obtain the historical pedestrian information in the historical affiliation set;

[0046] A judgment submodule, used to judge whether the historical pedestrian information has corresponding accessory information in the first accessory relationship set;

[0047] a correction submodule, configured to, if present, perform affiliation correction on the first affiliation set according to the distance between the historical pedestrian information and the corresponding affiliation information in the first affiliation set to obtain a second affiliation set;

[0048] The identification submodule is used to perform missed detection identification on the first affiliation set according to the historical accessory information corresponding to the historical pedestrian information in the historical affiliation set, if it does not exist, to obtain a second affiliation set.

[0049] Optionally, the correction submodule is specifically used for:

[0050] Determining whether the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is greater than a preset accessory relationship distance threshold;

[0051] If so, the affiliation between the historical pedestrian information and the corresponding attachment information in the first affiliation set is cleared to obtain a second affiliation set.

[0052] Optionally, the identification submodule includes:

[0053] An acquiring unit, configured to acquire historical attachment information corresponding to the historical pedestrian information in the historical attachment relationship set;

[0054] a judging unit, configured to judge whether the historical attachment information has corresponding pedestrian information in the first attachment relationship set;

[0055] The identification unit is used to identify the pedestrian information corresponding to the historical attachment information in the first attachment relationship set according to the historical target trajectory, if any, to obtain a second attachment relationship set.

[0056] Optionally, the identification unit is specifically used for:

[0057] Calculating an average distance between the historical pedestrian information and the corresponding historical accessory information according to the accessory trajectory corresponding to the historical accessory information and the pedestrian trajectory corresponding to the historical pedestrian information in the historical target trajectory;

[0058] If the average distance is less than a preset average distance threshold, pedestrian information corresponding to the historical attachment information in the first attachment relationship set is determined to obtain a second attachment relationship set.

[0059] Optionally, the verification module is specifically used for:

[0060] Performing historical target trajectory matching on the pedestrian information and accessory information in the second accessory relationship set to obtain a target pedestrian trajectory and a target accessory trajectory;

[0061] Verifying the consistency of the target pedestrian trajectory and the target accessory trajectory with the historical target trajectory to obtain a verification result;

[0062] The second affiliation set is corrected according to the verification result to obtain a third affiliation set.

[0063] The third aspect of the present invention provides a device for generating an affiliation, comprising: a memory and at least one processor, wherein the memory stores a computer program; the at least one processor calls the computer program in the memory so that the device for generating an affiliation executes the above-mentioned method for generating an affiliation.

[0064] A fourth aspect of the present invention provides a computer-readable storage medium, in which a computer program is stored, which, when executed on a computer, enables the computer to execute the above-mentioned method for generating affiliation relationships.

[0065] In the technical solution provided by the present invention, a first affiliation set between target pedestrian information and target accessory information in the target information is determined based on the target information observed in the current frame; a historical affiliation set is obtained, and the first affiliation set is subjected to missed detection identification and affiliation correction based on the historical affiliation set to obtain a second affiliation set; the second affiliation set is subjected to consistency verification based on the historical target trajectory, and a third affiliation set is generated based on the verification result. In an embodiment of the present invention, a preliminary identification of the relationship between pedestrians and accessories is first performed on the target information observed in the current frame to obtain a first affiliation set. In order to improve the accuracy of affiliation identification, the first affiliation set is subjected to missed detection identification and affiliation correction based on the historical affiliation set to obtain a second affiliation set with further improved accuracy. Finally, the second affiliation set is verified based on the historical target trajectory, and a verified third affiliation set is generated. The present invention can improve the stability of affiliation identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of an embodiment of a method for generating a dependency relationship in an embodiment of the present invention;

[0067] Figure 2 is a schematic diagram of another embodiment of a method for generating a dependency relationship in an embodiment of the present invention;

[0068] Figure 3 A schematic diagram of an embodiment of a device for generating a dependency relationship in an embodiment of the present invention;

[0069] Figure 4 It is a schematic diagram of another embodiment of a device for generating a dependency relationship in an embodiment of the present invention;

[0070] Figure 5 The figure is a schematic diagram of an embodiment of a device for generating a dependency relationship in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The embodiments of the present invention provide a method, apparatus, device and storage medium for generating affiliation relationships, which are used to improve the stability of affiliation relationship identification.

[0072] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0073] It is understandable that the execution subject of the present invention may be a device for generating a subordinate relationship, or may be a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0074] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of a method for generating a dependency relationship in an embodiment of the present invention includes:

[0075] 10. Determine a first attachment relationship set between target pedestrian information and target attachment information in the target information according to the target information observed in the current frame;

[0076] As an example but not limitation, the present invention can be applied to the field of autonomous driving, by collecting environmental information through an environmental information collection device (such as a camera, a lidar, etc.) installed on the autonomous driving vehicle, and performing target detection on the environmental information through a deep learning-based detection model to obtain target information observed in the current frame. For example, target detection is performed on the current bird's-eye view collected by a camera installed on the autonomous driving vehicle to obtain target information in the current bird's-eye view.

[0077] Among them, the target information can be obstacle information around the autonomous driving vehicle, or other preset object information, such as passenger information in station monitoring images, pedestrian information in road monitoring, etc., which is not limited here. The present invention is applicable to target detection and tracking scenarios with affiliated relationships.

[0078] In one embodiment, the target information is subjected to affiliation recognition through a trained affiliation recognition model to obtain a first affiliation set between the target pedestrian information and the target accessory information in the target information, such as a pedestrian dragging a suitcase or a cleaner holding a broom. The affiliation recognition model adopts a neural network structure model, which can improve the accuracy of affiliation recognition.

[0079] In one embodiment, since the detection model has a high accuracy in detecting pedestrians, in order to improve the accuracy of the identification of the subsidiary relationship, the target information is classified / screened for pedestrian target information to obtain the target pedestrian information in the target information, and then the non-pedestrian information within the preset search distance threshold range of the target pedestrian information is searched, and the non-pedestrian information closest to the preset search distance threshold range is set as the target subsidiary information corresponding to the target pedestrian information, and the first subsidiary relationship set between the target pedestrian information and the target subsidiary information in the target information is obtained. This embodiment searches for the subsidiary based on the pedestrian so that the identification of the subsidiary is not limited to the preset subsidiary type, and can identify the subsidiary information within the range of the non-preset subsidiary type, so that the intelligence of the subsidiary relationship detection is improved, thereby improving the accuracy of the subsidiary relationship generation.

[0080] 20. Obtain a historical affiliation set, and perform missed detection identification and affiliation correction on the first affiliation set according to the historical affiliation set to obtain a second affiliation set;

[0081] It should be noted that due to the occlusion of pedestrians or accessories, or the close distance between pedestrians and accessories causing the bounding box of one of them to be filtered by the threshold, there is a difference between the first affiliation set and the historical affiliation set. Therefore, in order to avoid missed or false detection of affiliations, the first affiliation set is subjected to missed identification and affiliation correction based on the historical affiliation set to obtain the second affiliation set, thereby improving the accuracy of affiliation generation.

[0082] In one embodiment, the historical affiliation set is the affiliation set observed in the historical frame. To improve data processing efficiency, the historical affiliation set is the affiliation set observed within a preset number of frames, such as the affiliation set within k=100 frames, which is not specifically limited here. This embodiment can filter data that has not appeared in the field of view for too long, thereby improving the efficiency of affiliation generation.

[0083] In one embodiment, the historical affiliation set includes a historical affiliation set of multiple frames, and each frame historical affiliation set is compared with the first affiliation set to obtain the difference affiliations in the multi-frame historical affiliation set that are different from the first affiliation set and the difference frame number corresponding to the difference affiliations, and determine whether the difference frame number is greater than a preset frame number threshold. If the difference frame number is greater than the preset frame number threshold, then, and perform missed detection identification and affiliation correction on the first affiliation set according to the corresponding difference affiliations to obtain a second affiliation set. This embodiment can detect the historical affiliation set with differences, and adjust the current (first) affiliation based on the historical affiliation set of sufficiently multiple frames, thereby improving the stability of the affiliation and avoiding repeated adjustments of the affiliation.

[0084] In one embodiment, historical accessory information in a historical accessory relationship set is obtained, and it is determined whether the historical accessory information has corresponding pedestrian information in a first accessory relationship set. If so, accessory correction is performed on the first accessory relationship set based on the distance between the historical accessory information and the corresponding pedestrian information in the first accessory relationship set to obtain a second accessory relationship set. If not, missed detection identification is performed on the first accessory relationship set based on the historical pedestrian information corresponding to the historical accessory information in the historical accessory relationship set to obtain a second accessory relationship set. For example, it is identified whether there is corresponding pedestrian information in the first attachment relationship set for historical accessory information A. If there is pedestrian information B corresponding to historical accessory information A in the first attachment relationship set, then, according to the current distance of pedestrian information B corresponding to historical accessory information A, it is re-verified whether the attachment relationship between the two is still established, thereby correcting the attachment relationship of pedestrian information B corresponding to historical accessory information A in the first attachment relationship set to obtain a second attachment relationship set. If there is no pedestrian information B corresponding to historical accessory information A in the first attachment relationship set, then it means that the first attachment relationship set may have missed the pedestrian information corresponding to historical accessory information A. Then, according to historical pedestrian information C corresponding to historical accessory information A in the historical attachment relationship set, the pedestrian information corresponding to historical accessory information A in the target information is re-identified, thereby performing missed detection identification on the first attachment relationship set to obtain a second attachment relationship set. This embodiment can identify missed detection and wrong detection in the current attachment relationship, thereby improving the accuracy of attachment relationship generation.

[0085] 30. According to the historical target trajectory, the second set of affiliations is verified for consistency, and a third set of affiliations is generated according to the verification result.

[0086] It should be noted that the historical target trajectory is used to indicate the historical status information of the target information (pedestrian information or accessory information), and the historical target trajectory includes but is not limited to the status information such as the position, size, speed and direction of the target information (pedestrian information or accessory information). The historical target trajectory includes the historical pedestrian trajectory and the historical accessory trajectory. In one embodiment, the historical target trajectory is matched with the second pedestrian information and the second accessory information in the second accessory relationship set by a preset bipartite graph matching algorithm to obtain the historical pedestrian trajectory corresponding to the second pedestrian information and the historical accessory trajectory corresponding to the second accessory information, and the second accessory relationship set is verified for consistency based on the historical pedestrian trajectory corresponding to the second pedestrian information and the historical accessory trajectory corresponding to the second accessory information to obtain a verification result. Among them, the preset bipartite graph matching algorithm can be any one of the Hungarian algorithm, the greedy algorithm and the KM (Kuhn-Munkres) algorithm, which is not specifically limited here.

[0087] Specifically, it is verified whether the tracking identifier (track id) in the second pedestrian information is consistent with the tracking identifier in the corresponding historical pedestrian trajectory, and it is verified whether the tracking identifier in the second accessory information is consistent with the tracking identifier in the corresponding historical accessory trajectory, and a verification result is obtained. The verification result is used to indicate whether the tracking identifier in the second pedestrian information is consistent / inconsistent with the tracking identifier in the corresponding historical pedestrian trajectory, and / or whether the tracking identifier in the second accessory information is consistent / inconsistent with the tracking identifier in the corresponding historical accessory trajectory.

[0088] Furthermore, if the verification result indicates that the tracking identifier in the second pedestrian information is inconsistent with the tracking identifier in the corresponding historical pedestrian trajectory, or the tracking identifier in the second accessory information is inconsistent with the tracking identifier in the corresponding historical accessory trajectory, then the tracking identifier in the historical target trajectory is used as a reference to correct the tracking identifier of the second affiliation set to obtain a third affiliation set. For example, the second affiliation set assigns a tracking identifier A to one of the second pedestrian information, and the identifier of the second pedestrian information in the historical pedestrian trajectory is B, which means that the tracking identifier in the second pedestrian information is inconsistent with the tracking identifier in the corresponding historical pedestrian trajectory. Then, the tracking identifier of the second pedestrian information in the second affiliation set is modified based on the tracking identifier in the historical pedestrian trajectory, that is, A is modified to B. This embodiment can avoid repeated jumps in the target trajectory, thereby improving the stability of affiliation generation.

[0089] In an embodiment of the present invention, a preliminary identification of the relationship between pedestrians and accessories is first performed on target information observed in the current frame to obtain a first accessory relationship set. In order to improve the accuracy of accessory relationship identification, missed detection identification and accessory relationship correction are performed on the first accessory relationship set based on the historical accessory relationship set, thereby obtaining a second accessory relationship set with further improved accuracy. Finally, the second accessory relationship set is verified based on the historical target trajectory, and a verified third accessory relationship set is generated. The present invention can improve the stability of accessory relationship identification.

[0090] See also Figure 2 Another embodiment of the method for generating affiliation in the embodiment of the present invention includes:

[0091] 201. Determine, according to target information observed in a current frame, a first attachment relationship set between target pedestrian information and target attachment information in the target information;

[0092] Specifically, step 201 includes: obtaining the target information observed in the current frame, and classifying the target information into target types to obtain target pedestrian information and target accessory information in the target information; determining a first set of affiliation relationships between the target pedestrian information and the target accessory information according to a preset search distance threshold. In this embodiment, the target information includes target type information, and the target type information includes but is not limited to pedestrian type information and accessory type information. For example, the accessory type information may be information on pedestrian accessories such as suitcases, bicycles, brooms, and pets, which are not specifically limited here. The target information is classified into types using the target type information in the target information to obtain target pedestrian information and target accessory information in the target information, thereby quickly identifying pedestrians and accessories, and thereby improving the efficiency of affiliation generation.

[0093] In one embodiment, determining a first set of affiliations between target pedestrian information and target accessory information according to a preset search distance threshold includes: determining a first set of affiliations between target pedestrian information and target accessory information within the preset search distance threshold. Since there may be multiple target accessory information within the preset search distance threshold, or multiple target pedestrian information may have the same target accessory information within the preset search distance threshold, the first set of affiliations in this embodiment is a candidate affiliation, which is used for further confirmation of subsequent affiliations.

[0094] Further, determining the first affiliation set between the target pedestrian information and the target accessory information according to the preset search distance threshold includes: determining whether the target accessory information in the target information contains pedestrian information within the preset search distance threshold; if the target accessory information in the target information contains pedestrian information within the preset search distance threshold, determining the first affiliation set between the corresponding target accessory information and the nearest target pedestrian information. This embodiment searches for the nearest pedestrian with an accessory, and can establish a one-to-one affiliation, thereby improving the accuracy of affiliation recognition.

[0095] 202. Obtain historical pedestrian information in the historical affiliation set;

[0096] Among them, historical pedestrian information refers to pedestrian information in a historical affiliation set, that is, pedestrian information with an affiliation in history. Specifically, historical pedestrian information can be a pedestrian identifier in a historical affiliation set. Historical pedestrian information can include historical information of multiple pedestrians, which is not limited here.

[0097] 203. Determine whether the historical pedestrian information has corresponding accessory information in the first accessory relationship set;

[0098] It should be noted that, since there may be differences between the first attachment relationship set and the historical attachment relationship set, it is necessary to identify missed detections and wrong detections in the first attachment relationship set by judging whether there is corresponding attachment information in the first attachment relationship set for the historical pedestrian information, and to promptly recall missed detections and wrong detections in the first attachment relationship set. Specifically, it is judged whether there is corresponding attachment information in the first attachment relationship set for the historical pedestrian information, wherein the attachment information corresponding to the historical pedestrian information in the first attachment relationship set may be the same as or different from the corresponding attachment information in the historical attachment relationship set, which is not limited here.

[0099] 204. If so, performing affiliation correction on the first affiliation set according to the distance between the historical pedestrian information and the corresponding affiliation information in the first affiliation set to obtain a second affiliation set;

[0100] If the historical pedestrian information has corresponding accessory information in the first accessory relationship set, then the first accessory relationship set is corrected according to the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set to obtain the second accessory relationship set. It should be noted that if the historical pedestrian information has corresponding accessory information in the first accessory relationship set, it means that the historical pedestrian information has a historical accessory relationship set and a first accessory relationship set, then it is necessary to re-verify whether the historical accessory relationship set of the historical pedestrian information and the first accessory relationship set are the same, that is, to verify whether the historical accessory relationship set of the historical pedestrian information still exists. Specifically, step 204 includes: if it exists, then determine whether the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is greater than a preset accessory relationship distance threshold; if so, then clear the accessory relationship between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set to obtain the second accessory relationship set. By verifying whether the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is greater than the preset accessory relationship distance threshold, if the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is greater than the preset accessory relationship distance threshold, it means that there is a false detection in the historical accessory relationship set of the historical pedestrian information, then, the accessory relationship between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is cleared to obtain the second accessory relationship set, if the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is less than the preset accessory relationship distance threshold, it means that the historical accessory relationship set of the historical pedestrian information is detected correctly, then the first accessory relationship set can be directly set to the second accessory relationship set.

[0101] 205. If it does not exist, performing missed detection identification on the first attachment relationship set according to the historical attachment information corresponding to the historical pedestrian information in the historical attachment relationship set to obtain a second attachment relationship set;

[0102] If the historical pedestrian information does not have corresponding accessory information in the first accessory relationship set, then the first accessory relationship set is identified for missed detection based on the historical accessory information corresponding to the historical pedestrian information in the historical accessory relationship set to obtain the second accessory relationship set. It should be noted that if the historical pedestrian information does not have corresponding accessory information in the first accessory relationship set, it means that there may be missed detection in the first accessory relationship set, then it is further verified whether the historical accessory information corresponding to the historical pedestrian information in the historical accessory relationship set has corresponding pedestrian information in the first accessory relationship set. Specifically, step 205 includes: if it does not exist, then obtain the historical accessory information corresponding to the historical pedestrian information in the historical accessory relationship set; determine whether the historical accessory information has corresponding pedestrian information in the first accessory relationship set; if it exists, then identify the pedestrian information corresponding to the historical accessory information in the first accessory relationship set based on the historical target trajectory to obtain the second accessory relationship set. It should be noted that if there is corresponding pedestrian information in the first attachment relationship set for the historical accessory information, it means that the pedestrian bounding box or the accessory bounding box in the first attachment relationship set may be missed due to threshold filtering because the distance is too close. In this case, it is necessary to re-identify the bounding box of the pedestrian information corresponding to the historical accessory information according to the historical target trajectory to obtain the second attachment relationship set.

[0103] Furthermore, pedestrian information corresponding to historical accessory information in the first accessory relationship set is identified according to the historical target trajectory to obtain a second accessory relationship set, including: calculating an average distance between the historical pedestrian information and the corresponding historical accessory information according to the accessory trajectory corresponding to the historical accessory information in the historical target trajectory and the pedestrian trajectory corresponding to the historical pedestrian information; if the average distance is less than a preset average distance threshold, determining the pedestrian information corresponding to the historical accessory information in the first accessory relationship set to obtain a second accessory relationship set. In this implementation, since the historical target trajectory can truly reflect the historical movement state of pedestrians or accessories, the average distance between the historical pedestrian information and the corresponding historical accessory information is calculated according to the accessory trajectory and the pedestrian trajectory in the historical target trajectory, and it is determined whether the average distance between the historical pedestrian information and the corresponding historical accessory information is less than a preset average distance threshold. If the average distance between the historical pedestrian information and the corresponding historical accessory information is less than the preset average threshold, it means that there is an attachment relationship between the historical pedestrian information and the corresponding historical accessory information, and the pedestrian information corresponding to the historical accessory information in the first attachment relationship set is determined to obtain a second attachment relationship set. If the average distance between the historical pedestrian information and the corresponding historical accessory information is greater than the preset average threshold, it means that there is no attachment relationship between the historical pedestrian information and the corresponding historical accessory information, and the attachment relationship between the historical pedestrian information and the corresponding historical accessory information in the first attachment relationship set is cleared to obtain a second attachment relationship set.

[0104] 206. Perform consistency verification on the second set of affiliated relationships according to the historical target trajectory, and generate a third set of affiliated relationships according to the verification result.

[0105] Specifically, step 206 includes: matching the pedestrian information and accessory information in the second affiliation set with the historical target trajectory to obtain the target pedestrian trajectory and the target accessory trajectory; verifying the consistency of the target pedestrian trajectory and the target accessory trajectory with the historical target trajectory to obtain a verification result; correcting the second affiliation set according to the verification result to obtain a third affiliation set. In one embodiment, the Hungarian algorithm is used to match the pedestrian information and the accessory information in the second affiliation set with the historical target trajectory to obtain the target pedestrian trajectory and the target accessory trajectory. The Hungarian algorithm can improve the accuracy of target matching, thereby improving the accuracy of affiliation generation. Then verify whether the target pedestrian trajectory and the target accessory trajectory are consistent with the pedestrian trajectory and the accessory trajectory in the historical target trajectory to obtain a verification result. If the verification result indicates that the target pedestrian trajectory and the target accessory trajectory are inconsistent with the historical target trajectory, the second affiliation set is corrected based on the historical target trajectory to obtain a third affiliation set.

[0106] In an embodiment of the present invention, a preliminary identification of the relationship between pedestrians and accessories is first performed on target information observed in the current frame to obtain a first accessory relationship set. In order to improve the accuracy of accessory relationship identification, missed detection identification and accessory relationship correction are performed on the first accessory relationship set based on historical pedestrian information in the historical accessory relationship set, thereby obtaining a second accessory relationship set with further improved accuracy. Finally, the second accessory relationship set is verified based on the historical target trajectory, and a verified third accessory relationship set is generated. The present invention can improve the stability of accessory relationship identification.

[0107] The above describes the method for generating the dependent relationship in the embodiment of the present invention. The following describes the device for generating the dependent relationship in the embodiment of the present invention. Figure 3 In one embodiment of the present invention, a device for generating a dependency relationship includes:

[0108] A determination module 301 is used to determine a first attachment relationship set between target pedestrian information and target attachment information in the target information according to the target information observed in the current frame;

[0109] A correction module 302, configured to obtain a historical affiliation set, and perform missed detection identification and affiliation correction on the first affiliation set according to the historical affiliation set to obtain a second affiliation set;

[0110] The verification module 303 is used to perform consistency verification on the second affiliation set according to the historical target trajectory, and generate a third affiliation set according to the verification result.

[0111] In an embodiment of the present invention, a preliminary identification of the relationship between pedestrians and accessories is first performed on target information observed in the current frame to obtain a first accessory relationship set. In order to improve the accuracy of accessory relationship identification, missed detection identification and accessory relationship correction are performed on the first accessory relationship set based on the historical accessory relationship set, thereby obtaining a second accessory relationship set with further improved accuracy. Finally, the second accessory relationship set is verified based on the historical target trajectory, and a verified third accessory relationship set is generated. The present invention can improve the stability of accessory relationship identification.

[0112] See also Figure 4 Another embodiment of the device for generating a dependency relationship in the embodiment of the present invention includes:

[0113] A determination module 301 is used to determine a first attachment relationship set between target pedestrian information and target attachment information in the target information according to the target information observed in the current frame;

[0114] A correction module 302, configured to obtain a historical affiliation set, and perform missed detection identification and affiliation correction on the first affiliation set according to the historical affiliation set to obtain a second affiliation set;

[0115] The verification module 303 is used to perform consistency verification on the second affiliation set according to the historical target trajectory, and generate a third affiliation set according to the verification result.

[0116] Optionally, the determining module 301 includes:

[0117] The classification unit 3011 is used to obtain target information observed in the current frame, and classify the target information into target types to obtain target pedestrian information and target accessory information in the target information;

[0118] The determining unit 3012 is configured to determine a first set of affiliation relationships between the target pedestrian information and the target accessory information according to a preset search distance threshold.

[0119] Optionally, the determining unit 3012 is specifically configured to:

[0120] Determining whether there is pedestrian information in the target accessory information in the target information within a preset search distance threshold range;

[0121] If the target accessory information in the target information has pedestrian information within the preset search distance threshold range, a first affiliation relationship set between the corresponding target accessory information and the nearest target pedestrian information is determined.

[0122] Optionally, the correction module 302 includes:

[0123] The acquisition submodule 3021 is used to acquire historical pedestrian information in the historical affiliation set;

[0124] A judgment submodule 3022, used to judge whether the historical pedestrian information has corresponding accessory information in the first accessory relationship set;

[0125] a correction submodule 3023, configured to, if present, perform affiliation correction on the first affiliation set according to the distance between the historical pedestrian information and the corresponding affiliation information in the first affiliation set, to obtain a second affiliation set;

[0126] The identification submodule 3024 is configured to perform missed detection identification on the first affiliation set according to the historical accessory information corresponding to the historical pedestrian information in the historical affiliation set, if the affiliation set does not exist, to obtain a second affiliation set.

[0127] Optionally, the correction submodule 3023 is specifically used for:

[0128] Determining whether the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is greater than a preset accessory relationship distance threshold;

[0129] If so, the affiliation between the historical pedestrian information and the corresponding attachment information in the first affiliation set is cleared to obtain a second affiliation set.

[0130] Optionally, the identification submodule 3024 includes:

[0131] An acquiring unit 30241 is used to acquire historical attachment information corresponding to the historical pedestrian information in the historical attachment relationship set;

[0132] A judging unit 30242, configured to judge whether the historical attachment information has corresponding pedestrian information in the first attachment relationship set;

[0133] The identification unit 30243 is configured to identify the pedestrian information corresponding to the historical attachment information in the first attachment relationship set according to the historical target trajectory, if any, to obtain a second attachment relationship set.

[0134] Optionally, the identification unit 30243 is specifically used for:

[0135] Calculating an average distance between the historical pedestrian information and the corresponding historical accessory information according to the accessory trajectory corresponding to the historical accessory information and the pedestrian trajectory corresponding to the historical pedestrian information in the historical target trajectory;

[0136] If the average distance is less than a preset average distance threshold, pedestrian information corresponding to the historical attachment information in the first attachment relationship set is determined to obtain a second attachment relationship set.

[0137] Optionally, the verification module 303 is specifically used for:

[0138] Performing historical target trajectory matching on the pedestrian information and accessory information in the second accessory relationship set to obtain a target pedestrian trajectory and a target accessory trajectory;

[0139] Verifying the consistency of the target pedestrian trajectory and the target accessory trajectory with the historical target trajectory to obtain a verification result;

[0140] The second affiliation set is corrected according to the verification result to obtain a third affiliation set.

[0141] In an embodiment of the present invention, a preliminary identification of the relationship between pedestrians and accessories is first performed on target information observed in the current frame to obtain a first accessory relationship set. In order to improve the accuracy of accessory relationship identification, missed detection identification and accessory relationship correction are performed on the first accessory relationship set based on historical pedestrian information in the historical accessory relationship set, thereby obtaining a second accessory relationship set with further improved accuracy. Finally, the second accessory relationship set is verified based on the historical target trajectory, and a verified third accessory relationship set is generated. The present invention can improve the stability of accessory relationship identification.

[0142] above Figure 3 and Figure 4 The device for generating the dependency relationship in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The device for generating the dependency relationship in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0143] Figure 5 1 is a schematic diagram of the structure of a device for generating an affiliation relationship provided in an embodiment of the present invention. The device 500 for generating an affiliation relationship may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 may be temporary storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of computer program operations in the device 500 for generating an affiliation relationship. Furthermore, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of computer program operations in the storage medium 530 on the device 500 for generating an affiliation relationship.

[0144] The device 500 for generating the attachment relationship may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the device for generating a dependency relationship shown does not constitute a limitation on the device for generating a dependency relationship, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0145] The present invention also provides a computer device, which includes a memory and a processor, wherein a computer-readable computer program is stored in the memory, and when the computer-readable computer program is executed by the processor, the processor executes the steps of the method for generating the affiliation in the above-mentioned embodiments.

[0146] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program runs on a computer, the computer executes the steps of the method for generating the affiliation.

[0147] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0148] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several computer programs to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0149] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating affiliation, characterized in that: The method for generating the affiliation relationship includes: Determine, according to the target information observed in the current frame, a first set of affiliation relationships between target pedestrian information and target accessory information in the target information; Get the historical pedestrian information in the historical affiliation collection; Determine whether the historical pedestrian information has corresponding accessory information in the first accessory relationship set; If so, performing affiliation correction on the first affiliation set according to the distance between the historical pedestrian information and the corresponding affiliation information in the first affiliation set to obtain a second affiliation set; If not, performing missed detection identification on the first affiliation set according to the historical attachment information corresponding to the historical pedestrian information in the historical affiliation set to obtain a second affiliation set; According to the historical target trajectory, the second attachment relationship set is verified for consistency, and a third attachment relationship set is generated according to the verification result, wherein the historical target trajectory includes the historical pedestrian trajectory and the historical attachment trajectory.

2. The method for generating affiliation according to claim 1, characterized in that: The step of determining, based on the target information observed in the current frame, a first set of affiliation relationships between the target pedestrian information and the target accessory information in the target information includes: Obtaining target information observed in the current frame, and classifying the target information into target types, to obtain target pedestrian information and target accessory information in the target information; A first affiliation relationship set between the target pedestrian information and the target accessory information is determined according to a preset search distance threshold.

3. The method for generating affiliation according to claim 2, characterized in that: The determining, according to a preset search distance threshold, a first set of affiliation relationships between the target pedestrian information and the target accessory information comprises: Determining whether there is pedestrian information in the target accessory information in the target information within a preset search distance threshold range; If the target accessory information in the target information has pedestrian information within the preset search distance threshold range, a first accessory relationship set between the corresponding target accessory information and the nearest target pedestrian information is determined.

4. The method for generating affiliation according to claim 1, characterized in that: The step of performing affiliation correction on the first affiliation set according to the distance between the historical pedestrian information and the corresponding affiliation information in the first affiliation set to obtain a second affiliation set includes: Determining whether the distance between the historical pedestrian information and the corresponding accessory information in the first accessory relationship set is greater than a preset accessory relationship distance threshold; If so, the affiliation between the historical pedestrian information and the corresponding attachment information in the first affiliation set is cleared to obtain a second affiliation set.

5. The method for generating affiliation according to claim 1, characterized in that: The step of performing missed detection identification on the first affiliation set according to the historical affiliation information corresponding to the historical pedestrian information in the historical affiliation set to obtain a second affiliation set includes: Obtaining historical attachment information corresponding to the historical pedestrian information in the historical attachment relationship set; Determining whether the historical attachment information has corresponding pedestrian information in the first attachment relationship set; If so, pedestrian information corresponding to the historical attachment information in the first attachment relationship set is identified according to the historical target trajectory to obtain a second attachment relationship set.

6. The method for generating affiliation according to claim 5, characterized in that: The step of identifying pedestrian information corresponding to the historical attachment information in the first attachment relationship set according to the historical target trajectory to obtain a second attachment relationship set includes: Calculating an average distance between the historical pedestrian information and the corresponding historical accessory information according to the accessory trajectory corresponding to the historical accessory information and the pedestrian trajectory corresponding to the historical pedestrian information in the historical target trajectory; If the average distance is less than a preset average distance threshold, pedestrian information corresponding to the historical attachment information in the first attachment relationship set is determined to obtain a second attachment relationship set.

7. The method for generating affiliation according to claim 1, characterized in that: According to the historical target trajectory, consistency verification is performed on the second set of affiliations, and a third set of affiliations is generated according to the verification result, including: Performing historical target trajectory matching on the pedestrian information and accessory information in the second accessory relationship set to obtain a target pedestrian trajectory and a target accessory trajectory; Verifying the consistency of the target pedestrian trajectory and the target accessory trajectory with the historical target trajectory to obtain a verification result; The second affiliation set is corrected according to the verification result to obtain a third affiliation set.

8. A device for generating an affiliation relationship, characterized in that: The generation device of the affiliation relationship includes: A determination module, configured to determine, based on target information observed in a current frame, a first set of affiliation relationships between target pedestrian information and target accessory information in the target information; a correction module, configured to obtain historical pedestrian information in a historical attachment relationship set; determine whether the historical pedestrian information has corresponding attachment information in the first attachment relationship set; if so, perform attachment correction on the first attachment relationship set according to the distance between the historical pedestrian information and the corresponding attachment information in the first attachment relationship set to obtain a second attachment relationship set; if not, perform missed detection identification on the first attachment relationship set according to the historical attachment information corresponding to the historical pedestrian information in the historical attachment relationship set to obtain a second attachment relationship set; A verification module is used to verify the consistency of the second set of affiliations according to historical target trajectories, and generate a third set of affiliations according to the verification result, wherein the historical target trajectories include historical pedestrian trajectories and historical accessory trajectories.

9. A device for generating an affiliation relationship, characterized in that: The device for generating the affiliation relationship includes: a memory and at least one processor, wherein the memory stores a computer program; The at least one processor calls the computer program in the memory to enable the affiliation generation device to execute the affiliation generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for generating a dependency relationship as described in any one of claims 1 to 7 is implemented.

Images