A tracking method, device, vehicle and storage medium based on multi-model detection

Abstract

The present invention discloses a tracking method, device, vehicle and storage medium based on multi-model detection, the method comprising: obtaining sensor data of obstacles collected by unmanned driving equipment; detecting the sensor data and outputting a first tracking implicit characteristic of the obstacle; tracking the trajectory of the sensor data and outputting the trajectory implicit characteristic of the obstacle; inputting the first tracking implicit characteristic and the trajectory implicit characteristic into a first preset network model, and outputting a first metric of similarity between the tracking implicit characteristic and the trajectory implicit characteristic. Through the embodiments of the present invention, accurate tracking effect is achieved while providing output of the object credibility state; not only the characteristics of the object itself are considered, but also other objects and map information are included, and the extraction of case characteristics is more diversified; in terms of the types of features, not only case features are included, but also the features of grid points under global coordinates are included, so that the overall model has good generalization ability.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and in particular to a tracking method, device, vehicle and storage medium based on multi-model detection. Background Art

[0002] Unmanned driving object tracking refers to matching, predicting and determining the state of objects across frames based on the perception detection results of unmanned driving, and providing the trajectory information and confidence information of the objects. There are two main object tracking methods in the field of unmanned driving. One is to predict the position and state of objects based on Kalman filtering, match the results of object detection in different frames and estimate the trajectory state based on the predicted position and other information, and then use the predicted position or area overlap ratio as a metric to calculate the optimal matching result using a similar Hungarian algorithm. This method is very dependent on accurate estimation of speed, acceleration or target frame. If the perception detection has a relatively large error, the tracking result will have corresponding errors. The other is to calculate the similarity information between objects based on general object characteristics, obtain the similarity matrix based on the distance of similarity, and then use the optimal matching or greedy algorithm to calculate the tracking relationship between objects in different frames. This type of algorithm is relatively limited in terms of object feature extraction. Generally, it is difficult to give the optimal matching distance calculation based on heuristic methods; and the greedy algorithm sometimes cannot give the optimal matching option. In addition, the existing tracking models cannot combine trajectory information well and are strongly dependent on the detection results of the previous frame; they cannot provide good information about the existence of objects, which limits the task scope of the module; at the same time, the existing tracking algorithms cannot combine the outputs of dynamic and static objects at the same time. Summary of the invention

[0003] The main purpose of the present invention is to provide a tracking method, device, vehicle and storage medium based on multi-model detection, aiming to solve the problems in the prior art that it is impossible to track the input of multiple sensory detection models and it is impossible to track the coupled dynamic and static obstacles.

[0004] To achieve the above object, the present invention provides a tracking method based on multi-model detection, the method comprising the following steps:

[0005] Obtain sensor data of obstacles collected by unmanned driving equipment;

[0006] detecting the sensing data and outputting a first tracking implicit characteristic of the obstacle;

[0007] Tracking the sensor data and outputting implicit characteristics of the obstacle's trajectory;

[0008] The first tracking implicit characteristic and the trajectory implicit characteristic are input into a first preset network model, and a first metric of similarity between the tracking implicit characteristic and the trajectory implicit characteristic is output.

[0009] Optionally, the method further comprises the following steps:

[0010] Inputting the first tracking implicit characteristic and the trajectory implicit characteristic into a second preset network model, and outputting an updated second tracking implicit characteristic and a current trajectory confidence;

[0011] The second tracked implicit feature is used as an input of the first preset network model;

[0012] The current trajectory confidence is used to determine the matching degree between the obstacle and the trajectory;

[0013] The second preset network model is a long short-term memory deep learning model, which estimates the similarity and confidence across time by recording implicit characteristics.

[0014] Optionally, the detecting the sensor data and outputting a first tracking implicit characteristic of the obstacle comprises the following steps:

[0015] Performing machine learning-based detection on the sensor data to output object tracking implicit characteristics of the obstacle;

[0016] Performing rule-based detection on the sensor data and outputting grid tracking implicit characteristics of obstacles;

[0017] The first tracking implicit characteristics include: object tracking implicit characteristics and grid tracking implicit characteristics.

[0018] Optionally, the performing machine learning-based detection on the sensor data and outputting object tracking implicit characteristics of the obstacle comprises the following steps:

[0019] Performing machine learning-based detection on the sensor data and outputting display characteristics of obstacles; the machine learning-based detection includes a convolutional network-based and / or point-based network-based detection method;

[0020] Inputting the display characteristics of the obstacle into a second preset network model, and outputting the object tracking implicit characteristics of the obstacle;

[0021] The second preset network model uses the display characteristics of the obstacle to perform deep learning training based on the annotated data of multiple frames;

[0022] The deep learning training uses a long short-term memory model to compare the similarity between the implicit characteristics of an object and the implicit characteristics of a newly detected object.

[0023] Optionally, performing rule-based detection on the sensor data and outputting grid tracking implicit characteristics of obstacles comprises the following steps:

[0024] The rule-based detection includes an empirically based output based on objects above ground height, the output results including protruding objects, roadside objects and drivable areas;

[0025] The explicit features of the output results at different dimensions are converted into implicit features for matching through a specific feature extraction module.

[0026] Optionally, the object tracking implicit feature is an example-based feature extracted based on the display feature of the object combined with the combined information of the object and the map and the relationship information of other surrounding objects;

[0027] The grid tracking implicit feature is a feature generated based on grid information in a global world coordinate system; the grid information includes time information, features of obstacles occupied by the grid, attributes perceived by sensors contained in the grid, and attributes of the map contained in the grid.

[0028] Optionally, the method further comprises the following steps:

[0029] The first preset network model is a deep learning network model;

[0030] Normalizing the first metric to obtain a second metric so that the similarities have comparable sizes;

[0031] The second metric is used as a unit quantity of the matching algorithm to output the final matching result.

[0032] In addition, to achieve the above object, the present invention also proposes a tracking device based on multi-model detection, the tracking device based on multi-model detection comprising:

[0033] A data acquisition unit, used to acquire sensor data of obstacles collected by the unmanned driving equipment;

[0034] A data detection unit, used to detect the sensing data and output a first tracking implicit characteristic of the obstacle;

[0035] A trajectory tracking unit, used to track the trajectory of the sensor data and output the implicit trajectory characteristics of the obstacle;

[0036] The feature matching unit is used to input the first tracking implicit feature and the trajectory implicit feature into a first preset network model, and output a first metric of similarity between the tracking implicit feature and the trajectory implicit feature.

[0037] In addition, to achieve the above-mentioned objectives, the present invention also proposes a vehicle, which includes: a memory, a processor, and a tracking program based on multi-model detection stored in the memory and executable on the processor, wherein the tracking program based on multi-model detection is configured to implement the steps of the tracking method based on multi-model detection as described above.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the tracking method based on multi-model detection as described above are implemented.

[0039] The present invention can bring accurate tracking effect through multi-detection source and multi-task tracking module method, and provide output of object credibility status at the same time, and the two functions are completed simultaneously in one module. Not only the characteristics of the object itself are taken into account, but also other objects and map information are included; in the extraction of case characteristics, it is more diversified. In terms of feature types, there are not only case features, but also grid features under global coordinates, so that the overall model has good generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic flow chart of a tracking method based on multi-model detection provided by the present invention.

[0041] Figure 2 Another schematic flow chart of a tracking method based on multi-model detection provided by the present invention.

[0042] Figure 3 A schematic diagram of a flow chart for obtaining tracking implicit characteristics provided by the present invention.

[0043] Figure 4 A schematic diagram of a flow chart for obtaining implicit characteristics of object tracking provided by the present invention.

[0044] Figure 5 A schematic diagram of a flow chart for obtaining implicit characteristics of grid tracking provided by the present invention.

[0045] Figure 6 It is a structural block diagram of an embodiment of a tracking device based on multi-model detection of the present invention.

[0046] Figure 7 It is a schematic diagram of the structure of a transportation tool of the hardware operating environment involved in the embodiment of the present invention.

[0047] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0049] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0051] In one embodiment, Figure 1 As shown, the present invention provides a tracking method based on multi-model detection, the method comprising:

[0052] Step S101: Obtain sensor data of obstacles collected by unmanned driving equipment.

[0053] When an unmanned driving device (such as an unmanned vehicle) is driving, there will be various obstacles interacting with it. In order to ensure the safe driving of the vehicle, the vehicle will monitor the surrounding obstacles and obtain data such as the status information of each obstacle for analysis. The status information may include: the coordinates (x, y) of the obstacle, the speed (v) of the obstacle, the acceleration (a) of the obstacle, etc.

[0054] It should be noted that the obstacles interacting with the vehicle around it are changing dynamically, that is, if there are three obstacles a, b, and c interacting with the vehicle in a certain time period, then the obstacles interacting with the vehicle in another time period may become four obstacles a, c, d, and e. Therefore, the vehicle needs to monitor the obstacles interacting with it in real time and update the collected data in a timely manner.

[0055] Step S102: Detect the sensor data and output a first tracking implicit characteristic of the obstacle.

[0056] The obstacle sensing data collected by the unmanned driving equipment is detected to obtain the first tracking implicit characteristics including: Figure 3 The process shown.

[0057] Step S201: Perform machine learning-based detection on the sensor data and output object tracking implicit characteristics of the obstacle.

[0058] According to the obstacle sensing data collected by the unmanned driving equipment, the object tracking implicit characteristics of the obstacle are output based on machine learning detection. Figure 4 The process shown is completed.

[0059] Step S301: perform machine learning-based detection on the sensor data and output display characteristics of obstacles; the machine learning-based detection includes a convolutional network-based and / or point-based network-based detection method.

[0060] Step S302: input the display characteristics of the obstacle into a second preset network model, and output the object tracking implicit characteristics of the obstacle.

[0061] Step S303: The second preset network model uses the display characteristics of the obstacle to perform deep learning training based on the labeled data of multiple frames.

[0062] Step S304: The deep learning training uses a long short-term memory model to compare the similarity between the implicit characteristics of the object and the implicit characteristics of the newly detected object.

[0063] Machine learning-based detection includes detection methods based on convolutional networks and point networks. The main outputs are information such as the box, category, and location of the obstacle. The categories include various types of motor vehicles, pedestrians, bicycles, and various types of static obstacles on the road and on the roadside. The corresponding detection results will generate the output results of these detection models based on the data used for multi-frame annotation to train deep learning. The training will use a long short-term memory model to compare the similarity between the implicit characteristics of the tracked object and the implicit characteristics of the newly detected object. The final cross-frame matching result is completed through the predicted similarity. The display characteristics such as box, location, and category will be used as input, and then the statistical normalization results will be given to generate implicit features with sufficient digital length through the feature extraction module. The implicit feature will be output for subsequent feature matching processes.

[0064] The implicit feature of object tracking is a case-based feature extracted based on some displayed features of the object, such as frame, position, type, speed, direction, etc., combined with the combined information of the object and the map and the relationship information of other objects around. In addition to the original numbers, these features will be processed nonlinearly to achieve a better search of the feature space. After passing through the feature extraction module, each object instance has a dimension of 2561024. At the same time, these features have been normalized. Avoid abnormal data values ​​generated by the deep learning prediction module.

[0065] Step S202: Perform rule-based detection on the sensor data and output implicit grid tracking characteristics of obstacles.

[0066] According to the obstacle sensor data collected by the unmanned driving equipment, the grid tracking implicit characteristics of the obstacle are output based on rule detection. Figure 5 The process shown is completed.

[0067] Step S401, the rule-based detection includes output based on experience of objects above the ground, and the output results include protruding objects, roadside objects and drivable areas.

[0068] Step S402: converting the explicit features of the output results at different dimensions into implicit features for matching through a specific feature extraction module.

[0069] Rule-based detection includes empirically based outputs based on objects above the ground, including outputs for protruding objects, roadside objects, and drivable areas. These outputs do not have a fixed and unified format. Through a specific feature extraction module, the explicit features of such objects at different latitudes are converted into implicit features for matching. Generally, rule-based detection has the characteristics of high output recall and does not need to distinguish object types. Here, the possibility of missed detection that may be caused by machine learning-based detection methods is taken into account to ensure the continuity of the trajectory. This part will also generate the required implicit features through discrete feature extraction modules.

[0070] The implicit characteristics of grid tracking are based on the grid information in the global world coordinate system. The grid information includes time information, the characteristics of the obstacles occupied by the grid, the sensor perception attributes contained in the grid, and the map attributes contained in the grid. By tracking the implicit characteristics of each grid point, the grid information is reflected on the instance characteristics of the object through attention. This makes the objects and tracking characteristics used in the calculation have global perception capabilities.

[0071] The first tracking implicit characteristic includes: object tracking implicit characteristic and grid tracking implicit characteristic.

[0072] Step S103: Track the trajectory of the sensor data and output the implicit trajectory characteristics of the obstacle.

[0073] According to the historical trajectory information and current trajectory information of the obstacle, the implicit trajectory characteristics of the obstacle are obtained. The implicit trajectory characteristics are based on some displayed characteristics of the historical trajectory of the obstacle, such as frame, position, type, speed, direction and other different types of information, combined with the combined information of the object and the map and the relationship information of other objects around it, to extract a case-based feature.

[0074] Step S104: input the first tracking implicit characteristic and the trajectory implicit characteristic into a first preset network model, and output a first metric of similarity between the tracking implicit characteristic and the trajectory implicit characteristic.

[0075] The first preset network model is a deep learning network model; the first metric is normalized to obtain a second metric so that the similarity has comparable size; the second metric is used as a unit quantity of the matching algorithm to output the final matching result.

[0076] Similarity matching includes calculating a similarity metric based on the first tracking implicit characteristics generated by detection and the trajectory implicit characteristics generated by tracking. The parameters of this metric are not limited to matrices in the usual sense, but are generated through deep learning. When the first tracking implicit characteristics and the trajectory implicit characteristics are used as input, the similarity information is output in the form of a number. The output digital information will be used as a unit quantity of the matching algorithm (such as the Hungarian algorithm and the KM algorithm) to output the final matching result. Here, the metric based on deep learning is normalized so that the similarity has comparable size, which is convenient for calculating the optimal result of the whole play.

[0077] This embodiment uses a multi-detection source, multi-task tracking module method to achieve accurate tracking results and provide output of the object's credibility status. Both functions are completed in one module. Not only the characteristics of the object itself are taken into account, but also other objects and map information; the extraction of case characteristics is more diverse. In terms of feature types, there are not only case features, but also grid features under global coordinates, which makes the model as a whole have good generalization ability.

[0078] In one embodiment, Figure 2 Provided Figure 1 The specific process of updating the implicit characteristics after step S104 of the embodiment.

[0079] Step S105: input the first tracking implicit characteristic and the trajectory implicit characteristic into a second preset network model, and output an updated second tracking implicit characteristic and a current trajectory confidence.

[0080] Implicit feature updates include the first tracking implicit features generated based on detection and the trajectory implicit features generated based on tracking, which are used as the input for updating the deep learning time series model, and generate implicit tracking features for the input of the next frame matching model. At the same time, the update process will generate data on the current trajectory confidence, which is used to judge the degree of match between obstacles and trajectories. This optimizes the accuracy index of the final output. The update process here uses a deep learning model with long short-term memory, and estimates the similarity and confidence across time through the recorded implicit features. This update process enables the overall module to have the multi-task capability of predicting similarity and predicting confidence at the same time. It provides accuracy optimization while ensuring the recall rate.

[0081] The update process of this embodiment uses a long short-term memory deep learning model, and estimates the similarity and confidence across time by recording implicit characteristics. This update process enables the overall module to have the multi-task ability of predicting similarity and predicting confidence at the same time. It also provides accuracy optimization while ensuring the recall rate.

[0082] In addition, the embodiment of the present invention also proposes a tracking device based on multi-model detection, referring to Figure 5 , the tracking device based on multi-model detection includes:

[0083] A data acquisition unit 10 is used to acquire sensor data of obstacles collected by the unmanned driving equipment;

[0084] A data detection unit 20, configured to detect the sensing data and output a first tracking implicit characteristic of the obstacle;

[0085] A trajectory tracking unit 30, used to track the trajectory of the sensor data and output the implicit trajectory characteristics of the obstacle;

[0086] The feature matching unit 40 is used to input the first tracking implicit feature and the trajectory implicit feature into a first preset network model, and output a first metric of similarity between the tracking implicit feature and the trajectory implicit feature.

[0087] This embodiment uses a multi-detection source, multi-task tracking module method to achieve accurate tracking results and provide output of the object's credibility status. Both functions are completed in one module. Not only the characteristics of the object itself are taken into account, but also other objects and map information; the extraction of case characteristics is more diverse. In terms of feature types, there are not only case features, but also grid features under global coordinates, which makes the model as a whole have good generalization ability.

[0088] It should be noted that each unit in the above-mentioned device can be used to implement each step in the above-mentioned method and achieve corresponding technical effects, which will not be described in detail in this embodiment.

[0089] Reference Figure 6 , Figure 6 A schematic diagram of the structure of a vehicle in the hardware operating environment involved in an embodiment of the present invention.

[0090] like Figure 6As shown, the vehicle may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as WI-FI, 4G, 5G interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0091] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the vehicle, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0092] like Figure 6 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a tracking program based on multi-model detection.

[0093] exist Figure 6 In the vehicle shown, the network interface 1004 is mainly used for data communication with an external network; the user interface 1003 is mainly used for receiving user input instructions; the vehicle calls the tracking program based on multi-model detection stored in the memory 1005 through the processor 1001, and performs the following operations:

[0094] To achieve the above object, the present invention provides a tracking method based on multi-model detection, the method comprising the following steps:

[0095] Obtain sensor data of obstacles collected by unmanned driving equipment;

[0096] detecting the sensing data and outputting a first tracking implicit characteristic of the obstacle;

[0097] Tracking the sensor data and outputting implicit characteristics of the obstacle's trajectory;

[0098] The first tracking implicit characteristic and the trajectory implicit characteristic are input into a first preset network model, and a first metric of similarity between the tracking implicit characteristic and the trajectory implicit characteristic is output.

[0099] Optionally, the method further comprises the following steps:

[0100] Inputting the first tracking implicit characteristic and the trajectory implicit characteristic into a second preset network model, and outputting an updated second tracking implicit characteristic and a current trajectory confidence;

[0101] The second tracked implicit feature is used as an input of the first preset network model;

[0102] The current trajectory confidence is used to determine the matching degree between the obstacle and the trajectory;

[0103] The second preset network model is a long short-term memory deep learning model, which estimates the similarity and confidence across time by recording implicit characteristics.

[0104] Optionally, the detecting the sensor data and outputting a first tracking implicit characteristic of the obstacle comprises the following steps:

[0105] Performing machine learning-based detection on the sensor data to output object tracking implicit characteristics of the obstacle;

[0106] Performing rule-based detection on the sensor data and outputting grid tracking implicit characteristics of obstacles;

[0107] The first tracking implicit characteristics include: object tracking implicit characteristics and grid tracking implicit characteristics.

[0108] Optionally, the performing machine learning-based detection on the sensor data and outputting object tracking implicit characteristics of the obstacle comprises the following steps:

[0109] Performing machine learning-based detection on the sensor data and outputting display characteristics of obstacles; the machine learning-based detection includes a convolutional network-based and / or point-based network-based detection method;

[0110] Inputting the display characteristics of the obstacle into a second preset network model, and outputting the object tracking implicit characteristics of the obstacle;

[0111] The second preset network model uses the display characteristics of the obstacle to perform deep learning training based on the annotated data of multiple frames;

[0112] The deep learning training uses a long short-term memory model to compare the similarity between the implicit characteristics of an object and the implicit characteristics of a newly detected object.

[0113] Optionally, performing rule-based detection on the sensor data and outputting grid tracking implicit characteristics of obstacles comprises the following steps:

[0114] The rule-based detection includes an empirically based output based on objects above ground height, the output results including protruding objects, roadside objects and drivable areas;

[0115] The explicit features of the output results at different dimensions are converted into implicit features for matching through a specific feature extraction module.

[0116] Optionally, the object tracking implicit feature is an example-based feature extracted based on the display feature of the object combined with the combined information of the object and the map and the relationship information of other surrounding objects;

[0117] The grid tracking implicit feature is a feature generated based on grid information in a global world coordinate system; the grid information includes time information, features of obstacles occupied by the grid, attributes perceived by sensors contained in the grid, and attributes of the map contained in the grid.

[0118] Optionally, the method further comprises the following steps:

[0119] The first preset network model is a deep learning network model;

[0120] Normalizing the first metric to obtain a second metric so that the similarities have comparable sizes;

[0121] The second metric is used as a unit quantity of the matching algorithm to output the final matching result.

[0122] This embodiment uses a multi-detection source, multi-task tracking module method to achieve accurate tracking results and provide output of the object's credibility status. Both functions are completed in one module. Not only the characteristics of the object itself are taken into account, but also other objects and map information; the extraction of case characteristics is more diverse. In terms of feature types, there are not only case features, but also grid features under global coordinates, which makes the model as a whole have good generalization ability.

[0123] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a tracking program based on multi-model detection is stored. When the tracking program based on multi-model detection is executed by a processor, the following operations are implemented:

[0124] Obtain sensor data of obstacles collected by unmanned driving equipment;

[0125] detecting the sensing data and outputting a first tracking implicit characteristic of the obstacle;

[0126] Tracking the sensor data and outputting implicit characteristics of the obstacle's trajectory;

[0127] The first tracking implicit characteristic and the trajectory implicit characteristic are input into a first preset network model, and a first metric of similarity between the tracking implicit characteristic and the trajectory implicit characteristic is output.

[0128] Optionally, the method further comprises the following steps:

[0129] Inputting the first tracking implicit characteristic and the trajectory implicit characteristic into a second preset network model, and outputting an updated second tracking implicit characteristic and a current trajectory confidence;

[0130] The second tracked implicit feature is used as an input of the first preset network model;

[0131] The current trajectory confidence is used to determine the matching degree between the obstacle and the trajectory;

[0132] The second preset network model is a long short-term memory deep learning model, which estimates the similarity and confidence across time by recording implicit characteristics.

[0133] Optionally, the detecting the sensor data and outputting a first tracking implicit characteristic of the obstacle comprises the following steps:

[0134] Performing machine learning-based detection on the sensor data to output object tracking implicit characteristics of the obstacle;

[0135] Performing rule-based detection on the sensor data and outputting grid tracking implicit characteristics of obstacles;

[0136] The first tracking implicit characteristics include: object tracking implicit characteristics and grid tracking implicit characteristics.

[0137] Optionally, the performing machine learning-based detection on the sensor data and outputting object tracking implicit characteristics of the obstacle comprises the following steps:

[0138] Performing machine learning-based detection on the sensor data and outputting display characteristics of obstacles; the machine learning-based detection includes a convolutional network-based and / or point-based network-based detection method;

[0139] Inputting the display characteristics of the obstacle into a second preset network model, and outputting the object tracking implicit characteristics of the obstacle;

[0140] The second preset network model uses the display characteristics of the obstacle to perform deep learning training based on the annotated data of multiple frames;

[0141] The deep learning training uses a long short-term memory model to compare the similarity between the implicit characteristics of an object and the implicit characteristics of a newly detected object.

[0142] Optionally, performing rule-based detection on the sensor data and outputting grid tracking implicit characteristics of obstacles comprises the following steps:

[0143] The rule-based detection includes an empirically based output based on objects above ground height, the output results including protruding objects, roadside objects and drivable areas;

[0144] The explicit features of the output results at different dimensions are converted into implicit features for matching through a specific feature extraction module.

[0145] Optionally, the object tracking implicit feature is an example-based feature extracted based on the display feature of the object combined with the combined information of the object and the map and the relationship information of other surrounding objects;

[0146] The grid tracking implicit feature is a feature generated based on grid information in a global world coordinate system; the grid information includes time information, features of obstacles occupied by the grid, attributes perceived by sensors contained in the grid, and attributes of the map contained in the grid.

[0147] Optionally, the method further comprises the following steps:

[0148] The first preset network model is a deep learning network model;

[0149] Normalizing the first metric to obtain a second metric so that the similarities have comparable sizes;

[0150] The second metric is used as a unit quantity of the matching algorithm to output the final matching result.

[0151] This embodiment uses a multi-detection source, multi-task tracking module method to achieve accurate tracking results and provide output of the object's credibility status. Both functions are completed in one module. Not only the characteristics of the object itself are taken into account, but also other objects and map information; the extraction of case characteristics is more diverse. In terms of feature types, there are not only case features, but also grid features under global coordinates, which makes the model as a whole have good generalization ability.

[0152] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0153] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0154] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, controller, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0155] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A tracking method based on multi-model detection, characterized in that: The method comprises the following steps: Obtain sensor data of obstacles collected by unmanned driving equipment; detecting the sensing data and outputting a first tracking implicit characteristic of the obstacle; Tracking the sensor data and outputting implicit characteristics of the obstacle's trajectory; Inputting the first tracking implicit characteristic and the trajectory implicit characteristic into a first preset network model, and outputting a first metric of similarity between the tracking implicit characteristic and the trajectory implicit characteristic; Wherein, the first tracking implicit characteristic includes: object tracking implicit characteristic and grid tracking implicit characteristic; The object tracking implicit feature is an example-based feature extracted based on the display feature of the object combined with the combined information of the object and the map and the relationship information of other objects around it; The grid tracking implicit feature is a feature generated based on the grid information in the global world coordinate system; the grid information also includes time information, characteristics of obstacles occupied by the grid, attributes perceived by sensors contained in the grid, and attributes of the map contained in the grid.

2. The method according to claim 1, characterized in that: The method further comprises the following steps: Inputting the first tracking implicit characteristic and the trajectory implicit characteristic into a second preset network model, and outputting an updated second tracking implicit characteristic and a current trajectory confidence; The second tracked implicit feature is used as an input of the first preset network model; The current trajectory confidence is used to determine the matching degree between the obstacle and the trajectory; The second preset network model is a long short-term memory deep learning model, which estimates the similarity and confidence across time by recording implicit characteristics.

3. The method according to claim 1, characterized in that: The detecting the sensor data and outputting the first tracking implicit characteristic of the obstacle comprises the following steps: Performing machine learning-based detection on the sensor data to output object tracking implicit characteristics of the obstacle; The sensor data is subjected to rule-based detection, and grid tracking implicit characteristics of obstacles are output.

4. The method according to claim 3, characterized in that: The detecting of the sensor data based on machine learning and outputting the object tracking implicit characteristics of the obstacle comprises the following steps: Performing machine learning-based detection on the sensor data and outputting display characteristics of obstacles; the machine learning-based detection includes a convolutional network-based and / or point-based network-based detection method; Inputting the display characteristics of the obstacle into a second preset network model, and outputting the object tracking implicit characteristics of the obstacle; The second preset network model uses the display characteristics of the obstacle to perform deep learning training based on the annotated data of multiple frames; The deep learning training uses a long short-term memory model to compare the similarity between the implicit characteristics of an object and the implicit characteristics of a newly detected object.

5. The method according to claim 3, characterized in that: Performing rule-based detection on the sensor data and outputting grid tracking implicit characteristics of obstacles includes the following steps: The rule-based detection includes an empirically based output based on objects above ground height, the output results including protruding objects, roadside objects and drivable areas; The explicit features of the output results at different dimensions are converted into implicit features for matching through a specific feature extraction module.

6. The method according to claim 1, characterized in that: The method further comprises the following steps: The first preset network model is a deep learning network model; Normalizing the first metric to obtain a second metric so that the similarities have comparable sizes; The second metric is used as a unit quantity of the matching algorithm to output the final matching result.

7. A tracking device based on multi-model detection, characterized in that: The tracking device based on multi-model detection includes: A data acquisition unit, used to acquire sensor data of obstacles collected by the unmanned driving equipment; A data detection unit, used to detect the sensing data and output a first tracking implicit characteristic of the obstacle; A trajectory tracking unit, used to track the trajectory of the sensor data and output the implicit trajectory characteristics of the obstacle; a characteristic matching unit, configured to input the first tracking implicit characteristic and the trajectory implicit characteristic into a first preset network model, and output a first metric of similarity between the tracking implicit characteristic and the trajectory implicit characteristic; Wherein, the first tracking implicit characteristic includes: object tracking implicit characteristic and grid tracking implicit characteristic; The object tracking implicit feature is an example-based feature extracted based on the display feature of the object combined with the combined information of the object and the map and the relationship information of other objects around it; The grid tracking implicit feature is a feature generated based on the grid information in the global world coordinate system; the grid information also includes time information, characteristics of obstacles occupied by the grid, attributes perceived by sensors contained in the grid, and attributes of the map contained in the grid.

8. A means of transport, characterized in that: The vehicle comprises: a memory, a processor, and a tracking program based on multiple model detection stored in the memory and executable on the processor, wherein the tracking program based on multiple model detection is configured to implement the steps of the tracking method based on multiple model detection according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the tracking method based on multi-model detection according to any one of claims 1 to 6 are implemented.

Images