Intelligent 3D multi-target tracking method and system based on vehicle-cloud cooperation

Abstract

The application discloses a kind of intelligent 3D multi-target tracking method and system of car cloud cooperation, and the method comprises: detecting vehicle target, obtains the detection state of each vehicle under initial time scene, establishes initial track information;According to the target information of the tracking result under initial time scene, the state of target time is predicted by Kalman filtering algorithm;Vehicle object is detected, the detection state of each vehicle is obtained, and then the data correlation degree between the detection frame information of all detection objects under target time and the predicted state of target under initial time is calculated;According to data correlation degree, similarity matching is carried out to all detection objects and predicted object, and matching result is obtained;Target state management is carried out to matching result, until tracking target is completed.The application can quickly complete tracking confirmation to multiple target vehicles, improves the accuracy of tracking prediction based on vehicle cloud cooperation vehicle data, and can be widely applied in vehicle cloud cooperation control technical field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle-cloud collaborative control, and in particular to a vehicle-cloud collaborative intelligent 3D multi-target tracking method and system. BACKGROUND

[0002] In vehicle-road-cloud collaborative and automatic driving tasks, a 3-dimensional (3D) target tracking method is widely used to capture dynamic information of vehicles and pedestrians in a complex scene, and can also provide accurate position and acceleration and other motion information of objects in the scene.

[0003] According to different data used, current tracking methods can be divided into three categories: image data-based tracking methods, 3D tracking methods based on radar point cloud data, and multi-modal tracking methods based on image and radar data fusion. Although the image data-based tracking method has achieved a lot of significant results, the image data-based tracking method cannot effectively extract and utilize 3D data information to perceive stereo information and complex scenes due to the lack of depth information of the image data. Compared with 2D image data, 3D point cloud data can supplement more rich stereo scene information. The tracking method based on multi-modal data makes full use of image data and point cloud data to achieve more accurate and more robust tracking effect; but in most multi-modal tracking methods, data needs to be calibrated for different modal data fusion, and in actual engineering applications, it is difficult to effectively maintain the stability of the calibration parameters for a long time. Therefore, the tracking method based on 2D image data and multi-modal data fusion has certain limitations in the actual engineering background. The target tracking method based on point cloud data can supplement stereo scene information and enhance tracking effect while ensuring the stability of the method in actual engineering applications.

[0004] Current tracking methods can be roughly divided into two categories according to stages: end-to-end tracking methods and detection-tracking two-stage tracking methods. In the end-to-end method, all modules are integrated into a complete process, which optimizes the learning process of the algorithm, but the final sub-module lacks certain theoretical support, which leads to the problem that its safety and reliability is a matter of concern. Current 3D target detection methods have significantly improved effect, which can provide sufficient reliable information for target tracking, so the two-stage tracking method is more valuable in actual application. SUMMARY

[0005] Therefore, the embodiments of the present application provide a fast and accurate vehicle-cloud collaborative intelligent 3D multi-target tracking method and system.

[0006] An aspect of the embodiments of the present application provides a vehicle-cloud collaborative intelligent 3D multi-target tracking method, comprising:

[0007] Detecting a vehicle target, obtaining a detection state of each vehicle in an initial time scene, and then establishing initial track information according to the detection state and a vehicle identity; wherein the detection state comprises vehicle detection box information and a detection confidence score;

[0008] According to the target information of the tracking result in the initial time scene, predicting a state of a target time through a Kalman filtering algorithm;

[0009] Detecting a vehicle object, obtaining a detection state of each vehicle in a target time scene, and then calculating a data correlation degree between detection box information of all detection objects in the target time and a predicted state of the target in the initial time;

[0010] According to the data correlation degree, performing similarity matching on all detection objects and predicted objects to obtain a matching result;

[0011] Managing the matching result to complete tracking of the target.

[0012] Optionally, the detecting a vehicle object, obtaining a detection state of each vehicle in a target time scene, and then calculating a data correlation degree between detection box information of all detection objects in the target time and a predicted state of the target in the initial time comprises:

[0013] According to the detection states of the vehicle object in the front and rear time, estimating a speed of the vehicle object;

[0014] According to the estimated speeds in the front and rear time and detection angle information, determining a measure of motion difference between the front and rear time;

[0015] According to the detection information in the target time scene and the predicted state, determining a measure of geometric difference;

[0016] According to the measure of motion difference and the measure of geometric difference, determining a similarity measure of the target in the front and rear time;

[0017] The motion difference comprises an angle difference of speed between the front and rear time, a numerical difference of speed between the front and rear time, and an offset of vehicle orientation between the front and rear time.

[0018] Optionally, the step of performing similarity matching on all detection objects and predicted objects according to the data correlation degree to obtain a matching result specifically comprises: generating a tracking matching result according to the similarity measures of all targets, comprising:

[0019] Obtaining the similarity measures of all detection targets;

[0020] According to the similarity measures, matching all detection targets in the front and rear time through a Hungarian algorithm to obtain a tracking result.

[0021] Optionally, the formula for calculating the estimated speed of the vehicle object is:

[0022]

[0023] wherein, represents the estimated speed of the vehicle object; t represents the time; (x t ,y t ) represents the spatial position of the detection target at time t; represents the spatial position vector differentiated with respect to time.

[0024] Optionally, the expression for measuring the motion difference between the previous and subsequent time is:

[0025]

[0026] wherein, S M represents the measurement of the motion difference between the previous and subsequent time; represents the estimated speed of the vehicle object at time t; represents the estimated speed of the vehicle object at time t-1; θ t represents the orientation angle of the vehicle.

[0027] Optionally, the target state management on the matching result comprises:

[0028] obtaining the matching result in the data association stage;

[0029] when the matching is successful, configuring the state of the vehicle object as Tracking, and assigning the same identity to the detection target and the prediction target;

[0030] when the matching fails, if the detection target comes from the previous time, updating the life cycle of the detection target; if the target comes from the subsequent time, confirming that the target just appears in the tracking field of view, and configuring the identity, life cycle and state of the target;

[0031] completing the tracking task of the target according to the life cycle of the detection target.

[0032] Optionally, the completing the tracking task of the target according to the life cycle of the detection target comprises:

[0033] when the life cycle of the detection target is less than 1, configuring the state of the detection target as Death;

[0034] when the life cycle of the detection target is greater than or equal to 1, configuring the state of the detection target as Miss.

[0035] Optionally, the formula for calculating the life cycle of the target when the matching fails is:

[0036]

[0037] Wherein, life' represents the life cycle of the target of the matching failure; life represents a fixed life cycle threshold; a represents a scale factor; and b represents an offset factor; C t represents a confidence score of the target being successfully detected.

[0038] Another aspect of the embodiment of the present application further provides a vehicle-cloud cooperative intelligent 3D multi-target tracking system, comprising:

[0039] A first module is configured to detect vehicle targets, acquire detection states of each vehicle in an initial time scene, and then establish initial track information according to the detection states and vehicle identities; wherein the detection states include vehicle detection box information and detection confidence scores.

[0040] A second module is configured to predict the state of a target in a target time according to target information of a tracking result in the initial time scene through a Kalman filtering algorithm.

[0041] A third module is configured to detect vehicle objects, acquire detection states of each vehicle in a target time scene, and then calculate a data association degree between detection box information of all detection objects in the target time and the predicted state of the target in the initial time.

[0042] A fourth module is configured to perform similarity matching on all detection objects and predicted objects according to the data association degree, and obtain a matching result.

[0043] A fifth module is configured to perform target state management on the matching result until the tracking target is completed.

[0044] Another aspect of the embodiment of the present application further provides an electronic device, comprising a processor and a memory.

[0045] The memory is configured to store a program.

[0046] The processor executes the program to implement the method as described above.

[0047] The embodiment of the present application further discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method as described above.

[0048] The embodiment of the application detects vehicle targets, obtains detection states of each vehicle in an initial time scene, and then establishes initial trajectory information according to the detection states and vehicle identities; according to target information of tracking results in the initial time scene, the state of a target time is predicted through a Kalman filtering algorithm; vehicle objects are detected, detection states of each vehicle in a target time scene are obtained, and then data correlation degrees between detection box information of all detection objects in the target time and the predicted state of the target in the initial time are calculated; according to the data correlation degrees, similarity matching is performed on all detection objects and predicted objects, and a matching result is obtained; target state management is performed on the matching result until tracking targets are completed. The application can quickly complete tracking confirmation of multiple target vehicles, improves the accuracy of tracking prediction based on vehicle cloud collaborative vehicle data. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0050] Fig. 1 The overall step flowchart provided by the embodiment of the present application is provided.

[0051] Fig. 2 The target life cycle and state adaptive management flowchart based on detection confidence provided by the present application is provided.

[0052] Fig. 3 The intelligent 3D multi-target tracking system flowchart provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0054] In view of the problems in the prior art, the present application realizes multi-target tracking by adopting a two-stage tracking paradigm based on single point cloud data. In addition, the present application also supports online updating of a better target detection model in combination with the data interaction advantages between a vehicle cloud collaborative system, and realizes online adjustment and upgrading functions of an offline tracking system.

[0055] Specifically, the present application provides an intelligent 3D multi-target tracking method based on vehicle cloud collaboration, comprising:

[0056] Detecting a vehicle target, obtaining a detection state of each vehicle in an initial time scene, and then establishing initial track information according to the detection state and a vehicle identity; wherein the detection state comprises vehicle detection box information and a detection confidence score;

[0057] According to the target information of the tracking result in the initial time scene, predicting a state of a target time through a Kalman filtering algorithm;

[0058] Detecting a vehicle object, obtaining a detection state of each vehicle in a target time scene, and then calculating a data correlation degree between detection box information of all detection objects in the target time and a predicted state of the target in the initial time;

[0059] According to the data correlation degree, performing similarity matching on all detection objects and predicted objects to obtain a matching result;

[0060] Managing the target state of the matching result until the tracking target is completed.

[0061] Optionally, the detecting a vehicle object, obtaining a detection state of each vehicle in a target time scene, and then calculating a data correlation degree between detection box information of all detection objects in the target time and a predicted state of the target in the initial time comprises:

[0062] According to the detection states of the vehicle object in the front and rear time, estimating a speed of the vehicle object;

[0063] According to the estimated speeds in the front and rear time and detection angle information, determining a measure of motion difference between the front and rear time;

[0064] According to the detection information in the target time scene and the predicted state, determining a measure of geometric difference;

[0065] According to the measure of motion difference and the measure of geometric difference, determining a similarity measure of the target in the front and rear time;

[0066] The motion difference comprises an angle difference of the speed between the front and rear time, a numerical difference of the speed between the front and rear time, and an offset of the vehicle orientation between the front and rear time.

[0067] Optionally, the step of performing similarity matching on all detection objects and predicted objects according to the data correlation degree to obtain a matching result specifically comprises: generating a tracking matching result according to the similarity measures of all targets, comprising:

[0068] Obtaining the similarity measures of all detection targets;

[0069] According to the similarity measures, matching all detection targets in the front and rear time through a Hungarian algorithm to obtain a tracking result.

[0070] Optionally, the calculation formula of the estimated speed of the vehicle object is:

[0071]

[0072] wherein, represents the estimated speed of the vehicle object; t represents the time; (x t ,y t ) represents the spatial position of the detection target at time t; represents the spatial position vector operation of time differentiation.

[0073] Optionally, the expression of the measurement of the motion difference between the front and rear time is:

[0074]

[0075] wherein, S M represents the measurement of the motion difference between the front and rear time; represents the estimated speed of the vehicle object at time t; represents the estimated speed of the vehicle object at time t-1; θ t represents the orientation angle of the vehicle.

[0076] Optionally, the target state management on the matching result comprises:

[0077] obtaining the matching result in the data association stage;

[0078] when the matching is successful, configuring the state of the vehicle object as Tracking, and assigning the same identity to the detection target and the prediction target;

[0079] when the matching fails, if the detection target comes from the front time, updating the life cycle of the detection target; if the target comes from the rear time, confirming that the target just appears in the tracking field of view, and configuring the identity, life cycle and state of the target;

[0080] completing the tracking task of the target according to the life cycle of the detection target.

[0081] Optionally, the completing the tracking task of the target according to the life cycle of the detection target comprises:

[0082] when the life cycle of the detection target is less than 1, configuring the state of the detection target as Death;

[0083] when the life cycle of the detection target is greater than or equal to 1, configuring the state of the detection target as Miss.

[0084] Optionally, the calculation formula of the life cycle of the matching failed target is:

[0085]

[0086] wherein, life' represents the life cycle of the target of the matching failure; life represents a fixed life cycle threshold; a represents a scale factor; β represents an offset factor; C t represents a confidence score of the target being successfully detected.

[0087] Another aspect of the embodiment of the present application further provides a vehicle-cloud cooperative intelligent 3D multi-target tracking system, comprising:

[0088] A first module is configured to detect vehicle targets, acquire detection states of each vehicle in an initial time scene, and then establish initial track information according to the detection states and vehicle identities; wherein the detection states comprise vehicle detection box information and detection confidence scores.

[0089] A second module is configured to predict the state of a target in a target time according to target information of a tracking result in the initial time scene through a Kalman filtering algorithm.

[0090] A third module is configured to detect vehicle objects, acquire detection states of each vehicle in a target time scene, and then calculate a data association degree between detection box information of all detection objects in the target time and the predicted state of the target in the initial time.

[0091] A fourth module is configured to perform similarity matching on all detection objects and predicted objects according to the data association degree, and obtain a matching result.

[0092] A fifth module is configured to perform target state management on the matching result until the tracking target is completed.

[0093] Another aspect of the embodiment of the present application further provides an electronic device comprising a processor and a memory.

[0094] The memory is configured to store a program.

[0095] The processor executes the program to implement the method as described above.

[0096] The embodiment of the present application further discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method as described above.

[0097] The specific implementation process of the present application will be described in detail in combination with the accompanying drawings of the specification:

[0098] This invention proposes a vehicle-cloud collaborative intelligent 3D multi-target tracking method and online update system, which solves the problems that 2D image data cannot provide scene stereo information and multimodal data tracking is difficult to use in practical applications, and supports online calibration and upgrade of offline tracking systems.

[0099] A vehicle-cloud collaborative intelligent 3D multi-target tracking method is implemented through the following steps:

[0100] Step 1: Use the target detection module to detect vehicle targets and obtain the detection state D of each vehicle in the scene at the initial time t-1. t-1 This includes vehicle detection frame information (size, position, orientation) B. t-1 And the detection confidence score C t-1 It also initializes the vehicle target identity ID and establishes initial trajectory information.

[0101] Step 2: Using the Kalman filter algorithm, target information B from the tracking results at time t-1 is used. t-1 Predict its state P at time t t .

[0102] Step 3: Use the vehicle detection module to obtain the vehicle detection status D in the scene at time t. t Based on the proposed metric method based on target motion and geometric information, the detection box information B of all detected targets at time t is calculated. t and the predicted state P of the target at time t-1 t The degree of data correlation S between them.

[0103] Step 4: Based on the obtained data correlation S, the Hungarian algorithm is used to perform similarity matching on all detected targets and predicted targets to obtain matching results.

[0104] Step 5: The adaptive target state management module is used to manage the vehicle target state of the matching results. If the match is successful, the target state is set to Tracking, and the detected target is assigned the same identity ID as the predicted target; if the match fails, the state is managed according to its lifespan.

[0105] Step 6: Repeat steps 2 through 5 until the tracking is complete.

[0106] A smart 3D multi-target tracking system that supports vehicle-cloud collaborative online updates includes a vehicle-cloud collaborative online update module, a vehicle target detection module, a vehicle identity correlation calculation module, a vehicle identity matching module, and a vehicle status management module.

[0107] The vehicle cloud cooperation online updating module adjusts and upgrades the vehicle target detection pre-training model in the offline tracking system through the cloud training of the latest and optimal vehicle target detection model.

[0108] The vehicle detection module detects the vehicle target in the input 3D radar point cloud frame data by using the pre-trained vehicle target detection model, and obtains the 3D detection frame, the orientation angle and the detection confidence score of the vehicle.

[0109] The vehicle identity correlation degree calculation module is used for calculating the identity correlation degree between vehicles at different time points. Specifically, according to the target similarity proposed in the application, the similarity matrix between all targets at different time points is calculated.

[0110] The vehicle identity matching module, based on the similarity matrix between targets at different time points obtained by the vehicle identity correlation degree calculation module, uses the Hungarian algorithm to perform maximum matching between targets at different time points, and realizes vehicle identity correlation.

[0111] The vehicle state management module performs adaptive management of the life cycle and state of the vehicle target in the tracking process based on the vehicle identity correlation result at different time points. Specifically, the vehicle state management is performed according to the management strategy proposed in the application, and the final tracking result is updated.

[0112] Specifically, the complete implementation process of the embodiment is explained in combination with Fig. 1 , Fig. 2 and Fig. 3 .

[0113] A vehicle cloud cooperation intelligent 3D multi-target tracking algorithm, specifically including three specific algorithm processes of target (vehicle) detection, target (vehicle) identity correlation and target (vehicle) state management.

[0114] I. Target detection

[0115] The application realizes multi-target real-time tracking based on the tracking-by-detection tracking paradigm, and target detection is the first step of the algorithm. A cascaded 3D R-CNN model is used as the target detection algorithm, the point cloud data at time t is input, and the final detection result D t is obtained through the 3D point cloud convolution network and the cascaded multi-detection head network, wherein B t , C t ], wherein B t = [x t , y t , z t , w t , h t , l t , θ t ] represents the center position (xt ,y t ,z t ), width, height and length (w) t ,h t ,l t ) and orientation angle (θ) t ); C t This represents the detection confidence score.

[0116] II. Target Identity Association

[0117] At the initial time t-1, the target detection module obtains all target detection results D. t-1 And initialize identity IDs for all targets. Based on the detection box information B in the detection state at time t-1. t-1 Kalman filtering is used to predict the state at time t.

[0118] At time t, the target detection state is D. t Estimate its speed based on the detected state at previous and subsequent times:

[0119]

[0120] Based on the estimated velocity and detection angle information at different time points, a measure of the motion difference between the two time points is defined:

[0121]

[0122] The first term in the formula measures the angular difference in velocity between two moments (frames); the second term measures the numerical difference in velocity between two moments; and the third term measures the offset of the target's orientation between two moments.

[0123] Based on the detection information B at time t t and predicted state P t The geometric difference is defined based on complete IoU.

[0124]

[0125] Where ρ 2 Indicates the center position of the detected vehicle (x) t ,y t ,z t The target center location predicted by the Kalman filter The Euclidean distance between them, c 2 Indicates that the detection bounding box B is included. t And predicted bounding box P t The minimum diagonal length, α is used to adjust the weight between IoU and spatial volume metric, and ν is the parameter that measures spatial volume metric, defined as follows:

[0126]

[0127]

[0128] Finally, the similarity between the current and previous time target is measured as:

[0129] S = S M + S G (5)

[0130] After obtaining the similarity measure of all targets between the current and previous time, the Hungarian algorithm is used to match all detected targets with all targets at the previous time (including the detected targets at the previous time and the failed matching targets) to obtain the tracking results. The matching target function is:

[0131]

[0132] where X is an assignment matrix with only one element of 1 in each row and column, S ij represents the similarity between the target i at the previous time and the detected target j at the current time. x ij = 1, indicating a successful match, otherwise a failed match.

[0133] III. Target state management

[0134] For the results of the data association stage matching, the state management strategy is used to adaptively manage the life cycle and state of the target.

[0135] If the match is successful, the target state is set to Tracking, and the same identity ID is given to the detected target and the predicted target; otherwise, the failed matching target enters the state management stage. If the target comes from the previous time, its life cycle is updated according to equation (7), and if life' < 1, the target state is set to Death, considering that the target has left the tracking field; otherwise, the target state is set to Miss, and the target will have a chance to be successfully tracked in the next survival period, and the problem of tracking failure caused by short-term occlusion is solved. If the target comes from time t, the target just appears in the tracking field, its identity ID is initialized, its life cycle life' is updated, and its state is set to Birth.

[0136] Specifically, the life cycle of the predicted target that is not successfully matched will be updated as follows:

[0137]

[0138] where life is a fixed life cycle threshold, a is a scale factor, and β is an offset factor. C t is the confidence score of the target being successfully detected output by the target detection module.

[0139] The targets in the states of Miss, Birth and Tracking update the tracking result T at time t together t .

[0140] In combination Figs. 1 to 3 Another embodiment of the application is an intelligent 3D multi-target tracking system supporting online updating of vehicle-cloud collaboration, specifically comprising a vehicle-cloud collaborative online updating module, a vehicle target detection module, a vehicle identity correlation degree calculation module, a vehicle identity matching module, and a vehicle state management module.

[0141] The vehicle-cloud collaborative online updating module uploads data to the cloud through a vehicle-cloud collaborative communication system, selects different vehicle target detection methods and models for online training in the cloud, and if the model trained in the cloud is better than the vehicle target detection model in the current offline tracking system in terms of performance (detection accuracy and speed, etc.), the model is downloaded from the cloud to update the vehicle target detection model in the offline tracking system. To ensure that the updated model can fit the offline tracking system, the vehicle detection model needs to meet the requirements that the input is 3D radar point cloud data and the output is vehicle target detection results (including but not limited to 3D detection box, orientation angle, and detection confidence score).

[0142] The vehicle detection module detects vehicle targets in the input 3D radar point cloud data to obtain the 3D detection box, orientation angle, and detection confidence score of the vehicle. Specifically, in the system, a 3D R-CNN cascade detection model with three detection heads is initially used, and the fixed IoU threshold is set to 0.75.

[0143] The vehicle identity correlation degree calculation module is used to calculate the identity correlation degree between vehicles at different times. Specifically, according to the target similarity described in formula (5), the similarity matrix between all targets at different times is calculated.

[0144] The vehicle identity matching module, based on the similarity matrix between targets at different times obtained by the vehicle identity correlation degree calculation module, uses the Hungarian algorithm to perform maximum matching between targets at different times to achieve vehicle identity correlation.

[0145] The vehicle state management module is used to adaptively manage the vehicle target life cycle and state based on the vehicle identity correlation results at different times, and to update the final tracking results. In formula (7), the fixed life cycle threshold, scale factor, and offset factor are set to 5, 8, and -0.5, respectively.

[0146] In summary, the tracking method of the present application, which integrates vehicle detection, data association and target life state management method, uses the detected target motion and geometric information to realize multi-target real-time tracking in complex scenes; in the case of target occlusion or false detection, the target re-matching and false detection result screening are realized by using the vehicle state management strategy. The tracking system described in the present application is based on the tracking method, combined with the vehicle cloud collaborative system to support online adjustment and upgrade of the tracking system. Therefore, the advantages of the present application are as follows:

[0147] (1) An intelligent 3D multi-target tracking method is proposed, which includes designing a data association method based on target motion and geometric information to realize fast and accurate multi-target tracking; designing an adaptive target state management method to reduce target mismatching and effectively solve the problem of tracking interruption caused by short-term occlusion of the vehicle.

[0148] (2) Based on the proposed intelligent 3D multi-target tracking method, combined with the vehicle cloud collaborative system, an online updated multi-target real-time tracking system is proposed, which makes the system support online remote adjustment and upgrade for long-term stable use.

[0149] In some alternative embodiments, the functions / operations mentioned in the block diagram can not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially simultaneously or the blocks can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flowcharts of the present application are provided by way of example, with the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and in which sub-operations described as part of larger operations are independently executed.

[0150] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the described functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It can also be understood that a detailed discussion of the actual implementation of each module is unnecessary for an understanding of the present application. Rather, given the properties, functions and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be within the routine skill of the engineer, given the present disclosure. Therefore, those skilled in the art, using ordinary skill, can implement the present application as set forth in the claims without undue experimentation, given the present disclosure. It can also be understood that the disclosed specific concepts are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0151] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions of the present application can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0152] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, and can be specifically embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with these instructions execution systems, apparatuses, or devices. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport programs for use by an instruction execution system, apparatus, or device, or in conjunction with these instruction execution systems, apparatuses, or devices.

[0153] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by editing, interpreting, or otherwise processing, if necessary, in other suitable ways, to be electronically obtained, and then stored in the computer memory.

[0154] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware which is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, can be used: a combination of discrete logic circuits having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having logic gates, field programmable gate arrays (FPGA), or other components, in combination or as the case can be.

[0155] In the description of the present application, reference has been made to descriptive terms including "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that a particular feature, structure, material or characteristic is included in at least one embodiment or example of the present application. The appearances of the above terms in various places in the specification are not necessarily all referring to the same embodiment or example. Moreover, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0156] Although embodiments of the present application have been shown and described, it would be appreciated by those skilled in the art that changes can be made in these embodiments without departing from the principles and spirit of the application, the scope of which is defined in the claims and their equivalents.

[0157] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. A vehicle-cloud collaborative intelligent 3D multi-target tracking method, characterized in that, include: Vehicle targets are detected, and the detection status of each vehicle in the initial scene is obtained. Then, initial trajectory information is established based on the detection status and vehicle identity. The detection status includes vehicle detection box information and detection confidence score. Based on the target information from the tracking results in the initial time scenario, the state of the target at that time is predicted using a Kalman filter algorithm. Vehicle targets are detected, the detection status of each vehicle in the target time scene is obtained, and then the data correlation between the detection box information of all detected targets in the target time and the predicted state of the targets in the initial time is calculated. Based on the data correlation, similarity matching is performed on all detected targets and predicted targets to obtain matching results; The matching results are then managed to maintain their target status until the target is successfully tracked. The step of detecting vehicle targets, obtaining the detection status of each vehicle in the target time scene, and then calculating the data correlation between the detection box information of all detected targets in the target time and the predicted state of the targets in the initial time includes: Estimate the speed of the vehicle target based on the detection status of the vehicle target at previous and subsequent time points; Based on the estimated velocity and detection angle information at the time points before and after, determine the measure of the motion difference between the time points before and after. Based on the detection information and prediction state in the target time scenario, and combined with complete-IoU, a measure of geometric difference is determined; Based on the measurement of motion difference and the measurement of geometric difference, a similarity measurement of the targets at different times is determined; The motion differences include the angular difference in velocity between two consecutive moments, the numerical difference in velocity between two consecutive moments, and the shift in vehicle orientation between two consecutive moments. The expression for measuring the difference in motion between the preceding and following moments is: ； in, A measure representing the difference in motion between two points in time; Indicates the vehicle target is in Estimated speed at any given moment; Indicates the vehicle target is in Estimated speed at any given moment; This indicates the vehicle's orientation angle.

2. The intelligent 3D multi-target tracking method with vehicle-cloud collaboration according to claim 1, characterized in that, The step of performing similarity matching on all detected and predicted targets based on the data correlation to obtain matching results specifically involves: generating tracking matching results based on the similarity measure of all targets, including: Obtain the similarity measure for all detected targets; Based on the similarity metric, the Hungarian algorithm is used to match all detected targets at different time points to obtain the tracking results.

3. The intelligent 3D multi-target tracking method with vehicle-cloud collaboration according to claim 1, characterized in that, The formula for calculating the estimated speed of the vehicle target is: ； in, Indicates the estimated speed of the vehicle target; Indicates time; express Detect the spatial position of the target at any given moment; This represents the operation of differentiating a spatial position vector with respect to time.

4. The intelligent 3D multi-target tracking method with vehicle-cloud collaboration according to claim 1, characterized in that, The target state management of the matching results includes: Obtain the matching results from the data association phase; When a match is successful, the vehicle target's status is configured as Tracking, and the detected target and the predicted target are assigned the same identity. When a match fails, if the detected target is from an earlier time, the lifecycle of the detected target is updated; if the detected target is from a later time, it is confirmed that the detected target has just appeared in the tracking field of view, and the target's identity, lifecycle, and status are configured. The target tracking task is completed based on the life cycle of the target being detected.

5. The intelligent 3D multi-target tracking method with vehicle-cloud collaboration according to claim 4, characterized in that, The step of completing the target tracking task based on the lifecycle of the detected target includes: When the lifetime of the target being detected is less than 1, the state of the target being detected is configured as Death; When the lifetime of the target to be detected is greater than or equal to 1, the state of the target to be detected will be configured as Miss.

6. The intelligent 3D multi-target tracking method with vehicle-cloud collaboration according to claim 5, characterized in that, The formula for calculating the lifecycle of the target that failed to match is: ； in, Indicates the lifecycle of the target that failed to match; Indicates a fixed lifecycle threshold; Represents the scale factor; Represents the offset factor; The confidence score represents the success of the target detection.

7. A vehicle-cloud collaborative intelligent 3D multi-target tracking system, characterized in that, include: The first module is used to detect vehicle targets, obtain the detection status of each vehicle in the initial scene, and then establish initial trajectory information based on the detection status and vehicle identity; wherein, the detection status includes vehicle detection box information and detection confidence score; The second module is used to predict the state of the target at the initial moment using a Kalman filter algorithm based on the target information of the tracking results in the initial moment scenario. The third module is used to detect vehicle targets, obtain the detection status of each vehicle in the target time scene, and then calculate the data correlation between the detection box information of all detected targets in the target time and the predicted state of the targets in the initial time. The fourth module is used to perform similarity matching on all detected targets and predicted targets based on the data correlation to obtain matching results; The fifth module is used to manage the target status of the matching results until the tracking target is completed; The step of detecting vehicle targets, obtaining the detection status of each vehicle in the target time scene, and then calculating the data correlation between the detection box information of all detected targets in the target time and the predicted state of the targets in the initial time includes: Estimate the speed of the vehicle target based on the detection status of the vehicle target at previous and subsequent time points; Based on the estimated velocity and detection angle information at the time points before and after, determine the measure of the motion difference between the time points before and after. Based on the detection information and prediction state in the target time scenario, and combined with complete-IoU, a measure of geometric difference is determined; Based on the measurement of motion difference and the measurement of geometric difference, a similarity measurement of the targets at different times is determined; The motion differences include the angular difference in velocity between two consecutive moments, the numerical difference in velocity between two consecutive moments, and the shift in vehicle orientation between two consecutive moments. The expression for measuring the difference in motion between the preceding and following moments is: ； in, A measure representing the difference in motion between two points in time; Indicates the vehicle target is in Estimated speed at any given moment; Indicates the vehicle target is in Estimated speed at any given moment; This indicates the vehicle's orientation angle.

8. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 6.

Images