A multi-target tracking method, apparatus, computing device, and storage medium

Abstract

The present disclosure provides a multi-target tracking method, device, computing equipment and storage medium, which is used to solve the problem of inaccurate multi-target tracking in the prior art. The target tracking method comprises: obtaining a prediction box of each target in a current observation frame according to a trajectory segment of one or more targets in an existing observation frame; performing target detection on the current observation frame to obtain one or more detection boxes, the detection box being divided into a high-quality box, a medium-quality box and a low-quality box; matching each prediction box with a detection box according to the similarity of the prediction box and the detection box; and if a certain prediction box is not matched with a high-quality box but is matched with a medium-quality box, determining that the target corresponding to the prediction box is in a tracking state in the current observation frame. The embodiment of the present disclosure can maintain the life cycle of the object based on the matched medium-quality detection box, associate the trajectory segments, and update the motion state of the object only according to the matched high-quality detection box, thereby ensuring the accuracy of the trajectory segments.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer vision, and particularly relates to a multi-target tracking method and device, a computing device and a storage medium. BACKGROUND

[0002] The main task of multi-target tracking (MOT or MTT) is to locate and label all targets in a sequence of observation frames, to correspond one-to-one between different observation frames, to maintain the same label for the same target, and to form the motion trajectory of different targets. However, the current multi-target tracking method is prone to target tracking errors and trajectory tracking errors, and thus the accuracy of multi-target tracking needs to be improved. SUMMARY

[0003] Embodiments of the present disclosure provide a multi-target tracking method, device, computing device and storage medium to improve the accuracy of multi-target tracking.

[0004] To achieve the above-mentioned purpose, the embodiments of the present disclosure adopt the following technical solutions:

[0005] In a first aspect, the embodiments of the present disclosure provide a multi-target tracking method, comprising:

[0006] obtaining a prediction box of each target in a current observation frame according to a trajectory segment of one or more targets in an existing observation frame;

[0007] performing target detection on the current observation frame to obtain one or more detection boxes, the detection boxes being divided into high-quality boxes, medium-quality boxes and low-quality boxes;

[0008] matching each prediction box with a detection box according to the similarity between the prediction box and the detection box; and

[0009] if a prediction box is not matched with a high-quality box but is matched with a medium-quality box, determining that the target corresponding to the prediction box is in a tracking state in the current observation frame.

[0010] In a second aspect, the embodiments of the present disclosure provide a multi-target tracking device, comprising:

[0011] a prediction module adapted to obtain a prediction box of each target in a current observation frame according to a trajectory segment of one or more targets in an existing observation frame;

[0012] a detection module adapted to perform target detection on the current observation frame to obtain one or more detection boxes, the detection boxes being divided into high-quality boxes, medium-quality boxes and low-quality boxes;

[0013] The matching module is adapted to match each prediction box with a detection box according to the similarity between the prediction box and the detection box.

[0014] The analysis module is adapted to determine that a target corresponding to a certain prediction box is in a tracking state in the current observation frame when the prediction box is not matched with a high-quality box but is matched with a medium-quality box.

[0015] In a third aspect, the present disclosure provides a computing device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor; wherein the processor, when executing the computer program, implements the multi-target tracking method as described above.

[0016] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multi-target tracking method as described above.

[0017] The technical solution provided by the present disclosure can maintain the life cycle of a moving object when a prediction box predicted according to a trajectory segment is matched with a medium-quality detection box obtained by a target detection method, thereby avoiding false detection operations caused by maintaining the life cycle only according to high-quality detection boxes. In addition, the present disclosure updates the motion state of an object only according to high-quality detection boxes that are matched, and does not update the motion state according to medium-quality detection boxes, thereby ensuring the accuracy of the maintained trajectory segment. Furthermore, the present disclosure proposes a plurality of new distance measurement values of prediction boxes and detection boxes, thereby improving tracking performance. By maintaining a speed item in the state quantity, the speed item is multiplied by a timestamp between two frames to represent the displacement between the two frames, thereby avoiding errors in maintaining the motion state caused when a sensor loses a frame or an object is not perceived by a perception algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0019] Figure 1 A structural diagram of a vehicle 100 is provided for the embodiments of the present disclosure.

[0020] Figure 2 A flowchart of a multi-target tracking method 200 is provided for the embodiments of the present disclosure.

[0021] Figure 3 A schematic diagram of a multi-target tracking method is provided for the embodiments of the present disclosure.

[0022] Figure 4A and 4B are respectively a schematic diagram of a multi-target tracking mode of an embodiment of the present disclosure;

[0023] Figures 5A-5C are respectively a conversion relationship matrix in an embodiment of the present disclosure;

[0024] Figure 6 is a structural diagram of a multi-target tracking device 600 provided by an embodiment of the present disclosure;

[0025] Figure 7 is a structural diagram of a computing device 700 provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0027] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] In order for those skilled in the art to better understand the present disclosure, some technical terms appearing in the embodiments of the present disclosure will be explained as follows:

[0029] Point cloud: data of the surrounding environment collected by a point cloud collection device (such as a laser radar, a millimeter wave radar, etc.), which is marked by a set of sparse three-dimensional space points.

[0030] Frame: measurement data received by a sensor in one observation, such as one frame of data of a camera being one picture and one frame of data of a point cloud collection device being a set of point clouds.

[0031] Target: a target object in each frame of data, which can include static objects and dynamic objects, such as pedestrians, vehicles, animals, obstacles, traffic lights, road signs, etc.

[0032] Target detection: finding the location of a target object in sensor data by an algorithm, usually represented as a rectangle or cuboid in 2D or 3D space.

[0033] Target tracking: for a given target object and a sequence of sensor input data, computing the state of the given target object at each time instant.

[0034] Tracklet: historical information and trajectory of a target in historical observation frames, which can include the state of the target in the historical observation frames.

[0035] State: position information of an object in a three-dimensional world, including position parameters and / or angle parameters, i.e., position and / or pose in a pose. The state can also include information of a bounding box, such as position information and dimension information of the bounding box.

[0036] Figure 1 is a schematic diagram of a vehicle 100 in which various techniques disclosed herein can be implemented. The vehicle 100 can be a car, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawnmower, an excavator, a snowmobile, an aircraft, a recreational vehicle, an amusement park vehicle, a farm device, a construction device, a tram, a golf cart, a train, a trolley, or other vehicle. The vehicle 100 can operate in an autonomous driving mode completely or partially. The vehicle 100 in the autonomous driving mode can control itself, e.g., the vehicle 100 can determine a current state of the vehicle and a current state of an environment in which the vehicle is located, determine a predicted behavior of at least one other vehicle in the environment, determine a trust level corresponding to a likelihood that the at least one other vehicle performs the predicted behavior, and control the vehicle 100 itself based on the determined information. While in the autonomous driving mode, the vehicle 100 can operate without human interaction.

[0037] The vehicle 100 can include various vehicle systems, such as a drive system 142, a sensor system 144, a control system 146, a user interface system 148, a control computer system 150, and a communication system 152. The vehicle 100 can include more or fewer systems, and each system can include multiple units. Further, each system and unit of the vehicle 100 can be interconnected. For example, the control computer system 150 can be in data communication with one or more of the vehicle systems 142-148 and 152. Thus, one or more of the described functionalities of the vehicle 100 can be divided into additional functional components or physical components, or combined into fewer functional components or physical components. In further examples, additional functional components or physical components can be added to the example as shown in Figure 1 ​

[0038] The drive system 142 can include a plurality of operable components (or units) that provide motive power for the vehicle 100. In one embodiment, the drive system 142 can include an engine or motor, wheels, a transmission, an electronic system, and a power source (or power supply). The engine or motor can be any combination of an internal combustion engine, an electric motor, a steam engine, a fuel cell engine, a propane engine, or other forms of engines or motors. In some embodiments, the engine can convert a power source into mechanical energy. In some embodiments, the drive system 142 can include a plurality of engines or motors. For example, a hybrid vehicle can include a gasoline engine and an electric motor, and other cases are possible.

[0039] The wheels of the vehicle 100 can be standard wheels. The wheels of the vehicle 100 can be a plurality of forms of wheels, including unicycle, bicycle, tricycle, or four-wheel forms, such as four wheels on a car or truck. Other numbers of wheels are possible, such as six or more wheels. One or more wheels of the vehicle 100 can be operated to rotate in a direction different from other wheels. The wheels can be at least one wheel that is fixedly connected to the transmission. The wheels can include a combination of metal and rubber, or other combinations of materials. The transmission can include units that are operable to transmit mechanical power from the engine to the wheels. For this purpose, the transmission can include a gear box, a clutch, a differential gear, and a drive shaft. The transmission can also include other units. The drive shaft can include one or more axles that are adapted to the wheels. The electronic system can include units that are used to transmit or control electronic signals for the vehicle 100. These electronic signals can be used to activate a plurality of lights, a plurality of servos, a plurality of motors, and other electronic driving or control devices in the vehicle 100. The power source can be an energy source that provides, in whole or in part, power for the engine or motor. That is, the engine or motor is capable of converting the power source into mechanical energy. Illustratively, the power source can include gasoline, oil, petroleum-based fuels, propane, other compressed gas fuels, ethanol, fuel cells, solar panels, batteries, and other sources of electrical energy. The power source can additionally or alternatively include any combination of fuel tanks, batteries, capacitors, or flywheels. The power source can also provide energy for other systems of the vehicle 100.

[0040] The sensor system 144 can include a plurality of sensors for sensing information of the environment and conditions of the vehicle 100. For example, the sensor system 144 can include an inertial measurement unit (IMU), a global positioning system (GPS) transceiver, a radar (RADAR) unit, a laser rangefinder / LIDAR unit (or other distance measuring device), an acoustic sensor, and a camera or image capture device. The sensor system 144 can include a plurality of sensors for monitoring the vehicle 100 (e.g., an oxygen (O2) monitor, an oil gauge sensor, an engine oil pressure sensor, etc.). Other sensors can also be configured. One or more of the sensors included in the sensor system 144 can be individually or collectively actuated to update the position, orientation, or both of the one or more sensors.

[0041] The IMU can include a combination of sensors (e.g., accelerometers and gyroscopes) for sensing changes in position and orientation of the vehicle 100 based on inertial acceleration. The GPS transceiver can be any sensor for estimating the geographical position of the vehicle 100. For this purpose, the GPS transceiver can include a receiver / transmitter to provide position information of the vehicle 100 relative to the earth. It is noted that GPS is an example of a global navigation satellite system, and thus, in some embodiments, the GPS transceiver can be replaced with a Beidou satellite navigation system transceiver or a Galileo satellite navigation system transceiver. The radar unit can use radio signals to sense objects in the environment in which the vehicle 100 is located. In some embodiments, in addition to sensing objects, the radar unit can also be used to sense the speed and heading of objects approaching the vehicle 100. The laser rangefinder or LIDAR unit (or other distance measuring device) can be any sensor that uses laser light to sense objects in the environment in which the vehicle 100 is located. In one embodiment, the laser rangefinder / LIDAR unit can include a laser source, a laser scanner, and a detector. The laser rangefinder / LIDAR unit can operate in a continuous (e.g., using heterodyne detection) or discontinuous detection mode. The camera can include a device for capturing a plurality of images of the environment in which the vehicle 100 is located. The camera can be a still image camera or a dynamic video camera.

[0042] The control system 146 is used to control the operation of the vehicle 100 and its components (or units). Accordingly, the control system 146 can include various units, such as a steering unit, a power control unit, a braking unit, and a navigation unit.

[0043] The steering unit may be a combination of mechanisms for adjusting the forward direction of vehicle 100. A power control unit (e.g., a throttle) may be used to control the engine speed, thereby controlling the speed of vehicle 100. The braking unit may include a combination of mechanisms for decelerating vehicle 100. The braking unit may utilize friction to decelerate the vehicle in a standard manner. In other embodiments, the braking unit may convert the kinetic energy of the wheels into electrical current. The braking unit may also take other forms. The navigation unit may be any system that determines a driving path or route for vehicle 100. The navigation unit may also dynamically update the driving path as vehicle 100 travels. The control system 146 may also additionally or optionally include other components (or units) not shown or described.

[0044] User interface system 148 can be used to allow vehicle 100 to interact with external sensors, other vehicles, other computer systems, and / or the user of vehicle 100. For example, user interface system 148 may include standard visual display devices (e.g., plasma displays, liquid crystal displays (LCDs), touchscreen displays, head-mounted displays, or other similar displays), speakers or other audio output devices, microphones or other audio input devices. For example, user interface system 148 may also include navigation interfaces and interfaces for controlling the internal environment of vehicle 100 (e.g., temperature, fan, etc.).

[0045] Communication system 152 can provide vehicle 100 with a means of communicating with one or more devices or other vehicles in the vicinity. In an exemplary embodiment, communication system 152 can communicate with one or more devices directly or through a communication network. Communication system 152 can be, for example, a wireless communication system. For example, the communication system can use 3G cellular communication (e.g., CDMA, EVDO, GSM / GPRS) or 4G cellular communication (e.g., WiMAX or LTE), and can also use 5G cellular communication. Optionally, the communication system can communicate with a wireless local area network (WLAN) (e.g., using...). In some embodiments, the communication system 152 can communicate directly with one or more devices or other vehicles in the vicinity, for example, using infrared light. Or ZigBee. Other wireless protocols, such as various vehicular communication systems, are also within the scope of this application. For example, the communication system may include one or more Dedicated Short Range Communication (DSRC) devices, V2V devices, or V2X devices that conduct public or private data communication with vehicles and / or roadside stations.

[0046] The control computer system 150 can control some or all of the functions of the vehicle 100. An autonomous driving control unit in the control computer system 150 can be used to identify, evaluate, and avoid or navigate around potential obstacles in the environment in which the vehicle 100 is located. Generally, the autonomous driving control unit can be used to control the vehicle 100 without a driver, or to provide assistance to a driver controlling the vehicle. In some embodiments, the autonomous driving control unit is used to combine data from a GPS transceiver, radar data, LIDAR data, camera data, and data from other vehicle systems to determine a travel path or trajectory for the vehicle 100. The autonomous driving control unit can be activated to enable the vehicle 100 to be driven in an autonomous driving mode.

[0047] The control computer system 150 can include at least one processor (which can include at least one microprocessor) that executes processing instructions (i.e., machine executable instructions) stored in a non-transitory computer readable medium (e.g., a data storage device or memory). The memory stores at least one machine executable instruction that, when executed by the processor, implements functionality including a map engine, a localization module, a perception module, a navigation or path module, and an autonomous control module. The map engine and the localization module are used to provide map information and localization information. The perception module is used to perceive things in the environment in which the vehicle is located based on information obtained from sensor systems and map information provided by the map engine. The navigation or path module is used to plan a travel path for the vehicle based on processing results of the map engine, the localization module, and the perception module. The autonomous control module inputs and parses decision information from the navigation or path module and other modules into control command outputs for the vehicle control system, and sends the control commands to corresponding components in the vehicle control system through a vehicle network (e.g., a vehicle internal electronic network system implemented through a CAN bus, a local area inter-network, a multimedia orientation system transmission, etc.) to achieve autonomous control of the vehicle; the autonomous control module can also obtain information of components in the vehicle through the vehicle network.

[0048] The control computer system 150 can also be multiple computing devices that distribute control of components or systems of the vehicle 100. In some embodiments, the memory can contain processing instructions (e.g., program logic) that are executed by the processor to implement various functions of the vehicle 100. In one embodiment, the control computer system 150 is capable of data communication with the systems 142, 144, 146, 148, and / or 152. Interfaces in the control computer system are used to facilitate data communication between the control computer system 150 and the systems 142, 144, 146, 148, and 152.

[0049] The memory can also include other instructions, including instructions for data transmission, instructions for data reception, instructions for interaction, or instructions for controlling the drive system 140, the sensor system 144, or the control system 146 or the user interface system 148.

[0050] In addition to storing processing instructions, the memory can store a variety of information or data, such as image processing parameters, road maps, and path information. These information can be used by the vehicle 100 and the control computer system 150 during operation of the vehicle 100 in an autonomous manner, a semi-autonomous manner, and / or a manual mode.

[0051] Although the autonomous driving control unit is shown as being separate from the processor and the memory, it should be understood that in some embodiments, some or all of the functionality of the autonomous driving control unit can be implemented with program code instructions resident in one or more memories (or data storage devices) and executed by one or more processors, and the autonomous driving control unit can in some cases be implemented using the same processor and / or memory (or data storage device). In some embodiments, the autonomous driving control unit can be implemented at least partially using various specialized circuit logic, various processors, various field programmable gate arrays (“FPGAs”), various application specific integrated circuits (“ASICs”), various real-time controllers, and hardware.

[0052] The control computer system 150 can control the functions of the vehicle 100 in accordance with inputs received from various vehicle systems (e.g., the drive system 142, the sensor system 144, and the control system 146), or inputs received from the user interface system 148. For example, the control computer system 150 can use inputs from the control system 146 to control the steering unit to avoid obstacles detected by the sensor system 144. In one embodiment, the control computer system 150 can be used to control multiple aspects of the vehicle 100 and its systems.

[0053] Although Figure 1 The various components (or units) shown in FIG. 1 integrated into the vehicle 100, one or more of these components (or units) can be onboarded to the vehicle 100 or separately associated with the vehicle 100. For example, the control computer system can exist partially or entirely independently of the vehicle 100. Thus, the vehicle 100 can exist in the form of separate or integrated equipment units. The equipment units that make up the vehicle 105 can communicate with each other in a wired or wireless manner. In some embodiments, additional components or units can be added to or removed from the various systems (e.g., the LiDAR or radar shown). Figure 1

[0054] ​Figure 1 A flowchart of a multi-target tracking method 200 according to an embodiment of the present disclosure is shown, which can be applied to online scenarios and offline scenarios. As shown in the figure, the method comprises the following steps. Figure 2

[0055] In step S201, a predicted bounding box of each target in a current observation frame is obtained according to a trajectory segment of one or more targets in an existing observation frame.

[0056] In some embodiments, the observation frame can be an image frame of an image sequence or a point cloud frame of a point cloud sequence. The trajectory segment contains the state quantity and the detection bounding box of each target in the existing observation frame. According to the state quantity and the motion model of the target in the existing observation frame, the state quantity and the predicted bounding box of the target in the current observation frame can be obtained. The motion model can be a Kalman filter, which can include a prior value or an estimated value of the displacement of the target between two adjacent frames, and the latest state of the target can be obtained based on the Kalman filter.

[0057] In some embodiments, the present disclosure determines the latest trajectory segment once after target detection for each frame, and the updating method of the trajectory segment can refer to Figure 2 for understanding. In some embodiments, if a new target is detected, a trajectory segment is initialized for the new target; if a target does not appear in a plurality of consecutive frames in the past, the trajectory segment corresponding to the target is removed from the trajectory segment pool. Therefore, the existing observation frame can be the last observation frame, that is, the present disclosure determines the predicted bounding box of the target in the current observation frame based on the trajectory segment of the target in the last observation frame. It should be understood that one of the important tasks of multi-target tracking is to associate the frames, that is, to correspond each target detection bounding box to the trajectory segment for the target detection bounding box of the current frame and the trajectory segment of the last frame. When target detection is performed on the initial frame, a trajectory segment is initialized for each detected target.

[0058] In step S202, target detection is performed on the current observation frame to obtain one or more detection bounding boxes, which are divided into high-quality bounding boxes, medium-quality bounding boxes and low-quality bounding boxes.

[0059] It should be understood that a person skilled in the art can select a target detection algorithm for target detection according to needs, such as a single-stage target detection algorithm and a two-stage target detection algorithm, and the present disclosure does not limit this. The target detection algorithm outputs one or more detection bounding boxes and an evaluation index (such as a score value) of each detection bounding box, and each detection bounding box can correspond to a target. Here, the evaluation index of the detection bounding box represents the quality of the detection bounding box, and a person skilled in the art can select a corresponding evaluation index according to needs, for example, confidence, accuracy, and intersection over union with a true value bounding box, and the present disclosure does not limit this.

[0060] ​Based on the evaluation index of the bounding box, each frame of the output bounding box can be divided into high-quality, medium-quality and low-quality bounding boxes. Among them, the high-quality bounding box is the bounding box with a score value greater than or equal to the first threshold value, the medium-quality bounding box is the bounding box with a score value between the first threshold value and the second threshold value, and the low-quality bounding box is the bounding box with a score value less than the second threshold value, and the second threshold value is greater than 0. Specifically, the medium-quality bounding box is the bounding box with a score value less than the first threshold value but greater than or equal to the second threshold value. For different score types such as confidence and accuracy, a corresponding score threshold value can be set by those skilled in the art, and the specific value of each score threshold value is not limited in the present disclosure. For example, the value range of the first threshold value is [0.5, 0.7], and the specific value can be 0.5; the value range of the second threshold value is [0.1, 0.3], and the specific value can be 0.3, but it is not limited thereto.

[0061] In some embodiments, the predicted bounding box and the detected bounding box can be two-dimensional boxes (such as rectangular boxes) or three-dimensional boxes (such as cuboids). Both types of boxes can be represented by coordinate values of key points and / or box dimensions, for example, can be represented by coordinates of a center point and length and width values. Those skilled in the art can select the representation of the box as long as it can uniquely represent the box. For example, both types of boxes can be represented by coordinate values of diagonal vertices and / or length and width values, and can also be represented by coordinate values of four vertices. For another example, both types of boxes can be represented by coordinates of a center point and length, width and height values, can be represented by coordinate values of body diagonal vertices and / or length, width and height values, and can also be represented by coordinate values of eight vertices.

[0062] Step S203, matching each predicted bounding box with a detected bounding box according to the similarity between the predicted bounding box and the detected bounding box.

[0063] In some embodiments, the similarity is a measure value of the relative position relationship between the predicted bounding box and the detected bounding box, and all measure values that can represent the relative position relationship between the predicted bounding box and the detected bounding box belong to the protection scope of the similarity of the present disclosure. For example, the similarity can be a distance measure value (such as Euclidean distance, Mahalanobis distance, etc.) of the two boxes, an intersection-over-union of area (i.e. a ratio of intersection area to union area), an intersection-over-union of volume (i.e. a ratio of intersection volume to union volume), and a similarity of image features corresponding to the two boxes. In addition, the present disclosure also provides some more accurate similarity parameters, which will be described in detail later.

[0064] In some embodiments, the present disclosure adopts the method of matching high-quality bounding boxes first and then matching medium-quality bounding boxes to improve the accuracy of matching and target detection. Specifically, step S203 includes: extracting high-quality bounding boxes in the current observation frame to perform primary matching with each trajectory segment; and extracting medium-quality bounding boxes in the current observation frame to perform secondary matching with the trajectory segments that are not matched.

[0065] For example, assuming there are 10 trajectory segments in the current trajectory segment pool, and 5 high-quality detection boxes and 10 medium-quality detection boxes in the current observation frame, the 5 high-quality boxes are first matched with the existing trajectory segments. Then, the 10 medium-quality detection boxes are matched with the unmatched trajectory segments. It should be noted that those skilled in the art can select the matching algorithm for the predicted and detected boxes as needed, such as a bipartite graph matching algorithm, specifically the Hungarian algorithm, but not limited to these.

[0066] Step S204: If a predicted box does not match a high-quality box but matches a medium-quality box, then it is determined that the target corresponding to the predicted box is in a tracking state in the current observation frame.

[0067] In some embodiments, if a predicted bounding box does not match a high-quality bounding box and a medium-quality bounding box, it is determined that the target corresponding to the predicted bounding box is in a lost state in the current observation frame; if a target is in a lost state in multiple consecutive frames, it is determined that the target is in a disappeared state, and the trajectory segment corresponding to the target is cleared.

[0068] In some embodiments, if a predicted bounding box matches a medium-quality bounding box, the state variables of the corresponding trajectory segment remain unchanged; if a predicted bounding box matches a high-quality bounding box, it is determined that the target corresponding to the predicted bounding box is in a tracking state in the current observation frame, and the state variables of the corresponding trajectory segment are updated according to the high-quality bounding box.

[0069] As can be seen, the target's state in the observation frame according to this disclosure includes a tracking state, a lost state, and a vanished state. A tracking state indicates that the target should still exist in the current observation frame; a lost state indicates that the target does not exist in the current observation frame; and a vanished state indicates that the target has disappeared from the observation field of view. Furthermore, when matching high-quality and medium-quality boxes, the object is determined to be in a tracking state. However, the presence determined by high-quality boxes has higher accuracy, so high-quality boxes can maintain the target's lifetime while updating the target's motion model (e.g., a Kalman filter), that is, updating the target's latest state quantity to the state quantity in the current observation frame. The presence determined by medium-quality boxes has relatively lower accuracy; therefore, medium-quality boxes only maintain the target's lifetime but do not update the target's motion model, meaning the target's latest state quantity remains the state quantity from the previous trajectory segment.

[0070] Here, as Figure 3 As shown, if the target lifecycle is updated solely based on matching high-quality detection boxes, the corresponding trajectory segment will be cleared whenever no high-quality box is matched. This approach can lead to the premature and incorrect deletion of trajectory segments when the target detection quality is poor or the vehicle is occluded. The target tracking method used in this disclosure, however... Figure 4AAs shown, in addition to using high threshold to screen high-quality detection boxes, medium threshold is also used to screen medium-quality target detection boxes which can represent "there is an object here". These medium-quality detection boxes can effectively maintain the life status of the trajectory segment, but will not be used to update the motion model to avoid causing confusion of the actual state of the target.

[0071] In some embodiments, each trajectory segment has a corresponding target identification, and the method 200 can further include: for high-quality boxes and medium-quality boxes that have been matched in the current observation frame, establishing the association between the detection box and the corresponding trajectory segment, and outputting the target identification of the associated trajectory segment for the detection box. That is, one target corresponds to one trajectory segment, and after the matching of the detection box and the prediction box, the detection box is thus ensured to have the same identification in the same sequence of observation frames, avoiding the situation where the same target has different identifications.

[0072] In some embodiments, if a detection box is not matched with any prediction box and the detection box is a high-quality box, a new trajectory segment is generated for the high-quality box, and a new target identification is generated for the trajectory segment. Here, only high-quality detection boxes are initialized with a trajectory segment, and if a medium-quality detection box is not matched with any prediction box, it will not be initialized with a trajectory segment, ensuring the accuracy of the trajectory segments in the trajectory segment pool.

[0073] In some embodiments, when a target is first determined to be in a lost state, that is, a trajectory segment first does not match a detection box, a countdown module is started for the target, and the initial countdown count is set to k (k is an integer greater than or equal to 2). If the target is still determined to be in a lost state in the next frame, that is, the trajectory segment still does not match a detection box, the countdown count is reduced by 1 (that is, the number of frames in which the target does not appear is increased by 1). In this way, when the countdown count becomes 0, that is, the target is in a lost state for k consecutive frames, it is determined that the target is in a disappeared state.

[0074] If, during the countdown, a target is re-determined to be in a tracking state, that is, the corresponding trajectory segment matches a detection box, the countdown count is reset to k and the counting is restarted. That is, the present disclosure only determines that a target is disappeared when the target is in a lost state for k consecutive frames, and if the target is in a tracking state for at least one frame in the k frames, it will not be determined to be in a disappeared state.

[0075] In some embodiments, the method 200 can further determine the bounding boxes to be output according to the scores of the bounding boxes. Specifically, for each matched bounding box, information of the matched bounding box and information of each bounding box in the track segment associated with the matched bounding box are input into a preset learning model to obtain an output probability of the bounding box. When the output probability of a bounding box is greater than or equal to a preset threshold, the state quantity of the bounding box is output. The information includes the state quantity of the bounding box and / or the bounding box quality, for example, the score value of the bounding box. The matched bounding box refers to a bounding box that is matched and whose bounding box quality satisfies a predetermined condition.

[0076] Here, if only the matched high-score bounding boxes are output, some medium-score bounding boxes that are relatively close to the ground truth bounding boxes can be ignored. Therefore, the present disclosure designs an output evaluation strategy for bounding boxes, corrects the score values of the bounding boxes based on a learning model, and determines which bounding boxes need to be output. The input of the learning model is information of a current bounding box and information of each bounding box in a track segment associated with the current bounding box, and the output is an output probability of whether the current bounding box needs to be output. Then, whether the current bounding box needs to be output can be determined according to the output probability.

[0077] Further, the input of the learning model is a score value of a current bounding box output by a target detection algorithm and score values of each bounding box in a track segment associated with the current bounding box, and the output is a corrected score value of the current bounding box. Then, whether the current bounding box needs to be output can be determined according to the corrected score value. A person skilled in the art can select the type, parameters and hyperparameters of the learning model according to needs, and the present disclosure does not limit the same.

[0078] In some embodiments, the bounding box quality satisfies a predetermined condition, for example, the bounding box is a high-quality bounding box and / or a medium-quality bounding box, or the score value of the bounding box is greater than or equal to a predetermined threshold. In an implementation, the present disclosure only inputs the matched bounding boxes with confidence greater than or equal to 0.5 into the learning model to determine whether to output the bounding boxes, and does not input the matched bounding boxes with confidence less than 0.5 into the learning model for determination.

[0079] In addition, when the similarity between the predicted bounding box and the detection bounding box uses Mahalanobis distance, Euclidean distance, etc., a large number of matching errors can be caused due to the inability to judge the object overlap relationship. When the area intersection-over-union is used, it can fail on fast-moving vehicles. Therefore, the present disclosure uses a new distance metric GIOU (Generalized IOU) to measure the distance between the predicted bounding box and the detection bounding box, thereby significantly improving the tracking performance.

[0080] In some embodiments, when both the predicted bounding box and the detected bounding box are two-dimensional boxes, the similarity between the predicted bounding box and the detected bounding box is related to the intersection area, the union area, and the area of ​​the circumscribed convex polygon of the two boxes.

[0081] In some embodiments, when both the predicted bounding box and the detected bounding box are 3D boxes, the similarity between the predicted bounding box and the detected bounding box is related to the intersection volume, the union volume, and the volume of the circumscribed convex polygon of the two boxes.

[0082] Furthermore, the steps for calculating the similarity of the 3D bounding boxes include:

[0083] 1) Generate the first and second projection boxes of the predicted and detected bounding boxes on the two-dimensional plane, respectively:

[0084] Assuming the predicted bounding box and the detected bounding box are 3D BBox A and 3D BBox B respectively, the two-dimensional projection boxes of the two 3D boxes in the top view are denoted as 2D BBox A' and 2D BBox B' respectively.

[0085] 2) Calculate the intersection area of ​​the first and second projection frames and the area of ​​the circumscribed convex polygon:

[0086] The intersection region of two-dimensional boxes A' and B' in the top view is I, and the area of ​​the intersection region is S. I ;

[0087] The circumscribed convex polygon of two-dimensional frames A' and B' in the top view is C, and the area of ​​this circumscribed convex polygon is S. C ;

[0088] 3) Calculate the intersection height and union height of the predicted bounding box and the detected bounding box on the vertical axis:

[0089] The height of the intersection of 3D bounding boxes A and B on the z-axis is:

[0090] H I =max(0,min((z A +h A / 2)-(z B -h B / 2),(z B +h B / 2)-(z A -h A / 2)))

[0091] The union height of 3D bounding boxes A and B on the z-axis is:

[0092] H C =max((z A +h A / 2)-(z B -h B(z B +h B (z A -h A (z

[0093] wherein z A and z B are coordinate values of key points of three-dimensional boxes A and B on the z-axis, and can be coordinate values of center points of the boxes on the z-axis. h A and h B are height values of three-dimensional boxes A and B on the z-axis.

[0094] 4) Calculate the intersection volume according to the intersection area and the intersection height of the first projection box and the second projection box:

[0095] The intersection volume of three-dimensional boxes A and B is V I =S I *H I

[0096] 5) Calculate the volume of the circumscribed convex polyhedron according to the area of the circumscribed convex polygon and the union height:

[0097] The volume of the circumscribed convex polyhedron of three-dimensional boxes A and B is V C =S C *H C

[0098] 6) Calculate the union volume according to the volume of the prediction box, the volume of the detection box, and the intersection volume:

[0099] The union volume of three-dimensional boxes A and B is V U =w A *h A *l A +w B *h B *l B -V I

[0100] 7) Calculate the GIOU of the two boxes according to the intersection volume, the union volume, and the volume of the circumscribed convex polyhedron:

[0101] GIOU(A, B) = V I / V U -(V c -V U ) / V C

[0102] In some embodiments, the similarity of the present disclosure can also employ DIOU (Distance-IOU) based on three-dimensional boxes, which considers the distance and overlap rate of the predicted box and the detection box. Specifically, the DIOU is calculated according to the volume intersection over union of the two boxes, the Euclidean distance d1 between the center points of the two boxes, and the diagonal distance d2 of the convex hull of the two boxes. The volume intersection over union of the two boxes can be directly obtained by dividing the intersection volume by the union volume, or can be the new GIOU calculated above. In one implementation, DIOU = volume intersection over union - d1 / d2 = GIOU - d1 / d2.

[0103] In some embodiments, the similarity of the present disclosure can also employ CIOU (Complete-IOU) based on three-dimensional boxes, which considers the overlap rate, center point distance, and size ratio of the predicted box and the detection box, i.e., an additional loss term of aspect ratio based on DIOU. Specifically, CIOU = DIOU + av, where a is a weight, and v is used to measure the similarity of the size ratio (e.g., aspect ratio, length-height ratio, width-height ratio, or the average of the three ratios).

[0104] As can be seen, the present disclosure sets multiple metric values of three-dimensional boxes to measure the similarity of the predicted box and the detection box, which improves the generalizability of multi-target tracking relative to Mahalanobis distance and Euclidean distance, and eliminates the difficulty of re-adjusting the distance metric threshold for different associations in each new scenario.

[0105] In addition, as described above, the present disclosure can employ a Kalman filter to maintain the motion state of the target. In the Kalman filter, x represents the motion state of the target, such as the position and velocity of the target; z represents the observation, such as the detection box corresponding to the target. In the Kalman filter, the motion state of the object is described by the following two formulas: t x t = F t-1 x t + w t and z t = H t x t + v t The former describes the state transition of the target at adjacent time points, w t represents noise, and F t is a state transition transformation relationship, which can be a vector or a matrix, of course not limited thereto. The latter describes the relationship between the observation z t and the state quantity x t , wherein H t is an observation transformation relationship, which can also be a vector or a matrix, and v

[0106] Generally, the state quantity xt including but not limited to the position parameter of the target, the angle parameter and the size of the corresponding detection frame. The position parameter includes at least one of the first coordinate, the second coordinate and the third coordinate of the key point of the target in the spatial coordinate system. The first coordinate, the second coordinate and the third coordinate can correspond to the x-axis coordinate, the y-axis coordinate and the z-axis coordinate in the spatial coordinate system. The angle parameter includes at least one of the pitch angle, the yaw angle and the roll angle. The key point can be one or more, for example, can be the center point of the object or the center point of a specific part of the object, or a set of these points. For example, for a vehicle, the key point can be the center point of the vehicle, or the center point of the vehicle head, or the center points of the rear wheels on both sides, or a set of multiple body points, and the number and position of the key point are not limited by the present disclosure.

[0107] In some embodiments, the state quantity x t is [x, y, z, yaw, l, w, h, Δx, Δy, Δz], representing the position, size and displacement between two frames of the object; the observation quantity z t is [x, y, z, yaw, l, w, h], representing the target detection frame of each frame. Therefore, H t , F t are shown in Figure 4B and 5B respectively. If the state quantity is an n-dimensional vector, then H t is an n*(n-3) matrix, and F t is an n*n matrix. The first n-3 columns of the H t matrix have diagonal values of 1, and other values are 0; the last 3 columns of the matrix have values of 0. The F t matrix has diagonal values, and the values of the 1st row, n-2th column, 2nd row, n-1th column and 3rd row, nth column are all 1, and other values are 0.

[0108] However, considering that if only the position of each object and its displacement between two frames are maintained in the Kalman filter, when the sensor loses the middle frame or the perception algorithm fails to perceive the object for multiple consecutive frames, the Kalman filter will maintain an error for the motion state of the target. Therefore, the state quantity x t maintained by the present disclosure is no longer the displacement of the target object between two frames, but the speed of the target, and at this time the state quantity becomes [x, y, z, yaw, l, w, h, v x , v y , v z ]. Therefore, F t becomes a matrix as shown in Figure 5A , that is, the values of the 1st row, n-2th column, 2nd row, n-1th column and 3rd row, nth column are all replaced by time_lag, which represents the time difference between two observations.

[0109] Based on this, the method 200 can further include the steps of:

[0110] calculating the state quantity of the target in the next observation frame according to the target in the state quantity of the current observation frame and the preset state transition transformation relationship; and

[0111] calculating the prediction box information of the target in the next observation frame according to the state quantity of the target in the next observation frame and the preset observation transformation relationship.

[0112] Further, the state quantity includes position and velocity, and the velocity includes velocity components on three coordinate axes. The calculation of the state quantity of the target in the next observation frame according to the target in the state quantity of the current observation frame and the preset state transition transformation relationship includes: calculating the state quantity of the target in the next observation frame according to the state transition transformation relationship, the position, velocity of the target in the current observation frame, and the time difference between the adjacent two frames.

[0113] It can be seen that, by maintaining the target position and the target velocity, the object displacement between two observations is represented by multiplying the velocity by the time stamp to obtain the latest position of the target, thereby improving the robustness when the sensor loses frames or the perception algorithm loses observations.

[0114] Figure 5C A schematic diagram of a multi-target tracking device 600 according to an embodiment of the present disclosure is shown. As shown, the device 600 includes: Figure 6

[0115] a prediction module 601 adapted to obtain the prediction box of each target in the current observation frame according to the trajectory segment of one or more targets in the existing observation frame;

[0116] a detection module 602 adapted to perform target detection on the current observation frame to obtain one or more detection boxes, wherein the detection boxes are divided into high-quality boxes, medium-quality boxes, and low-quality boxes;

[0117] a matching module 603 adapted to match each prediction box with a detection box according to the similarity of the prediction box and the detection box; and

[0118] an analysis module 604 adapted to determine that the target corresponding to a certain prediction box is in a tracking state in the current observation frame when the prediction box is not matched with a high-quality box but is matched with a medium-quality box.

[0119] In some embodiments, the matching module 603 is further adapted to: perform primary matching of the high-quality boxes in the current observation frame with each trajectory segment; and perform secondary matching of the medium-quality boxes in the current observation frame with the trajectory segments that are not matched. The calculation process of the similarity has been disclosed in detail in the description based on the method 200, and will not be expanded here.​

[0120] In some embodiments, the analysis module 604 is further adapted to:

[0121] when a prediction box matches a medium quality box, keeping the state quantity of the corresponding track segment unchanged;

[0122] when a prediction box matches a high quality box, determining that the target corresponding to the prediction box is in a tracking state in the current observation frame, and updating the state quantity of the corresponding track segment according to the high quality box;

[0123] when a detection box does not match any prediction box and the detection box is a high quality box, newly generating a track segment for the high quality box, and newly generating a target identifier for the track segment;

[0124] when a prediction box does not match a high quality box and a medium quality box, determining that the target corresponding to the prediction box is in a lost state in the current observation frame; and

[0125] when a target is in a lost state in consecutive multiple frames, determining that the target is in a disappeared state, and clearing the corresponding track segment.

[0126] In some embodiments, each track segment has a corresponding target identifier, and the device 600 further comprises an output module (not shown in the figure) adapted to: for a high quality box and a medium quality box that have been matched in the current observation frame, establishing an association between the detection box and the corresponding track segment, and outputting the target identifier of the track segment associated with the detection box for the detection box.

[0127] In some embodiments, the output module is further adapted to: for each matched detection box, inputting information of the matched detection box and information of each detection box in the track segment associated with the detection box into a preset learning model to obtain an output probability of the detection box; when the output probability of a detection box is greater than or equal to a preset threshold, outputting the state quantity of the detection box. The information includes the state quantity of the detection box and / or the quality of the detection box. The matched detection box refers to a detection box that has been matched and whose quality meets a predetermined condition.

[0128] In some embodiments, the output module is further adapted to: calculating the state quantity of a target in a next observation frame according to the state quantity of the target in the current observation frame and a preset state transition transformation relationship; and calculating prediction box information of the target in the next observation frame according to the state quantity of the target in the next observation frame and a preset observation transformation relationship. The state quantity includes position and velocity, and the output module is further adapted to calculate the state quantity of the target in the next observation frame according to the state transition transformation relationship, the position and velocity of the target in the current observation frame, and a time difference between adjacent two frames.

[0129] According to the multi-target tracking device 600 of the present disclosure, the specific details have been disclosed in the description based on other drawings, and will not be repeated here.

[0130] In addition, the embodiment of the present disclosure also provides a computer readable storage medium, comprising a program or instructions, when the program or instructions are executed on a computer, the multi-target tracking method as described above is realized.

[0131] In addition, the embodiment of the present disclosure also provides a computer readable storage medium, comprising a program or instructions, when the program or instructions are executed on a computer, the multi-target tracking method as described above is realized. Figure 6 As shown in the computing device 700, comprising a memory 701, and one or more processors 702 in communication connection with the memory. The memory 701 stores instructions executable by the one or more processors 702, and the instructions are executed by the one or more processors 702 to enable the one or more processors 702 to implement the point cloud processing method as described above. The computing device 700 can further comprise a communication interface 703, which can implement one or more communication protocols (LTE, Wi-Fi, etc.).

[0132] According to the technical scheme of the present disclosure, the prediction box of each target in the current frame is predicted according to the existing trajectory segment, the detection box of the current frame is obtained according to the target detection algorithm, and then the life cycle of the object is updated according to the matching result of the prediction box and the detection box. The medium quality box matched only maintains the life cycle but does not update the latest state quantity of the trajectory segment, and the high quality matched maintains the life cycle and updates the latest state quantity of the trajectory segment, which avoids the situation that the trajectory segment is removed in advance when the target detection is unstable or the vehicle is blocked, and ensures the high accuracy of the motion model. Moreover, for each observation frame, the position and number of each target object are calculated and output, so that the same object has the same number in the associated frame.

[0133] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0134] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 7 one or more functions specified in the flowchart or multiple flows and / or blocks. Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks.

[0135] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device that implements the function specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks. Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the function specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks. Figure 1 Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks.

[0137] The principles and implementation of the present disclosure are described in the specific embodiments in the present disclosure, and the above embodiment description is only for helping to understand the method of the present disclosure and its core idea; at the same time, for those skilled in the art, according to the idea of the present disclosure, there will be changes in specific implementation and application range, and the above description should not be understood as a limitation of the present disclosure.

Claims

1. A multi-target tracking method, comprising: obtaining a predicted bounding box of each target in a current observation frame according to a trajectory segment of one or more targets in a previous observation frame; performing target detection on the current observation frame to obtain one or more detected bounding boxes, the detected bounding boxes being classified into high-quality bounding boxes, medium-quality bounding boxes and low-quality bounding boxes; matching each predicted bounding box with a detected bounding box according to a similarity between the predicted bounding box and the detected bounding box; and if a predicted bounding box is not matched with a high-quality bounding box but is matched with a medium-quality bounding box, determining that a target corresponding to the predicted bounding box is in a tracking state in the current observation frame. 2.The method of claim 1, further comprising: if a predicted bounding box is matched with a medium-quality bounding box, keeping a state quantity of a corresponding trajectory segment unchanged; and if a predicted bounding box is matched with a high-quality bounding box, determining that a target corresponding to the predicted bounding box is in a tracking state in the current observation frame, and updating the state quantity of the corresponding trajectory segment according to the high-quality bounding box. Each trajectory segment has a corresponding target identifier, and the method further comprises:

3. The method of claim 1, wherein, for a high-quality bounding box and a medium-quality bounding box that have been matched in the current observation frame, establishing an association between the detected bounding box and the corresponding trajectory segment, and outputting a target identifier of the associated trajectory segment for the detected bounding box. 4.The method of claim 3, further comprising: for each matched detected bounding box, inputting information of the matched detected bounding box and information of each detected bounding box in a trajectory segment associated with the detected bounding box into a preset learning model to obtain an output probability of the detected bounding box; wherein the information includes a state quantity of the detected bounding box and / or a quality of the detected bounding box. 5.The method of claim 4, further comprising: when the output probability of a detected bounding box is greater than or equal to a preset threshold, outputting a state quantity of the detected bounding box. The matched detected bounding box refers to a detected bounding box that has been matched and whose quality meets a predetermined condition. 6.The method of claim 1, further comprising: if a predicted bounding box is not matched with a high-quality bounding box and a medium-quality bounding box, determining that a target corresponding to the predicted bounding box is in a lost state in the current observation frame; and if a target is in a lost state in consecutive multiple frames, determining that the target is in a disappeared state, and clearing a corresponding trajectory segment. The matching each predicted bounding box with a detected bounding box according to a similarity between the predicted bounding box and the detected bounding box comprises:

7. The method of claim 1, wherein, performing a first matching between high-quality bounding boxes in the current observation frame and each trajectory segment; and for a trajectory segment that has not been matched, performing a second matching between medium-quality bounding boxes in the current observation frame and the trajectory segment that has not been matched. 8.The method of claim 1, wherein: the similarity is a measure value of a relative position relationship between a predicted bounding box and a detected bounding box; when the predicted bounding box and the detected bounding box are both two-dimensional boxes, the similarity is related to an intersection area, a union area and an area of an outer convex polygon of the two boxes; when the predicted bounding box and the detected bounding box are both three-dimensional boxes, the similarity is related to an intersection volume, a union volume and a volume of an outer convex polyhedron of the two boxes. 9.The method of claim 1, further comprising a calculation step of a similarity of a three-dimensional bounding box: generating a first projection box and a second projection box of a predicted bounding box and a detected bounding box on a two-dimensional plane, respectively. ​ calculating an intersection area of the first projection box and the second projection box and an area of an outer convex polygon of the intersection area; calculating an intersection height and a union height of the predicted box and the detection box in a vertical axis.

10. The method of claim 9, wherein, The similarity calculation of the three-dimensional box further comprises: calculating an intersection volume according to the intersection area and the intersection height of the first projection box and the second projection box; calculating a volume of the outer convex polygon according to the area of the outer convex polygon and the union height; calculating a union volume according to the volume of the predicted box, the volume of the detection box and the intersection volume.

11. The method of claim 1, further comprising: calculating a state quantity of the target in a next observation frame according to a state quantity of the target in a current observation frame and a preset state transition transformation relationship; and calculating a predicted box information of the target in the next observation frame according to the state quantity of the target in the next observation frame and a preset observation transformation relationship.

12. The method of claim 11, wherein the state quantity comprises a position and a velocity, and the calculating of the state quantity of the target in the next observation frame according to the state quantity of the target in the current observation frame and the preset state transition transformation relationship comprises: calculating the state quantity of the target in the next observation frame according to the state transition transformation relationship, the position, the velocity of the target in the current observation frame and a time difference between adjacent two frames.

13. The method of claim 1, wherein: the high-quality box is a detection box with a score value greater than or equal to a first threshold value; the medium-quality box is a detection box with a score value between the first threshold value and a second threshold value; and the low-quality box is a detection box with a score value less than the second threshold value, and the second threshold value is greater than 0.

14. A multi-target tracking device, comprising: a prediction module adapted to obtain a predicted box of each target in a current observation frame according to a trajectory segment of one or more targets in an existing observation frame; a detection module adapted to perform target detection on the current observation frame to obtain one or more detection boxes, the detection boxes being divided into high-quality boxes, medium-quality boxes and low-quality boxes; a matching module adapted to match each predicted box with a detection box according to a similarity of the predicted box and the detection box; and an analysis module adapted to determine that a target corresponding to a predicted box is in a tracking state in the current observation frame when the predicted box is not matched with a high-quality box but is matched with a medium-quality box.

15. A computing device, comprising: a processor, a memory and a computer program stored on the memory and executable on the processor; wherein the processor, when executing the computer program, performs the method of any one of claims 1-13.

16. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method of any one of claims 1-13. ​

Images