Target tracking method and device, medium and equipment

Abstract

The embodiment of the invention discloses a target tracking method and device, a medium and equipment, and the method comprises the steps: determining the first target information of at least one target observed through a first observation mode at a first moment; determining at least one first observation sequence set after tracking at a second moment before the first moment; each target corresponds to one first observation sequence set; based on the first target information and the first observation sequence sets, determining a first association relationship between the target and each first observation sequence set; updating the first observation sequence set based on the first target information and the first association relationship to obtain a second observation sequence set; determining a second incidence relation between each observation sequence in the second observation sequence set and each second observation sequence set; and based on the second association relationship, updating the second observation sequence set to obtain a third observation sequence set for determining a target tracking result at the first moment. According to the embodiment of the invention, the accuracy and effectiveness of target tracking can be improved.

Description

Technical Field

[0001] The present disclosure relates to intelligent driving technology, and in particular, to a target tracking method, apparatus, medium, and device. Background Art

[0002] In the intelligent driving scenario, sensors play a crucial role. Different types of sensors can provide different information. For example, cameras can provide image information, radars can provide distance and speed information, lidars can provide high-precision distance information, etc. By fusing the information of multiple sensors, the accuracy and robustness of target detection and tracking can be improved. In related technologies, usually, the observation information of a single-frame sensor is associated and matched with the target state of historical observations, which easily leads to missed associations or false associations, resulting in relatively low accuracy of target tracking results. Summary of the Invention

[0003] Embodiments of the present disclosure provide a target tracking method, apparatus, medium, and device to improve the accuracy and effectiveness of target tracking, thereby enhancing the driving safety of vehicles.

[0004] In a first aspect of the embodiments of the present disclosure, a target tracking method is provided, including: determining first target information of at least one target observed by a first observation method at a first moment; determining at least one first observation sequence set after tracking at least one target at a second moment; wherein the second moment is before the first moment; each target corresponds to one first observation sequence set, and the first observation sequence set includes at least one observation sequence; based on the first target information and the first observation sequence set, determining a first association relationship between the target and each first observation sequence set; based on the first target information and the first association relationship, updating the first observation sequence set to obtain a second observation sequence set; determining a second association relationship between each observation sequence in the second observation sequence set and each second observation sequence set; based on the second association relationship, updating the second observation sequence set to obtain a third observation sequence set; based on the third observation sequence set, determining a target tracking result at the first moment.

[0005] In a second aspect of the embodiments of the present disclosure, a target tracking device is provided, including: a first processing module, configured to determine first target information of at least one target observed by a first observation method at a first moment; a second processing module, configured to determine at least one first observation sequence set after tracking at least one target at a second moment; wherein the second moment is before the first moment; each target corresponds to one first observation sequence set, and the first observation sequence set includes at least one observation sequence; a third processing module, configured to determine a first association relationship between the target and each first observation sequence set based on the first target information and the first observation sequence set; a fourth processing module, configured to update the first observation sequence set based on the first target information and the first association relationship to obtain a second observation sequence set; a fifth processing module, configured to determine a second association relationship between each observation sequence in the second observation sequence set and each second observation sequence set; a sixth processing module, configured to update the second observation sequence set based on the second association relationship to obtain a third observation sequence set; a seventh processing module, configured to determine a target tracking result at the first moment based on the third observation sequence set.

[0006] In a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, where the storage medium stores a computer program, and the computer program is used to execute the target tracking method in any one of the above embodiments of the present disclosure.

[0007] In a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, where the electronic device includes: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the target tracking method in any one of the above embodiments of the present disclosure.

[0008] In a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, where when the instructions in the computer program product are executed by a processor, the target tracking method provided in any one of the above embodiments of the present disclosure is executed.

[0009] Based on the target tracking method, device, medium, and equipment provided in the above embodiments of the present disclosure, when obtaining the first target information of the target observed at the first moment, by temporally correlating the observed target with each observation sequence set (i.e., the first observation sequence set) after target tracking at the second moment, the first correlation relationship between the target and each first observation sequence set is obtained. The first observation sequence set represents a set of observation sequences obtained by observing the same target through one or more observation methods, and the first correlation relationship represents whether the target observed at the first moment belongs to the first observation sequence set. Thus, the temporal correlation can be fully utilized to update the first target information of the observed target to the corresponding first observation sequence set, obtaining the second observation sequence set, effectively improving the accuracy and effectiveness of the temporal tracking result. Furthermore, by determining the second correlation relationship between each observation sequence in the second observation sequence set and each second observation sequence set, spatial correlation is achieved, enabling the uncorrelated observation sequences to be incorporated into the corresponding second observation sequence set and the mis-correlated observation sequences to be deleted from the second observation sequence set. Therefore, the accuracy and effectiveness of the target tracking result can be effectively improved, providing a reliable target tracking result for downstream planning and control, thereby enhancing the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is an exemplary application scenario of the target tracking method provided by the present disclosure;

[0011] Figure 2 is a flowchart of the target tracking method provided by an exemplary embodiment of the present disclosure;

[0012] Figure 3 is a flowchart of the target tracking method provided by another exemplary embodiment of the present disclosure;

[0013] Figure 4 is a flowchart of the target tracking method provided by still another exemplary embodiment of the present disclosure;

[0014] Figure 5 is a flowchart of the target tracking method provided by yet another exemplary embodiment of the present disclosure;

[0015] Figure 6 is a flowchart of the target tracking method provided by still another exemplary embodiment of the present disclosure;

[0016] Figure 7 is a flowchart of the target tracking method provided by yet another exemplary embodiment of the present disclosure;

[0017] Figure 8 is a flowchart block diagram of the target tracking method provided by an exemplary embodiment of the present disclosure;

[0018] Figure 9It is a schematic structural diagram of a target tracking device provided by an exemplary embodiment of the present disclosure;

[0019] Figure 10 It is a schematic structural diagram of a target tracking device provided by another exemplary embodiment of the present disclosure;

[0020] Figure 11 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0021] To explain the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited by the exemplary embodiments.

[0022] It should be noted that: unless otherwise specifically stated, the relative arrangements, numerical expressions, and numerical values of the components and steps described in these embodiments do not limit the scope of the present disclosure.

[0023] Overview of the present disclosure

[0024] In the process of implementing the present disclosure, the inventors found that in related intelligent driving technologies, the target tracking result is usually obtained by associating and matching the observation information of a single-frame sensor with the target state of historical observations, which is likely to cause missed associations or incorrect associations, thereby resulting in a low accuracy of the target tracking result. Among them, a missed association means that the observations of the same target through multiple observation methods are not successfully associated, resulting in one target being split into multiple targets. For example, when observing target A through the front view camera observation method, target information 1 is obtained, and when observing target A through the BEV (Bird’s Eye View) observation method, target information 2 is obtained. Due to the influence of observation errors, target tracking algorithm errors, environmental factors, etc., target information 2 and target information 1 are not successfully associated, and in the target tracking result, target information 1 and target information 2 will be regarded as two targets, that is, target A is split into two targets. An incorrect association means that the observations that do not belong to the same target are associated together, resulting in multiple targets being incorrectly merged into one target.

[0025] Exemplary overview

[0026] Figure 1 It is an exemplary application scenario of the target tracking method provided by the present disclosure. As Figure 1As shown in the figure, in the intelligent driving scenario, during the driving process of the vehicle (i.e., the host vehicle) 11 on the road, the surrounding environment is perceived (or observed) through the sensor 12 on the vehicle 11. For any target (such as other vehicles like target 13 and target 14 in the figure, as well as pedestrians, cyclists, cones, traffic signs, etc.) observed at any moment (which can be referred to as the first moment), the target tracking method of the embodiments of the present disclosure can be used to perform target tracking to obtain the target tracking result. Specifically, the first target information of at least one target observed by the first observation method at the first moment can be determined; at least one first observation sequence set after tracking at least one target at the second moment can be determined, where the second moment is before the first moment; each target corresponds to a first observation sequence set, and the first observation sequence set includes at least one observation sequence; based on the first target information and the first observation sequence set, the first association relationship between the target and each first observation sequence set can be determined; based on the first target information and the first association relationship, the first observation sequence set can be updated to obtain the second observation sequence set; the second association relationship between each observation sequence in the second observation sequence set and each second observation sequence set can be determined; based on the second association relationship, the second observation sequence set can be updated to obtain the third observation sequence set; based on the third observation sequence set, the target tracking result at the first moment can be determined. Since each target corresponds to an observation sequence set, not only the observation sequence sets of the targets observed at the first moment and the targets at the second moment (the historical moment relative to the first moment) are temporally associated (referred to as temporal association) and updated, but also each observation sequence can be associated (referred to as spatial association) and updated with each observation sequence set, so as to make full use of the temporal correlation and spatial correlation of the observation results. Through the association and update in two dimensions of time and space, the target tracking result at the first moment can be obtained, avoiding or reducing the problems of missed association and misassociation, effectively improving the accuracy and effectiveness of the target tracking result, providing a reliable target tracking result for downstream planning and control, and thus improving the safety of vehicle driving.

[0027] The target tracking method of the embodiments of the present disclosure is not limited to the intelligent driving scenario of vehicles, and can also be applied to the target tracking scenarios of other arbitrary autonomous mobile devices, such as robots, drones, etc.

[0028] Exemplary method

[0029] Figure 2 is a schematic flowchart of the target tracking method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, specifically, for example, on an edge device of a computing platform (or terminal) of an autonomous mobile device. This embodiment can also be applied to a server, such as a cloud server. By communicating with the autonomous mobile device, it obtains perception data, performs target tracking on the server side, and then transmits the target tracking result to the autonomous mobile device. AsFigure 2 As shown, the method of the embodiments of the present disclosure may include the following steps:

[0030] Step 210, determining first target information of at least one target observed by a first observation method at a first moment.

[0031] Among them, the first moment may be any moment. For example, the first moment may be the current moment or any historical moment. In the intelligent driving scenario, the first moment is, for example, the current moment during the driving process of an autonomous mobile device (or called an agent). The autonomous mobile device may include a vehicle, a robot, a drone, etc. The first observation method may be any one of the observation methods determined according to requirements on the autonomous mobile device. The first target information includes the state information of the target, and the state information may, for example, include one or more of information such as the identifier of the target (such as the ID of the target), type, position, size, direction, speed, etc. The number of targets observed by the first observation method at the first moment is determined according to the environment where the autonomous mobile device is located.

[0032] In some alternative embodiments, each observation method may correspond to one or more sensors. The observation method may, for example, include one or more of observations such as forward view observation, rear view observation, side view observation, panoramic view observation, BEV observation, etc.

[0033] In some alternative embodiments, the observation method may be determined in combination with the sensor type. For example, forward camera observation, rear camera observation, lidar observation, ultrasonic radar observation, millimeter wave radar observation, and so on. The specific observation method is not limited.

[0034] In some alternative embodiments, for any observation method, it depends on the sensor data collected by the sensor corresponding to the observation method at its inherent frame rate, and then the collected sensor data is perceptually processed through the corresponding perception algorithm model to obtain an observation result (or called a perception result). The first moment may refer to the moment when the sensor corresponding to any observation method collects data. The observation result at the first moment includes the first target information of at least one target (or called an observation target, target object, target body, etc.) observed or perceived by the first observation method.

[0035] In some alternative embodiments, the observation results can be filtered to filter out unstable or less confident targets, thereby improving the effectiveness and reliability of the observation results. The confidence level can be obtained through a perception algorithm model. For example, the perception algorithm model processes sensor data and outputs observation results, where the observation results include the first target information and the confidence level of each of at least one observed target. The confidence level characterizes the credibility of the target, and thus targets with a lower confidence level can be filtered out based on the confidence levels of the observed targets. For example, for targets observed by an observation method such as radar observation, targets with a confidence level less than the confidence level threshold can be filtered out according to the confidence level output by the perception algorithm model.

[0036] In some alternative embodiments, the availability of targets can be classified according to the characteristics of the sensors corresponding to the observation methods. Further, the availability of each target can be classified according to the distance between each observed target and the autonomous mobile device. For targets with a distance exceeding the threshold, the availability level is unavailable, and the unavailable targets are filtered out. The thresholds for different observation methods can be set according to the characteristics of the sensors.

[0037] In some alternative embodiments, the first target information can be represented as structured information of a specified data structure. For example, the first target information can be represented as structured information such as vectors, arrays, etc., and the specific data structure is not limited.

[0038] In some alternative embodiments, if the data structure of the target information in the observation results does not meet the requirements of target tracking, the target information can be extracted from the observation results and encapsulated according to the data structure required for target tracking to obtain the first target information of the target.

[0039] Step 220, determine at least one first observation sequence set after tracking at least one target at the second moment.

[0040] Among them, the second moment is before the first moment; each target corresponds to a first observation sequence set, and the first observation sequence set includes at least one observation sequence. The target corresponding to the first observation sequence set can be called the second target, historical target or historical object, that is, relative to the first moment, the first observation sequence set belongs to the observation sequence set after historical moment target tracking. Each first observation sequence set corresponds to a historical target, that is, the observation sequences in the first observation sequence set are the observation sequences of the same target under different observation methods. Each observation sequence includes the state information of the target observed at at least one third moment, and the third moment is before the first moment. The at least one third moment may include the second moment. For example, if the first moment is the t moment and the second moment is the t-1 moment, the at least one third moment may include at least one of the t-1 moment, t-2 moment, t-3 moment, etc. For the sake of distinction, the state information of the target observed at the third moment can be called the second target information (or historical target information). For example, a certain first observation sequence set B is the observation sequence set corresponding to the target C. The first observation sequence set B includes two observation sequences, namely observation sequence 0 and observation sequence 1. Observation sequence 0 is the observation sequence obtained by the front view observation method, and observation sequence 1 is the observation sequence obtained by the BEV observation method. The first moment is the t moment. Observation sequence 0 includes the state information of target C corresponding to each of the t-n moment, t-n+1 moment, …, t-1 moment. The state information at any moment includes at least one of the information such as the ID, type, position, size, direction, speed, etc. of target C observed at that moment. The sequence formed by the state information at multiple moments is the observation sequence corresponding to target C, and the set of observation sequences of multiple observation methods is the first observation sequence set corresponding to target C.

[0041] In some alternative embodiments, the coordinate system corresponding to any one of the first target information and the second target information may include one or more of the vehicle coordinate system, the world coordinate system, the image coordinate system, etc. The coordinate systems can be transformed with each other. For example, the coordinate system transformation is performed based on the transformation parameters between the coordinate systems. For example, for the observation method depending on the camera perception (observation) result, the corresponding target information may include information such as ID, type, position, size, direction, speed, etc. in the vehicle coordinate system or the world coordinate system, and the two-dimensional detection box in the image coordinate system. The specific content of the target information is not limited.

[0042] In some alternative embodiments, the set of observation sequences after target tracking can be maintained in real time, that is, the life cycle of the set of observation sequences and the observation sequences in the set of observation sequences can be managed according to the observation results. For any set of observation sequences, the set of observation sequences corresponds to a target. If, in the observation of the target, the observation results of a preset number of frames do not have the observation of a certain observation method, the observation sequence corresponding to this observation method in the set of observation sequences is deleted. The preset number of frames can be 1 frame, 2 frames, 3 frames, …. If the number of frames when the observation sequences in the set of observation sequences are empty reaches the preset number of frames, indicating that the observation results of the preset number of frames do not include this target, the set of observation sequences can be deleted to ensure the effectiveness and reliability of the target tracking result. On this basis, the set of observation sequences after target tracking at the second moment can include the sets of observation sequences corresponding to all targets within the overall perception range.

[0043] In some alternative embodiments, all the sets of observation sequences after target tracking at the second moment can be used as the first set of observation sequences.

[0044] In some alternative embodiments, some sets of observation sequences can be selected from all the sets of observation sequences as the first set of observation sequences according to specified screening conditions. For example, according to the first observation method, some sets of observation sequences that may belong to the same target as the target observed at the first moment can be selected from all the sets of observation sequences as the first set of observation sequences to reduce the computational amount of correlation matching and improve the target tracking efficiency. For example, if the first observation method is forward observation, the target observed at the first moment through the first observation method should be the target within the forward observation range of the autonomous mobile device. Based on this, the sets of observation sequences that may belong to the target within the forward observation range are selected from the sets of observation sequences after tracking at the second moment as the first set of observation sequences. Optionally, the sets of observation sequences that may belong to the target within the forward observation range can be selected from each set of observation sequences by combining the target information (status information) of the target corresponding to each set of observation sequences at the moment closest to the first moment and the motion state of the autonomous mobile device.

[0045] Step 230: Based on the first target information and the first set of observation sequences, determine the first association relationships between the target and each first set of observation sequences.

[0046] Among them, the first association relationship between any target and any first set of observation sequences includes one of the target being associated with the first set of observation sequences and not being associated (or not being correlated, not being relevant). Being associated means that the target and the first set of observation sequences belong to the same target, and not being associated means that the target and the first set of observation sequences do not belong to the same target.

[0047] In some alternative embodiments, for each of at least one observed target, a first association relationship between the target and each first observation sequence set can be determined based on the first target information and the first observation sequence set of the target.

[0048] In some alternative embodiments, for any one of at least one observed target, the first target information of the target can be respectively matched with the second target information at at least one moment in each first observation sequence set, and based on the matching result, the first association relationship between the target and the first observation sequence set can be determined. In the process of matching the first target information with the second target information in the first observation sequence set, the second target information can be aligned to the first moment and matched with the first target information in the same space. For a static target, the second target information can be aligned to the first moment through coordinate system transformation. For a dynamic target, through target state prediction, the second target information can be aligned to the first moment. For example, based on the second target information, motion state information such as position, velocity, acceleration, etc. can be extracted, and through a kinematic model, the state information of the target from the moment corresponding to the second target information to the first moment can be predicted for matching with the first target information.

[0049] In some alternative embodiments, target feature information for matching with the first target information can be extracted from the first observation sequence set, and based on the matching between the first target information and the target feature information, the first association relationship between the target and the first observation sequence set can be determined. The target feature information can refer to the state information of predicting the second target at the first moment based on the second target information of at least one observation sequence in the first observation sequence set.

[0050] In some alternative embodiments, for any one first observation sequence set, a main observation sequence (or the first observation sequence) can be determined from each observation sequence in the first observation sequence set in a specified manner, and based on the second target information in the main observation sequence, the target feature information can be determined. Optionally, when the first observation sequence set includes the observation sequences corresponding to the first observation method (which can be called the second observation sequences), the target feature information can be determined based on the second target information in the second observation sequences. Or, based on the second target information in the first observation sequence, the first target feature is determined, and based on the second target information in the second observation sequence, the second target feature is determined. The first target feature and the second target feature are determined as the target feature information, and the target feature information is matched with the first target information to determine the first association relationship between the target and the first observation sequence set.

[0051] In some alternative embodiments, if the observation sequence corresponding to the first observation method is determined as the main observation sequence, then the above-mentioned second observation sequence and the first observation sequence are the same observation sequence, and the first target feature is determined based on the second target information in the observation sequence and used as the target feature information.

[0052] Step 240: Update the first observation sequence set based on the first target information and the first association relationship to obtain a second observation sequence set.

[0053] Among them, after obtaining the first association relationship between the target and each first observation sequence set, the first target information can be updated to the associated first observation sequence set according to the first association relationship, that is, the first target information is added to the corresponding observation sequence in the first observation sequence set to update the first observation sequence set and obtain a second observation sequence set. In the case of multiple targets, the first observation sequence set associated with the target can be updated according to the first target information and the first association relationship of each target to obtain a second observation sequence set.

[0054] In some alternative embodiments, for the first observation sequence set with associated targets, write the first target information of the associated targets into the first observation sequence set, and use the updated first observation sequence set as the second observation sequence set. For the first observation sequence set without associated targets, use the first observation sequence set as the second observation sequence set.

[0055] In some alternative embodiments, for a target without an associated first observation sequence set, a second observation sequence set corresponding to the target can be created based on the first target information of the target, that is, an empty observation sequence set is created, and the first target information of the target is used as the target information of the first frame of the observation sequence corresponding to the first observation method to update the newly created empty observation sequence set, and the updated observation sequence set is used as the second observation sequence set corresponding to the target.

[0056] In some alternative embodiments, it is possible to determine whether to create a second observation sequence set corresponding to the target according to the confidence level and / or availability level of the target. For example, for a target whose confidence level and / or availability level meet the conditions, create a second observation sequence set corresponding to the target, and for a target whose confidence level and / or availability level do not meet the conditions, the target can be deleted.

[0057] In some alternative embodiments, for the case where the second observation sequence corresponding to the first observation mode is included in the first observation sequence set, if the first observation sequence set is associated with the target, the second observation sequence is updated based on the first target information. For the case where the second observation sequence corresponding to the first observation mode is not included in the first observation sequence set, a second observation sequence is created in the first observation sequence set, and the second observation sequence is updated based on the first target information. By updating the second observation sequence, the second observation sequence includes the target information at the first moment, so that based on the first association relationship, the temporal fusion of the observed target and the first observation sequence set is realized.

[0058] In some alternative embodiments, for the case where the second observation sequence corresponding to the first observation mode is included in the first observation sequence set, if there is no target associated with the first observation sequence set, it indicates that the observation life cycle of the target by the first observation mode may end, that is, the target cannot be observed by the first observation mode, and the second observation sequence can be deleted from the first observation sequence set to obtain a second observation sequence set. Or, the second observation sequence can be temporarily retained, and when it is determined that the target has not been observed in a preset number of frames, the second observation sequence is deleted, so as to avoid the situation of being misdeleted due to environmental factors or other factors that cause temporary non-observation, and ensure the reliability of the observation sequence set.

[0059] Step 250, determine the second association relationships between each observation sequence in the second observation sequence set and each second observation sequence set.

[0060] Among them, the second association relationship between any observation sequence and any second observation sequence set indicates whether the observation sequence and the second observation sequence set belong to the same target. For any second observation sequence set, there may be observation sequences with incorrect associations, that is, the target corresponding to the observation sequence and the targets corresponding to other observation sequences in the second observation sequence set do not belong to the same target, and they are incorrectly associated together during the previous target tracking process. Or, multiple observation sequences that originally belonged to the same target are incorrectly considered to be different targets or are not associated temporarily, so that multiple observation sequences are located in different second observation sequence sets. As time goes by, the second association relationship between the observation sequence and the second observation sequence set can be changed by combining the change in the matching degree between the observation sequence and the second observation sequence set.

[0061] In some alternative embodiments, the second association relationship between the observation sequence and the second observation sequence set can be determined according to the matching degree between the target at at least one moment in the observation sequence and the second observation sequence set. The matching degree can be represented by similarity. The matching degree between the target at any moment and the second observation sequence set can be obtained and stored during the process of determining the association relationship between the target at this moment and the observation sequence set. For example, during the process of determining the first association relationship described above, the matching degree between the observed target and each first observation sequence set is determined and stored. When the target is added to the associated first observation sequence set to obtain the second observation sequence set, the matching degree between the target and the first observation sequence set is the matching degree between the target at the first moment of the corresponding observation sequence in the second observation sequence set and the second observation sequence set. In this way, the matching degree between the target at each moment in the observation sequence and each second observation sequence set can be obtained from the storage area, and thus can be used to determine the second association relationship between each observation sequence in the second observation sequence set and each second observation sequence set. As new target information at different moments is continuously added to the observation sequence, the matching degree between the targets at different moments and the second observation sequence set also changes. Therefore, the matching degree of the overall observation sequence and the second observation sequence set changes, so by determining the second association relationship between each observation sequence in the second observation sequence set and each second observation sequence set during each target tracking process, the effectiveness of the observation sequences in the observation sequence set can be effectively ensured.

[0062] In some alternative embodiments, since at the first moment, only some of the second observation sequence sets may be updated relative to the first observation sequence set, and only some of the observation sequences in the second observation sequence set may be updated. For the unupdated observation sequences and the unupdated second observation sequence sets, their association relationships do not change. Based on this, when determining the second association relationship, for the unupdated observation sequences, only the second association relationship with the updated second observation sequence set can be calculated; for the updated observation sequences, the second association relationship with each second observation sequence set can be calculated to reduce the computational amount.

[0063] In some alternative embodiments, the observation sequences to be associated (referred to as the third observation sequences) and the second set of observation sequences can be selected according to specified conditions. For example, the states of the set of observation sequences can be marked, and the states of the set of observation sequences can include a stable state and a floating state. The floating state means that the set of observation sequences only includes one floating observation sequence, and the floating observation sequence refers to the observation sequence of an unstable target. For an observed unstable target, it is maintained through the floating observation sequence, that is, the second set of observation sequences in the floating state only includes floating observation sequences. The type of observation method refers to the observation method corresponding to the observation sequence. The observation sequences to be associated and the second set of observation sequences can be selected according to the type of the observation sequence and the state of the set of observation sequences. For example, for the second set of observation sequences in the floating state, the floating observation sequences in the second set of observation sequences are used as the observation sequences to be associated. These observation sequences may belong to the same target as other second sets of observation sequences, and then these observation sequences can be merged with the associated second set of observation sequences. For another example, for the second set of observation sequences that only includes the observation sequences corresponding to the first observation method, the observation sequences in the second set of observation sequences are used as the observation sequences to be associated, and the second association relationships between the observation sequences and other second sets of observation sequences are determined to judge whether the observation sequences can be merged into other second sets of observation sequences. For another example, for the second set of observation sequences to which the first target information at the first moment is added according to the first association relationship, the observation sequences to which the first target information is added can be used as the observation sequences to be associated, and the association relationships between the observation sequences and the second set of observation sequences to which they currently belong and other second sets of observation sequences are determined. It is judged whether the association relationship between the observation sequences and the second set of observation sequences to which they currently belong has changed. If the observation sequences are not associated with the second set of observation sequences to which they currently belong, the observation sequences can be removed from the second set of observation sequences to which they currently belong, or removed after it is determined that they are not associated for multiple consecutive frames. If the observation sequences are associated with a certain other second set of observation sequences, they can be moved from the second set of observation sequences to which they currently belong to the other second set of observation sequences. If the observation sequences are not associated with any second set of observation sequences, it means that the life cycle of the observation sequences ends, and the observation sequences can be deleted.

[0064] Step 260, based on the second association relationship, update the second set of observation sequences to obtain a third set of observation sequences.

[0065] Among them, the update of the second observation sequence set based on the second association relationship may include merging (or reorganization), splitting, life cycle management, etc. Reorganization means moving the observation sequences in the second observation sequence set corresponding to target A to the second observation sequence set corresponding to target B for reorganization based on the second association relationship to obtain a third observation sequence set. Splitting means moving the observation sequences in the second observation sequence set corresponding to target D out of the second observation sequence set based on the second association relationship. Optionally, the moved observation sequence can be used as a free observation sequence, and a corresponding free-state observation sequence set can be created as the third observation sequence set. Life cycle management means determining whether to continue maintaining the second observation sequence set or end the second observation sequence set according to whether there are still remaining observation sequences in the second observation sequence set after reorganization and splitting. For example, if a target gradually cannot be observed by at least one observation method, the number of observation sequences in the second observation sequence set corresponding to the target will become fewer and fewer until the target cannot be observed by all observation methods and the number of observation sequences becomes zero, and the second observation sequence set becomes an empty set, indicating that the life cycle of the observation sequence set of the target ends, and the second observation sequence set can be deleted, or if it is determined that the second observation sequence set is an empty set for multiple consecutive frames, the second observation sequence set is deleted.

[0066] In some alternative embodiments, similar to the update of the first observation sequence set, there may be a second observation sequence set that needs to be updated in the second observation sequence set. The updated observation sequence set is used as the third observation sequence set, and for the second observation sequence set that is not updated, the second observation sequence set is used as the third observation sequence set.

[0067] Step 270, determine the target tracking result at the first moment based on the third observation sequence set.

[0068] Among them, the third observation sequence set corresponding to each target represents the state trajectory of the target at different observation methods and different moments. Therefore, based on the third observation sequence set, the target tracking result at the first moment can be determined, and the state information of the target at the first moment and each historical moment before the first moment can be obtained for downstream planning and control to enable the autonomous mobile device to drive safely.

[0069] The target method provided in this embodiment, when obtaining the first target information of the target observed at the first moment, obtains the first association relationship between the target and each observed sequence set (i.e., the first observed sequence set) after tracking the target at the second moment through temporal association, where the first observed sequence set represents a set of observed sequences obtained by observing the same target through one or more observation methods, and the first association relationship represents whether the target observed at the first moment belongs to the first observed sequence set. Thus, the temporal correlation can be fully utilized to update the first target information of the observed target to the corresponding first observed sequence set for temporal fusion to obtain the second observed sequence set, effectively improving the accuracy and effectiveness of the tracking result. Furthermore, by determining the second association relationship between each observed sequence in the second observed sequence set and each second observed sequence set, spatial association is achieved, enabling the missed-associated observed sequences to be incorporated into the corresponding second observed sequence set and the mis-associated observed sequences to be split out from the second observed sequence set for spatial fusion. Therefore, the accuracy and effectiveness of the target tracking result can be effectively improved, providing a reliable target tracking result for downstream planning and control, thereby enhancing the safety of vehicle driving.

[0070] Figure 3 It is a flowchart of a target tracking method provided in another exemplary embodiment of the present disclosure.

[0071] In some optional embodiments, based on the above Figure 2 illustrated embodiment, as Figure 3 illustrated, determining at least one first observed sequence set after tracking at least one target at the second moment in step 220 may include:

[0072] Step 2201: Based on the first observation method, determine, from each observed sequence set after tracking at least one target at the second moment, the observed sequence set that meets the first condition as the first observed sequence set.

[0073] Among them, the first observation method can be referred to the foregoing embodiments, and the first condition can refer to the relative position condition. Different observation methods correspond to a certain observation range. According to the first observation method, the corresponding observation range can be determined, that is, the observation target at the first moment is the target within the observation range. Each observation sequence set belongs to the historical state of the historical target. Combining the relative position relationship between the historical state of the historical target and the observation range at the first moment, the observation sequence set that meets the first condition is determined from each observation sequence set as the first observation sequence set. For example, if the first observation method is the forward-looking observation method, the target observed at the first moment through the first observation method should be the target within the forward observation range of the autonomous mobile device. Based on this, the observation sequence set of the target that may belong to the forward observation range is selected from the observation sequence set after tracking at the second moment as the first observation sequence set. Optionally, the observation sequence set of the target that may belong to the forward observation range can be selected from each observation sequence set by combining the target information (status information) of the target corresponding to each observation sequence set at the moment closest to the first moment and the motion state of the autonomous mobile device. For example, based on the forward observation range, the selection range is expanded according to the motion state of the autonomous mobile device, and whether the state of the target corresponding to the observation sequence set at the nearest moment is within the selection range is used to determine the observation sequence that meets the first condition. Optionally, for the observation sequence set corresponding to the dynamic target, the motion state of the dynamic target corresponding to each observation sequence set can also be combined to predict the state of the dynamic target at the first moment, and then according to the predicted state at the first moment and the selection range, the observation sequence that meets the first condition is selected.

[0074] In the embodiments of the present disclosure, by determining the observation sequence set that meets the first condition from each observation sequence set after target tracking at the second moment as the first observation sequence set, for the observation sequence set that cannot be within the observation range of the first observation method, subsequent target matching, association and other processes can be avoided, which can effectively reduce the target tracking calculation amount and improve the target tracking processing efficiency.

[0075] Figure 4 It is a schematic flowchart of the target tracking method provided by another exemplary embodiment of the present disclosure.

[0076] In some optional embodiments, based on any of the above embodiments, step 230 of determining the first association relationship between the target and each first observation sequence set based on the first target information and the first observation sequence set may include:

[0077] Step 2310, determining the target feature information corresponding to each first observation sequence set.

[0078] Among them, the target feature information includes the target features of at least one observation sequence in the first observation sequence set. There is no limit on the number of observation sequences included in the at least one observation sequence. For example, the target feature information may include the target features of each observation sequence in the first observation sequence set, or the target feature information includes the target features of the first observation sequences whose observation confidence levels (or simply confidence levels) in the first observation sequence set meet the confidence conditions (i.e., the first target features), or the target feature information includes the target features of the second observation sequences corresponding to the first observation method in the first observation sequence set (i.e., the second target features), or the target feature information includes the first target features and the second target features. The specific target feature information can be referred to in the foregoing embodiments. The confidence level is determined according to the distance between the target and the autonomous mobile device and the sensor performance of different observation methods. For example, for a target at a long distance, the confidence level of the lidar observation result is higher, and for a target at a short distance, the confidence level of the camera observation result is higher. Based on this, the observation sequence with the highest confidence level in the first observation sequence set is used as the first observation sequence.

[0079] Step 2320, determine the first similarity between the first target information and each piece of target feature information.

[0080] Among them, the first similarity between the first target information and any piece of target feature information characterizes the matching degree between the first target information and the target feature information. The higher the first similarity, the higher the matching degree, indicating that the target corresponding to the first target information and the target corresponding to the target feature information are more likely to belong to the same target.

[0081] In some optional embodiments, the first similarity between the first target information and the target feature information can be determined based on the Intersection over Union (IOU), Mahalanobis distance, cosine similarity, etc. The specific way to determine the first similarity can be determined according to the specific content of the first target information. For example, if the first target information includes a two-dimensional detection box in the image coordinate system and the extracted target feature information is a two-dimensional projection box in the image coordinate system, the IOU between the two-dimensional detection box of the first target information and the two-dimensional projection box of the target feature information can be determined based on the two-dimensional IOU as the first similarity. If the first target information includes the state information in the three-dimensional coordinate system and the target feature information is the feature extracted in the three-dimensional coordinate system, the first similarity between the first target information and the target feature information can be determined based on the multi-dimensional Mahalanobis distance, cosine similarity, etc. The multi-dimensional Mahalanobis distance can include the three-dimensional Mahalanobis distance, four-dimensional Mahalanobis distance, etc. For example, for the first target information of a target observed by Radar, based on the horizontal and vertical coordinates (X, Y) and the longitudinal velocity V of the target in the first target information X, the horizontal and vertical coordinates and vertical speed in the target feature information are used to determine the first similarity through the three-dimensional Mahalanobis distance. For another example, for pedestrians observed visually, the first similarity can be determined through the four-dimensional Mahalanobis distance based on the horizontal and vertical coordinates and horizontal and vertical speeds. For other vehicles observed visually, the first similarity can be determined through the IOU in the image coordinate system.

[0082] Step 2330, based on the first similarity, determine the first association relationship between the target and each first observation sequence set.

[0083] Among them, the first similarity characterizes the matching degree between the target and the first observation sequence set. Therefore, based on the first similarity, the first association relationship between the target and each first observation sequence set can be determined. For example, if the first similarity between the target and any first observation sequence set is greater than the similarity threshold, it is determined that the target is associated with the first observation sequence set.

[0084] In some optional embodiments, according to the first similarities between the target and each first observation sequence set, the first observation sequence set with the largest first similarity can be selected as the first observation sequence set associated with the target, and it is determined that the other first observation sequence sets are not associated with the target. Or, further based on the similarity threshold, the first observation sequence set with the largest first similarity is judged. If the largest first similarity is greater than or equal to the similarity threshold, it is determined that the first observation sequence set is associated with the target. If the largest first similarity is less than the similarity threshold, it indicates that the target may be a new target, or there is a large observation error due to environmental factors, etc., and the target can be temporarily determined as not matching the associated first observation sequence set. Optionally, a corresponding free observation sequence can be created for the target that does not match the associated first observation sequence set, and a corresponding observation sequence set can be created.

[0085] In the embodiments of the present disclosure, by matching the first target information of the target observed at the first moment with the target feature information of the first observation sequence set, the first similarity is obtained, providing an accurate and effective similarity reference for determining the first association relationship between the target and the first observation sequence set, thereby ensuring the effectiveness of the first association relationship.

[0086] In some optional embodiments, on the basis of the above embodiments, determining the target feature information corresponding to each first observation sequence set in step 2310 may include:

[0087] For any first observation sequence set in each first observation sequence set, determine the first target feature of the first observation sequence in the first observation sequence set and the second target feature of the second observation sequence as the target feature information; the first observation sequence is an observation sequence that satisfies the confidence condition; the second observation sequence is the observation sequence corresponding to the first observation method.

[0088] Among them, the confidence condition refers to the condition related to the observation confidence level. For example, the confidence condition is that the observation confidence level in the first observation sequence set is the highest. That is, the observation sequence with the highest observation confidence level in the first observation sequence set is used as the first observation sequence. The observation sequence corresponding to the first observation method is an observation sequence formed by the target information observed at different times through the first observation method. The first target feature of the first observation sequence is the target feature aligned to the first time determined based on the target information at at least one time in the first observation sequence. Similarly, the second target feature of the second observation sequence is the target feature aligned to the first time determined based on the target information at at least one time in the second observation sequence. For the specific alignment method, refer to the foregoing embodiments.

[0089] In some alternative embodiments, the first observation sequence and the second observation sequence may be the same observation sequence, and then the first target feature and the second target feature are the same target feature.

[0090] In some alternative embodiments, for any first observation sequence set, the second observation sequence may not be included in the first observation sequence set, and then the second target feature is empty or there is no need to determine the second target feature.

[0091] In some alternative embodiments, for the case where the first observation sequence set only includes free observation sequences, it can be determined that the first observation sequence (i.e., the main observation sequence) is not included in the first observation sequence set, or the free observation sequences can be used as the first observation sequence.

[0092] In the embodiments of the present disclosure, since the first observation sequence in the first observation sequence set is an observation sequence with a relatively high confidence level, and the second observation sequence is the observation sequence corresponding to the first observation method, the confidence and effectiveness of the first target feature of the determined first observation sequence and the second target feature of the second observation sequence are higher. Furthermore, the first target feature and the second target feature are used as target feature information to be matched with the first target information, and the first association relationship between the target and the first observation sequence set is obtained, which can improve the accuracy and effectiveness of the first association relationship.

[0093] In some alternative embodiments, determining the target feature information corresponding to each first observation sequence set in step 2310 includes:

[0094] For any first observation sequence set, based on the target information observed at at least one third time in the first observation sequence set, predict the target information at the first time as the target feature information.

[0095] Among them, the third moment is before the first moment, that is, the third moment is a historical moment relative to the first moment. The number of third moments can be any number, which can be specifically set according to actual needs. For example, if the first moment is the t moment, any third moment can be one of the moments such as t - 1 moment, t - 2 moment, t - 3 moment, etc.

[0096] In some alternative embodiments, when the first observation sequence set includes only one observation sequence, the target information (which can be referred to as the second target information) observed at at least one third moment in this observation sequence can be used to predict the target information (which can be referred to as the target prediction information) of the target (i.e., the historical target corresponding to this observation sequence) at the first moment as the target feature information. It should be emphasized that a certain third moment among at least one third moment can be consistent with the second moment. For example, if the second moment is the t - 1 moment, at least one third moment can include the t - 1 moment, t - 2 moment, and t - 3 moment.

[0097] In some alternative embodiments, when the first observation sequence set includes multiple observation sequences, the target information of the target at the first moment can be predicted respectively based on the first observation sequence and the second observation sequence in this first observation sequence set as the first target feature and the second target feature, and the first target feature and the second target feature are determined as the target feature information, referring to the foregoing embodiments. Among them, the first target feature is predicted based on the target information observed at at least one third moment in the first observation sequence, and the second target feature is predicted based on the target information observed at at least one third moment in the second observation sequence.

[0098] In some alternative embodiments, for the specific operation of predicting the target information of the target at the first moment based on the target information observed at at least one third moment, reference may be made to the operation of aligning the second target information to the first moment in the foregoing embodiments. That is, for a static target, the target information of the static target at the first moment can be determined through coordinate system transformation. For a dynamic target, the target information of the dynamic target at the first moment can be predicted through a kinematic model. For example, for a static target, based on the transformation relationship between the coordinate system at the third moment and the coordinate system at the first moment, the target information of the static target can be transformed from the coordinate system at the third moment to the coordinate system at the first moment to achieve the unification of the coordinate systems of the target information. For example, it is transformed from the ego-vehicle coordinate system at the third moment to the ego-vehicle coordinate system at the first moment. When the autonomous mobile device is in a moving state, the poses of the ego-vehicle coordinate systems at different moments are different relative to the global coordinate system (such as the world coordinate system or the ego-vehicle coordinate system at the starting position of the autonomous mobile device). The pose of the autonomous mobile device in the global coordinate system is determined through the sensing device, the ego-vehicle coordinate systems at different moments are determined based on the poses of the autonomous mobile device at different moments, and the transformation of the coordinate systems is achieved by transforming the ego-vehicle coordinate system at the third moment to the global coordinate system and then from the global coordinate system to the ego-vehicle coordinate system at the first moment. Details are not described herein again. For a dynamic target, the target information of the dynamic target at the first moment can be predicted based on the relative motion of the dynamic target relative to the autonomous mobile device and in combination with a kinematic model. The kinematic model may include, for example, a uniform motion model, a uniformly accelerated motion model, a variable accelerated motion model, etc.

[0099] In the embodiments of the present disclosure, by predicting the target information of the historical target corresponding to the first observation sequence set at the first moment as the target feature information, the target feature information can effectively represent the state of the historical target at the first moment, improve the effectiveness of the target feature information, and thus unify the historical target and the observed target at the first moment into the same space, improving the accuracy of the first similarity between the first target information and the target feature information.

[0100] In some alternative embodiments, determining the first similarity between the first target information and each target feature information in step 2320 may include:

[0101] Determining the first sub-similarity between the first target information and the first target feature, and the second sub-similarity between the first target information and the second target feature; and determining the first sub-similarity and the second sub-similarity as the first similarity.

[0102] In some alternative embodiments, the first sub-similarity and the second sub-similarity may be determined based on one or more of IOU, Mahalanobis distance, cosine similarity, etc. The specific principle is not described herein again.

[0103] In an embodiment of the present disclosure, since the first sub-similarity characterizes the matching degree between the target and the first observation sequence in the first observation sequence set, and the second sub-similarity characterizes the matching degree between the target and the second observation sequence in the first observation sequence set, determining the first association relationship between the target and the first observation sequence set by combining the matching degrees of the target with the first observation sequence and the second observation sequence respectively helps to improve the effectiveness of the first association relationship.

[0104] In some alternative embodiments, determining the first similarities between the first target information and each target feature information in step 2320 includes:

[0105] Determining the type of the first observation method; based on the type of the first observation method, determining the similarity calculation method; based on the similarity calculation method, determining the first similarities between the target and each target feature information respectively.

[0106] Among them, the type of the first observation method may include forward view observation, rear view observation, BEV observation method, lidar (Lidar) observation, ultrasonic radar (Ultrasonic Sensor System, abbreviated as: USS) observation, millimeter wave radar observation, etc. The similarity calculation method may include calculating similarity based on tracking ID, calculating similarity based on IOU, calculating similarity based on Mahalanobis distance, and so on. For some types of observation methods, the tracking ID of the target may be included in the observation result, and then the first similarity may be calculated according to the tracking ID of the target and the tracking ID in the target feature information. For the observation methods whose observation results do not include the tracking ID, corresponding similarity calculation methods may be specified respectively, and according to the type of the first observation method and the similarity calculation method specified for each type of observation method, the similarity calculation method corresponding to the first observation method is determined. Then, based on the similarity calculation method, the first similarities between the target and each target feature information are determined. For example, for forward view observation and rear view observation, the first similarity may be determined by the IOU in the image coordinate system. For BEV observation, the first similarity may be determined by the two-dimensional projection IOU in the BEV coordinate system. For Radar observation, the first similarity is determined based on one or more of three-dimensional Mahalanobis distance, cosine similarity, etc. Radar observation may include at least one of lidar observation, ultrasonic radar observation, millimeter wave radar observation, etc. Alternatively, in the case where the similarity calculated by the tracking ID cannot associate the target with the first observation sequence set, the first similarity may be further calculated based on IOU, Mahalanobis distance, etc.

[0107] In some alternative embodiments, the similarity calculation method can be determined in combination with the type of the first observation method and the type of the observation target. For example, for a pedestrian target observed visually, the first similarity is determined based on the four-dimensional Mahalanobis distance. For other vehicle targets observed visually, the first similarity is determined based on the two-dimensional IOU in the BEV and image coordinate systems.

[0108] In some alternative embodiments, for the case where the observation result includes a tracking ID, the target feature information may include the tracking IDs of historical targets extracted from the first observation sequence set.

[0109] In the embodiments of the present disclosure, corresponding similarity calculation methods are respectively used for different observation methods and different targets to determine the first similarity, which can ensure the accuracy and effectiveness of the first similarity.

[0110] In some alternative embodiments, on the basis of the above embodiments, determining the first association relationship between the target and each first observation sequence set based on the first similarity in step 2330 may include:

[0111] Determine the type of the first observation method; in response to the type of the first observation method being the first type, determine the first association relationship between the target and each first observation sequence set based on each second sub-similarity; in response to the type of the first observation method being the second type, determine the first association relationship between the target and each first observation sequence set based on each second sub-similarity and each first sub-similarity.

[0112] Among them, the first sub-similarity is the similarity between the first target information and the first target feature corresponding to the first observation sequence. The second sub-similarity is the similarity between the first target information and the second target feature corresponding to the second observation sequence. The first observation sequence is the observation sequence in the first observation sequence set that meets the confidence condition. The second observation sequence is the observation sequence in the first observation sequence set corresponding to the first observation method. The type of the first observation method can be referred to the foregoing embodiments. The first type may include the Radar observation method, the second type may include the visual observation method, and the visual observation method may include the front view observation, the rear view observation, the BEV observation, etc. For the Radar observation method, the first association relationship between the target and each first observation sequence set can be determined only based on the second sub-similarity. For example, based on the second sub-similarities between the target and each first observation sequence set, the first association relationship between the target and each first observation sequence set is obtained through the greedy algorithm. The greedy algorithm means taking the first observation sequence set with the largest second sub-similarity as the first observation sequence set associated with the target. For the visual observation method, the first association relationship between the target and each first observation sequence set can be determined by combining the second sub-similarity and the first sub-similarity. For the case where multiple targets are observed through the first observation method, for example, first, based on the second sub-similarity, the greedy algorithm can be used to determine the first association relationship between each target and each first observation sequence set. For the unassociated observation targets and the first observation sequence sets, the first sub-similarity is used for supplementary association. For example, based on the first sub-similarity, the KM (Kuhn-Munkres Algorithm, weighted Hungarian algorithm) algorithm is used to determine the first observation sequence sets associated with each target respectively, and the first association relationship between the target and each first observation sequence set is obtained. The KM algorithm is a weighted bipartite graph matching algorithm. The first sub-similarity is used as the weight, and the first association relationship between each target and each first observation sequence set is determined through the KM algorithm.

[0113] It can be understood that in practical applications, the observation method may not be distinguished, and the first sub-similarity may be used to determine the first association relationship, or the second sub-similarity may be used to determine the first association relationship, or both the first sub-similarity and the second sub-similarity may be used to determine the first association relationship.

[0114] In the embodiments of the present disclosure, by using the respective more optimal sub-similarities for different types of observation methods to determine the first association relationship, the effectiveness of the first association relationship can be further improved.

[0115] Figure 5 It is a schematic flowchart of a target tracking method provided by another exemplary embodiment of the present disclosure.

[0116] In some optional embodiments, on the basis of any of the above embodiments, such as Figure 5As shown in the figure, the step 240 of updating the first observation sequence set based on the first target information and the first association relationship to obtain the second observation sequence set may include:

[0117] Step 2410, in response to determining, based on the first association relationship, a fourth observation sequence set associated with the target from at least one first observation sequence set, updating the fourth observation sequence set to obtain the second observation sequence set.

[0118] Among them, the fourth observation sequence set associated with the target refers to the first observation sequence set with which the first association relationship is associated. Since the first association relationship between the target and the first observation sequence set represents whether the target is associated with the first observation sequence set, therefore, based on the first association relationship, the first observation sequence set associated with the target can be obtained. The first observation sequence set associated with the target is used as the fourth observation sequence set. For each observation target, the number of the determined fourth observation sequence sets associated with the observation target may be 0 or 1.

[0119] In some optional embodiments, updating the fourth observation sequence set may include updating the fourth observation sequence set based on the first target information of the target. For example, writing the first target information into the observation sequences in the fourth observation sequence set to obtain the second observation sequence set.

[0120] Step 2420, in response to not determining a first observation sequence set associated with the target from at least one first observation sequence set, creating an observation sequence set corresponding to the target as the second observation sequence set.

[0121] Among them, for any observation target, if no first observation sequence set associated with the target is determined, it means that the target may be a newly emerged target, or the first target information of the observed target is affected by environmental factors or other factors, etc., affecting the association result. In this case, an observation sequence set corresponding to the target can be created as the second observation sequence set of the target to participate in subsequent processing.

[0122] In the embodiments of the present disclosure, by updating the target associated with the first observation sequence set to the associated first observation sequence set and creating an observation sequence set corresponding to the target that is not associated with the first observation sequence set, the effectiveness of the observation sequence set is ensured.

[0123] In some optional embodiments, the updating of the fourth observation sequence set in step 2410 may include: in response to the second observation sequence being included in the fourth observation sequence set, writing the first target information into the second observation sequence to obtain the second observation sequence set. In response to the second observation sequence not being included in the fourth observation sequence set, creating a second observation sequence, writing the first target information into the second observation sequence to obtain the second observation sequence set.

[0124] Among them, the second observation sequence is the observation sequence corresponding to the first observation method.

[0125] In some optional embodiments, for any fourth observation sequence set, if the second observation sequence is included in the fourth observation sequence set, it indicates that the target has been observed by the first observation method before the first moment. The first target information of the target can be written into the corresponding second observation sequence to extend the life cycle of the second observation sequence, and the observation sequence set after writing the first target information is used as the second observation sequence set.

[0126] In some optional embodiments, if the second observation sequence is not included in the fourth observation sequence set, it indicates that the target has not been observed by the first observation method before the first moment, and the first moment is the first time the target is observed. A second observation sequence corresponding to the first observation method can be created in the fourth observation sequence set, the first target information is written into the second observation sequence, and the updated observation sequence set is used as the second observation sequence set.

[0127] In the embodiments of the present disclosure, by updating the observed target to the second observation sequence in the first observation sequence set associated therewith, the real-time management of the life cycle of the observation sequence is realized, and the effectiveness of the target tracking result is ensured.

[0128] In some optional embodiments, on the basis of the above embodiments, creating the observation sequence set corresponding to the target in step 2420 may include: creating the observation sequence set corresponding to the target based on the first target information as the second observation sequence set of the target.

[0129] Among them, for any observed target, if the first observation sequence set associated with the target is not determined, it indicates that the target may be a newly emerged target, or the first target information of the observed target is affected by environmental factors or other factors, etc., affecting the association result. In this case, the observation sequence set corresponding to the target can be created as the second observation sequence set of the target to participate in subsequent processing. The second observation sequence set only contains the first target information of the target at the first moment, and the first target information is used as the target information at the first moment of the second observation sequence corresponding to the first observation method. The second observation sequence can be marked as a free observation sequence, or the second observation sequence set can be marked as a free observation sequence set.

[0130] In the embodiments of the present disclosure, for the target that fails to match the first observation sequence set, the life cycle management of the observation sequence set of the target is realized by creating the observation sequence set of the target, and the effectiveness of the observation sequence set is ensured.

[0131] In some alternative embodiments, based on any of the above embodiments, creating the observation sequence set corresponding to the target in step 2420 may include:

[0132] In response to the type of the first observation method being the second type, determine the confidence level of the target. In response to the confidence level meeting the preset level condition, create the observation sequence set corresponding to the target as the second observation sequence set of the target.

[0133] Wherein, the second type may refer to the visual observation method.

[0134] In some alternative embodiments, the confidence level of the target may be obtained by classifying the confidence level of the target determined by the perception algorithm model. For example, when the confidence level of the target is less than the first threshold, it is level I; when it is greater than the first threshold and less than the second threshold, it is level II; when it is greater than the second threshold, it is level III. The specific level classification is not limited.

[0135] In some alternative embodiments, the confidence level of the target may be determined according to the performance of the sensor corresponding to the observation method for observing the target. Different types of sensors may use different methods to determine the confidence level of the target. Optionally, the confidence level of the target may be obtained by classifying the confidence level according to the distance between the target and the autonomous mobile device and the performance of the sensor corresponding to the observation method. For example, the confidence level in the short-distance segment is level III, the confidence level in the middle-distance segment is level II, and the confidence level in the long-distance segment is level I. The short-distance segment, middle-distance segment, and long-distance segment corresponding to different observation methods or different types of sensors may be different. For example, for the first camera and the second camera with different FOVs (Field of View), if the FOV of the first camera is greater than that of the second camera, and the first camera and the second camera observe the same target at a long distance, the confidence level of the target observed by the first camera is less than the confidence level of the target observed by the second camera. Optionally, the number of confidence levels corresponding to different observation methods or different types of sensors may be different. The specific number of confidence levels divided and the level classification method are not limited. Based on this, the confidence level of the target may be determined according to the performance of the sensor corresponding to the first observation method. When the confidence level of the target meets the preset level condition, create the observation sequence set corresponding to the target as the second observation sequence set of the target.

[0136] In some alternative embodiments, the preset level condition may be that the confidence level is higher than a specified level. For example, the confidence levels include level III (or the third level), level II (or the second level), and level I (or the first level). The preset level condition may be level II and above, or above level II, where the levels are ranked from high to low as level III, level II, and level I. The specific preset level condition may be set according to actual requirements. When the confidence level of the target meets the preset level condition, an observation sequence set corresponding to the target is created as the second observation sequence set of the target.

[0137] In some alternative embodiments, in response to the confidence level not meeting the preset level condition, or the type of the first observation method being the first type, the first target information is deleted.

[0138] Among them, if the confidence level of the target does not meet the preset level condition, it means that the confidence level of the target is relatively low, and the corresponding observation sequence set may not be created temporarily. For example, if the distance between the target and the autonomous mobile device is very far, and it is determined that the confidence level of the target does not meet the preset level condition, since it will not have an impact on the autonomous mobile device temporarily, the observation sequence set may not be created temporarily. The first type is the Radar observation method. If the type of the first observation method is the first type and the first observation sequence set associated with the target has not been determined, it means that there may be a false detection situation for the target, and the first target information is temporarily deleted. Even if the target actually exists, it can be associated with the corresponding observation sequence set in a timely manner during the target tracking process at a subsequent moment.

[0139] In the embodiments of the present disclosure, by further determining whether to create the second observation sequence set of the target according to the confidence level of the target, the corresponding observation sequence set is created only for the target with a higher confidence level, and the corresponding observation sequence set may not be created temporarily for the target with a lower confidence level, which can improve the reliability and effectiveness of the second observation sequence set.

[0140] In some alternative embodiments, based on any of the above embodiments, it further includes:

[0141] Step 310, for any first observation sequence set in at least one first observation sequence set, in response to the target associated with the first observation sequence set not being determined, the second observation sequence in the first observation sequence set is deleted to obtain a second observation sequence set.

[0142] Among them, the second observation sequence is the observation sequence corresponding to the first observation method.

[0143] In some alternative embodiments, if the target associated with the first observation sequence set is not determined and the first observation sequence set includes a second observation sequence, it indicates that it may no longer be possible to observe the target corresponding to the first observation sequence set through the first observation method, that is, the life cycle of the second observation sequence in the first observation sequence set ends. The second observation sequence can be deleted from the current first observation sequence set to obtain a second observation sequence set. For example, the first observation method is a forward-looking observation method, and the first observation sequence set corresponds to target D. Before the first moment, target D can be observed through the forward-looking observation method, and there is a second observation sequence in the first observation sequence set. At the first moment, target D is not observed. For example, as the autonomous mobile device moves, target D changes from in front of the autonomous mobile device to on the right side of the autonomous mobile device. At the first moment, target D cannot be observed through the first observation method, so the target associated with the first observation sequence set cannot be determined, and the second observation sequence can be deleted to end the observation of target D by the first observation method.

[0144] In some alternative embodiments, when the target associated with the first observation sequence set is not determined, the consecutive frame number in which the target associated with the first observation sequence set is not determined through the first observation method can be further determined. In response to the consecutive frame number exceeding the specified frame number threshold, the second observation sequence in the first observation sequence set is deleted to obtain a second observation sequence set. In response to the consecutive frame number not exceeding the specified frame number threshold, the second observation sequence can be temporarily retained to avoid the situation of erroneously deleting the second observation sequence due to environmental factors or other factors that cause the target to be temporarily unobserved or the observed target has a large error and is temporarily not associated, ensuring the effectiveness and reliability of the life cycle of the observation sequence.

[0145] Figure 6 It is a schematic flowchart of the target tracking method provided by another exemplary embodiment of the present disclosure.

[0146] In some alternative embodiments, based on any of the above embodiments, the observation sequence includes second target information of the target observed at at least one moment by the same observation method.

[0147] Based on any of the above embodiments, as Figure 6 shown, determining the second association relationships between the observation sequences in the second observation sequence set and the second observation sequence sets respectively in step 250 may include:

[0148] Step 2510, determine a third observation sequence to be associated from each of the second observation sequence sets.

[0149] Among them, according to pre-configured screening rules, third observation sequences to be associated can be determined from each second observation sequence set. The number of third observation sequences can be one or more. The screening rules can be set according to the principle that there may be requirements for recombination and splitting. For example, the screening rules can include: determining the observation sequences or the changed partial observation sequences in the second observation sequence set that have changed during the update based on the first association relationship as the third observation sequences, and / or determining the free observation sequences in the second observation sequence set that only contains free observation sequences as the third observation sequences, and / or taking the second observation sequence in the second observation sequence set that only contains the second observation sequence corresponding to the first observation method as the third observation sequence, and so on. There is a possibility of recombination with other second observation sequence sets or splitting from the currently affiliated second observation sequence set for these observation sequences. Therefore, these observation sequences are used as the third observation sequences to be associated. For other observation sequences that completely have no possibility of recombination or splitting, they can not participate in the current secondary association (i.e., the association between the observation sequence and the second observation sequence set) to reduce the amount of calculation and improve the processing efficiency of target tracking.

[0150] Step 2520: Determine the second similarities between the second target information of the targets observed at at least one moment in the third observation sequences and each second observation sequence set.

[0151] Among them, the second similarity between the second target information at any moment and any second observation sequence set can be obtained and stored during the target tracking process at that moment. The specific principle can refer to the similarity between the first target information and the first target feature (i.e., the first sub-similarity) and / or the second sub-similarity between the first target information and the second target feature at the first moment in the foregoing embodiment, which will not be elaborated here. For example, if the third observation sequence is the observation sequence corresponding to the first observation method, after the above first association update (i.e., updating the first observation sequence set based on the first association relationship), the first target information of the observed target at the first moment is included in the third observation sequence. After the above first association, the first sub-similarity between the target and each first observation sequence set and / or the second sub-similarity between the target and each first observation sequence set can be stored, and the first sub-similarity and / or the second sub-similarity are used as the second similarity between the target and the corresponding second observation sequence sets. The first moment is one of the at least one moment in step 2520, and the first target information of the observed target at the first moment is the second target information in step 2520.

[0152] In some alternative embodiments, for the newly created second observation sequence set in the first association update process, the historical stored similarity may not yet include the similarity between the targets in other second observation sequence sets and this newly created second observation sequence set. The second similarity between the target and this second observation sequence set can be calculated according to the above similarity calculation process. For example, for the second target information at any moment in the third observation sequence, according to the above similarity calculation operation, such as the calculation operation of the first sub-similarity, the second similarity between this second target information and this newly created second observation sequence set is determined, and details are not described herein again.

[0153] In some alternative embodiments, the second similarity between the second target information at one or more moments with the latest time in the third observation sequence and each second observation sequence set can be determined to ensure timeliness.

[0154] Step 2530, based on the second similarity, determine the third similarity between the third observation sequence and each second observation sequence set.

[0155] Among them, for any second observation sequence set, the second similarity at at least one moment in the third observation sequence can be weighted according to a certain weight to obtain the third similarity between this third observation sequence and this second observation sequence set. The weights of each moment can be set according to the rule that the closer to the first moment, the greater the weight. For example, the first moment is the t moment, the weight of the t moment is greater than the weight of the t - 1 moment, the weight of the t - 1 moment is greater than the weight of the t - 2 moment, and so on, so that the similarity between the target information at the latest moment and the second observation sequence set contributes the most to the similarity between the second observation sequence and the second observation sequence set, and decays reversely with time to ensure the timeliness of the third similarity.

[0156] Step 2540, based on the third similarity, determine the second association relationship between the third observation sequence and each second observation sequence set.

[0157] In some alternative embodiments, for the case where the number of third observation sequences is 1, the second observation sequence set with the largest third similarity can be determined from each second observation sequence set based on the third similarity as the second observation sequence set associated with this third observation sequence. Or, in combination with the largest third similarity and the similarity threshold, it is determined whether the second observation sequence set with the largest third similarity is associated with this third observation sequence. For example, if the largest third similarity is less than the similarity threshold, it is determined that no second observation sequence set associated with this third observation sequence is matched.

[0158] In some alternative embodiments, for the case where the number of third observation sequences is multiple, a one-to-one global optimal matching may be implemented based on the third similarity using a pre-configured matching algorithm to obtain the second association relationships between each third observation sequence and each second observation sequence set. The matching algorithm may be, for example, the KM algorithm.

[0159] In some alternative embodiments, in response to the type of the first observation method being the first type, during the process of determining the second association relationship based on the third similarity, for key targets within a specified range, lower matching conditions may be adopted to determine the second association relationship between the third observation sequence and each second observation sequence set, so that the third observation sequence has a greater possibility of being associated with the second observation sequence set.

[0160] In the embodiments of the present disclosure, by screening the third observation sequences to be associated, the number of observation sequences participating in the secondary association can be effectively reduced, thereby improving the association efficiency.

[0161] In some alternative embodiments, determining the third observation sequence to be associated from each second observation sequence set in step 2510 may include:

[0162] Determining the types of the observation sequences in each second observation sequence set and / or the states of each second observation sequence set; the state includes a stable state and a free state; based on the types of the observation sequences and / or the states of each second observation sequence set, determining the second observation sequence sets that meet the first condition from each second observation sequence set; and determining the observation sequences in the second observation sequence sets that meet the first condition as the third observation sequences.

[0163] Among them, the type of the observation sequence may include the type of the observation method corresponding to the observation sequence. The first condition may include at least one of a type condition and a state condition. The type condition may include, for example, only including the observation sequences corresponding to the first observation method (i.e., not including other observation sequences except the observation sequences corresponding to the first observation method), etc. The state condition may include, for example, the second observation sequence sets belonging to the free state. Based on the first condition, determining the second observation sequence sets that meet the first condition from each second observation sequence set, and determining the observation sequences in the second observation sequence sets that meet the first condition as the third observation sequences to be associated.

[0164] In some alternative embodiments, the first condition may further include the condition that the second observation sequence corresponding to the first observation method is updated during the first association update process. That is, the changed second observation sequences in the second observation sequence sets including multiple observation sequences are determined as the third observation sequences to be associated. Since the second observation sequence is updated, it may affect the association relationship between the second observation sequence and the current second observation sequence set to which it belongs and other second observation sequence sets. Therefore, as the third observation sequence to be associated, it participates in the secondary association.

[0165] In an embodiment of the present disclosure, based on the type of the observation sequence, the state of the observation sequence set, and the first condition, the observation sequences that may be recombined with other second observation sequence sets or split from the currently affiliated second observation sequence set are screened out as the third observation sequences to be associated, which can avoid performing secondary association on all the observation sequences, effectively reduce the amount of calculation, and improve the processing efficiency of target tracking.

[0166] In some optional embodiments, determining the third similarity between the third observation sequence and each second observation sequence set based on the second similarity in step 2530 may include:

[0167] Determine the types of the observation sequences in each second observation sequence set; based on the types of the observation sequences, determine the second observation sequence sets that meet the second condition from each second observation sequence set as the fifth observation sequence sets; based on the second similarity, determine the third similarity between the third observation sequence and each fifth observation sequence set.

[0168] Among them, the second condition can be set according to the rules that may merge with the observation sequences in other second observation sequence sets or may split the observation sequences in the second observation sequence set. For example, the second condition may include: the second observation sequence set does not include the second observation sequence corresponding to the first observation method, and the second observation sequence set includes one or more other observation method observation sequences except the second observation sequence. Based on the types of the observation sequences in the second observation sequence set and the second condition, the second observation sequence sets that meet the second type condition are screened out, and the second observation sequence sets that meet the second condition are used as the fifth observation sequence sets to participate in the secondary association, that is, based on the second similarity, determine the third similarity between the third observation sequence and each fifth observation sequence set, and then based on the third similarity, determine the second association relationship between the third observation sequence and each fifth observation sequence set.

[0169] In an embodiment of the present disclosure, by screening out the second observation sequence sets participating in the secondary association through the type of the observation sequence and the second condition, the number of the second observation sequence sets participating in the secondary association can be effectively reduced, thereby reducing the amount of calculation and improving the processing efficiency of target tracking.

[0170] In some optional embodiments, the type of the observation sequence, the first condition, and the second condition can be combined to determine the third observation sequence and the fifth observation sequence set to be associated; then, for each third observation sequence, based on the second similarity between the target at at least one moment in the third observation sequence and each fifth observation sequence set, determine the third similarity between the third observation sequence and each fifth observation sequence set, which can further reduce the amount of calculation.

[0171] In some alternative embodiments, based on the type of the observation sequence and / or the state of the second set of observation sequences, on the basis of determining a third observation sequence that meets the first condition, for each third observation sequence, a fifth set of observation sequences to be secondarily associated with the third observation sequence may be determined, and then, based on the second similarity, the third similarity between the third observation sequence and each fifth set of observation sequences may be determined. For example, based on the type of the observation sequence, a second observation sequence updated at a first moment may be determined from each second set of observation sequences as the third observation sequence to be associated; based on the state of each second set of observation sequences, a free observation sequence in a free-state second set of observation sequences may be determined as the third observation sequence to be associated. The second observation sequence updated at the first moment includes a second observation sequence in which first target information at the first moment is written in an existing second observation sequence, and a newly created second observation sequence in which the first target information is written. For a non-free observation sequence that is not updated at the first moment, no secondary association may be performed. For the case where the second observation sequence updated at the first moment is used as the third observation sequence, the second set of observation sequences where the third observation sequence is located and other second sets of observation sequences that do not include the second observation sequence corresponding to the first observation method may be used as the fifth set of observation sequences to be secondarily associated with the third observation sequence. For the case where a free observation sequence is used as the third observation sequence, each second set of observation sequences may be used as the fifth set of observation sequences to be secondarily associated with the third observation sequence.

[0172] Figure 7 It is a schematic flowchart of a target tracking method provided by another exemplary embodiment of the present disclosure.

[0173] In some alternative embodiments, on the basis of any of the above embodiments, as Figure 7 shown, the updating of the second set of observation sequences based on the second association relationship in step 260 to obtain a third set of observation sequences may include:

[0174] Step 2610, for each observation sequence in the second set of observation sequences, based on the second association relationship, determine a second set of observation sequences associated with the observation sequence as the sixth set of observation sequences.

[0175] Wherein, the second association relationship between any observation sequence and any second set of observation sequences indicates whether the observation sequence is associated with the second set of observation sequences. Therefore, based on the second association relationship, a second observation sequence associated with the observation sequence may be determined, and the second observation sequence associated with the observation sequence is used as the sixth observation sequence.

[0176] In some alternative embodiments, for the case of determining the third observation sequence to be associated in the above embodiments, for each third observation sequence, based on the second association relationships between the third observation sequence and each second observation sequence set respectively, determine the second observation sequence set associated with the third observation sequence as the sixth observation sequence set.

[0177] In some alternative embodiments, for the case of determining the fifth observation sequence set in the above embodiments, for each observation sequence, based on the second association relationships between the observation sequence and each fifth observation sequence set respectively, determine the fifth observation sequence set associated with the observation sequence as the sixth observation sequence set.

[0178] In some alternative embodiments, for each third observation sequence, based on the second association relationships between the third observation sequence and each fifth observation sequence set respectively, determine the fifth observation sequence set associated with the third observation sequence as the sixth observation sequence set.

[0179] Step 2620, in response to the observation sequence not belonging to the sixth observation sequence set, reorganize the observation sequence and the sixth observation sequence set to obtain a reorganized sequence set.

[0180] Among them, for each observation sequence, after obtaining the sixth observation sequence set associated with the observation sequence, if the observation sequence does not belong to the sixth observation sequence set, it means that the observation sequence and the sixth observation sequence set belong to the same target, and the observation sequence and the sixth observation sequence set can be reorganized to obtain a reorganized sequence set. It can be understood that reorganizing the observation sequence and the sixth observation sequence set includes moving the observation sequence from the current observation sequence set to the sixth observation sequence set.

[0181] In some alternative embodiments, if the current observation sequence set to which the observation sequence belongs only includes the observation sequence, after the observation sequence is removed, the observation sequence set becomes an empty set, and the empty set can be deleted.

[0182] Step 2630, for each second observation sequence set, based on the second association relationship, determine the fourth observation sequences among the observation sequences in the second observation sequence set that are not associated with the second observation sequence set.

[0183] Among them, for each second observation sequence set, based on the second association relationship between each observation sequence in the second observation sequence set and the second observation sequence set, an observation sequence that is not associated with the second observation sequence set can be determined as the fourth observation sequence. The fact that the observation sequence is not associated with the second observation sequence set means that the observation sequence may have been wrongly associated with the second observation sequence set before the first moment, or it means that the target can no longer be observed through the observation method corresponding to the observation sequence, resulting in the end of the life cycle of the observation sequence. The observation sequence is used as the fourth observation sequence for subsequent corresponding processing.

[0184] Step 2640, delete the fourth observation sequence from the second observation sequence set to obtain the remaining sequence set.

[0185] Among them, since the fourth observation sequence is not associated with the second observation sequence set to which it currently belongs, the fourth observation sequence can be deleted from the second observation sequence set to obtain the remaining sequence set.

[0186] In some optional embodiments, if each fourth observation sequence includes the observation sequence recombined with the sixth observation sequence set as described above. In this case, the above recombination operation may only include the operation of writing the observation sequence into the sixth observation sequence set. Then, through step 2630 and step 2640, the observation sequence will be deleted from the observation sequence set to which it originally belonged, and the remaining sequence set will be obtained.

[0187] In some optional embodiments, if the fourth observation sequence is not recombined with other observation sequence sets, the fourth observation sequence can be temporarily used as a free observation sequence, and a corresponding observation sequence set can be created to delete the observation sequence or the observation sequence set when the life cycle end condition is met at a subsequent moment.

[0188] It should be noted that steps 2610 to 2620 and steps 2630 to 2640 do not have a sequential order.

[0189] Step 2650, determine the third observation sequence set based on the recombined sequence set and the remaining sequence set.

[0190] Among them, each recombined sequence set can be used as the third observation sequence set. Each remaining sequence set can be used as the third observation sequence set.

[0191] In some optional embodiments, for any second observation sequence set, if there are both recombination operations and operations to delete unassociated observation sequences, on the basis of obtaining the recombined sequence set through recombination, unassociated observation sequences can be deleted to obtain the remaining sequence set as the third observation sequence set, or recombination can be performed on the basis of the remaining observation sequences obtained by deleting unassociated observation sequences to obtain the recombined sequence set as the third observation sequence set.

[0192] In an embodiment of the present disclosure, through the recombination and splitting of the observation sequence set, a secondary association (or spatial association) and update are performed between the observation sequence and the observation sequence set. The observation sequences that are not associated are recombined into the corresponding observation sequence set, and the observation sequences that are mis-associated or have ended are removed from the observation sequence set, ensuring the accuracy and reliability of the target tracking result.

[0193] In some optional embodiments, it further includes: for each third observation sequence set, in response to the third observation sequence set not including an observation sequence, the third observation sequence set is deleted.

[0194] Among them, after the third observation sequence set is obtained by recombination and splitting, the deletion of the observation sequence may cause the observation sequence set to become an empty set, that is, it does not include an observation sequence or the number of observation sequences is 0. The empty set indicates the end of the life cycle of the observation sequence set. Therefore, the third observation sequence set that becomes an empty set can be deleted to realize the real-time maintenance of the life cycle of the observation sequence set and further improve the effectiveness of the observation sequence set.

[0195] In some optional embodiments, Figure 8 is a flowchart of a target tracking method provided by an exemplary embodiment of the present disclosure. As Figure 8 shown, the target tracking method includes:

[0196] Step 510, observation preprocessing.

[0197] Among them, the observation refers to the target (i.e., the observed target) observed by the first observation method (i.e., observation method 1) at the first moment (i.e., time T). Observation preprocessing refers to preprocessing the observed target, and the preprocessing includes target filtering, transformation of the target information data structure, etc. Target filtering means filtering out unstable or low-confidence targets. The transformation of the target information data structure means transforming the data structure of the target information output by the perception algorithm model into the data structure supported by target tracking. The transformation of the target information data structure may include extracting the state information of the target from the observation result of the perception algorithm model, and encapsulating the extracted state information according to the specified data structure to obtain an observation set to be subjected to target tracking, or a target set, a target collection. As shown in the figure, the observed target *M indicates that the target set includes M targets.

[0198] Step 520, observation sequence set preprocessing.

[0199] Among them, the set of observation sequences is the set of observation sequences after target tracking at the second moment (taking the moment T-1 in the figure as an example), and may include all sets of observation sequences maintained in real time. The preprocessing of the set of observation sequences may include screening, feature extraction, time alignment, etc. for the set of observation sequences. Screening the set of observation sequences means screening out at least one first set of observation sequences participating in the first association. In the figure, the first set of observation sequences *N represents N first sets of observation sequences. Each first set of observation sequences includes one or more observation sequences, such as observation sequence 10, observation sequence 11,... in the figure, representing the observation sequences in a first set of observation sequences. Observation sequence 20 and observation sequence 21 are the observation sequences in another first set of observation sequences. Feature extraction means extracting the target information of one or more moments with the most recent time from the set of observation sequences. Time alignment means predicting the target feature information of the target at the first moment based on the target information of one or more moments, such as the first target feature and the second target feature in the figure. The first target feature is the target feature corresponding to the first observation sequence (taking observation sequence 10 in the figure as an example) whose observation confidence level meets the level condition. The second target feature is the target feature of the second observation sequence. See the foregoing embodiments for details.

[0200] Step 530, similarity estimation.

[0201] Among them, similarity estimation includes determining the first similarity between each target observed at the first moment and each first set of observation sequences. As Figure 8 shown, based on the first target feature and the second target feature respectively corresponding to each first set of observation sequences, and the first target information of each target, the similarity matrix (or similarity tensor) corresponding to the first similarity has a structure of N*M*2. If the horizontal direction is represented as width W, the vertical direction is represented as height H, and the depth direction is represented as C, W = M represents the number of targets, H = 2 represents two types of similarities, namely the first sub-similarity and the second sub-similarity, and C = N represents the number of first sets of observation sequences, that is, each target corresponds to the first sub-similarity and the second sub-similarity with each first set of observation sequences. See the foregoing embodiments for details.

[0202] Step 540, association assignment.

[0203] Among them, the associated assignment refers to determining the first association relationship between each target and each first observation sequence set based on the similarity estimation result in step 530. According to the first association relationship, unassociated observation targets, associated observation targets, associated observation sequence sets (i.e., the first observation sequence sets), and unassociated observation sequence sets can be determined. An unassociated observation target refers to a target in the target set for which no associated first observation sequence set is determined; an associated observation target refers to a target for which an associated first observation sequence set is determined; an associated observation sequence set refers to the first observation sequence set for which an associated target is determined; and an unassociated observation sequence set refers to the first observation sequence set for which no associated target is determined.

[0204] Step 550, Observation Sequence Set Update and Observation Sequence Lifecycle Management.

[0205] Among them, the update of the observation sequence set includes updating the first observation sequence set based on the first target information and the first association relationship in the above embodiments. Observation sequence lifecycle management refers to creating a corresponding observation sequence for a target first observed by the first observation method, and deleting the observation sequence for a target whose observation has ended, etc. In practical applications, the operations of observation sequence lifecycle management can be included in the update operation of the observation sequence set. Through the update of the observation sequence set and the observation sequence lifecycle management, the second observation sequence set at time T is obtained.

[0206] Step 560, Similarity Cache Update.

[0207] Among them, similarity cache update means adding the similarity matrix at time T obtained by the similarity estimation in step 530 above to the cached similarity matrices at times T - 1 and T - 2. Figure 8 Caching the similarity matrix corresponding to the first sub - similarity means using the first sub - similarity for calculation when determining the similarity between an observation sequence and an observation sequence set.

[0208] It should be noted that steps 540 to 550 and step 560 can be performed in any order.

[0209] Step 570, Observation Sequence and Observation Sequence Set Selection.

[0210] Among them, the selection of the observation sequence and the observation sequence set refers to the operation of determining the third observation sequence to be associated and the fifth observation sequence set for secondary association with the third observation sequence. For details, refer to the foregoing embodiments. In the figure, the third observation sequence *K represents determining K third observation sequences, and the fifth observation sequence set *J represents determining J fifth observation sequence sets.

[0211] It should be noted that step 570 and step 560 can be performed in any order.

[0212] Step 580, sequence similarity estimation.

[0213] Among them, the sequence similarity refers to the similarity between the observed sequence and the set of observed sequences. The sequence similarity estimation means that for each third observed sequence, based on the second similarity between the targets observed at at least one moment in the third observed sequence and each fifth observed sequence set, the third similarity between the third observed sequence and each fifth observed sequence set is determined, and a sequence similarity matrix with a J*K structure is obtained.

[0214] Step 590, association assignment.

[0215] Among them, the association assignment means determining the second association relationships between the third observed sequence and each fifth observed sequence set based on the third similarity. Based on the second association relationships, the unassociated observed sequences (i.e., the third observed sequence), the associated observed sequences, the associated observed sequence sets (i.e., the fifth observed sequence sets), and the unassociated observed sequence sets can be obtained.

[0216] Step 5100, update of the observed sequence set and life cycle management.

[0217] Among them, the update of the observed sequence set means updating the second observed sequence set or the fifth observed sequence set based on the second association relationships to obtain a third observed sequence set. The life cycle management of the observed sequence set means deleting the observed sequence sets that become empty, and the life cycle ends. Based on the remaining third observed sequence sets, the target tracking result at time T is determined.

[0218] In some alternative embodiments, after step 5100, for each observed sequence set, the first observed sequence and the second observed sequence in the observed sequence set can be re-determined according to the observation confidence levels of the observed sequences in the observed sequence set, so as to prepare for the next target tracking.

[0219] For the specific operations of the above steps 510 to 5100, reference can be made to the relevant content of the foregoing embodiments, which will not be elaborated here.

[0220] The target tracking method provided by the embodiments of the present disclosure can effectively improve the accuracy and effectiveness of the target tracking result by performing temporal correlation update between the target and the observation sequence set, and spatial secondary correlation update between the observation sequence and the observation sequence set, making full use of the temporal correlation. Moreover, each target tracking can only process the target observed by one observation method at one moment, realizing the association between one-frame observation and the real-time maintained observation sequence set, enabling different observation methods to have different frame rates, which helps to set the most suitable frame rate for different observation methods, and can reduce the related processing or hardware for sensor synchronization. In addition, each target tracking can be based on the previous tracking result and combine a new frame of observation to perform new target tracking, which has good inheritance and ensures the accuracy and effectiveness of the target tracking result.

[0221] In the technical solution of the present disclosure, the processing of collection, storage, use, processing, transmission, provision, and disclosure of user personal information complies with the provisions of relevant laws and regulations and does not violate public order and good customs. Moreover, in the technical solution of the present disclosure, the collection and use of user personal information are carried out with the knowledge and authorization of the user, and do not involve illegal collection and illegal use of user personal information.

[0222] The above embodiments of the present disclosure can be implemented separately or combined in any combination without conflict, which can be specifically set according to actual needs, and the present disclosure does not make any limitation.

[0223] Any target tracking method provided by the embodiments of the present disclosure can be executed by any suitable electronic device with data processing capabilities, including but not limited to: electronic devices such as terminal devices and servers. Alternatively, any target tracking method provided by the embodiments of the present disclosure can be executed by a processor. For example, the processor executes any target tracking method mentioned in the embodiments of the present disclosure by calling the corresponding instructions stored in the memory. This will not be elaborated below.

[0224] Exemplary device

[0225] Figure 9 It is a schematic structural diagram of a target tracking device provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding method embodiment of the present disclosure, such as Figure 9 The device shown may include: a first processing module 71, a second processing module 72, a third processing module 73, a fourth processing module 74, a fifth processing module 75, a sixth processing module 76, and a seventh processing module 77.

[0226] The first processing module 71 is configured to determine first target information of at least one target observed by a first observation method at a first moment.

[0227] The second processing module 72 is configured to determine at least one first observation sequence set after tracking at least one target at a second moment.

[0228] Wherein, the second moment is before the first moment; each target corresponds to a first observation sequence set, and the first observation sequence set includes at least one observation sequence.

[0229] The third processing module 73 is configured to determine a first association relationship between the target and each first observation sequence set respectively based on the first target information and the first observation sequence set.

[0230] The fourth processing module 74 is configured to update the first observation sequence set based on the first target information and the first association relationship to obtain a second observation sequence set.

[0231] The fifth processing module 75 is configured to determine a second association relationship between each observation sequence in the second observation sequence set and each second observation sequence set respectively.

[0232] The sixth processing module 76 is configured to update the second observation sequence set based on the second association relationship to obtain a third observation sequence set.

[0233] The seventh processing module 77 is configured to determine the target tracking result at the first moment based on the third observation sequence set.

[0234] Figure 10 It is a schematic structural diagram of a target tracking device provided by another exemplary embodiment of the present disclosure.

[0235] In some optional embodiments, on the basis of the above Figure 9 illustrated embodiment, as Figure 10 shown, the second processing module 72 may include: a first determination unit 721.

[0236] The first determination unit 721 is configured to determine, based on the first observation method, an observation sequence set that meets the first condition from each observation sequence set after tracking at least one target at the second moment as the first observation sequence set.

[0237] In some optional embodiments, on the basis of any of the above embodiments, the third processing module 73 may include: a second determination unit 731, a third determination unit 732, and a fourth determination unit 733.

[0238] The second determination unit 731 is configured to determine the target feature information corresponding to each first observation sequence set respectively.

[0239] Wherein, the target feature information includes the target features of at least one observation sequence in the first observation sequence set.

[0240] A third determination unit 732, configured to determine a first similarity between the first target information and each piece of target feature information.

[0241] A fourth determination unit 733, configured to determine a first association relationship between the target and each first observation sequence set based on the first similarity.

[0242] In some optional embodiments, based on the above embodiments, the second determination unit 731 is specifically configured to:

[0243] For any one of the first observation sequence sets in each first observation sequence set, determine a first target feature of the first observation sequence in the first observation sequence set and a second target feature of the second observation sequence as target feature information; the first observation sequence is an observation sequence that satisfies a confidence condition; the second observation sequence is an observation sequence corresponding to the first observation method.

[0244] In some optional embodiments, the second determination unit 731 is specifically configured to:

[0245] For any one of the first observation sequence sets, predict the target information of the target at the first moment based on the target information observed at at least one third moment in the first observation sequence set as the target feature information; the third moment is before the first moment.

[0246] In some optional embodiments, the third determination unit 732 is specifically configured to:

[0247] Determine a first sub - similarity between the first target information and the first target feature, and a second sub - similarity between the first target information and the second target feature; determine the first sub - similarity and the second sub - similarity as the first similarity.

[0248] In some optional embodiments, the third determination unit 732 is specifically configured to:

[0249] Determine the type of the first observation method; based on the type of the first observation method, determine a similarity calculation method; based on the similarity calculation method, determine the first similarity between the target and each piece of target feature information.

[0250] In some optional embodiments, based on the above embodiments, the fourth determination unit 733 is specifically configured to:

[0251] Determine the type of the first observation method; in response to the type of the first observation method being the first type, determine the first association relationship between the target and each first observation sequence set based on each second sub - similarity; in response to the type of the first observation method being the second type, determine the first association relationship between the target and each first observation sequence set based on each second sub - similarity and each first sub - similarity.

[0252] In some alternative embodiments, based on any of the above embodiments, such as Figure 10 shown, the fourth processing module 74 may include: a first processing unit 741, a second processing unit 742, and a third processing unit 743.

[0253] The first processing unit 741 is configured to update the fourth observation sequence set in response to determining, based on a first association relationship, a fourth observation sequence set associated with the target from at least one first observation sequence set.

[0254] The second processing unit 742 is configured to create an observation sequence set corresponding to the target in response to not determining a first observation sequence set associated with the target from at least one first observation sequence set.

[0255] In some alternative embodiments, the first processing unit 741 is specifically configured to:

[0256] In response to the second observation sequence being included in the fourth observation sequence set, write the first target information into the second observation sequence to obtain a second observation sequence set. The second observation sequence is the observation sequence corresponding to the first observation method. In response to the second observation sequence not being included in the fourth observation sequence set, create a second observation sequence, write the first target information into the second observation sequence, and obtain a second observation sequence set.

[0257] In some alternative embodiments, based on the above embodiments, the second processing unit 742 is specifically configured to: in response to not determining a first observation sequence set associated with the target, create an observation sequence set corresponding to the target based on the first target information as the second observation sequence set of the target.

[0258] In some alternative embodiments, based on any of the above embodiments, the second processing unit 742 is specifically configured to: in response to not determining a first observation sequence set associated with the target and the type of the first observation method being the second type, determine the confidence level of the target. In response to the confidence level meeting a preset level condition, create an observation sequence set corresponding to the target as the second observation sequence set of the target.

[0259] In some alternative embodiments, the second processing unit 742 is further configured to: in response to the confidence level not meeting the preset level condition or the type of the first observation method being the first type, delete the first target information.

[0260] In some alternative embodiments, based on any of the above embodiments, the second processing unit 742 is further configured to:

[0261] For any first observation sequence set in at least one first observation sequence set, in response to the failure to determine the target associated with the first observation sequence set, delete the second observation sequences in the first observation sequence set to obtain a second observation sequence set.

[0262] Wherein, the second observation sequence is the observation sequence corresponding to the first observation method.

[0263] In some optional embodiments, based on any of the above embodiments, the observation sequence includes second target information of the target observed at at least one moment by the same observation method.

[0264] Based on any of the above embodiments, as Figure 10 shown, the fifth processing module 75 may include: a fifth determination unit 751, a sixth determination unit 752, a seventh determination unit 753, and an eighth determination unit 754.

[0265] The fifth determination unit 751 is configured to determine, from each second observation sequence set, the third observation sequence to be associated.

[0266] The sixth determination unit 752 is configured to determine the second similarity degrees between the second target information of the target observed at at least one moment in the third observation sequence and each second observation sequence set respectively.

[0267] The seventh determination unit 753 is configured to determine the third similarity degrees between the third observation sequence and each second observation sequence set respectively based on the second similarity degrees.

[0268] The eighth determination unit 754 is configured to determine the second association relationships between the third observation sequence and each second observation sequence set respectively based on the third similarity degrees.

[0269] In some optional embodiments, the fifth determination unit 751 is specifically configured to:

[0270] Determine the type of the observation sequences in each second observation sequence set and / or the state of each second observation sequence set; the state includes a stable state and a free state; based on the type of the observation sequences, determine the second observation sequence sets that meet the first condition from each second observation sequence set; and determine the observation sequences in the second observation sequence sets that meet the first condition as the third observation sequence.

[0271] In some optional embodiments, the seventh determination unit 753 is specifically configured to:

[0272] Determine the type of the observation sequences in each second observation sequence set; based on the type of the observation sequences, determine the second observation sequence sets that meet the second condition from each second observation sequence set as the fifth observation sequence sets; and determine the third similarity degrees between the third observation sequence and each fifth observation sequence set respectively based on the second similarity degrees.

[0273] In some alternative embodiments, based on any of the above embodiments, such as Figure 10 shown, the sixth processing module 76 may include: a fourth processing unit 761, a fifth processing unit 762, a sixth processing unit 763, a seventh processing unit 764, and an eighth processing unit 765.

[0274] The fourth processing unit 761 is configured to, for each observation sequence in the second observation sequence set, determine, based on the second association relationship, a second observation sequence set associated with the observation sequence as the sixth observation sequence set.

[0275] The fifth processing unit 762 is configured to, in response to the observation sequence not belonging to the sixth observation sequence set, recombine the observation sequence with the sixth observation sequence set to obtain a recombined sequence set.

[0276] The sixth processing unit 763 is configured to, for each second observation sequence set, determine, based on the second association relationship, a fourth observation sequence in each observation sequence in the second observation sequence set that is not associated with the second observation sequence set.

[0277] The seventh processing unit 764 is configured to delete the fourth observation sequence from the second observation sequence set to obtain a remaining sequence set.

[0278] The eighth processing unit 765 is configured to determine a third observation sequence set based on the recombined sequence set and the remaining sequence set.

[0279] In some alternative embodiments, the sixth processing module 76 is further configured to: for each third observation sequence set, in response to the third observation sequence set not including an observation sequence, delete the third observation sequence set.

[0280] Each of the above embodiments of the present disclosure may be implemented alone or in any combination without conflict, and may be specifically set according to actual needs, and the present disclosure does not make any limitations.

[0281] For the beneficial technical effects corresponding to the exemplary embodiments of the present device, reference may be made to the corresponding beneficial technical effects in the above exemplary method section, and details are not described herein again.

[0282] Exemplary electronic device

[0283] Figure 11 is a structural diagram of an electronic device provided by an embodiment of the present disclosure, including at least one processor 91 and a memory 92.

[0284] The processor 91 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0285] The memory 92 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage media, and the processor 91 may run one or more computer program instructions to implement the methods of the various embodiments of the present disclosure above and / or other desired functions.

[0286] In one example, the electronic device 90 may further include: an input device 93 and an output device 94, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0287] The input device 93 may further include, for example, a touch screen, a microphone, various sensors, and so on. The sensors may include, for example, an image sensor (such as a camera, a webcam, etc.), lidar, millimeter-wave radar, ultrasonic radar, a positioning sensor, a pressure sensor, an air quality sensor, a temperature sensor, etc. The image sensor, lidar, millimeter-wave radar, ultrasonic radar, etc. may be used for the perception of the surrounding environment, that is, to detect the static and dynamic objects in the surrounding environment. The static and dynamic objects may include, for example, static objects such as lane lines, curbs, arrows, signs, trees, buildings, etc., and dynamic objects such as surrounding vehicles, pedestrians, cyclists, etc. The positioning sensor is used to implement the positioning of the autonomous mobile device (such as a self-vehicle, a robot, etc.) where the electronic device is located. The positioning sensor may include, for example, an Inertial Measurement Unit (IMU for short), a Global Positioning System (GPS for short), etc. The pressure sensor may be used to detect the seat pressure. The temperature sensor may be used to detect the temperature inside the vehicle cockpit. The air quality sensor may be used to detect the air quality inside the vehicle cockpit.

[0288] The output device 94 may output various information to the outside, which may include, for example, a display, a speaker, and a communication network and its connected remote output devices, etc.

[0289] Of course, for simplicity, Figure 11 only some of the components related to the present disclosure in the electronic device 90 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to the specific application situation, the electronic device 90 may further include any other appropriate components.

[0290] Exemplary computer program product and computer-readable storage medium

[0291] In addition to the above methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when run by a processor, cause the processor to execute the steps in the methods of various embodiments of the present disclosure described in the above "Exemplary Method" section.

[0292] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The programming code may be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0293] Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when run by a processor, cause the processor to execute the steps in the methods of various embodiments of the present disclosure described in the above "Exemplary Method" section.

[0294] The computer-readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium, for example but not limited to, includes systems, devices or components of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0295] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that they are essential for each embodiment of the present disclosure. In addition, the above disclosed specific details are only for the purposes of illustration and facilitating understanding, rather than limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

[0296] Those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure also intends to include these changes and modifications.

Claims

1. A target tracking method, comprising: Determining first target information of at least one target observed by a first observation method at a first moment; Determining at least one set of first observation sequences after tracking at least one target at a second moment; wherein, the second moment is before the first moment; each target corresponds to one set of first observation sequences, and the set of first observation sequences includes at least one observation sequence; Based on the first target information and the set of first observation sequences, determining a first association relationship between the target and each set of first observation sequences; Based on the first target information and the first association relationship, updating the set of first observation sequences to obtain a set of second observation sequences; Determining a second association relationship between each observation sequence in the set of second observation sequences and each set of second observation sequences; Based on the second association relationship, updating the set of second observation sequences to obtain a set of third observation sequences; Based on the set of third observation sequences, determining a target tracking result at the first moment.

2. The method according to claim 1, wherein The determining of at least one set of first observation sequences after tracking at least one target at a second moment includes: Based on the first observation method, determining, from each set of observation sequences after tracking at least one target at the second moment, a set of observation sequences that meets a first condition as the set of first observation sequences.

3. The method according to claim 1, wherein The determining of the first association relationship between the target and each set of first observation sequences based on the first target information and the set of first observation sequences includes: Determining target feature information corresponding to each set of first observation sequences; the target feature information includes target features of at least one observation sequence in the set of first observation sequences; Determining a first similarity between the first target information and each piece of target feature information; Based on the first similarity, determining the first association relationship between the target and each set of first observation sequences.

4. The method according to claim 3, wherein, The determining of the target feature information corresponding to each set of first observation sequences includes: For any set of first observation sequences in each set of first observation sequences, determining a first target feature of a first observation sequence in the set of first observation sequences and a second target feature of a second observation sequence as the target feature information; the first observation sequence is an observation sequence that meets a confidence condition; the second observation sequence is an observation sequence corresponding to the first observation method; The determining of the first similarity between the first target information and each piece of target feature information includes: Determining a first sub-similarity between the first target information and the first target feature, and a second sub-similarity between the first target information and the second target feature; Determining the first sub-similarity and the second sub-similarity as the first similarity.

5. The method according to claim 4, wherein The determining of the first association relationship between the target and each set of first observation sequences based on the first similarity includes: Determining the type of the first observation method; In response to the type of the first observation method being the first type, determine the first association relationship between the target and each of the first observation sequence sets based on each of the second sub-similarities; In response to the type of the first observation method being the second type, determine the first association relationship between the target and each of the first observation sequence sets based on each of the second sub-similarities and each of the first sub-similarities.

6. The method according to claim 3, wherein The determining of the target feature information corresponding to each of the first observation sequence sets includes: For any one of the first observation sequence sets, predict the target information of the target at the first moment based on the target information observed at at least one third moment in the first observation sequence set as the target feature information; the third moment is before the first moment.

7. The method according to claim 1, wherein The updating of the first observation sequence set based on the first target information and the first association relationship to obtain a second observation sequence set includes: In response to determining, based on the first association relationship, a fourth observation sequence set associated with the target from the at least one first observation sequence set, update the fourth observation sequence set; or, In response to not determining a first observation sequence set associated with the target from the at least one first observation sequence set, create an observation sequence set corresponding to the target.

8. The method according to claim 7, wherein The updating of the fourth observation sequence set includes: In response to the second observation sequence being included in the fourth observation sequence set, write the first target information into the second observation sequence; the second observation sequence is the observation sequence corresponding to the first observation method; or, In response to the second observation sequence not being included in the fourth observation sequence set, create the second observation sequence and write the first target information into the second observation sequence.

9. The method according to claim 7, wherein The creating of the observation sequence set corresponding to the target includes: Create an observation sequence set corresponding to the target based on the first target information; or, In response to the type of the first observation method being the second type, determine the confidence level of the target; In response to the confidence level meeting the preset level condition, create an observation sequence set corresponding to the target.

10. The method according to claim 7, wherein Further includes: For any one of the at least one first observation sequence sets, in response to not determining a target associated with the first observation sequence set, delete the second observation sequence in the first observation sequence set; the second observation sequence is the observation sequence corresponding to the first observation method.

11. According to the method described in any one of claims 1 to 10, wherein, The observation sequence includes second target information of the target observed at at least one moment by the same observation method; The determining of the second association relationship between each of the observation sequences in the second observation sequence set and each of the second observation sequence sets includes: Determine a third observation sequence to be associated from each of the second observation sequence sets; Determine the second similarity between the second target information of the target observed at at least one moment in the third observation sequence and each of the second observation sequence sets; Based on the second similarity, determine the third similarity between the third observation sequence and each of the second observation sequence sets; Based on the third similarity, determine the second association relationships between the third observation sequence and each of the second observation sequence sets.

12. The method according to claim 11, wherein Determining the third observation sequence to be associated from each of the second observation sequence sets includes: Determining the types of the observation sequences in each of the second observation sequence sets and / or the states of each of the second observation sequence sets; the states include a stable state and a free state; Based on the types of the observation sequences and / or the states of each of the second observation sequence sets, determine second observation sequence sets that meet the first condition from each of the second observation sequence sets; Determine the observation sequences in the second observation sequence sets that meet the first condition as the third observation sequence.

13. The method according to claim 11, wherein, Based on the second similarity, determining the third similarity between the third observation sequence and each of the second observation sequence sets includes: Determining the types of the observation sequences in each of the second observation sequence sets; Based on the types of the observation sequences, determine second observation sequence sets that meet the second type condition from each of the second observation sequence sets as the fifth observation sequence sets; Based on the second similarity, determine the third similarity between the third observation sequence and each of the fifth observation sequence sets.

14. According to the method of any one of claims 1-10, wherein Based on the second association relationship, updating the second observation sequence sets to obtain third observation sequence sets includes: For each observation sequence in the second observation sequence sets, based on the second association relationship, determine the second observation sequence set associated with the observation sequence as the sixth observation sequence set; In response to the observation sequence not belonging to the sixth observation sequence set, recombine the observation sequence and the sixth observation sequence set to obtain a recombined sequence set; For each of the second observation sequence sets, based on the second association relationship, determine fourth observation sequences in each observation sequence in the second observation sequence set that are not associated with the second observation sequence set; Delete the fourth observation sequences from the second observation sequence sets to obtain a remaining sequence set; Based on the recombined sequence set and the remaining sequence set, determine the third observation sequence sets.

15. The method according to claim 14, wherein, Further includes: For each of the third observation sequence sets, in response to the third observation sequence set not including an observation sequence, delete the third observation sequence set.

16. A target tracking device, comprising: A first processing module, configured to determine first target information of at least one target observed by a first observation method at a first moment; A second processing module, configured to determine at least one first observation sequence set after tracking at least one target at a second moment; wherein, the second moment is before the first moment; each target corresponds to one first observation sequence set, and the first observation sequence set includes at least one observation sequence; A third processing module, configured to determine a first association relationship between the target and each of the first observation sequence sets based on the first target information and the first observation sequence set; A fourth processing module, configured to update the first observation sequence set based on the first target information and the first association relationship to obtain a second observation sequence set; A fifth processing module, configured to determine a second association relationship between each of the observation sequences in the second observation sequence set and each of the second observation sequence sets; A sixth processing module, configured to update the second observation sequence set based on the second association relationship to obtain a third observation sequence set; A seventh processing module, configured to determine a target tracking result at the first moment based on the third observation sequence set.

17. A computer-readable storage medium storing a computer program for executing the method according to any one of claims 1-15 above.

18. An electronic device, comprising: A processor; A memory for storing executable instructions of the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method according to any one of claims 1-15 above.

Images