Target tracking method and target tracking apparatus

Abstract

A target tracking method and a target tracking device in the field of artificial intelligence, the method comprising: acquiring actual observation of a moving object collected by a sensor; in a case where a collection time of the actual observation is earlier than an update time of a current state quantity of the moving object, obtaining an estimated observation according to a state transition model corresponding to the moving object; and updating the current state quantity according to the actual observation and the estimated observation, and taking the updated state quantity as a state quantity corresponding to the update time. The target tracking can be realized in a case where the collection time of the observation is disordered, and the time stamp of the state quantity is prevented from jumping disorderly with the disordered change of the time stamp of the observation.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence, and more specifically, to a target tracking method and a target tracking device. BACKGROUND

[0002] The target tracking method can be applied to many scenarios, such as target detection, lane line detection, positioning, etc. Common target tracking methods need to use the measurement data of sensors, combined with the state transition model of the tracked object itself to perform Bayesian estimation. For example, a Bayesian filter can be used to implement target tracking. Specifically, the measurement data of the sensor is input into the Bayesian filter to obtain an estimate of the state of the tracked object. However, in the case of multiple sensors, due to transmission path and other reasons, the measurement data of a sensor may arrive at the Bayesian filter later than it should. Figure 1 is a schematic diagram of measurement data input into a Bayesian filter appearing out of order in time. Figure 1 The horizontal coordinate in the diagram represents the real time, and the data transmitted by the sensor includes a time stamp (header: X ms) and measurement data (data), where the time stamp is used to indicate that the measurement or collection time of the measurement data is X ms. The measurement data measured by sensor 0 at 1 ms is input into the Bayesian filter at 3 ms, and the measurement data measured by sensor 1 at 0 ms is input into the Bayesian filter at 4 ms, causing the measurement data input into the Bayesian filter to appear out of order in time. In this case, the Bayesian filter first outputs the state quantity with a time stamp of 1 ms calculated based on the measurement data measured at 1 ms, and then outputs the state quantity with a time stamp of 0 ms calculated based on the measurement data measured at 0 ms, resulting in the time stamp of the output state quantity jumping with the change in the time stamp of the measurement data.

[0003] Two methods can be generally used to solve the above problem. One method is to determine whether the time stamp of the measurement data is later than the time stamp of the state quantity updated last time in the Bayesian filter, and only input the measurement data collected at a time later than the update time of the state quantity updated last time into the Bayesian filter. However, this method will directly cause the missing of the observation quantity, resulting in the decrease of the confidence of the output result. The other method is to extrapolate the late observation data to the current time. However, extrapolation in the observation space relies on strong assumptions and is prone to error amplification. SUMMARY

[0004] This application provides a target tracking method and a target tracking device that can achieve target tracking even when the acquisition time of the observations is out of order, and avoids the out-of-order jump of the timestamp of the state quantity due to the out-of-order change of the timestamp of the observation.

[0005] Firstly, a target tracking method is provided, comprising: acquiring actual observations of a moving object collected by a sensor; when the acquisition time of the actual observations is earlier than the update time of the current state of the moving object, obtaining estimated observations based on the state transition model corresponding to the moving object; updating the current state based on the actual observations and the estimated observations, and using the updated state as the state corresponding to the update time.

[0006] The observed quantities can include the velocity, acceleration, or position of a moving object. For example, the observed quantities can include the position and / or velocity of a moving object as measured by millimeter-wave radar.

[0007] State variables can include the velocity, acceleration, or position of a moving object.

[0008] The update time of the current state quantity of a moving object can be understood as the latest update time of the state quantity of the moving object.

[0009] Optionally, the state transition model of the moving object includes a kinematic model of the moving object. For example, the kinematic model may include a uniform linear motion model, etc.

[0010] According to the scheme of this application embodiment, state transition is performed based on the state transition model corresponding to the moving object, and then the current state quantity is updated, which can obtain a more accurate state quantity corresponding to the latest update time. At the same time, the updated state quantity is the state quantity corresponding to the latest update time, rather than the state quantity corresponding to the acquisition time, so as to avoid out-of-order jumps in the output state quantity in time.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, obtaining the estimated observation based on the state transition model corresponding to the moving object includes: obtaining the likelihood function of the observation corresponding to the acquisition time based on the state transition model corresponding to the moving object; the estimated observation is determined based on the likelihood function of the observation corresponding to the acquisition time; the state transition model is determined based on the first probability density function, which is the probability density function for transitioning the current state quantity to the state quantity corresponding to the acquisition time.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the current state quantity is updated based on the actual observations and estimated observations, and the updated state quantity is used as the state quantity corresponding to the update time. This includes: updating the probability density function of the current state quantity based on the likelihood function of the observations corresponding to the acquisition time; and determining the state quantity corresponding to the update time based on the probability density function of the updated state quantity.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the likelihood function of the observation corresponding to the acquisition time satisfies:

[0014] g(z k-1 |X k ')=∫g(z k-1 |X)f k-1|k (X|X k ')dX

[0015] Among them, g(z) k-1 |X k ') indicates about t k-1 The observation z corresponding to time t k-1 The likelihood function, X represents t k-1 The state variable X that is updated in real time k 'Indicates the absence of z k-1 In the case of t k The state variable that is updated in real time, g(z) k-1 |X) represents the z k-1 For t k-1 Regarding z, in the case of constantly updating the state quantity k-1 The likelihood function, f k-1|k (X|X k ') is the first probability density function, which is the value of t k The state variable X that is updated at all times k 'Transfer to t k-1 The probability density function of the state variables that are updated at any given time.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the probability density function of the updated state quantity satisfies:

[0017]

[0018] Among them, f k|k (X k |z k ) represents t k The state variable X that is updated at all times k The probability density function, f' k|k (X k '|z k ,zk-2 ) represents the probability density function of the state quantity X k-1 updated at time t k , z k represents the observation quantity collected at time t k , z k represents the observation quantity collected at time t k , z k-2 represents the set of observation quantities {z k-1 , z k} collected at k-2 time points. k-2 k-2} collected at k-2 time points.

[0019] With reference to the first aspect, in some implementations of the first aspect, the updating of the current state quantity according to the actual observation quantity and the estimated observation quantity, and the taking of the updated state quantity as the state quantity corresponding to the updating time, comprises:

[0020] determining an expectation of the state quantity corresponding to the updating time according to the actual observation quantity and the estimated observation quantity;

[0021] taking the expectation of the state quantity corresponding to the updating time as the state quantity corresponding to the updating time;

[0022] wherein the expectation of the state quantity corresponding to the updating time is related to a Kalman gain value, and the Kalman gain value is related to a first covariance, an observation matrix at the collection time, a covariance of the observation matrix, and a variance of the observation quantity at the collection time, and the first covariance refers to a covariance of the state quantity transferred from the updating time to the collection time.

[0023] With reference to the first aspect, in some implementations of the first aspect, the Kalman gain value satisfies:

[0024]

[0025] Var(z k-1 ) satisfies:

[0026]

[0027] P k-1|k satisfies:

[0028]

[0029] wherein P k-1|k represents a covariance of the state quantity transferred from the updating time t k to the collection time t k-1 , P' k|k represents the observation quantity collected at time t k-1 .​k-1 the state quantity updated at time t k the covariance of the state quantity updated at time t k-1|k the state transition matrix from time t k to time t k-1 the state transition matrix from time t k the covariance of the prediction matrix, k-1 the observation matrix at time t k-1 the observation matrix at time t k-1 the variance of the observation quantity z k-1 collected at time t k-1 the variance of the observation quantity z k-1 the covariance of the observation matrix.

[0030] With reference to the first aspect, in some implementations of the first aspect, the expectation of the state quantity corresponding to the updating time satisfies:

[0031]

[0032] wherein x k|k represents the expectation of the state quantity updated at time t k , and represents the estimated observation quantity at time t k-1 .

[0033] With reference to the first aspect, in some implementations of the first aspect, the estimated value of the observation quantity corresponding to the collection time satisfies:

[0034]

[0035] wherein represents the estimated observation quantity at time t k-1 , H k-1 represents the observation matrix at time t k-1 , F k-1|k represents the state transition matrix from time t k to time t k-1 , and x' k|k represents the expectation of the state quantity updated at time t k-1 in the case of lacking the observation quantity z k-1 collected at time t k .

[0036] With reference to the first aspect, in some implementations of the first aspect, in the case that the collection time of the actual observation quantity is earlier than the updating time of the current state quantity of the moving object, the estimated observation quantity is obtained according to the state transition model corresponding to the moving object, comprising: in the case that the collection time of the observation quantity is earlier than the updating time of the current state quantity of the moving object, and the time difference between the collection time and the updating time is less than or equal to a threshold, the estimated observation quantity is obtained according to the state transition model corresponding to the moving object.

[0037] According to the scheme of the embodiments of the present application, the state transition is performed only in the case where the time difference between the collection time and the update time is less than or equal to the threshold, and then the state quantity is updated, so that in the case where the time difference is too large, the confidence of the updated state quantity is reduced due to the use of the observation quantity to update the state quantity.

[0038] In a second aspect, a target tracking device is provided, including an acquisition module and a processing module, wherein the acquisition module is configured to acquire an actual observation quantity of a moving object collected by a sensor; and the processing module is configured to: in the case where a collection time of the actual observation quantity is earlier than an update time of a current state quantity of the moving object, obtain an estimated observation quantity according to a state transition model corresponding to the moving object; and update the current state quantity according to the actual observation quantity and the estimated observation quantity, and take an updated state quantity as a state quantity corresponding to the update time.

[0039] According to the scheme of the embodiments of the present application, the state transition is performed according to the state transition model corresponding to the moving object, and then the current state quantity is updated, so that a more accurate state quantity corresponding to the latest update time can be obtained. Meanwhile, the updated state quantity is the state quantity corresponding to the latest update time, instead of the state quantity corresponding to the collection time, so that the output state quantity is prevented from appearing out of order in time.

[0040] With reference to the second aspect, in some implementations of the second aspect, the processing module is configured to: obtain a likelihood function about the observation quantity corresponding to the collection time according to the state transition model corresponding to the moving object, the estimated observation quantity is determined according to the likelihood function about the observation quantity corresponding to the collection time, and the state transition model is determined according to the first probability density function, which is a probability density function of the transition of the current state quantity to the state quantity corresponding to the collection time.

[0041] With reference to the second aspect, in some implementations of the second aspect, the processing module is configured to: update the probability density function of the current state quantity according to the likelihood function about the observation quantity corresponding to the collection time; and determine the state quantity corresponding to the update time according to the probability density function of the updated state quantity.

[0042] With reference to the second aspect, in some implementations of the second aspect, the likelihood function about the observation quantity corresponding to the collection time satisfies:

[0043] g(z k-1 |X k ')=∫g(z k-1 |X)f k-1|k (X|X k ')dX

[0044] wherein g(z k-1 |Xk represents a likelihood function of the observation z k-1 corresponding to the time t k-1 , X represents the state quantity updated at the time t k-1 , X k ' represents the state quantity updated at the time t k-1 in the absence of the observation z k , g(z k-1 |X) represents a likelihood function of the observation z k-1 in the case of updating the state quantity at the time t k-1 , f k-1 (X k-1|k |z k ) is a first probability density function, and the first probability density function is a probability density function of transferring the state quantity X k updated at the time t k to the state quantity updated at the time t k-1 .

[0045] With reference to the second aspect, in some implementations of the second aspect, the probability density function of the updated state quantity satisfies:

[0046]

[0047] wherein f k|k (X k |z k ) represents a probability density function of the state quantity X k updated at the time t k , f' k|k (X k '|z k ,z k-2 ) represents a probability density function of the state quantity X k-1 updated at the time t k in the absence of the observation z k , z k represents the observation collected at the time t k , z k represents a set {z1, z2,..., z k-2 , z k-1 , z k} of observations collected at k times, and z k-2 represents a set {z1, z2,..., z k-2} of observations collected at k-2 times.

[0048] In some implementations of the second aspect, in combination with the second aspect, the processing module is configured to: determine an expectation of the state quantity corresponding to the update time according to the actual observation quantity and the estimated observation quantity; and take the expectation of the state quantity corresponding to the update time as the state quantity corresponding to the update time, wherein the expectation of the state quantity corresponding to the update time is related to a Kalman gain value, and the Kalman gain value is related to a first covariance, an observation matrix at the collection time, a covariance of the observation matrix, and a variance of the observation quantity at the collection time, and the first covariance refers to a covariance of the state quantity transferred from the update time to the collection time.

[0049] In some implementations of the second aspect, in combination with the second aspect, the Kalman gain value satisfies:

[0050]

[0051] Var(z k-1 ) satisfies:

[0052]

[0053] P k-1|k satisfies:

[0054]

[0055] wherein P k-1|k represents a covariance of the state quantity transferred from the update time t k to the collection time t k-1 , P' k|k represents a covariance of the state quantity updated at the time t k-1 in the case that the observation quantity z k-1 collected at the time t k is absent, F k-1|k represents a state transition matrix from the time t k to the time t k-1 , Q k represents a covariance of a prediction matrix, H k-1 represents an observation matrix at the time t k-1 , Var(z k-1 ) represents a variance of the observation quantity z k-1 collected at the time t k-1 , and R k-1 represents a covariance of the observation matrix.

[0056] In some implementations of the second aspect, in combination with the second aspect, the expectation of the state quantity corresponding to the update time satisfies:

[0057]

[0058] wherein x k|k represents the state quantity corresponding to the update time, and x k represents the state quantity corresponding to the collection time.an expectation of the state quantity updated at the time t an expectation of the state quantity updated at the time t k-1 an estimated observation quantity at the time t

[0059] In some implementations of the second aspect, in combination with the second aspect, the estimated observation quantity corresponding to the time t satisfies:

[0060]

[0061] an expectation of the state quantity updated at the time t an expectation of the state quantity updated at the time t k-1 an estimated observation quantity at the time t k-1 an estimated observation quantity at the time t k-1 an observation matrix at the time t k-1|k an observation matrix at the time t k an observation matrix at the time t k-1 an observation matrix at the time t k|k an observation matrix at the time t k-1 an observation matrix at the time t k-1 an observation matrix at the time t k an expectation of the state quantity updated at the time t

[0062] In some implementations of the second aspect, in combination with the second aspect, the processing module is configured to: in a case where the time of collection of the actual observation quantity is earlier than the time of update of the current state quantity of the moving object, and the time difference between the time of collection and the time of update is less than or equal to the threshold value, obtain the estimated observation quantity according to the state transition model corresponding to the moving object.

[0063] It should be understood that the above-mentioned extensions, limitations, explanations and descriptions of related contents in the first aspect are also applicable to the same contents in the second aspect.

[0064] In a third aspect, a target tracking device is provided, which comprises: a memory configured to store a program; and a processor configured to execute the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute the method in the first aspect.

[0065] In a fourth aspect, a computer program product is provided, which comprises: computer program code, when the computer program product is executed on a computer, the computer program code causes the computer to execute the method in the first aspect.

[0066] In a fifth aspect, a computer readable storage medium is provided, which stores a computer program, when the computer program is executed on a computer, the computer program causes the computer to execute the method in the first aspect.

[0067] It should be understood that, in this application, the method of the first aspect can specifically refer to the method in the first aspect and any one of the various implementations of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 is a schematic diagram of measurement data inputting into a Bayesian filter;

[0069] Figure 2 is a structural schematic diagram of a vehicle provided by an embodiment of the present application;

[0070] Figure 3 is a structural schematic diagram of a computer system provided by an embodiment of the present application;

[0071] Figure 4 is an application schematic diagram of instructing an autonomous vehicle from a cloud side provided by an embodiment of the present application;

[0072] Figure 5 is a structural schematic diagram of a target tracking device provided by an embodiment of the present application;

[0073] Figure 6 is a schematic flow chart of a target tracking method provided by an embodiment of the present application;

[0074] Figure 7 is a structural schematic diagram of another target tracking device provided by an embodiment of the present application;

[0075] Figure 8 is a structural schematic diagram of still another target tracking device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0077] Figure 2 is a functional block diagram of a vehicle 100 provided by an embodiment of the present application.

[0078] The vehicle 100 can be a manually driven vehicle, or the vehicle 100 can be configured to be in a fully or partially autonomous driving mode.

[0079] In one example, the vehicle 100 can control the ego vehicle while being in the autonomous driving mode, and can determine a current state of the vehicle and its surrounding environment through human operation, determine possible behaviors of at least one other vehicle in the surrounding environment, and determine a confidence level corresponding to a possibility that the other vehicle performs the possible behavior, and control the vehicle 100 based on the determined information. When the vehicle 100 is in the autonomous driving mode, the vehicle 100 can be configured to operate without human interaction.

[0080] The vehicle 100 can include various subsystems, such as a travel system 110, a sensing system 120, a control system 130, one or more peripheral devices 140, and a power source 160, a computer system 150, and a user interface 170.

[0081] Optionally, the vehicle 100 can include more or fewer subsystems, and each subsystem can include multiple elements. Additionally, each subsystem and element of the vehicle 100 can be interconnected by wire or wirelessly.

[0082] Illustratively, the travel system 110 can include components for providing powered movement to the vehicle 100. In one embodiment, the travel system 110 can include an engine 111, a transmission 112, an energy source 113, and wheels 114 / tires. The engine 111 can be a combustion engine, an electric motor, an air compression engine, or other types of engine combinations; for example, a hybrid engine composed of a gasoline engine and an electric motor, a hybrid engine composed of a combustion engine and an air compression engine. The engine 111 can convert the energy source 113 into mechanical energy.

[0083] Illustratively, the energy source 113 can include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source 113 can also provide energy to other systems of the vehicle 100.

[0084] Illustratively, the transmission 112 can include a gearbox, a differential, and a drive shaft; the transmission 112 can transmit mechanical power from the engine 111 to the wheels 114.

[0085] In one embodiment, the transmission 112 can also include other devices, such as a clutch. The drive shaft can include one or more shafts that can be coupled to one or more wheels 114.

[0086] Illustratively, the sensing system 120 can include several sensors that sense information about the environment surrounding the vehicle 100.

[0087] For example, the sensing system 120 may include a positioning system 121 (e.g., GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU) 122, a radar 123, a laser rangefinder 124, and a camera 125. The sensing system 120 may also include sensors from the internal systems of the monitored vehicle 100 (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of the autonomous vehicle 100.

[0088] The positioning system 121 can be used to estimate the geographical location of the vehicle 100. The IMU 122 can be used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 122 can be a combination of an accelerometer and a gyroscope.

[0089] For example, radar 123 can use radio signals to sense objects in the surrounding environment of vehicle 100. In some embodiments, in addition to sensing objects, radar 123 can also be used to sense the speed and / or direction of travel of objects.

[0090] For example, the laser rangefinder 124 can use a laser to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the laser rangefinder 124 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0091] For example, camera 125 can be used to capture multiple images of the surrounding environment of vehicle 100. For example, camera 125 can be a still camera or a video camera.

[0092] like Figure 2 As shown, the control system 130 controls the operation of the vehicle 100 and its components. The control system 130 may include various components, such as a steering system 131, an accelerator 132, a braking unit 133, a computer vision system 134, a route control system 135, and an obstacle avoidance system 136.

[0093] For example, the steering system 131 can be operated to adjust the forward direction of the vehicle 100. For example, in one embodiment, it can be a steering wheel system. The throttle 132 can be used to control the operating speed of the engine 111 and thus the speed of the vehicle 100.

[0094] For example, braking unit 133 can be used to control the deceleration of vehicle 100; braking unit 133 can use friction to slow down wheel 114. In other embodiments, braking unit 133 can convert the kinetic energy of wheel 114 into electric current. Braking unit 133 can also take other forms to slow down the rotational speed of wheel 114 to control the speed of vehicle 100.

[0095] like Figure 2 As shown, the computer vision system 134 is operable to process and analyze images captured by the camera 125 to identify objects and / or features in the environment surrounding the vehicle 100. These objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 134 may use object recognition algorithms, structure-from-motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 134 may be used to map the environment, track objects, estimate object velocities, and so on.

[0096] For example, the route control system 135 can be used to determine the driving route of the vehicle 100. In some embodiments, the route control system 135 can combine data from sensors, GPS, and one or more predetermined maps to determine the driving route of the vehicle 100.

[0097] like Figure 2 As shown, obstacle avoidance system 136 can be used to identify, assess and avoid or otherwise traverse potential obstacles in the environment of vehicle 100.

[0098] In one instance, the control system 130 may include additional or alternative components besides those shown and described. Alternatively, some of the components shown above may be reduced.

[0099] like Figure 2 As shown, vehicle 100 can interact with external sensors, other vehicles, other computer systems or users through peripheral device 140; wherein peripheral device 140 may include wireless communication system 141, on-board computer 142, microphone 143 and / or speaker 144.

[0100] In some embodiments, peripherals 140 can provide a means for vehicle 100 to interface with user interface 170. For example, onboard computer 142 can provide information to a user of vehicle 100. User interface 116 can also operate onboard computer 142 to receive input from a user; onboard computer 142 can be operated via a touchscreen, in other cases, peripherals 140 can provide a means for vehicle 100 to communicate with other devices located within the vehicle. For example, microphone 143 can receive audio (e.g., voice commands or other audio input) from a user of vehicle 100. Similarly, speaker 144 can output audio to a user of vehicle 100.

[0101] As Figure 2 indicated, wireless communication system 141 can wirelessly communicate with one or more devices directly or via a communication network. For example, wireless communication system 141 can use 3G cellular communication; e.g., code division multiple access (CDMA), EVDO, global system for mobile communications (GSM) / general packet radio service (GPRS), or 4G cellular communication; e.g., long term evolution (LTE); or 5G cellular communication. Wireless communication system 141 can utilize wireless fidelity (WiFi) and wireless local area network (WLAN) communication.

[0102] In some embodiments, wireless communication system 141 can utilize an infrared link, Bluetooth, or ZigBee to communicate directly with devices; other wireless protocols, such as various vehicle communication systems, e.g., wireless communication system 141 can include one or more dedicated short range communications (DSRC) devices, which can include public and / or private data communication between vehicles and / or roadside stations.

[0103] As Figure 2 indicated, power source 160 can provide power to various components of vehicle 100. In one embodiment, power source 160 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such a battery can be configured as a power source to provide power to various components of vehicle 100. In some embodiments, power source 160 and energy source 113 can be implemented together, as in some all-electric vehicles.

[0104] By way of example, portions or all of the functionality of the vehicle 100 can be controlled by a computer system 150, which can include at least one processor 151 that executes instructions 153 stored in non-transitory computer readable media, such as memory 152. The computer system 150 can also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0105] For example, the processor 151 can be any conventional processor, such as a commercially available CPU.

[0106] Alternatively, the processor can be a special purpose device, such as an ASIC or other hardware-based processor. Although Figure 2 Although the processor, memory, and other elements of the computer are functionally illustrated as being within the same block, it will be understood by those of ordinary skill in the art that the processor, computer, or memory can actually include multiple processors, computers, or memories that can or can not be stored in the same physical housing. For example, memory can be a hard drive or other storage medium located in a housing different from that of the computer. Reference to the processor or computer will be understood to encompass reference to a collection of processors or computers or memories that can or can not operate in parallel. Rather than using a single processor to perform the steps described herein, such as the steering assembly and the deceleration assembly, some components can each have their own processor that only performs calculations related to the functionality specific to that component.

[0107] In various aspects described herein, the processor can be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0108] In some embodiments, the memory 152 can contain instructions 153 (e.g., program logic) that can be executed by the processor 151 to perform various functions of the vehicle 100, including those described above. The memory 152 can also contain additional instructions, such as instructions to send data to, receive data from, interact with, and / or control one or more of the travel system 110, the sensing system 120, the control system 130, and the peripherals 140.

[0109] By way of example, the memory 152 can store data, in addition to the instructions 153, such as road maps, route information, the location, orientation, speed, and other such vehicle data of the vehicle, and other information. Such information can be used by the vehicle 100 and the computer system 150 during operation of the vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0110] As shown, user interface 170 can be used to provide information to or receive information from a user of vehicle 100. Optionally, user interface 170 can include one or more input / output devices within the set of peripheral devices 140, such as wireless communication system 141, on-board computer 142, microphone 143, and speaker 144. Figure 2

[0111] In embodiments of the application, computer system 150 can control the functions of vehicle 100 based on inputs received from various subsystems (e.g., travel system 110, sensing system 120, and control system 130) and from user interface 170. For example, computer system 150 can utilize inputs from control system 130 in order to control brake unit 133 to avoid an obstacle detected by sensing system 120 and obstacle avoidance system 136. In some embodiments, computer system 150 can be operable to provide control over many aspects of vehicle 100 and its subsystems.

[0112] Optionally, one or more of the above-described components can be installed separately from or associated with vehicle 100. For example, memory 152 can exist partially or entirely separately from vehicle 100. The above-described components can be communicatively coupled together in a wired and / or wireless manner.

[0113] Optionally, the above-described components are just one example, and in actual applications, components in each module described above can be added or deleted according to actual needs, Figure 2 This should not be understood as a limitation on the embodiments of the application.

[0114] Optionally, vehicle 100 can be an autonomous vehicle that travels on a roadway and can identify objects within its surrounding environment to determine adjustments to a current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently and, based on respective characteristics of the object, such as its current speed, acceleration, spacing from the vehicle, etc., can be used to determine a speed at which the autonomous vehicle is to adjust.

[0115] Optionally, vehicle 100 or a computing device associated with vehicle 100 (e.g., computer system 150, computer vision system 134, memory 152 of Figure 2 may predict behavior of the identified object based on characteristics of the identified object and a state of the surrounding environment (e.g., traffic, rain, ice on the roadway, etc.).

[0116] ​Optionally, each of the identified objects behaves in dependence on the others, so all of the identified objects can also be considered together to predict the behavior of a single identified object. Vehicle 100 can adjust its speed based on the predicted behavior of the identified object. In other words, the autonomous vehicle can determine that the vehicle will need to adjust to a steady state (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the object. Other factors can also be considered in determining the speed of vehicle 100 in this process, such as the lateral position of vehicle 100 in the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, and the like.

[0117] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of vehicle 100 to cause the autonomous vehicle to follow a given trajectory and / or maintain a safe lateral and longitudinal distance from objects in the vicinity of the autonomous vehicle (e.g., a sedan in an adjacent lane on the road).

[0118] The vehicle 100 described above can be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, trolley, golf cart, train, and handcart, among others, without limitation.

[0119] In one possible implementation, the vehicle 100 described above Figure 2 The vehicle 100 shown can be an autonomous vehicle, and the autonomous system is described in detail below.

[0120] Figure 3 is a schematic diagram of an autonomous system provided by embodiments of the present application.

[0121] As shown in FIG. 1, the autonomous system includes a vehicle 100, a server 102, and a mobile device 104. The vehicle 100 can be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, trolley, golf cart, train, and handcart, among others, without limitation. The vehicle 100 can include a vehicle controller 110, a vehicle sensor 112, a vehicle camera 114, a vehicle GPS 116, a vehicle communication device 118, and a vehicle display 120. Figure 3The illustrated autonomous driving system includes a computer system 201, which includes a processor 203 coupled with a system bus 205. The processor 203 can be one or more processors, each of which can include one or more processor cores. A video adapter 207 can drive a display 209 coupled to the system bus 205 via the video adapter 207. The system bus 205 can be coupled with a bus bridge 211 and an input / output (I / O) bus 213 to which are coupled I / O devices 215, such as input devices 217 (e.g., keyboard, mouse, touchscreen, etc.), a media tray 221 (e.g., CD-ROM, DVD, etc.), and a transceiver 223 that can transmit and / or receive radio communication signals. A camera 255 can capture still and dynamic digital video images. The interface to the I / O devices 215 can be a USB port 225.

[0122] The processor 203 can be any conventional processor, such as a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination of such processors.

[0123] Alternatively, the processor 203 can be a special purpose device such as an application specific integrated circuit (ASIC), a neural network processor, or a combination of such conventional processors and neural network processors.

[0124] Alternatively, in various embodiments described herein, the computer system 201 can be located remotely from the autonomous vehicle and can communicate wirelessly with the autonomous vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the autonomous vehicle, and others are performed by a remote processor, including taking the actions required to perform a single maneuver.

[0125] The computer system 201 can communicate with a software deployment server 249 via a network interface 229. The network interface 229 can be a hardware network interface, such as a network card. The network 227 can be an external network, such as the Internet, or an internal network, such as an Ethernet or virtual private network (VPN). Alternatively, the network 227 can be a wireless network, such as a wifi network, a cellular network, or the like.

[0126] like Figure 3 As shown, the hard disk drive interface is coupled to the system bus 205, the hardware driver interface 231 can be connected to the hard disk drive 233, and the system memory 235 is coupled to the system bus 205. Data running in the system memory 235 can include the operating system 237 and applications 243. The operating system 237 can include an interpreter 239 (shell) and a kernel 241. The shell 239 is an interface between the user and the operating system kernel. The shell can be the outermost layer of the operating system; it manages the interaction between the user and the operating system, such as waiting for user input, interpreting user input for the operating system, and processing various operating system outputs. The kernel 241 can consist of the parts of the operating system used to manage memory, files, peripherals, and system resources. Interacting directly with the hardware, the operating system kernel typically runs processes and provides inter-process communication, CPU time slice management, interrupts, memory management, I / O management, etc. Application 243 includes programs related to controlling autonomous driving, such as programs managing the interaction between the autonomous vehicle and obstacles on the road, programs controlling the autonomous vehicle's route or speed, and programs controlling the interaction between the autonomous vehicle and other autonomous vehicles on the road. Application 243 also exists on the system of software deployment server 249. In one embodiment, when autonomous driving-related programs 247 need to be executed, computer system 201 can download the application from software deployment server 249.

[0127] For example, sensor 253 may be associated with computer system 201 and may be used to detect the environment around computer 201.

[0128] For example, sensor 253 can detect animals, cars, obstacles, and pedestrian crossings. Furthermore, the sensor can also detect the environment around the aforementioned animals, cars, obstacles, and pedestrian crossings, such as the environment around the animals, for example, other animals around the animals, weather conditions, and ambient light levels.

[0129] Alternatively, if the computer system 201 is located on an autonomous vehicle, the sensors may be cameras, infrared sensors, chemical detectors, microphones, etc.

[0130] For example, there can be multiple sensors 253. Multiple sensors can be used to detect the positions of obstacles around the vehicle, and the positions of the obstacles can be obtained based on the data acquired by the multiple sensors. Specifically, obtaining the positions of obstacles based on the data acquired by the multiple sensors can be implemented by the target tracking method of this application embodiment.

[0131] In one example, Figure 2 The computer system 150 shown can also receive information from or transfer information to other computer systems. Alternatively, sensor data collected from the sensor system 120 of the vehicle 100 can be transferred to another computer for processing of this data.

[0132] For example, as Figure 4 Data from the computer system 312 can be transmitted to a server 320 on the cloud side via a network for further processing, as shown. The network and intermediate nodes can include a wide variety of configurations and protocols including the Internet, World Wide Web, intranets, virtual private networks, wide area networks, local networks, private networks using a company's proprietary communication protocol, Ethernet, WiFi and HTTP, and various combinations of the foregoing; such communications can be facilitated by any devices including modems and wireless interfaces capable of transmitting and receiving data from other computer systems.

[0133] In one example, the server 320 can include a server having multiple computers, such as a load-balanced server farm, that exchanges information with different nodes of the network for the purpose of receiving, processing, and transmitting data to and from the computer system 312. The server can be configured similarly to the computer system 312, with a processor 330, a memory 340, instructions 350, and data 360.

[0134] Exemplarily, the data 360 of the server 320 can include information related to road conditions around the vehicle. For example, the server 320 can receive, detect, store, update, and transmit information related to road conditions of the vehicle.

[0135] The following will make a detailed introduction to the related content of the Bayesian filter.

[0136] The state space refers to a set of state variables that describe all possible states of a system. A car can be regarded as a system, and the user's operation of the car can be regarded as an input variable. When there is an operation signal input, it will have a clear impact on variables such as the speed, acceleration, and angular velocity of the car, and these affected variables can all be used as state variable components of the system.

[0137] Observation refers to the process of directly or indirectly obtaining an estimated value of a state variable through a certain measurement method.

[0138] Bayesian estimation refers to the process of first calculating the probability distribution of the prior estimate of the state at the next time k, called prediction, starting from any time k-1; and then correcting the prior estimate obtained in the prediction step after obtaining the observation value at time k to obtain the posterior estimate of the state at time k, called update.

[0139] Bayesian recursive estimation, when applied to practical engineering, is called Bayesian filtering.

[0140] A Bayesian filter consists of two parts: prediction and update.

[0141] The prediction process satisfies formula (1):

[0142] f k+1|k (X|z k )=∫f k+1|k (X|X')f k|k (X'|z k )dX'(1)

[0143] The update process satisfies formulas (2) and (3):

[0144]

[0145] f k+1 (z k+1 |z k )=∫f k+1 (z k+1 |X)f k+1|k (X|z k )dX(3)

[0146] Where X represents the state variable, i.e., the output of the Bayesian filter, and the space containing the state variable is called the state space. z represents the observation variable, i.e., the input of the Bayesian filter, and the space containing the observation variable is called the observation space. k+1 For the (k+1)th observation, z k ={z0,z1,...,z k} represents the set of the first k observations; f(*) is the probability density function.

[0147] Figure 5 This is a schematic diagram of a target tracking device according to an embodiment of this application. The target tracking device 402 can be applied in a computer system 401. The target tracking device includes a prediction unit 410 and an update unit 420. The prediction unit 410 includes a judgment module 411, and the update unit 420 includes an observation transfer module 422.

[0148] Furthermore, the prediction unit 410 may also include a state transition module 412. Furthermore, the update unit 420 may also include an update module 421.

[0149] To better understand the execution process of the target tracking method in the embodiments of this application, the following will first describe... Figure 5 The functions of each module in the document will be briefly described.

[0150] The judging module 411 is configured to judge whether the collection time of the observation quantity is earlier than the update time of the current state quantity. That is, to judge whether the observation quantity is a late observation quantity. The current state quantity refers to the latest updated state quantity. Specifically, when the collection time of the observation quantity is earlier than the update time of the latest state quantity, the observation quantity is a late observation quantity.

[0151] It should be understood that, in the embodiment of the present application, the observation quantity collected by the sensor is the actual observation quantity.

[0152] The state transition module 412 is configured to obtain a predicted value of the state quantity to be updated according to the state quantity updated last time when the collection time of the observation quantity is not earlier than the update time of the current state quantity. The specific process satisfies formula (1).

[0153] The updating module 421 is configured to update the state quantity according to the predicted value of the state quantity to be updated obtained by the state transition module 412. The specific process satisfies formula (2) and formula (3).

[0154] The observation quantity transition unit 422 is configured to perform state transition on the observation quantity to obtain an estimated observation quantity when the collection time of the observation quantity is earlier than the update time of the current state quantity. The observation quantity transition unit 422 is further configured to update the current state quantity according to the actual observation quantity and the estimated observation quantity, and take the updated state quantity as the state quantity corresponding to the update time.

[0155] The target tracking method 500 of the embodiment of the present application will be described in detail below. Figure 6 The target tracking method 500 of the embodiment of the present application will be described in detail below. Figure 6 The target tracking method 500 of the embodiment of the present application will be described in detail below. Figure 6 The method shown in FIG. 5 can be performed by the target tracking device in the embodiment of the present application. The method 500 includes steps S510 to S560. The steps S510 to S560 will be described in detail below.

[0156] S510, obtaining an actual observation quantity of a moving object collected by a sensor. It should be understood that, in the embodiment of the present application, the observation quantity collected by the sensor is the actual observation quantity, which can also be referred to as an “observation quantity” in the embodiment of the present application. The observation quantity can include the speed of the moving object, the acceleration of the moving object, or the position of the moving object, etc. For example, the observation quantity can include the position and / or speed of the moving object measured by a millimeter wave radar, etc.

[0157] S520, judging whether the collection time of the observation quantity is earlier than the update time of the current state quantity of the moving object.

[0158] If the collection time of the observation is not earlier than the update time of the current state quantity of the moving object, step S530 is performed. If the collection time of the observation is earlier than the update time of the current state quantity of the moving object, step S540 is performed.

[0159] The collection time of the observation being earlier than the update time of the current state quantity of the moving object can be understood as that the observation can originally be used for the update of the current state quantity of the moving object, but actually fails to be used for the update of the current state quantity of the moving object.

[0160] The collection time of the observation can be indicated by a time stamp of the observation.

[0161] The state quantity can include a velocity of the moving object, an acceleration of the moving object, or a position of the moving object, etc. For example, when the method 500 is used for tracking a position of a target, the state quantity can be a result of the tracking, and the result of the tracking can be the position of the moving object.

[0162] The update time of the current state quantity of the moving object can be understood as a time when the state quantity of the moving object is updated for the last time.

[0163] The state quantity of the moving object and the update time of the state quantity can be saved in a tracking list in the Bayesian filter.

[0164] Specifically, judging whether the collection time of the observation is earlier than the update time of the current state quantity of the moving object can be judging whether a time stamp of the observation is earlier than the time when the tracking list is updated for the last time.

[0165] S530, updating the current state quantity of the moving object, and taking the updated state quantity as the state quantity corresponding to the collection time of the observation.

[0166] Specifically, the current state quantity of the moving object can be updated by the Bayesian filter. For example, the current state quantity of the moving object can be updated according to the above formula (1), formula (2) and formula (3).

[0167] S540, judging whether a time difference between the collection time of the observation and the update time of the current state quantity of the moving object is greater than a first threshold.

[0168] If the time difference is greater than the first threshold, the observation is discarded. That is, the input observation is not used for the update of the state quantity. In this way, it can be avoided that the confidence of the updated state quantity is reduced due to the update of the state quantity by the observation in the case that the time difference is too large.

[0169] If the time difference is less than or equal to the first threshold, step S550 is performed.

[0170] Optionally, step S540 can also be determining whether a time difference between the collection time of the observation and the update time of the current state quantity of the moving object is greater than or equal to a first threshold.

[0171] If the time difference is greater than or equal to the first threshold, the observation is discarded. That is, the input observation is not used for updating the state quantity. If the time difference is less than the first threshold, step S550 is performed.

[0172] It should be noted that steps S520, S530 and S540 are optional steps, and the method 500 of the embodiment of the present application can perform step S550 after step S510.

[0173] S550, in a case where the collection time of the observation is earlier than the update time of the current state quantity of the moving object, obtaining an estimated observation according to a state transition model corresponding to the moving object.

[0174] When the method 500 is used to track the position of a target, the state transition model can be a kinematic model of the moving object. For example, the kinematic model can include a uniform straight motion model, etc.

[0175] Specifically, the estimated observation is determined according to a likelihood function of the observation corresponding to the collection time, and the state transition model is determined according to a first probability density function, the first probability density function being a probability density function of transferring the current state quantity to a state quantity corresponding to the collection time.

[0176] Step S550 is described below in combination with formulas.

[0177] t k The observation z k is input into the Bayesian filter, and the Bayesian filter updates the state quantity according to the observation z k , and obtains a state quantity X k as the current state quantity X k . Accordingly, the latest update of the state quantity is at time t k , that is, the update time of the current state quantity X k is at time t k . After time t k , the observation z k-1 collected at time t k-1 is input into the Bayesian filter. The collection time of the observation being earlier than the update time of the current state quantity can be understood as that the Bayesian filter receives the observation z k after updating the state quantity and then receives the observation z k-1 .

[0178] If the observable z k-1 The input time to the Bayesian filter is earlier than the observation z. k When the input to the Bayesian filter is t, then k-1 The likelihood function g(z) at time t k-1 |X k-1 ).

[0179] If the observable z k-1 The input time to the Bayesian filter is no earlier than the observation z. k When the input to the Bayesian filter is t, then k Time to t k-1 The state transition model at time f is k-1|k (X|X k This state transition is a Markov transition. k-1|k (X|X k ') represents the first probability density function mentioned above. The first probability density function is the expression of t. k The state variable X that is updated at all times k 'Transfer to t k-1 The probability density function of the state variables that are updated at any given time.

[0180] The likelihood function of the observations at the corresponding acquisition time satisfies:

[0181] g(z k-1 |X k ')=∫g(z k-1 |X)f k-1|k (X|X k ')dX

[0182] Among them, g(z) k-1 |X k ') indicates about t k-1 The observation z corresponding to time t k-1 The likelihood function, X represents t k-1 The state variable X that is updated in real time k 'Indicates the absence of z k-1 In the case of t k The state variable that is updated in real time, g(z) k-1 |X) represents the z k-1 For t k-1 Regarding z, in the case of constantly updating the state quantity k-1 The likelihood function.

[0183] S560 updates the current state quantity based on the actual and estimated observations, and uses the updated state quantity as the state quantity corresponding to the update time.

[0184] Specifically, the probability density function of the current state quantity can be updated according to a likelihood function about the observation quantity corresponding to the collection time. The state quantity corresponding to the update time is determined according to the updated probability density function of the state quantity.

[0185] The step S560 is described below in combination with the formula.

[0186] The updated probability density function of the state quantity satisfies:

[0187]

[0188] wherein f k|k (X k |z k ) represents the probability density function of the updated state quantity X k at the time t k , f' k|k (X k '|z k ,z k-2 ) represents the probability density function of the updated state quantity X k-1 at the time t k in the absence of z k . That is, the posterior probability of the current state quantity X k '. z k represents the observation quantity collected at the time t k , z k represents the set {z1, z2,..., z k-2 , z k-1 , z k} of the observation quantities collected at the k times, and z k-2 represents the set {z1, z2,..., z k-2} of the observation quantities collected at the k-2 times.

[0189] Further, the Kalman filter is an implementation form of the Bayesian filter. The step S560 is described below taking the Kalman filter as an example.

[0190] The step S560 can include determining the expectation of the state quantity corresponding to the update time according to the actual observation quantity and the estimated observation quantity; and taking the expectation of the state quantity corresponding to the update time as the state quantity corresponding to the update time.

[0191] wherein the expectation of the state quantity corresponding to the update time is related to the Kalman gain value. The Kalman gain value is related to the first covariance, the observation matrix at the collection time, the covariance of the observation matrix, and the variance of the observation quantity at the collection time. The first covariance refers to the covariance of the state quantity transferred from the update time to the collection time.

[0192] Optionally, the Kalman gain value satisfies:

[0193]

[0194] Var(z k-1 ) satisfies:

[0195]

[0196] P k-1|k satisfies:

[0197]

[0198] where P k-1|k represents the covariance of the state quantity transferred from the update time t k to the collection time t k-1 , P' k|k represents the covariance of the state quantity updated at t k-1 in the case that the observation quantity z k-1 collected at t k is absent, F k-1|k represents the state transition matrix from t k to t k-1 , Q k represents the covariance of the prediction matrix, H k-1 represents the observation matrix at t k-1 , Var(z k-1 ) represents the variance of the observation quantity z k-1 collected at t k-1 , and R k-1 represents the covariance of the observation matrix.

[0199] The expectation of the state quantity corresponding to the update time satisfies:

[0200]

[0201] where x k|k represents the expectation of the state quantity updated at t k , and z k-1 represents the estimated observation quantity at t . The expectation x k|k of the state quantity corresponding to the update time is taken as the state quantity corresponding to the update time.

[0202] Optionally, the estimated value of the observation quantity corresponding to the collection time satisfies:

[0203]

[0204] where z represents the estimated observation quantity at t k-1 , and H k-1 represents the observation matrix at tk-1 The observation matrix at time F k-1|k Indicates from t k Time shifts to t k-1 The state transition matrix at time x' k|k This indicates that in the absence of the stated t k-1 Observations collected at different times z k-1 t under the circumstances k The expectation of constantly updated state quantities.

[0205] Indicates t k-1 The estimated observations at time can be understood as the state transition matrix F. k-1|k The expectation of the state variable x' k|k Transfer to t k-1 Time, and then based on the observation matrix H k-1 Get t k-1 Expected value of observations at time As the t k-1 Estimated observations of time.

[0206] Further, calculate t k The covariance of the state variables updated at each time step. t k The covariance of the state variables updated at each time step satisfies:

[0207] P k|k =(IK' k H k-1 )P k-1|k

[0208] Among them, P k|k Indicates t k The covariance of the state variables that are updated at any given time.

[0209] According to the scheme of this application embodiment, state transition is performed based on the state transition model corresponding to the moving object, and then the current state quantity is updated. This can obtain a more accurate state quantity corresponding to the latest update time, rather than the state quantity corresponding to the acquisition time, thus avoiding out-of-order jumps in the output state quantity in time.

[0210] The following explanation uses the application of method 500 to target tracking as an example to illustrate steps S550 and S560.

[0211] R represents the distance between the sensor and the target. x = (R) represents a one-dimensional vector.

[0212] In the absence of the aforementioned t k-1 Observations collected at different times z k-1 t in the case k The state variable x that is updated at all times k satisfy x k ~N(x;x' k|k ,P' k|k ) represents a random number whose distance between the sensor and the target is x' k|k , and whose variance is P' k|k . P' k|k may be 1.

[0213] x' k|k represents an expectation of the state quantity updated at time t k-1 in the case where the observation quantity z k-1 collected at time t k is absent. P' k|k represents a covariance of the state quantity updated at time t k-1 in the case where the observation quantity z k-1 collected at time t k is absent.

[0214] For example, x' k|k may be 3, P' k|k may be 1, and the observation quantity z k-1 collected at time t k-1 may be 4. In this case, P k-1|k satisfies:

[0215]

[0216] The variance Var(z k-1 ) of the observation quantity z k-1 collected at time t k-1 satisfies:

[0217]

[0218] The Kalman gain value satisfies:

[0219]

[0220] The estimated observation quantity at time t k-1 satisfies:

[0221]

[0222] The expectation x k of the state quantity updated at time t k|k satisfies:

[0223]

[0224] where x k|k may be output as a result, that is, the state quantity corresponding to time t k .

[0225] t k covariance P of the state quantity updated in real time k|k satisfies:

[0226] P k|k = (I - K k H k-1 ) P k-1|k = (1 - 1 * 1) * 1 = 0

[0227] It should be understood that the above examples are intended to help those skilled in the art understand the embodiments of the present application, and are not intended to limit the embodiments of the present application to the specific values or specific scenarios exemplified. Those skilled in the art can obviously make various equivalent modifications or changes according to the examples given, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0228] The target tracking method of the embodiments of the present application is described in detail above, and the device embodiments of the present application will be described in detail below. It should be understood that the target tracking device in the embodiments of the present application can perform the target tracking method of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments. Figure 6 Figures 7 to 8 The target tracking device in the embodiments of the present application can perform the target tracking method of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0229] Figure 7 is a schematic block diagram of the target tracking device provided by the embodiments of the present application. It should be understood that the target tracking device 1000 can perform the target tracking method shown in Figure 6 . The target tracking device 1000 comprises an acquisition unit 1010 and a processing unit 1020.

[0230] The acquisition unit 1010 is configured to acquire actual observation quantities of a moving object collected by a sensor. The processing unit 1020 is configured to, in a case where a collection time of the actual observation quantities is earlier than an update time of a current state quantity of the moving object, obtain an estimated observation quantity according to a state transition model corresponding to the moving object; and update the current state quantity according to the actual observation quantity and the estimated observation quantity, and take an updated state quantity as a state quantity corresponding to the update time.

[0231] Optionally, the processing unit 1020 is configured to obtain a likelihood function about an observation quantity corresponding to the collection time according to the state transition model corresponding to the moving object, and the estimated observation quantity is determined according to the likelihood function about the observation quantity corresponding to the collection time, and the state transition model is determined according to a first probability density function, and the first probability density function is a probability density function of transferring the current state quantity to a state quantity corresponding to the collection time.

[0232] ​Optionally, the processing unit 1020 is used to: update the probability density function of the current state quantity based on the likelihood function of the observation quantity corresponding to the acquisition time; and determine the state quantity corresponding to the update time based on the updated probability density function of the state quantity.

[0233] Optionally, the likelihood function of the observations at the acquisition time satisfies:

[0234] g(z k-1 |X k ')=∫g(z k-1 |X)f k-1|k (X|X k ')dX

[0235] Among them, g(z) k-1 |X k ') indicates about t k-1 The observation z corresponding to time t k-1 The likelihood function, X represents t k-1 The state variable X that is updated in real time k 'Indicates the absence of z k-1 In the case of t k The state variable that is updated in real time, g(z) k-1 |X) represents the z k-1 For t k-1 Regarding z, when the state is updated in real time k-1 The likelihood function, f k-1|k (X|X k ') is the first probability density function, which is the value of t k The state variable X that is updated at all times k 'Transfer to t k-1 The probability density function of the state variables that are updated at any given time.

[0236] Optionally, the probability density function of the updated state quantity satisfies:

[0237]

[0238] Among them, f k|k (X k |z k ) represents t k The state variable X that is updated at all times k The probability density function, f' k|k (X k '|z k ,z k-2 ) indicates the absence of z k-1 In the case of t k The state variable X that is updated at all times k The probability density function of z kIndicates t k The observations collected at any time, z k Let {z1, z2, ..., zk} be the set of observations collected at k time points. k-2 ,z k-1 ,z k}, z k-2 Let {z1, z2, ..., z2} be the set of observations collected at k-2 time points. k-2}

[0239] Optionally, the processing unit 1020 is configured to: determine the expectation of the state quantity corresponding to the update time based on the actual observation and the estimated observation; and use the expectation of the state quantity corresponding to the update time as the state quantity corresponding to the update time; wherein the expectation of the state quantity corresponding to the update time is related to the Kalman gain value, the Kalman gain value is related to the first covariance, the observation matrix at the acquisition time, the covariance of the observation matrix, and the variance of the observation at the acquisition time, and the first covariance refers to the covariance of the state quantity transferred from the update time to the acquisition time.

[0240] Optionally, the Kalman gain value satisfies:

[0241]

[0242] Var(z k-1 )satisfy:

[0243]

[0244] P k-1|k satisfy:

[0245]

[0246] Among them, P k-1|k Indicates the update time t k Transfer to acquisition time t k-1 Covariance of state quantities, P' k|k This indicates the absence of t. k-1 Observations collected at different times z k-1 In the case of t k The covariance of the state variables updated at each time, F k-1|k Indicates from t k Time shifts to t k-1 The state transition matrix at time Q k H represents the covariance of the prediction matrix. k-1 Indicates t k-1 The observation matrix at time, Var(z) k-1 ) represents t k-1 Observations collected at different times z k-1 The variance, R k-1A covariance of the observation matrix is denoted.

[0247] Optionally, the expectation of the state quantity at the updating time satisfies:

[0248]

[0249] where x k|k denotes the estimated observation quantity at the time t k , the expectation of the state quantity updated at the time t denotes the estimated observation quantity at the time t k-1 .

[0250] Optionally, the estimated value of the observation quantity at the collecting time satisfies:

[0251]

[0252] where, denotes the estimated observation quantity at the time t k-1 , H k-1 denotes the observation matrix at the time t k-1 , F k-1|k denotes the state transition matrix from the time t k to the time t k-1 , x' k|k denotes the state quantity at the time t k-1 , and z k-1 denotes the observation quantity collected at the time t k .

[0253] Optionally, the processing unit 1020 is configured to: in a case where the collecting time of the observation quantity is earlier than the updating time of the current state quantity of the moving object, and a time difference between the collecting time and the updating time is less than or equal to a threshold, obtain the estimated observation quantity according to the state transition model corresponding to the moving object.

[0254] It should be noted that the above target tracking apparatus 1000 is embodied in the form of functional units. The term “unit” herein can be implemented in the form of software and / or hardware, and is not limited specifically.

[0255] For example, the “unit” can be a software program, a hardware circuit or a combination of both, which realizes the above functions. The hardware circuit can include an application specific integrated circuit (ASIC), an electronic circuit, a processor (for example, a shared processor, a dedicated processor or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combination logic circuit and / or other suitable components supporting the described functions.

[0256] Therefore, the units of each example described in the embodiments of the present application can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0257] Figure 8 is a hardware structure schematic diagram of the target tracking device provided by the embodiments of the present application.

[0258] As shown in Figure 8 The target tracking device 1200 (the target tracking 1200 can be a computer device specifically) includes a memory 1201, a processor 1202, a communication interface 1203 and a bus 1204. Among them, the memory 1201, the processor 1202, the communication interface 1203 are communicated with each other through the bus 1204.

[0259] The memory 1201 can be a read only memory (read only memory, ROM), a static storage device, a dynamic storage device or a random access memory (random access memory, RAM). The memory 1201 can store programs, when the programs stored in the memory 1201 are executed by the processor 1202, the processor 1202 is used to execute each step of the target tracking method of the embodiments of the present application, for example, each step shown in Figure 6 .

[0260] It should be understood that the target tracking device shown in the embodiments of the present application can be a server, for example, can be a server in the cloud, or also can be a chip configured in the server in the cloud.

[0261] The processor 1202 can adopt a general central processing unit (central processing unit, CPU), a microprocessor, an application specific integrated circuit (application specific integrated circuit, ASIC) or one or more integrated circuits, used to execute related programs to realize the target tracking method of the method embodiments of the present application.

[0262] The processor 1202 can also be an integrated circuit chip with signal processing capability. In the implementation process, each step of the target tracking method of the present application can be completed by the integrated logic circuit of hardware in the processor 1202 or the instruction in the form of software.

[0263] The processor 1202 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium in the art. The storage medium is located in the memory 1201, and the processor 1201 reads the information in the memory 1201, and combines the hardware to complete the methods disclosed in the embodiments of the present application. Figure 7 The functions required to be performed by the units included in the target tracking device shown in the embodiments of the present application, or the steps of the target tracking method Figure 6 The target tracking method shown in the embodiments of the present application.

[0264] The communication interface 1203 uses a transceiver device such as but not limited to a transceiver to realize the communication between the target tracking device 1200 and other devices or communication networks.

[0265] The bus 1204 can include a path for transmitting information between various components (e.g., memory 1201, processor 1202, communication interface 1203) of the target tracking device 1200.

[0266] It should be noted that although the above-mentioned target tracking device 1200 only shows the memory, processor, communication interface, but in the specific implementation process, those skilled in the art should understand that the target tracking device 1200 can also include other devices necessary for normal operation. At the same time, according to the specific needs of those skilled in the art should understand that the above-mentioned target tracking device 1200 can also include hardware devices that realize other additional functions.

[0267] In addition, those skilled in the art should understand that the above-mentioned target tracking device 1200 can also only include the devices necessary to implement the embodiments of the present application, and does not have to include Figure 8 All the devices shown in the embodiments of the present application.

[0268] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0269] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0270] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0271] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0272] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0273] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or an access network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0274] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A target tracking method, characterized in that, include: Acquire the actual observations of moving objects collected by the sensor; When the actual observation is acquired earlier than the update time of the current state of the moving object, the estimated observation is obtained based on the state transition model corresponding to the moving object. The current state quantity is updated based on the actual observations and the estimated observations, and the updated state quantity is used as the state quantity corresponding to the update time. The step of obtaining the estimated observations based on the state transition model corresponding to the moving object includes: The likelihood function of the observation at the acquisition time is obtained based on the state transition model corresponding to the moving object. The estimated observation is determined based on the likelihood function of the observation at the acquisition time. The state transition model is determined based on the first probability density function, which is the probability density function for transitioning the current state quantity to the state quantity at the acquisition time. The step of updating the current state quantity based on the actual observation and the estimated observation, and using the updated state quantity as the state quantity corresponding to the update time, includes: The probability density function of the current state quantity is updated based on the likelihood function of the observation quantity corresponding to the acquisition time. The state quantity at the update time is determined based on the probability density function of the updated state quantity.

2. The method as described in claim 1, characterized in that, The likelihood function of the observations corresponding to the acquisition time satisfies: g(z k-1 |X k ′)=Ig(z k-1 |X)f k-1|k (X|X k ′)dX; Among them, g(z) k-1 |X k ′) indicates about t k-1 The observation z corresponding to time t k-1 The likelihood function, X represents t k-1 The state variable X that is updated in real time k ′ indicates the absence of z k-1 In the case of t k The state variable that is updated in real time, g(z) k-1 |X) represents the z k-1 For t k-1 Regarding z, in the case of constantly updating the state quantity k-1 The likelihood function, f k-1|k (X|X k ′) is the first probability density function, which is the expression of t k The state variable X that is updated at all times k 'Transfer to t k-1 The probability density function of the state variables that are updated at any given time.

3. The method as described in claim 2, characterized in that, The probability density function of the updated state quantity satisfies: Among them, f k|k (X k |z k ) represents t k The state variable X that is updated at all times k The probability density function, f′ k|k (X k ′|z k , z k-2 ) indicates the absence of z k-1 In the case of t k The state variable X that is updated at all times k The probability density function of z′ k Indicates t k The observations collected at any time, z k Let {z1, z2, ..., zk} be the set of observations collected at k time points. k-2 , z k-1 , z k }, z k-2 Let {z1, z2, ..., z2} be the set of observations collected at k-2 time points. k-2 } 4. The method as described in claim 1, characterized in that, The step of updating the current state quantity based on the actual observation and the estimated observation, and using the updated state quantity as the state quantity corresponding to the update time, includes: The expected state quantity corresponding to the update time is determined based on the actual observations and the estimated observations; The expected value of the state quantity corresponding to the update time is taken as the state quantity corresponding to the update time. The expected value of the state quantity corresponding to the update time is related to the Kalman gain value, which is related to the first covariance, the observation matrix at the acquisition time, the covariance of the observation matrix, and the variance of the observation quantity at the acquisition time. The first covariance refers to the covariance of the state quantity transferred from the update time to the acquisition time.

5. The method as described in claim 4, characterized in that, The Kalman gain value satisfies: Var(z k-1 )satisfy: P k-1|k satisfy: Among them, P k-1|k Indicates the update time t k Transfer to acquisition time t k-1 Covariance of state quantities, P′ k|k This indicates the absence of t. k-1 Observations collected at different times z k-1 In the case of t k The covariance of the state variables updated at each time, F k-1|k Indicates from t k Time shifts to t k-1 The state transition matrix at time Q k H represents the covariance of the prediction matrix. k-1 Indicates t k-1 The observation matrix at time, Var(z) k-1 ) represents t k-1 Observations collected at different times z k-1 The variance, R k-1 The covariance of the observation matrix is ​​represented.

6. The method as described in claim 5, characterized in that, The expected state quantity corresponding to the update time satisfies: Where, x k|k Indicates t k The expectation of the constantly updated state quantity, Indicates t k-1 The estimated observation at time x′ k|k This indicates that in the absence of the stated t k-1 Observations collected at different times z k-1 t under the circumstances k The expectation of constantly updated state quantities.

7. The method according to any one of claims 4 to 6, characterized in that, The estimated observation satisfies: in, Indicates t k-1 The estimated observation at time, H k-1 Indicates t k-1 The observation matrix at time F k-1|k Indicates from t k Time shifts to t k-1 The state transition matrix at time x′ k|k This indicates that in the absence of the stated t k-1 Observations collected at different times z k-1 t under the circumstances k The expectation of constantly updated state quantities.

8. The method according to any one of claims 1 to 7, characterized in that, When the acquisition time of the observed quantity is earlier than the update time of the current state quantity of the moving object, the estimated observed quantity is obtained according to the state transition model corresponding to the moving object, including: If the actual observation is acquired earlier than the current state of the moving object, and the time difference between the acquisition time and the update time is less than or equal to a threshold, the estimated observation is obtained based on the state transition model corresponding to the moving object.

9. A target tracking device, characterized in that, It includes an acquisition module and a processing module, wherein, The acquisition module is used to: acquire the actual observations of the moving object collected by the sensor; The processing module is used for: When the actual observation is acquired earlier than the update time of the current state of the moving object, the estimated observation is obtained based on the state transition model corresponding to the moving object. The current state quantity is updated based on the actual observations and the estimated observations, and the updated state quantity is used as the state quantity corresponding to the update time. The processing module is used for: The likelihood function of the observation at the acquisition time is obtained based on the state transition model corresponding to the moving object. The estimated observation is determined based on the likelihood function of the observation at the acquisition time. The state transition model is determined based on the first probability density function, which is the probability density function for transitioning the current state quantity to the state quantity at the acquisition time. The processing module is used for: The probability density function of the current state quantity is updated based on the likelihood function of the observation quantity corresponding to the acquisition time. The state quantity at the update time is determined based on the probability density function of the updated state quantity.

10. The apparatus as claimed in claim 9, characterized in that, The likelihood function of the observations corresponding to the acquisition time satisfies: g(z k-1 |X k ′)=∫g(z k-1 |X)f k-1|k (X|X k ′)dX; Among them, g(z) k-1 |X k ′) indicates about t k-1 The observation z corresponding to time t k-1 The likelihood function, X represents t k-1 The state variable X that is updated in real time k ′ indicates the absence of z k-1 In the case of t k The state variable that is updated in real time, g(z) k-1 |X) represents the z k-1 For t k-1 Regarding z, in the case of constantly updating the state quantity k-1 The likelihood function, f k-1|k (X|X k ′) is the first probability density function, which is the expression of t k The state variable X that is updated at all times k 'Transfer to t k-1 The probability density function of the state variables that are updated at any given time.

11. The apparatus as claimed in claim 10, characterized in that, The probability density function of the updated state quantity satisfies: Among them, f k|k (X k |z k ) represents t k The state variable X that is updated at all times k The probability density function, f′ k|k (X k ′|z k , z k-2 ) indicates the absence of z k-1 In the case of t k The state variable X that is updated at all times k The probability density function of z′ k Indicates t k The observations collected at any time, z k Let {z1, z2, ..., zk} be the set of observations collected at k time points. k-2 , z k-1 , z k }, z k-2 Let {z1, z2, ..., z2} be the set of observations collected at k-2 time points. k-2 } 12. The apparatus as claimed in claim 9, characterized in that, The processing module is used for: The expected state quantity corresponding to the update time is determined based on the actual observations and the estimated observations; The expected value of the state quantity corresponding to the update time is taken as the state quantity corresponding to the update time. The expected value of the state quantity corresponding to the update time is related to the Kalman gain value, which is related to the first covariance, the observation matrix at the acquisition time, the covariance of the observation matrix, and the variance of the observation quantity at the acquisition time. The first covariance refers to the covariance of the state quantity transferred from the update time to the acquisition time.

13. The apparatus as claimed in claim 12, characterized in that, The Kalman gain value satisfies: Var(z k-1 )satisfy: P k-1|k satisfy: Among them, P k-1|k Indicates the update time t k Transfer to acquisition time t k-1 Covariance of state quantities, P′ k|k This indicates the absence of t. k-1 Observations collected at different times z k-1 In the case of t k The covariance of the state variables updated at each time, F k-1|k Indicates from t k Time shifts to t k-1 The state transition matrix at time Q k H represents the covariance of the prediction matrix. k-1 Indicates t k-1 The observation matrix at time, Var(z) k-1 ) represents t k-1 Observations collected at different times z k-1 The variance, R k-1 The covariance of the observation matrix is ​​represented.

14. The apparatus as claimed in claim 13, characterized in that, The expected state quantity corresponding to the update time satisfies: Where, x k|k Indicates t k The expectation of the constantly updated state quantity, Indicates t k-1 The estimated observation at time x′ k|k This indicates that in the absence of the stated t k-1 Observations collected at different times z k-1 t under the circumstances k The expectation of constantly updated state quantities.

15. The apparatus as claimed in any one of claims 12 to 14, characterized in that, The estimated observation satisfies: in, Indicates t k-1 The estimated observation at time, H k-1 Indicates t k-1 The observation matrix at time F k-1|k Indicates from t k Time shifts to t k-1 The state transition matrix at time x′ k|k This indicates that in the absence of the stated t k-1 Observations collected at different times z k-1 t under the circumstances k The expectation of constantly updated state quantities.

16. The apparatus according to any one of claims 9 to 15, characterized in that, The processing module is used for: If the actual observation is acquired earlier than the current state of the moving object, and the time difference between the acquisition time and the update time is less than or equal to a threshold, the estimated observation is obtained based on the state transition model corresponding to the moving object.

17. A target tracking device, characterized in that, It includes at least one processor and a memory, the at least one processor being coupled to the memory for reading and executing instructions in the memory to perform the method as claimed in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 8.

Images