Vehicle target sensing method and device, storage medium and electronic device

Abstract

The embodiment of the invention provides a target sensing method and device of a vehicle, a storage medium and an electronic device. The method comprises the steps that first observation information corresponding to a first obstacle is acquired; under the condition that a group of known obstacles exist, matching the first obstacle with the group of known obstacles based on the first observation information and the observation information of the group of known obstacles; under the condition that a target obstacle matched with the first obstacle exists in the group of known obstacles, adding the first observation information into a target time sequence observation queue of the target obstacle according to a time sequence; under the condition that at least one piece of observation information exists in the target time sequence observation queue, the time difference between the observation moment and the observation moment of the queue tail observation information is greater than or equal to a time difference threshold value, the at least one piece of observation information is moved out of the target time sequence observation queue, and sequentially inputting the at least one piece of observation information into the target Kalman filter according to the time sequence so as to update the target Kalman filter.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of traffic control, and more specifically, to a method and device for sensing a target of a vehicle, a storage medium, and an electronic device. Background Art

[0002] Perception technology is a key technology in the field of intelligent driving, which can sense dynamic obstacles such as vehicles and pedestrians. For intelligent driving vehicles, they can sense obstacles through the sensing devices configured on them and track the sensed targets, thereby providing obstacle information to the intelligent driving vehicle or the driver of the intelligent driving vehicle for route planning.

[0003] The target objects that the sensing device can sense are the targets within its sensing range, but the sensing range of the sensing device is usually limited, and there are often blind spots. The targets in the blind spots may bring serious danger to the intelligent driving vehicle. Therefore, it can be seen that the target sensing method of the vehicle in the related art has the problem of poor vehicle driving safety due to the inability to sense the targets in the blind spots. Summary of the invention

[0004] The embodiments of the present application provide a vehicle target perception method and device, a storage medium and an electronic device to at least solve the technical problem that the vehicle target perception method in the related art has poor vehicle driving safety due to the inability to perceive target objects in the blind spot of the field of view.

[0005] According to one aspect of an embodiment of the present application, a vehicle target perception method is provided, comprising: obtaining first observation information corresponding to a first obstacle, wherein the first observation information is obstacle information of the first obstacle perceived by a perception device of a current vehicle or a perception device of a remote vehicle at a first moment; in the case where there is a group of known obstacles, matching the first obstacle with the group of known obstacles based on the first observation information and the observation information of the group of known obstacles, wherein the group of known obstacles is a collection of obstacles perceived by the perception device of the current vehicle and the perception device of the remote vehicle; in the case where there is a target obstacle matching the first obstacle in the group of known obstacles, matching the first observation information in a time sequence Added to the target time series observation queue of the target obstacle, wherein the target time series observation queue records the observation information corresponding to the target obstacle in time sequence; in the target time series observation queue, if there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold, the at least one observation information is removed from the target time series observation queue, and the at least one observation information is sequentially input into the target Kalman filter in time sequence to update the target Kalman filter, wherein the target Kalman filter is the Kalman filter corresponding to the target obstacle, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times.

[0006] According to another aspect of an embodiment of the present application, a vehicle target perception device is provided, comprising: a first acquisition unit, for acquiring first observation information corresponding to a first obstacle, wherein the first observation information is obstacle information of the first obstacle perceived by a perception device of a current vehicle or a perception device of a remote vehicle at a first moment; a first matching unit, for matching the first obstacle with a group of known obstacles based on the first observation information and the observation information of the group of known obstacles in the presence of a group of known obstacles, wherein the group of known obstacles is a collection of obstacles perceived by the perception device of the current vehicle and the perception device of the remote vehicle; and an adding unit, for matching the first obstacle with the group of known obstacles in a time sequence in the presence of a target obstacle matching the first obstacle in the group of known obstacles. The first observation information is added to a target time series observation queue of the target obstacle, wherein the target time series observation queue records the observation information corresponding to the target obstacle in time sequence; a first execution unit is used to remove the at least one observation information from the target time series observation queue if there is at least one observation information in the target time series observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to a time difference threshold, and input the at least one observation information into the target Kalman filter in sequence in time sequence to update the target Kalman filter, wherein the target Kalman filter is a Kalman filter corresponding to the target obstacle, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times.

[0007] In an exemplary embodiment, the first matching unit includes: a copying module, used to copy the Kalman filter corresponding to each known obstacle in the group of known obstacles respectively, to obtain a copy of the Kalman filter corresponding to each known obstacle; an input module, used to input the observation information of the time series observation queue of each known obstacle, whose observation time is before the first time, into the copy of the Kalman filter corresponding to each known obstacle in sequence according to the time series, to obtain an updated copy of the Kalman filter corresponding to each known obstacle; a prediction module, used to predict the obstacle information of each known obstacle at the first time using the updated copy of the Kalman filter corresponding to each known obstacle, to obtain the predicted obstacle information of each known obstacle; a matching module, used to match the first obstacle with each known obstacle respectively based on the first observation information and the predicted obstacle information of each known obstacle.

[0008] In an exemplary embodiment, the device also includes: a first determining unit for, after the first observation information is added to the target timing observation queue of the target obstacle in a timing sequence, when the position where the first observation information is inserted is not the end of the target timing observation queue, determining that there is no observation information in the target timing observation queue whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold; a second determining unit for, when the position where the first observation information is inserted is the end of the target timing observation queue and the observation time of the head observation information of the target timing observation queue is the same as the first observation information When the time difference between the first and second moments is greater than or equal to the time difference threshold, it is determined that in the target timing observation queue, there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold; a third determination unit is used to determine that in the target timing observation queue, there is no observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold when the position where the first observation information is inserted is the tail of the target timing observation queue and the time difference between the observation time of the head observation information of the target timing observation queue and the first moment is less than the time difference threshold.

[0009] In an exemplary embodiment, the device also includes: a copying unit, which is used to copy the updated target Kalman filter to obtain a copy of the target Kalman filter after the at least one observation information is input into the target Kalman filter in sequence in time sequence to update the target Kalman filter, when the latest obstacle state of the target obstacle is to be output; a second execution unit, which is used to input the observation information in the target time series observation queue into the copy of the target Kalman filter in time sequence, and output the state information finally output by the copy of the target Kalman filter as the latest obstacle state of the target obstacle.

[0010] In an exemplary embodiment, the first acquisition unit includes: a receiving module, used to receive the first observation information sent by the remote vehicle through the V2X device of the remote vehicle through the V2X device of the current vehicle, wherein the first observation information is obstacle information of the first obstacle perceived by the perception device of the remote vehicle at the first moment; or, a perception module, used to perform obstacle perception through the perception device of the current vehicle to obtain the first observation information, wherein the first observation information is obstacle information of the first obstacle perceived by the perception device of the current vehicle at the first moment.

[0011] In an exemplary embodiment, the first observation information includes the posture information of the first obstacle; the perception module includes: a perception submodule, which is used to perform obstacle perception through the perception device of the current vehicle to obtain an obstacle perception result corresponding to the first obstacle; a conversion submodule, which is used to convert the posture information of the first obstacle in the obstacle perception result to a specified coordinate system to obtain updated posture information of the first obstacle; a generation submodule, which is used to generate the first observation information according to the updated posture information of the first obstacle; the device also includes: a propagation unit, which is used to broadcast the first observation information through the V2X device of the current vehicle after the obstacle perception is performed through the perception device of the current vehicle to obtain the first observation information, so as to transmit the first observation information to other vehicles except the current vehicle.

[0012] In an exemplary embodiment, the device also includes: a second acquisition unit, used to acquire second observation information corresponding to the second obstacle, wherein the second observation information is obstacle information of the second obstacle perceived by the perception device of the current vehicle or the perception device of the remote vehicle at the second moment; a second matching unit, used to match the second obstacle with the group of known obstacles based on the second observation information and the observation information of the group of known obstacles in the presence of the group of known obstacles; and a third execution unit, used to create a time series observation queue for the second obstacle, add the second observation information to the created time series observation queue, and create a Kalman filter for the second obstacle when there is no obstacle matching the second obstacle in the group of known obstacles.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above method embodiments when executing the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any of the above method embodiments.

[0016] In an embodiment of the present application, the observation information perceived by the perception device of the local vehicle (i.e., the current vehicle) and the observation information perceived by the perception device of the remote vehicle are combined to obtain first observation information corresponding to the first obstacle, wherein the first observation information is obstacle information of the first obstacle perceived by the perception device of the current vehicle or the perception device of the remote vehicle at the first moment; in the case of a group of known obstacles, based on the first observation information and the observation information of the group of known obstacles, the first obstacle is matched with a group of known obstacles, wherein the group of known obstacles is the group of known obstacles perceived by the perception device of the current vehicle and the perception device of the remote vehicle. a set of obstacles; in the case where there is a target obstacle matching the first obstacle in a set of known obstacles, the first observation information is added to the target time series observation queue of the target obstacle in time sequence, wherein the target time series observation queue records the observation information corresponding to the target obstacle in time sequence; in the target time series observation queue, in the case where there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold, the at least one observation information is removed from the target time series observation queue, and the at least one observation information is sequentially input into the target Kalman filter in time sequence to update the target Kalman filter, wherein, The target Kalman filter is a Kalman filter corresponding to the target obstacle. The state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times. Since the obstacle information of the same obstacle perceived by the perception device of the current vehicle and the perception device of the remote vehicle is input into the Kalman filter as observation information to predict the obstacle state of the obstacle at different times, the perception results of multiple perception devices can be fused. For targets that cannot be perceived by the perception device of the own vehicle, they can be observed by the perception devices of other vehicles, thereby reducing the limitations of the single-vehicle perception system and solving the target problem of vehicles in related technologies. The perception method has a technical problem of poor vehicle driving safety due to the inability to perceive targets in the blind spot of the field of vision, thereby achieving the technical effect of improving vehicle driving safety; in addition, the observation information of the same obstacle perceived by the perception devices of different vehicles is put into the same time series observation queue, and only the observation information whose time difference between the observation time and the observation time of the latest observation information exceeds the set time difference threshold is input into the Kalman filter, thereby solving the problem of low accuracy of obstacle state prediction caused by the inability to process some observation information due to the inability to ensure the time series of perception results of multiple perception devices, thereby achieving the technical effect of improving the accuracy of obstacle state prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a schematic diagram of an application scenario of a target perception method for a vehicle according to an embodiment of the present application.

[0019] Figure 2 It is a flow chart of an optional target perception method for a vehicle according to an embodiment of the present application.

[0020] Figure 3 It is a schematic diagram of an optional vehicle target perception method according to an embodiment of the present application.

[0021] Figure 4 It is a schematic diagram of another optional vehicle target perception method according to an embodiment of the present application.

[0022] Figure 5 It is a schematic diagram of another optional vehicle target perception method according to an embodiment of the present application.

[0023] Figure 6 It is a schematic diagram of another optional vehicle target perception method according to an embodiment of the present application.

[0024] Figure 7 It is a schematic diagram of another optional vehicle target perception method according to an embodiment of the present application.

[0025] Figure 8 It is a schematic diagram of another optional vehicle target perception method according to an embodiment of the present application.

[0026] Fig. 9 It is a flowchart of another optional vehicle target perception method according to an embodiment of the present application.

[0027] Fig.10 It is a structural block diagram of an optional vehicle target perception device according to an embodiment of the present application.

[0028] Fig.11 It is a block diagram of a computer system structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0031] According to one aspect of an embodiment of the present application, a vehicle target perception method is provided. Optionally, in this embodiment, the vehicle target perception method can be applied to Figure 1 In the hardware environment shown, a current vehicle 102 and a remote vehicle 104 are included. The current vehicle 102 may be a vehicle with a V2X (Vehicle to everything) communication device and a sensing device, and the remote vehicle 104 may be at least one vehicle with a V2X communication device and a sensing device. Figure 1 As shown, the current vehicle 102 and the remote vehicle 104 can sense obstacles through the perception device, and the current vehicle 102 and the remote vehicle 104 can communicate through the V2X communication device. After the remote vehicle 104 senses the obstacle in the blind spot of the perception device of the current vehicle 102 through the perception device, the remote vehicle 104 can send the information of the obstacle through the V2X communication device. After receiving the information of the obstacle from the remote vehicle 104, the current vehicle 102 can observe the obstacle in the blind spot of the perception device of the current vehicle 102.

[0032] The above-mentioned perception device may include but is not limited to at least one of the following: an optical camera, a millimeter-wave radar, an ultrasonic sensor, a lidar, etc.; the V2X communication device may be a communication device using C-V2X (Cellular Vehicle to everything) technology, and the message set SSM (Signal Strength Measurement) message may be used in the above-mentioned V2X communication.

[0033] The vehicle target perception method of the embodiment of the present application can be executed by the current vehicle 102, or by other devices (e.g., cloud servers) to which the current vehicle 102 is connected, or by the current vehicle 102 and other devices together. Taking the vehicle target perception method of the embodiment executed by the current vehicle 102 as an example, Figure 2 is a flow chart of an optional vehicle target perception method according to an embodiment of the present application, such as Figure 2 As shown, the above method includes steps S202 to S208.

[0034] Step S202: Acquire first observation information corresponding to the first obstacle, wherein the first observation information is obstacle information of the first obstacle sensed by a perception device of the current vehicle or a perception device of a remote vehicle at a first moment.

[0035] The target perception method of the vehicle in this embodiment can be applied to the field of intelligent driving, and can be used in scenarios where obstacle perception is performed through perception devices. Vehicle perception technology is a key technology in the field of intelligent driving, especially the perception of dynamic obstacles such as vehicles and pedestrians. However, single-vehicle perception technology often has blind spots (targets in the blind spots cannot be perceived), and obstacles in the blind spots may pose a greater risk to autonomous driving. In this regard, V2X technology can be used to communicate between vehicles and between vehicles and between roads to achieve perception sharing between vehicles and between vehicles and between roads; and through perception sharing between vehicles and between vehicles and between roads, targets that were originally in the blind spots of the vehicle and not discovered by the vehicle’s perception system may be discovered by the perception systems of other vehicles and sent over through V2X communication, and integrated with the vehicle’s perception results, thereby solving the blind spot problem of single-vehicle perception to a certain extent.

[0036] In this embodiment, the perception device of the remote vehicle (other perception devices, collectively referred to as remote devices) serializes its perception results and sends the serialized perception results to the current vehicle through the V2X device or other communication device. After deserialization, the vehicle perception module (i.e., the perception device of the current vehicle) uses it as an obstacle observed by the remote vehicle for subsequent processing. The perception device of the current vehicle can perceive obstacles and perform subsequent processing based on the perceived obstacles. For the current vehicle, it can obtain first observation information corresponding to the first obstacle, which is the obstacle information of the first obstacle perceived by the perception device of the current vehicle or the perception device of the remote vehicle at the first moment. In the case where the first obstacle is in the blind spot of the field of view of the perception device of the current vehicle, the first observation information can be perceived by the perception device of the remote vehicle and sent by the remote vehicle to the current vehicle.

[0037] Optionally, the data sent by the remote device may include: the positioning information and chassis information of the remote device; the unique device identifier (ID); the perception result, which may include all or part of the information of the size, position, speed, and confidence of each target object (obstacle), and may also include other information. The remote device may only send the target objects it perceives, but not the targets received by other devices. The first observation information may be the perception result of a certain obstacle, or it may be obtained after certain processing of the perception result of a certain obstacle.

[0038] Step S204, in the presence of a group of known obstacles, matching the first obstacle with a group of known obstacles based on the first observation information and the observation information of the group of known obstacles, wherein the group of known obstacles is a collection of obstacles perceived by the perception device of the current vehicle and the perception device of the remote vehicle.

[0039] If there is a group of known obstacles, the first obstacle can be matched with the group of known obstacles based on the first observation information and the observation information of the group of known obstacles, so as to determine whether the first observation information is observation information that matches a certain known obstacle, and the group of known obstacles is a collection of obstacles that have been sensed by the sensing device of the current vehicle and the sensing device of the remote vehicle. In the above group of known obstacles, the obstacles included are different, and may include obstacles that match the first obstacle (that is, the first obstacle is a certain known obstacle), or may not include obstacles that match the first obstacle (that is, the first obstacle is not any known obstacle).

[0040] There are many ways to match the first obstacle with a group of known obstacles. The matching may be performed using obstacle features (for example, inputting the first observation information), or based on the degree of overlap of the space occupied by the obstacles. Obstacle matching may also be performed in other ways, which are not limited in this embodiment, as long as it can be accurately determined whether a known obstacle that matches the first obstacle exists in a group of known obstacles.

[0041] Optionally, if there is no known obstacle (for example, no obstacle is sensed since startup, or the sensed obstacles are outdated), a time observation queue can be established for the first obstacle, a Kalman filter can be configured, and the first obstacle can be matched as a known obstacle with subsequently observed obstacles. The matching process is similar to the above and will not be repeated here.

[0042] Step S206, when there is a target obstacle matching the first obstacle in a group of known obstacles, the first observation information is added to a target timing observation queue of the target obstacle in time sequence, wherein the target timing observation queue records the observation information corresponding to the target obstacle in time sequence.

[0043] For each perceived obstacle, a time series observation queue can be configured for it, and the time series observation queue can be used to record the observation information corresponding to the obstacle in time series. If there is a target obstacle that matches the first obstacle in a group of known obstacles, the first obstacle can be considered as the target obstacle. At this time, the first observation information can be added to the target time series observation queue of the target obstacle in time series. The target time series observation queue already contains at least one observation information corresponding to the target obstacle, and the first observation information can be added at the head, tail or middle of the target time series observation queue.

[0044] For each observation information, it can be the obstacle information of a certain obstacle perceived by a certain sensing device at a certain moment. Here, the sensing moment corresponding to the observation information is the observation moment of the observation information. The target time series observation queue includes the observation information corresponding to the target obstacle arranged in time sequence, that is, the observation information corresponding to the target obstacle arranged in the order of observation moment. The method of adding the first observation information to the target time series observation queue of the target obstacle in time sequence can be:

[0045] Starting from the head observation information of the target timing observation queue, the observation time of the observation information in the target timing observation queue is compared with the first time in sequence until the observation information with an observation time less than the first time is found, and the first observation information is inserted before the found observation information. If the observation information with an observation time less than the first time is still not found when the tail observation information is found, the first observation information is inserted into the tail of the target timing observation queue; or,

[0046] Starting from the observation information at the end of the target timing observation queue, the observation time of the observation information in the target timing observation queue is compared with the first time in sequence until the observation information with an observation time greater than the first time is found, and the first observation information is inserted after the found observation information. If the observation information at the head of the queue is found and the observation information with an observation time greater than the first time is still not found, the first observation information is inserted into the head of the target timing observation queue.

[0047] Optionally, if there is no obstacle matching the first obstacle in a group of known obstacles, a time series observation queue can be established for the first obstacle, a Kalman filter can be configured, and the first obstacle can be used as a known obstacle to match with subsequently observed obstacles. The matching process is similar to the above and will not be repeated here.

[0048] Step S208, in the target timing observation queue, if there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold, at least one observation information is removed from the target timing observation queue, and at least one observation information is input into the target Kalman filter in sequence according to the timing to update the target Kalman filter.

[0049] In this embodiment, a Kalman filter (including an extended Kalman filter, etc.) can be used for autonomous driving perception tracking. Since the Kalman filter requires that the input in the filter must be time-series, the perception results of the vehicle and the perception results sent by other perception devices cannot guarantee their time sequence. The design of the aforementioned time-series observation queue can solve the problem that the time sequence of the perception results of multiple perception devices cannot be guaranteed: the time-series observation queue can be understood as an observation cache queue. By caching the observation information, the observation information received within a certain period of time can be cached in time sequence, and the observation information whose time difference between the observation time and the observation time of the end of the queue is greater than or equal to the time difference threshold is input into the Kalman filter to ensure the timeliness of the Kalman filter update.

[0050] Here, Kalman filter is a highly efficient recursive filter, which can estimate the state of a dynamic system from a series of measurements that do not completely contain noise. Simple Kalman filter can be applied to linear systems that conform to Gaussian distribution, while the improved extended Kalman filter can be applied to dynamic systems with time nonlinearity. The basic idea of ​​extended Kalman filter is to linearize the nonlinear system and then perform Kalman filtering. The Kalman filter used in this embodiment can be an extended Kalman filter (Extended Kalman Filter, EKF filter).

[0051] The Kalman filter can estimate the state of a dynamic system (predict and update the state of a dynamic system) and is used in a target tracking system. For each trajectory, assume that the start time of this trajectory is t0 and the end time is t n , change t0 to t n The observations at each moment are sent to the EKF filter in sequence, and the filter update and prediction formulas are as follows:

[0052] x′ k =Fx k-1 +u

[0053] P′ k =FP k-1 F T +Q

[0054] y k = z - Hx′ k

[0055] S=HP′ k H T +R

[0056] K=P′ k H T S -1

[0057] x k= x′ k +Ky k

[0058] P k =(I-KH)P′ k

[0059] Among them, x k-1 is the state of the filter at time k-1, x′ k is the state at time k predicted by the filter through the state at time k-1, x k is the state at time k after filtering output by the final filter (i.e., the updated state), where the state quantity x contains all or part of the following dimensions: coordinates, speed, direction in the map coordinate system, and length, width, and height of the target;

[0060] P is the state covariance matrix, and the initial moment is set to P0, and then P changes with time. k-1 is the state covariance of the filter at time k-1, P′ k is the covariance at the kth moment after the prediction, P k is the covariance of the final filter output at the filtered k-th moment;

[0061] Q is the process noise matrix, which is set as the unit matrix;

[0062] y k is the observation at time k;

[0063] F is the state transfer matrix, which represents the conversion relationship between the state at time k and the state at time k-1;

[0064] H is the observation matrix, which represents the conversion relationship between the observation quantity and the state quantity;

[0065] K is the Kalman gain.

[0066] For each obstacle, a trajectory can be managed, and each trajectory corresponds to a Kalman filter. In the single-vehicle perception system, the data input by the sensor at each moment will be passed into the deep learning detection model, and the model will output the information of the obstacle. The perception system will predict the Kalman filter of the managed trajectory to the moment of the current sensor input, and match it with the obstacle information output by the deep learning detection model. After matching, the obstacle information is used as observation input into the Kalman filter to update the Kalman filter. The Kalman filter predicts the trajectory, and the perception system inputs the new data into the Kalman filter for matching and updates the filter parameters.

[0067] However, when faced with the perception results sent by multiple perception devices, the timestamps of the sensor input data often cannot guarantee their timing. For example, the timestamp of the ego vehicle sensor reaches t cThe timestamp of the obstacle perception result sent by the remote vehicle, that is, the timestamp of the remote vehicle sensor input data, may be at t c Before the moment. If the Kalman filter has predicted and updated the vehicle t c At this moment, it can be seen from the above Kalman filter formula that the Kalman filter cannot process t c The previous observations cannot process the perception results sent by the remote vehicle. However, the above situation can be avoided to a certain extent by setting the above time series observation queue.

[0068] In this embodiment, each obstacle has its own corresponding time series observation queue and corresponding Kalman filter. For the target obstacle, its corresponding Kalman filter is the target Kalman filter, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times. The obstacle state may include but is not limited to at least one of the following: the coordinates, speed and direction of the obstacle in the map coordinate system, and the length, width and height of the obstacle.

[0069] In this embodiment, if there is at least one observation information in the target time series observation queue whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold, at least one observation information can be removed from the target time series observation queue, and at least one observation information can be sequentially input into the target Kalman filter in sequence according to the time sequence to update the target Kalman filter. The setting of the time difference threshold can ensure that the time difference between the observation information at the head and the end of the target time series observation queue is not greater than the above-mentioned time difference threshold.

[0070] Optionally, when new observation information that needs to be added to the target time series observation queue is obtained, the observation time of the above observation information can be compared with the observation time of the observation information at the end of the queue. If the time difference between the two is greater than the time difference threshold, the above observation information is directly discarded and not added to the target time series observation queue; if the time difference between the two is not greater than the time difference threshold, the above observation information can be added to the corresponding position in the target time series observation queue according to the time sequence, and then it is confirmed whether there is an observation information whose observation time and the observation time of the observation information at the end of the queue have a time difference greater than the time difference threshold. If so, these observation information are removed from the target time series observation queue and input into the target Kalman filter in sequence to update the target Kalman filter (the model parameters may be updated, such as state, covariance matrix, etc.).

[0071] For example, a trajectory management module may be provided, which may be used to manage the trajectory of a perceived target (ie, an obstacle). Figure 3As shown in Figure 1, for each target (i.e., perceived obstacle), the trajectory management module retains a time-series observation queue, i.e., the time-series observation queue. The observations in the time-series observation queue (i.e., the aforementioned observation information) include the observations of the ego vehicle and the remote device. When the observation at the head of the queue (i.e., Figure 3 The observation at time t0) and the tail of the queue (i.e., Figure 3 When the time difference of the observations (the observation at time tn in the queue) exceeds 0.5s, the observation at the head of the queue is removed from the time series observation queue and input into the Kalman filter. At this time, the last update time of the Kalman filter changes from tf to t0. For each time series observation queue, all observations with a time difference of more than 0.5s from the observation at the end of the queue are removed and input into the Kalman filter. Therefore, the time difference of the observations in the time series observation queue must be less than 0.5s. Here, 0.5s is the set time difference threshold, which can be set as needed and allowed to be adjusted.

[0072] Through the embodiments provided by the present application, first observation information corresponding to a first obstacle is obtained, wherein the first observation information is obstacle information of the first obstacle perceived by a perception device of the current vehicle or a perception device of a remote vehicle at the first moment; in the case where there is a group of known obstacles, based on the first observation information and the observation information of a group of known obstacles, the first obstacle is matched with a group of known obstacles, wherein a group of known obstacles is a collection of obstacles perceived by the perception device of the current vehicle and the perception device of the remote vehicle; in the case where there is a target obstacle matching the first obstacle in the group of known obstacles, the first observation information is added to a target time series observation queue of the target obstacle in time sequence, wherein the target time series observation queue records the observation information matching the target obstacle in time sequence. object; in the target time series observation queue, if there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold, the at least one observation information is removed from the target time series observation queue, and the at least one observation information is sequentially input into the target Kalman filter in time sequence to update the target Kalman filter, wherein the target Kalman filter is the Kalman filter corresponding to the target obstacle, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times, which can solve the technical problem of poor vehicle driving safety caused by the inability to perceive the target object in the blind spot of the field of view in the target perception method of the vehicle in the related art, thereby improving the safety of vehicle driving.

[0073] In an exemplary embodiment, based on the first observation information and the observation information of the group of known obstacles, matching the first obstacle with the group of known obstacles includes: respectively copying the Kalman filter corresponding to each known obstacle in the group of known obstacles to obtain a copy of the Kalman filter corresponding to each known obstacle; inputting the observation information whose observation time is before the first time in the time series observation queue of each known obstacle into the copy of the Kalman filter corresponding to each known obstacle in sequence according to the time series to obtain an updated copy of the Kalman filter corresponding to each known obstacle; using the updated copy of the Kalman filter corresponding to each known obstacle to predict the obstacle information of each known obstacle at the first time to obtain the predicted obstacle information of each known obstacle; and matching the first obstacle with each known obstacle respectively based on the first observation information and the predicted obstacle information of each known obstacle.

[0074] When matching the first obstacle with a group of known obstacles, the first observation information can be directly compared with the latest observation information of a group of known obstacles to determine whether there is a known obstacle that matches the first obstacle. However, in the above obstacle matching method, if the time difference between the observation moment of the latest observation information of the known obstacle and the first moment is long, then due to the movement of the vehicle or the movement of the obstacle itself, the above matching method has the problem of low matching accuracy.

[0075] In this embodiment, a Kalman filter can be used to predict the obstacle state of a known obstacle at the first moment, and matched with the first obstacle based on the prediction result. Since the obstacle state of the obstacle at the first moment is predicted based on the information of the perceived obstacle, the accuracy of the prediction result can be guaranteed, thereby improving the accuracy of obstacle matching; in addition, the prediction made using the Kalman filter can ensure the accuracy of obstacle matching.

[0076] Considering that there is a certain time difference between the obstacle state represented by the filter parameters of the Kalman filter corresponding to each known obstacle and the first moment. In order to improve the matching degree between the obstacle state and the first moment, the observation information in the time series observation queue of each known obstacle, whose observation time is before the first moment, can be sequentially input into the Kalman filter corresponding to each known obstacle according to the time sequence, and the updated Kalman filter corresponding to each known obstacle is obtained. At this time, the filter parameters of the Kalman filter corresponding to each known obstacle can represent the obstacle state closest to the first moment. At this time, the obstacle information (which can be state information, or the obstacle state of the known obstacle at the first moment) of each known obstacle at the first moment is predicted using the updated Kalman filter corresponding to each known obstacle, and the predicted obstacle information of each known obstacle is obtained; based on the first observation information and the predicted obstacle information of each known obstacle, the first obstacle is matched with each known obstacle respectively.

[0077] Optionally, considering that the observation time of the subsequently received observation information corresponding to the known obstacle may be before the first time, if the filter parameters of the Kalman filter corresponding to the known obstacle are directly updated, the newly received observation information may be discarded due to unavailability. In this regard, the Kalman filter corresponding to each known obstacle in a set of known obstacles may be copied respectively to obtain a copy of the Kalman filter corresponding to each known obstacle, and the copy of the copied Kalman filter is used for prediction in the prediction process to ensure that the prediction process does not change the filter parameters of the original Kalman filter.

[0078] For example, Figure 4 As shown, there are 5 known obstacles, which correspond to a time series observation queue and a Kalman filter respectively. The latest observation received is the remote vehicle observation, and the timestamp of the observation is t m (An example of the first moment), the Kalman filter corresponding to each known obstacle can be copied separately to obtain five Kalman filter copies, and then predicted to t m The corresponding prediction results are used to perform obstacle matching to obtain the obstacle matching results.

[0079] Optionally, in this embodiment, there may be one or more matching methods for obstacles, including but not limited to at least one of the following: IoU (Intersection over Union) matching, nearest neighbor matching, and an obstacle successfully matched by any of the matching methods may be considered as an obstacle that matches the first obstacle.

[0080] Among the above matching methods, IoU matching is one of the commonly used evaluation indicators in target detection, which is used to measure the degree of overlap between the predicted bounding box and the true bounding box. For the above obstacle matching, IoU matching refers to the intersection area of ​​the bounding boxes of two obstacles divided by their union area. The higher the IoU, the higher the matching degree of the obstacles. The nearest neighbor matching aims to pair individuals with the same or similar propensity scores as those in the treatment group. This method reduces confounding bias and simulates the effect of randomized controlled studies, thereby allowing a more reasonable comparison between the observation group and the control group.

[0081] For example, the timestamp of the remote vehicle observation is t m For each target trajectory, the state of the filter of each trajectory can be copied to predict the observed t m The timestamp is used to perform IoU matching and nearest neighbor matching with the observations. The observations that are successfully matched update the corresponding trajectory, and the observations that are not successfully matched create new trajectories.

[0082] like Figure 5 As shown in Figure 2, when matching, it is necessary to first copy the Kalman filter of the known obstacle, and then use the Kalman filter copy to predict to t m At the moment, the prediction result is used to perform IoU matching and nearest neighbor matching with the distant vehicle observation. If both matching fails, a new time series observation queue and Kalman filter are created for the distant vehicle observation. If any known obstacle successfully matches the distant vehicle observation in any matching method, the distant vehicle observation is added to the time series observation queue of the known obstacle to update the trajectory. The above steps can be performed by the trajectory matching module.

[0083] Through this embodiment, by copying the Kalman filter, updating the copied Kalman filter, predicting known obstacles, and then using the prediction results for matching, it is possible to complete the matching of obstacles without changing the original Kalman filter, thereby improving the convenience of obstacle matching.

[0084] In an exemplary embodiment, after the first observation information is added to the target time sequence observation queue of the target obstacle in time sequence, the above method also includes: when the position where the first observation information is inserted is not the end of the target time sequence observation queue, determining that there is no observation information in the target time sequence observation queue whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold; when the position where the first observation information is inserted is the end of the target time sequence observation queue, and the time difference between the observation time of the first observation information of the target time sequence observation queue and the first time is greater than or equal to the time difference threshold, determining that there is at least one observation information in the target time sequence observation queue whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold; when the position where the first observation information is inserted is the end of the target time sequence observation queue, and the time difference between the observation time of the first observation information of the target time sequence observation queue and the first time is less than the time difference threshold, determining that there is no observation information in the target time sequence observation queue whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold.

[0085] In this embodiment, considering that the observation information in the time series observation queue is arranged in time series, it can be determined whether there is observation information in the target time series observation queue whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold based on the insertion position of the observation information. The way to insert the observation information into the time series observation queue can be as follows: Figure 6 As shown, for successfully matched observations and trajectories, according to the timestamp t m , insert the observation into the time series observation queue, ensuring that the observation queue after insertion is still time series. That is, after insertion, the timestamps t1 and t2 of the observations on the left and right sides of the observation in the queue satisfy t1≤t m ≤t2. Each time an observation is inserted into the time series observation queue, the trajectory management module will determine whether the time difference between the observations at the head and tail of the queue exceeds 0.5s to ensure that the time difference of the observations in the time series observation queue is less than 0.5s. If the timestamp of the observation is earlier than t n -0.5s, it will not be inserted into the time series observation queue and will be directly discarded. The above process can be performed by the trajectory update module.

[0086] In this embodiment, the observation information in the target timing observation queue is time-series. If the first observation information is not inserted at the end of the target timing observation queue, and the observation information whose observation time is earlier than the observation time of the head observation information will be discarded, then the insertion position of the first observation information is not the head of the queue, then the time difference between the observation time of the end observation information and the observation time of the head observation information remains unchanged. Therefore, it can be directly determined that in the target timing observation queue, there is no observation information whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold.

[0087] If the position where the first observation information is inserted is the tail of the target timing observation queue, it can be compared whether the time difference between the observation time of the head observation information of the target timing observation queue and the first time is greater than the time difference threshold. If the above time difference is greater than or equal to the time difference threshold, it can be determined that there is at least one observation information in the target timing observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold; if the above time difference is less than the time difference threshold, it can be determined that there is no observation information in the target timing observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold.

[0088] Through this embodiment, by judging whether there is observation information whose time difference between the observation time and the observation time of the end observation information is greater than or equal to the time difference threshold according to the insertion position of the observation information, the process of determining whether there is observation information that needs to be removed from the timing observation queue can be simplified, thereby improving the efficiency of information processing.

[0089] In an exemplary embodiment, after at least one observation information is input into the target Kalman filter in sequence in time sequence to update the target Kalman filter, the above method also includes: when the latest obstacle state of the target obstacle is to be output, copying the updated target Kalman filter to obtain a copy of the target Kalman filter; inputting the observation information in the target time series observation queue into the copy of the target Kalman filter in time sequence, and outputting the state information finally output by the copy of the target Kalman filter as the latest obstacle state of the target obstacle.

[0090] In this embodiment, if it is necessary to output the latest obstacle state of the target obstacle (i.e., the trajectory state of the target obstacle), the observation information in the target time series observation queue can be sequentially input into the target Kalman filter in time sequence, and the state information finally output by the target Kalman filter is output as the latest obstacle state of the target obstacle. Similar to the above-mentioned embodiment, the above-mentioned update method may cause the observation information received subsequently to be unavailable. For this reason, the updated target Kalman filter can be copied first, and then the observation information in the target time series observation queue can be sequentially input into the copy of the target Kalman filter in time sequence, and the state information finally output by the copy of the target Kalman filter is output as the latest obstacle state of the target obstacle. The above-mentioned operation does not affect the original target Kalman filter.

[0091] For example, a trajectory output module may be provided, and the trajectory output module is used to output the trajectory of the obstacle. Figure 7 As shown in the figure, for all trajectories, when outputting the states of these trajectories, the state of the Kalman filter of the trajectory is copied, and the observations in the time series observation queue are sequentially input into the copy of the Kalman filter. At this time, the state of the copied Kalman filter is the state of the target after filtering at the latest moment, and the state is output and sent to the downstream module. Here, when a trajectory has no new observation data for a long time, it is considered that the target is lost and the trajectory can be deleted; the downstream modules can be set as needed, such as the prediction module, planning control module, etc.

[0092] Through this embodiment, when it is necessary to output the latest obstacle status of the obstacle, the Kalman filter is copied, and the observation information in the time series observation queue is input into the copy of the Kalman filter in sequence according to the time sequence to obtain the latest obstacle status. This can achieve the state update without changing the original Kalman filter, thereby improving the convenience of obstacle status determination.

[0093] In an exemplary embodiment, obtaining first observation information corresponding to a first obstacle includes: receiving, through a V2X device of a current vehicle, first observation information sent by a remote vehicle through a V2X device of the remote vehicle, wherein the first observation information is obstacle information of the first obstacle sensed by a perception device of the remote vehicle at a first moment; or, performing obstacle perception through a perception device of the current vehicle to obtain the first observation information, wherein the first observation information is obstacle information of the first obstacle sensed by the perception device of the current vehicle at the first moment.

[0094] In this embodiment, for the scenario where the first observation information is obstacle information of a first obstacle sensed by a perception device of a remote vehicle at a first moment, the perception device of the remote vehicle may send the above-mentioned first observation information through a V2X device of the remote vehicle. Correspondingly, the current vehicle may receive the first observation information sent by the remote vehicle through the V2X device of the remote vehicle through the V2X device of the current vehicle, thereby obtaining the above-mentioned first observation information. For the scenario where the first observation information is obstacle information of a first obstacle sensed by a perception device of the current vehicle at a first moment, the current vehicle may perform obstacle perception through the perception device of the current vehicle, thereby obtaining the above-mentioned first observation information.

[0095] For example, the perception device of the current vehicle is the vehicle perception module, the V2X device of the current vehicle is the vehicle V2X device, the perception device of the remote vehicle is the remote device perception module, and the V2X device of the remote vehicle is the remote V2X device. Figure 8 As shown, other perception devices (i.e., remote device perception modules) can perform model detection and tracking in a similar manner to the above to obtain perception results, and then serialize the perception results and send them to the ego vehicle through the remote V2X device. After deserializing the above perception results, the ego vehicle perception module predicts the timestamp of the obstacle observed by the ego vehicle through its trajectory when it is sent by the remote vehicle, and matches it with the timestamp. After matching, the obstacle observed by the remote vehicle is fused with the obstacle trajectory observed by the ego vehicle, thereby completing the fusion of the perception results of the target.

[0096] Through this embodiment, the perception results of perception devices of different vehicles are transmitted through V2X devices, and the perception results of multiple perception devices can be integrated into autonomous driving through V2X devices, thereby reducing the limitations of a single-vehicle perception system and improving the accuracy of target perception.

[0097] In an exemplary embodiment, the first observation information includes the position and posture information of the first obstacle. Considering that the perception results of the perception devices of different vehicles are based on their own coordinate systems, if the perception results are to be fused, the position and posture information of the obstacles perceived by different perception devices need to be converted to a unified coordinate system. The above conversion process can be performed by the vehicle on which the perception results are to be fused. However, in the above conversion method, there is a situation where the position and posture information of the obstacle perceived by the perception device of one vehicle is converted separately by different vehicles. In order to improve the convenience of the conversion of the position and posture information of the obstacle, the above conversion process can be performed by the perception device that perceives the position and posture information of the obstacle or the vehicle where the perception device is located.

[0098] Correspondingly, obstacle perception is performed through the perception device of the current vehicle to obtain the first observation information, including: obstacle perception is performed through the perception device of the current vehicle to obtain an obstacle perception result corresponding to the first obstacle; the posture information of the first obstacle in the obstacle perception result is converted to a specified coordinate system to obtain updated posture information of the first obstacle; and the first observation information is generated according to the updated posture information of the first obstacle.

[0099] The designated coordinate system to which the position and posture information of the perceived obstacle is converted may be a map coordinate system. The position and posture conversion may be accomplished using a coordinate conversion matrix between different coordinate systems. The generated first observation information may be transmitted by the current vehicle to other vehicles other than the current vehicle. The transmission of observation information between vehicles may be achieved by broadcasting through a V2X device. Correspondingly, after obtaining the first observation information by performing obstacle perception through the perception device of the current vehicle, the above method further includes: broadcasting the first observation information through the V2X device of the current vehicle to transmit the first observation information to other vehicles other than the current vehicle.

[0100] It should be noted that when the perception device sends its perception data, it can ensure that its positioning data and perception data are normal. Once it finds that its positioning data fails (for example, the signal is lost, etc.), it will no longer send the perception data; at the same time, for all perceived obstacles, only obstacles with a confidence level higher than 60% output by the convolutional neural network can be sent to ensure the accuracy of the sent perception data. In addition, when performing perception data fusion, any obstacle will be output only if it is observed several times within a certain period. The above method can reduce the impact of false detection on the vehicle to a certain extent; at the same time, an evaluation will be performed on each observation source during the fusion process. If the perception range of a remote device has a large overlapping area with the vehicle, but there are a large number of unmatched observations within the overlapping range, the remote device is considered unreliable and the perception data of the remote device is ignored.

[0101] In this embodiment, after the perception device of the current vehicle generates the first observation information, the first observation information can be broadcast through the V2X device, and the remote vehicle can obtain the first observation information after receiving the broadcast.

[0102] Through this embodiment, by converting the posture information of the perceived obstacle into a specified coordinate system and then generating observation information, the vehicle that perceives the obstacle can complete the conversion of the posture information, thereby improving the convenience of posture information conversion; in addition, by broadcasting observation information through V2X equipment, the transmission of perception data between different vehicles can be facilitated, thereby improving the efficiency of information transmission.

[0103] In an exemplary embodiment, the method further includes: obtaining second observation information corresponding to the second obstacle, wherein the second observation information is obstacle information of the second obstacle perceived by a perception device of the current vehicle or a perception device of a remote vehicle at a second moment; in the presence of a group of known obstacles, matching the second obstacle with a group of known obstacles based on the second observation information and the observation information of a group of known obstacles; in the absence of an obstacle matching the second obstacle in the group of known obstacles, creating a time series observation queue for the second obstacle, adding the second observation information to the created time series observation queue, and creating a Kalman filter for the second obstacle.

[0104] In this embodiment, the second observation information corresponding to the second obstacle can be obtained. The second obstacle, the second observation information, and the method for obtaining the second observation information are similar to those in the above-mentioned embodiment, and are not described in detail here. Here, the second observation information is the obstacle information of the second obstacle sensed by the sensing device of the current vehicle or the sensing device of the remote vehicle at the second moment. The second obstacle and the first obstacle can be the same obstacle or different obstacles, which is not limited in this embodiment.

[0105] If there is a group of known obstacles, the second obstacle can be matched with a group of known obstacles based on the second observation information and the observation information of the group of known obstacles. The above obstacle matching method is similar to that in the above embodiment and is not described in detail here. If there is no obstacle matching the second obstacle in the group of known obstacles, a time series observation queue can be created for the second obstacle, and the second observation information can be added to the created time series observation queue. In addition, a Kalman filter can be created for the second obstacle. Thereafter, the second obstacle can be used as a known obstacle for subsequent obstacle matching.

[0106] Through this embodiment, for a newly perceived obstacle for which there is no matching known obstacle, a time series observation queue and a Kalman filter can be created for it, and its observation information can be first placed in the created time series observation queue to improve the accuracy of obstacle state prediction; in addition, the created Kalman filter can facilitate subsequent obstacle matching and improve the efficiency of obstacle matching.

[0107] The target perception method of the vehicle in the embodiment of the present application is explained below in conjunction with an optional example. In this optional example, a target fusion scheme based on V2X communication and Kalman filter is provided. The vehicle equipped with a perception system and V2X communication equipment and the road-side perception equipment convert the position of the obstacle to a unified coordinate system after sensing the obstacle, and send it to the outside in the form of broadcast through the V2X equipment; and after the vehicle receives the perception results sent by one or more other perception devices, it fuses them with the perception results of the vehicle itself. The above scheme can be executed by a trajectory management module, a trajectory matching module, a trajectory update module and a trajectory output module, and the logic executed by different modules is the same or similar to that in the aforementioned embodiment.

[0108] Fig. 9 is an example flow chart of another optional vehicle target perception method according to an embodiment of the present application, such as Fig. 9 As shown, the process of the method includes steps S902 to S912.

[0109] In step S902, the perception device of the remote vehicle observes first observation information of the first obstacle, serializes the first observation information and sends it to the current vehicle through the V2X device.

[0110] Step S904: The V2X device of the current vehicle receives the serialized first observation information sent by the remote vehicle, and transmits it to the perception device of the current vehicle for deserialization.

[0111] Step S906: The matching unit of the current vehicle copies the Kalman filter of the known obstacle and predicts the known obstacle to the first moment.

[0112] Step S908, matching the first observation information with the known obstacle observation information at the first moment of the predicted value, if there is a known obstacle that successfully matches the first observation information, executing step S910, if there is no known obstacle that successfully matches the first observation information, executing step S912.

[0113] Step S910: insert the first observation information into the time sequence observation queue of the matching known obstacle in time sequence.

[0114] Step S912: create a new time series observation queue and a Kalman filter for the first observation information, and insert the first observation information into the newly created time series observation queue.

[0115] Through this optional example, by observing the target in the blind spot of the vehicle through other perception devices, the safety risk brought by obstacles not perceived by the vehicle's own perception system to the vehicle operation can be reduced to a certain extent, thereby improving the vehicle's driving safety.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application can be essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a terminal device to execute the method described in each embodiment of the present application, and the above storage medium can be ROM (Read-Only Memory, Read-Only Memory) / RAM (Random Access Memory, Random Access Memory), a disk or an optical disk, etc., and the above terminal device can be a mobile phone, a computer, a server or a network device, etc.

[0117] According to another aspect of the embodiments of the present application, a target perception device for a vehicle is also provided, and the device is used to implement the target perception method for the vehicle provided in the above-mentioned embodiments, which has been described and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0118] Fig.10 is a structural block diagram of an optional vehicle target sensing device according to an embodiment of the present application, such as Fig.10 As shown in , the device includes: a first acquiring unit 1002 , a first matching unit 1004 , an adding unit 1006 and a first executing unit 1008 .

[0119] The first acquisition unit 1002 is used to acquire first observation information corresponding to the first obstacle, wherein the first observation information is obstacle information of the first obstacle sensed by the sensing device of the current vehicle or the sensing device of the remote vehicle at the first moment. The first acquisition unit can be used to execute step S202.

[0120] The first matching unit 1004 is used to match the first obstacle with a group of known obstacles based on the first observation information and the observation information of the group of known obstacles when there is a group of known obstacles, wherein the group of known obstacles is a collection of obstacles that have been sensed by the sensing device of the current vehicle and the sensing device of the remote vehicle. The first matching unit 1004 can be used to execute step S204.

[0121] The adding unit 1006 is used to add the first observation information to the target time series observation queue of the target obstacle in a time sequence when there is a target obstacle matching the first obstacle in a group of known obstacles, wherein the target time series observation queue records the observation information corresponding to the target obstacle in a time sequence. The adding unit 1006 can be used to execute step S206.

[0122] The first execution unit 1008 is used to remove at least one observation information from the target time series observation queue when there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold in the target time series observation queue, and input the at least one observation information into the target Kalman filter in sequence according to the time sequence to update the target Kalman filter, wherein the target Kalman filter is a Kalman filter corresponding to the target obstacle, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times. The first execution unit 1008 can be used to execute step S208.

[0123] Through the embodiments provided by the present application, first observation information corresponding to a first obstacle is obtained, wherein the first observation information is obstacle information of the first obstacle perceived by a perception device of the current vehicle or a perception device of a remote vehicle at the first moment; in the case where there is a group of known obstacles, based on the first observation information and the observation information of a group of known obstacles, the first obstacle is matched with a group of known obstacles, wherein a group of known obstacles is a collection of obstacles perceived by the perception device of the current vehicle and the perception device of the remote vehicle; in the case where there is a target obstacle matching the first obstacle in the group of known obstacles, the first observation information is added to a target time series observation queue of the target obstacle in time sequence, wherein the target time series observation queue records the observation information matching the target obstacle in time sequence. object; in the target time series observation queue, if there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold, the at least one observation information is removed from the target time series observation queue, and the at least one observation information is sequentially input into the target Kalman filter in time sequence to update the target Kalman filter, wherein the target Kalman filter is the Kalman filter corresponding to the target obstacle, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times, which can solve the technical problem of poor vehicle driving safety caused by the inability to perceive the target object in the blind spot of the field of view in the target perception method of the vehicle in the related art, thereby improving the safety of vehicle driving.

[0124] In an exemplary embodiment, the first matching unit includes: a copy module, which is used to copy the Kalman filter corresponding to each known obstacle in a group of known obstacles respectively, and obtain a copy of the Kalman filter corresponding to each known obstacle; an input module, which is used to input the observation information of the time series observation queue of each known obstacle, whose observation time is before the first time, into the copy of the Kalman filter corresponding to each known obstacle in sequence according to the time sequence, and obtain an updated copy of the Kalman filter corresponding to each known obstacle; a prediction module, which is used to use the updated copy of the Kalman filter corresponding to each known obstacle to predict the obstacle information of each known obstacle at the first time, and obtain the predicted obstacle information of each known obstacle; a matching module, which is used to match the first obstacle with each known obstacle respectively based on the first observation information and the predicted obstacle information of each known obstacle.

[0125] In an exemplary embodiment, the above-mentioned device also includes: a first determination unit, which is used to determine, after adding the first observation information to the target time sequence observation queue of the target obstacle in time sequence, that there is no observation information in the target time sequence observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold when the position where the first observation information is inserted is not the tail of the target time sequence observation queue; a second determination unit, which is used to determine, when the position where the first observation information is inserted is the tail of the target time sequence observation queue and the time difference between the observation time of the head observation information of the target time sequence observation queue and the first time is greater than or equal to the time difference threshold, that there is at least one observation information in the target time sequence observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold when the position where the first observation information is inserted is the tail of the target time sequence observation queue and the time difference between the observation time of the head observation information of the target time sequence observation queue and the first time is less than the time difference threshold, that there is no observation information in the target time sequence observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold.

[0126] In an exemplary embodiment, the above-mentioned device also includes: a copying unit, which is used to input at least one observation information into the target Kalman filter in sequence in time sequence to update the target Kalman filter, and then, when the latest obstacle state of the target obstacle is to be output, copy the updated target Kalman filter to obtain a copy of the target Kalman filter; a second execution unit, which is used to input the observation information in the target time series observation queue into the copy of the target Kalman filter in time sequence, and output the state information finally output by the copy of the target Kalman filter as the latest obstacle state of the target obstacle.

[0127] In an exemplary embodiment, the first acquisition unit includes: a receiving module, which is used to receive first observation information sent by a remote vehicle through a V2X device of the remote vehicle through a V2X device of the current vehicle, wherein the first observation information is obstacle information of a first obstacle perceived by a perception device of the remote vehicle at a first moment; or, a perception module, which is used to perform obstacle perception through a perception device of the current vehicle to obtain the first observation information, wherein the first observation information is obstacle information of the first obstacle perceived by the perception device of the current vehicle at the first moment.

[0128] In an exemplary embodiment, the first observation information includes the posture information of the first obstacle; the perception module includes: a perception submodule, which is used to perform obstacle perception through the perception device of the current vehicle to obtain an obstacle perception result corresponding to the first obstacle; a conversion submodule, which is used to convert the posture information of the first obstacle in the obstacle perception result to a specified coordinate system to obtain updated posture information of the first obstacle; a generation submodule, which is used to generate the first observation information according to the updated posture information of the first obstacle; the device also includes: a propagation unit, which is used to broadcast the first observation information through the V2X device of the current vehicle after performing obstacle perception through the perception device of the current vehicle to obtain the first observation information, so as to transmit the first observation information to other vehicles except the current vehicle.

[0129] In an exemplary embodiment, the above-mentioned device also includes: a second acquisition unit, used to acquire second observation information corresponding to the second obstacle, wherein the second observation information is obstacle information of the second obstacle perceived by the perception device of the current vehicle or the perception device of the remote vehicle at the second moment; a second matching unit, used to match the second obstacle with a group of known obstacles based on the second observation information and the observation information of a group of known obstacles when there is a group of known obstacles; and a third execution unit, used to create a time series observation queue for the second obstacle when there is no obstacle matching the second obstacle in the group of known obstacles, add the second observation information to the created time series observation queue, and create a Kalman filter for the second obstacle.

[0130] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0131] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein the steps of any of the above method embodiments are executed when the program is run.

[0132] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0133] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the steps in any of the above method embodiments through the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0134] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0135] According to another aspect of the embodiment of the present application, a computer program product is also provided, which includes a computer program / instruction, and the computer program / instruction contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit 1101, various functions provided by the embodiment of the present application are executed. The above-mentioned serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0136] Fig.11 The computer system structure block diagram of the electronic device used to implement the embodiment of the present application is schematically shown. Fig.11 As shown, the computer system 1100 includes a CPU (Central Processing Unit) 1101, which can perform various appropriate actions and processes according to the program stored in the ROM 1102 or the program loaded from the storage part 1108 to the RAM 1103. Various programs and data required for system operation are also stored in the random access memory 1103. The central processing unit 1101, the read-only memory 1102 and the random access memory 1103 are connected to each other through a bus 1104. An I / O (Input / Output) interface 1105 is also connected to the bus 1104.

[0137] The following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed so that a computer program read therefrom is installed into the storage section 1108 as needed.

[0138] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processor 1101, various functions defined in the system of the present application are executed.

[0139] It should be noted that Fig.11 The computer system 1100 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0140] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0141] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle target perception method, It is characterized in that include: Acquire first observation information corresponding to a first obstacle, wherein the first observation information is obstacle information of the first obstacle sensed by a sensing device of the current vehicle or a sensing device of a remote vehicle at a first moment; In the case where there is a group of known obstacles, matching the first obstacle with the group of known obstacles based on the first observation information and the observation information of the group of known obstacles, wherein the group of known obstacles is a collection of obstacles perceived by the perception device of the current vehicle and the perception device of the remote vehicle; In the case where there is a target obstacle matching the first obstacle in the group of known obstacles, adding the first observation information to a target time series observation queue of the target obstacle in time sequence, wherein the target time series observation queue records the observation information corresponding to the target obstacle in time sequence; In the target time series observation queue, if there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold, the at least one observation information is removed from the target time series observation queue, and the at least one observation information is sequentially input into the target Kalman filter in time sequence to update the target Kalman filter, wherein the target Kalman filter is the Kalman filter corresponding to the target obstacle, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times.

2. The method according to claim 1, It is characterized in that The matching the first obstacle with the set of known obstacles based on the first observation information and the observation information of the set of known obstacles includes: Respectively copying the Kalman filter corresponding to each known obstacle in the set of known obstacles to obtain a copy of the Kalman filter corresponding to each known obstacle; Inputting the observation information whose observation time is before the first time in the time series observation queue of each known obstacle into the copy of the Kalman filter corresponding to each known obstacle in sequence according to the time series, so as to obtain an updated copy of the Kalman filter corresponding to each known obstacle; Predicting obstacle information of each known obstacle at the first moment using the updated copy of the Kalman filter corresponding to each known obstacle to obtain predicted obstacle information of each known obstacle; Based on the first observation information and the predicted obstacle information of each known obstacle, the first obstacle is matched with each known obstacle respectively.

3. The method according to claim 1, It is characterized in that After adding the first observation information to the target time sequence observation queue of the target obstacle in time sequence, the method further includes: In a case where the position where the first observation information is inserted is not the tail of the target time series observation queue, determining that there is no observation information in the target time series observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold; In the case where the first observation information is inserted at the tail of the target timing observation queue, and the time difference between the observation time of the head observation information of the target timing observation queue and the first time is greater than or equal to the time difference threshold, it is determined that in the target timing observation queue, there is at least one observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold; When the position where the first observation information is inserted is the tail of the target timing observation queue and the time difference between the observation time of the head observation information of the target timing observation queue and the first time is less than the time difference threshold, it is determined that in the target timing observation queue, there is no observation information whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to the time difference threshold.

4. The method according to claim 1, It is characterized in that After inputting the at least one observation information into the target Kalman filter in sequence to update the target Kalman filter, the method further includes: When the latest obstacle state of the target obstacle is to be output, copying the updated target Kalman filter to obtain a copy of the target Kalman filter; The observation information in the target time series observation queue is sequentially input into the copy of the target Kalman filter in time sequence, and the state information finally output by the copy of the target Kalman filter is output as the latest obstacle state of the target obstacle.

5. The method according to claim 1, It is characterized in that The acquiring first observation information corresponding to the first obstacle includes: receiving, through the V2X device of the current vehicle, the first observation information sent by the remote vehicle through the V2X device of the remote vehicle, wherein the first observation information is obstacle information of the first obstacle sensed by the sensing device of the remote vehicle at the first moment; or, The first observation information is obtained by performing obstacle perception through the perception device of the current vehicle, wherein the first observation information is obstacle information of the first obstacle perceived by the perception device of the current vehicle at the first moment.

6. The method according to claim 5, It is characterized in that The first observation information includes position information of the first obstacle; The performing obstacle sensing by the sensing device of the current vehicle to obtain the first observation information includes: performing obstacle sensing by the sensing device of the current vehicle to obtain an obstacle sensing result corresponding to the first obstacle; converting the position and posture information of the first obstacle in the obstacle sensing result to a specified coordinate system to obtain updated position and posture information of the first obstacle; and generating the first observation information according to the updated position and posture information of the first obstacle; After obtaining the first observation information by performing obstacle perception through the perception device of the current vehicle, the method further includes: broadcasting the first observation information through the V2X device of the current vehicle to transmit the first observation information to other vehicles except the current vehicle.

7. The method according to any one of claims 1 to 6, It is characterized in that The method further comprises: Acquire second observation information corresponding to a second obstacle, wherein the second observation information is obstacle information of the second obstacle sensed by a sensing device of the current vehicle or a sensing device of the remote vehicle at a second moment; In the case where the set of known obstacles exists, matching the second obstacle with the set of known obstacles based on the second observation information and the observation information of the set of known obstacles; When there is no obstacle matching the second obstacle in the set of known obstacles, a time series observation queue is created for the second obstacle, the second observation information is added to the created time series observation queue, and a Kalman filter is created for the second obstacle.

8. A target sensing device for a vehicle, It is characterized in that include: A first acquisition unit, configured to acquire first observation information corresponding to a first obstacle, wherein the first observation information is obstacle information of the first obstacle sensed by a perception device of a current vehicle or a perception device of a remote vehicle at a first moment; a first matching unit, configured to match the first obstacle with a group of known obstacles based on the first observation information and observation information of the group of known obstacles when there is a group of known obstacles, wherein the group of known obstacles is a collection of obstacles sensed by the sensing device of the current vehicle and the sensing device of the remote vehicle; an adding unit, configured to add the first observation information to a target time series observation queue of the target obstacle in a time sequence when there is a target obstacle matching the first obstacle in the group of known obstacles, wherein the target time series observation queue records the observation information corresponding to the target obstacle in a time sequence; A first execution unit is configured to remove at least one observation information from the target time series observation queue, and input the at least one observation information into the target Kalman filter in sequence in a time sequence to update the target Kalman filter, when there is at least one observation information in the target time series observation queue whose time difference between the observation time and the observation time of the tail observation information is greater than or equal to a time difference threshold. The target Kalman filter is a Kalman filter corresponding to the target obstacle, and the state information in the filter parameters of the target Kalman filter is used to represent the obstacle state of the target obstacle at different times.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of any one of the methods of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images