Obstacle state estimation method and device, storage medium and electronic equipment

Abstract

The invention discloses an obstacle state estimation method and device, a storage medium and electronic equipment. The method comprises the steps of determining a detection information table of a to-be-detected obstacle according to multiple frames of driving environment images collected by a vehicle in real time; on the basis of the multiple pieces of detection information in the detection information table, determining a satisfaction relation between the multiple pieces of detection information and a fitting condition; in response to the satisfaction relation that the multiple pieces of detection information meet the fitting condition, fitting is carried out based on the multiple pieces of detection information to obtain the fitting speed of the to-be-detected obstacle; adjusting a filter used for estimating the obstacle state based on the fitting speed to obtain a target filter; and estimating the state of the to-be-detected obstacle based on the target filter. According to the method, the multiple pieces of detection information meeting the fitting conditions are screened from the detection information table for fitting, the fitting speed with small errors can be obtained, the convergence speed of the filter can be increased, then the motion state of the obstacle can be estimated rapidly and accurately, and it is ensured that the vehicle runs safely and stably.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to an obstacle state estimation method, device, storage medium, and electronic device. Background Art

[0002] With the continuous development of intelligent driving technology, traffic safety issues are receiving more and more attention. In intelligent driving scenarios, once obstacles such as other vehicles or pedestrians appear within the field of view of the on-board camera of an intelligent driving vehicle (hereinafter referred to as the vehicle), the vehicle's intelligent driving system needs to quickly and accurately detect the initial state of the obstacle. Based on the initial state of the obstacle, the state estimation filter is used to estimate the motion state of the obstacle, and decisions are made and control instructions are issued accordingly. In related technologies, the accuracy of the detected initial velocity of the obstacle is low due to factors such as ambient light and weather, which affects the convergence speed of the filter, resulting in the inability to quickly and accurately estimate the motion state of the obstacle, affecting the safe and stable driving of the vehicle. Summary of the Invention

[0003] Obstacle state estimation methods provided in related technologies generally select the first several frames of images after an obstacle is detected, determine the obstacle distance corresponding to each frame based on each image, perform a straight line fit based on the obstacle distance corresponding to each frame, and use the slope of the line as the obstacle's initial velocity. Based on this initial velocity, a filter is then used to estimate the obstacle's motion state. Because these methods directly select the first several frames, if the detected obstacle distance error is too large due to factors such as light and interference, the fitted initial velocity will have a large error, slowing filter convergence and preventing rapid and accurate estimation of the obstacle's motion state, affecting the safe and stable driving of the vehicle.

[0004] To solve the above technical problems, the present disclosure provides an obstacle state estimation method, device, storage medium and electronic device to improve the accuracy of obstacle fitting speed and accelerate filter convergence, thereby achieving the purpose of quickly and accurately estimating the obstacle motion state.

[0005] An embodiment of the first aspect of the present disclosure provides an obstacle state estimation method, including:

[0006] Determine the detection information table of the obstacles to be detected based on the multiple frames of driving environment images collected by the vehicle in real time;

[0007] Based on the plurality of detection information in the detection information table, determining a satisfying relationship between the plurality of detection information and a fitting condition;

[0008] In response to the satisfying relationship being that the plurality of detection information satisfies the fitting condition, obtaining a fitting speed of the obstacle to be detected based on the plurality of detection information;

[0009] Adjusting a filter for estimating an obstacle state based on the fitting speed to obtain a target filter;

[0010] The state of the obstacle to be detected is estimated based on the target filter.

[0011] An embodiment of a second aspect of the present disclosure provides an obstacle state estimation device, comprising:

[0012] A first determination module is used to determine a detection information table of obstacles to be detected based on multiple frames of driving environment images collected in real time by the vehicle;

[0013] A second determining module is configured to determine, based on the plurality of detection information in the detection information table, whether the plurality of detection information satisfies a fitting condition;

[0014] a fitting module, configured to, in response to the satisfying relationship being that the plurality of detection information satisfy the fitting condition, obtain a fitting speed of the obstacle to be detected based on the plurality of detection information;

[0015] an adjustment module, configured to adjust a filter for estimating an obstacle state based on the fitting speed to obtain a target filter;

[0016] An estimation module is used to estimate the state of the obstacle to be detected based on the target filter.

[0017] An embodiment of the third aspect of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the obstacle state estimation method proposed in the embodiment of the first aspect of the present disclosure.

[0018] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:

[0019] processor;

[0020] a memory for storing instructions executable by the processor;

[0021] The processor is used to read the instructions from the memory and execute the instructions to implement the obstacle state estimation method proposed in the embodiment of the first aspect of the present disclosure.

[0022] An embodiment of the fifth aspect of the present disclosure provides a computer program product. When an instruction processor in the computer program product is executed, the obstacle state estimation method provided in the embodiment of the first aspect of the present disclosure is executed.

[0023] The obstacle state estimation method provided by the disclosed embodiments first determines a detection information table for obstacles to be detected based on multiple frames of driving environment images collected in real time by a vehicle. Then, based on the multiple detection information in the detection information table, the relationship between the multiple detection information and fitting conditions is determined. When the multiple detection information meet the fitting conditions, a fitting speed of the obstacle to be detected is obtained based on the multiple detection information. Based on the fitting speed, a filter used to estimate the obstacle state is adjusted to obtain a target filter. Finally, the state of the obstacle to be detected is estimated based on the target filter. This method preliminarily verifies the fitting conditions for the detection information in the detection information table, selects the detection information corresponding to several frames of driving environment images that meet the fitting conditions, and performs fitting. This method can obtain a fitting speed with a small error, thereby accelerating the filter convergence speed and enabling rapid and accurate estimation of the obstacle's motion state, ensuring safe and stable vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic structural block diagram of an obstacle state estimation system provided by an exemplary embodiment of the present disclosure;

[0025] Figure 2 is a flowchart of an obstacle state estimation method provided by an exemplary embodiment of the present disclosure;

[0026] Figure 3 is a flowchart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure;

[0027] Figure 4 is a flowchart of an obstacle state estimation method provided by yet another exemplary embodiment of the present disclosure;

[0028] Figure 5 is a flowchart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure;

[0029] Figure 6 is a flowchart of an obstacle state estimation method provided by yet another exemplary embodiment of the present disclosure;

[0030] Figure 7 is a flowchart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure;

[0031] Figure 8 is a flowchart of an obstacle state estimation method provided by yet another exemplary embodiment of the present disclosure;

[0032] Figure 9 is a flowchart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure;

[0033] Figure 10is a flowchart of an obstacle state estimation method provided by yet another exemplary embodiment of the present disclosure;

[0034] Figure 11 is a flowchart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure;

[0035] Figure 12 is a flowchart of an obstacle state estimation method provided by yet another exemplary embodiment of the present disclosure;

[0036] Figure 13 is a flowchart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure;

[0037] Figure 14 is a flowchart of an obstacle state estimation method provided by yet another exemplary embodiment of the present disclosure;

[0038] Figure 15 1 is a schematic diagram of the structure of an obstacle state estimation device provided by an exemplary embodiment of the present disclosure;

[0039] Figure 16 is a schematic diagram of the structure of an obstacle state estimation device provided by another exemplary embodiment of the present disclosure;

[0040] Figure 17 It is a schematic diagram of the composition structure of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0042] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0043] Application Overview

[0044] In intelligent driving scenarios, the accuracy and efficiency of obstacle distance and speed perception are key to ensuring safe vehicle driving. Perceiving obstacle distance and speed based on computer vision technology is one of the commonly used detection methods in the prior art. It should be noted that in the embodiments of this disclosure, obstacle distance refers to the relative distance between the obstacle (such as the vehicle in front) and the vehicle (i.e., the vehicle itself); obstacle speed refers to the relative speed between the obstacle and the vehicle.

[0045] When a vehicle detects a new obstacle in its field of view while driving, it can use computer vision technology to perceive the relative distance between the vehicle and the obstacle using a single frame image, thereby determining the obstacle distance. The obstacle's position can be determined based on the vehicle's position and the obstacle distance. Because the obstacle's speed cannot be determined from a single position, existing techniques capture several frames of images, determine the obstacle's positions based on these frames, and then perform a linear fit based on these positions and the acquisition time to obtain a fitted line. The slope of the fitted line is then used to determine the obstacle's speed. A filter estimates the obstacle's motion state based on the obstacle's speed and position, and uses this information to make decisions and control the vehicle's obstacle avoidance.

[0046] In real-world applications, noise signals can appear in captured images due to various factors such as road conditions, lighting, and interference. If interference is present in one of the preceding frames, the obstacle distance error corresponding to that frame can be excessively large. This can lead to significant errors in the initial velocity fitted, slowing filter convergence and preventing fast and accurate estimation of the obstacle's motion, impacting the vehicle's safe and stable operation.

[0047] To address the technical problem in existing technologies where the low accuracy of the fitted initial obstacle velocity results in slow filter convergence and an inability to quickly and accurately estimate the obstacle state, the present invention provides an obstacle state estimation method. This method, based on existing visual technology for sensing the initial value of the obstacle velocity, determines a detection information table for the obstacle based on multiple frames of real-time imagery. After verifying that multiple pieces of detection information in the detection information table meet fitting conditions, the method determines the obstacle's fitted velocity based on the multiple pieces of detection information that meet the fitting conditions. The method then adjusts the filter based on the fitted velocity to obtain a converged target filter. The obstacle state is estimated based on this target filter, and vehicle control decisions are made based on the state estimation results. Preliminary verification of this method demonstrates that the method selects detection information from several frames of images that meet the fitting conditions for fitting, resulting in a small fitted velocity error, enabling rapid convergence of the state estimation filter. This method, in turn, enables rapid and accurate estimation of the obstacle's motion state, ensuring safe and stable vehicle travel.

[0048] Exemplary Systems

[0049] Figure 1 FIG. 1 is a schematic diagram of a structure of an obstacle state estimation system provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the obstacle state estimation system 100 includes a collection device 101 , a data management device 102 , a fitting device 103 , an adjustment device 104 and a state estimation filter 105 .

[0050] The acquisition device 101 is used to collect data from the vehicle during its travel, obtaining vehicle travel data. The acquisition device 101 may include sensors for collecting vehicle travel data (such as a speed sensor, position sensor, and acceleration sensor) and sensors for collecting data about the vehicle's external driving environment (such as an onboard camera, lidar, and millimeter-wave radar). Accordingly, vehicle travel data may include vehicle speed, acceleration, vehicle position, images of the driving environment, laser signals, and millimeter-wave radar signals.

[0051] The data management device 102 is used to process and store the vehicle driving data collected by the acquisition device 101. For example, based on the driving environment image captured by the vehicle-mounted camera included in the acquisition device 101, it is determined whether a new obstacle has appeared in the vehicle's field of view. The obstacle can appear in the vehicle's field of view from far to near, from behind an obstruction, or from a distorted area at the edge of the field of view. The obstacle can be a moving object such as a moving vehicle or pedestrian, or a stationary object such as a parked vehicle, water barriers, or roadblocks.

[0052] When a new obstacle appears in the field of view of the vehicle according to the driving environment image, it is necessary to quickly and accurately predict the motion state of the obstacle (such as obstacle distance, speed and other state quantities) based on the detected information in order to plan the trajectory of the vehicle. The obstacle is the obstacle to be detected. The data management device 102 determines the detection information of the obstacle to be detected based on each frame of the driving environment image collected in real time by the on-board camera, and stores it in the detection information table. Among them, one detection information in the detection information table is determined based on one frame of the driving environment image. One detection information includes a position of the obstacle to be detected. The data management device 102 is also used to determine whether multiple detection information in the detection information table meets the fitting condition. When multiple detection information in the detection information table does not meet the fitting condition, it means that the detected detection information is too little and is not enough to fit a more accurate fitting speed. At this time, the driving environment image continues to be collected.

[0053] When the data management device 102 determines that multiple detection information in the detection information table meet the fitting conditions, it means that the currently detected detection information can fit a relatively accurate fitting speed. At this time, the fitting device 103 fits the multiple detection information to obtain the fitting speed of the obstacle to be detected.

[0054] In some embodiments, the fitting device 103 may obtain the fitting speed of the obstacle based on a Random Sample Consensus (RANSAC) algorithm. The RANSAC algorithm is an algorithm that calculates mathematical model parameters of data based on a set of sample data sets containing abnormal data to obtain valid sample data.

[0055] In some embodiments, the fitting device 103 can improve the RANSAC algorithm by removing abnormal data from the sample data set through data normalization, combined with a quality assessment function of the number of fitting frames and the fitting error, thereby obtaining an improved RANSAC algorithm. By fitting with the improved RANSAC algorithm, the fitting result is made closer to the actual value, thereby achieving the effect of improving the fitting accuracy. In practical applications, other fitting algorithms can also be used for fitting, which is not limited in the embodiments of the present disclosure.

[0056] Adjustment device 104 adjusts the unconverged state estimation filter 105 based on the fitting speed of the obstacle to be detected, and monitors the filter's convergence direction to ensure the correct adjustment direction. This allows the adjusted speed of filter 105 to increasingly approach the actual value, accelerating filter convergence. Once the filter meets the convergence conditions, the filter meeting the convergence conditions is used as the target filter for state estimation.

[0057] The state estimation filter 105 estimates the state of the obstacle to be detected based on the converged target filter, and completes the prediction of state quantities such as obstacle distance and speed.

[0058] In the disclosed embodiments, by initially verifying the fitting conditions for the detection information in the detection information table, and then selecting the detection information corresponding to several frames of driving environment images that meet the fitting conditions for fitting, a fitting speed with minimal error can be achieved. Adjusting the filter based on the minimal error fitting speed can accelerate filter convergence and produce the target filter, thereby enabling rapid and accurate estimation of the obstacle's motion state, ensuring safe and stable vehicle driving.

[0059] Exemplary Methods

[0060] Figure 2 The method provided by the embodiment of the present disclosure can be applied to vehicles, such as Figure 2 As shown, the method includes the following steps:

[0061] Step S201 : determining a detection information table of obstacles to be detected based on multiple frames of driving environment images collected by the vehicle in real time.

[0062] The disclosed embodiments can be performed by a vehicle (i.e., the vehicle itself) capable of detecting obstacle speed, and specifically by an obstacle state estimation device within the vehicle. Obstacles can include moving objects such as other vehicles or pedestrians, as well as stationary objects such as parked vehicles, roadblocks, and water barriers.

[0063] During vehicle travel, various sensors on the vehicle detect both the vehicle's own driving data and environmental data of the surrounding driving environment in real time. This environmental data includes multiple frames of driving environment images captured in real time by an image acquisition device installed on the vehicle (specifically, an onboard camera). The obstacle state estimation device identifies and processes each frame of the captured driving environment image to determine an obstacle to be detected and multiple pieces of detection information for the obstacle to be detected. This detection information is then stored in a data table, resulting in a detection information table for the obstacle to be detected.

[0064] Step S202: determining, based on the plurality of detection information in the detection information table, whether the plurality of detection information satisfies the fitting condition.

[0065] In actual applications, each time a frame of driving environment image is collected, it is subjected to recognition processing to obtain the detection information corresponding to the frame of driving environment image and store it in the detection information table. The obstacle state estimation device determines whether the multiple detection information meets the fitting conditions based on the multiple detection information stored in the detection information table. When the multiple detection information stored in the detection information table meets the fitting conditions, it means that the currently detected detection information can fit a relatively accurate fitting speed, and the process goes to step S203. When the multiple detection information stored in the detection information table does not meet the fitting conditions, it means that the currently detected detection information is not sufficient to fit a relatively accurate fitting speed. At this time, it is determined that the satisfaction relationship is that the multiple detection information does not meet the fitting conditions, and the driving environment image is continued to be collected for recognition, and new detection information is determined to update the detection information table.

[0066] Step S203 : In response to the satisfying relationship that the plurality of detection information satisfy the fitting condition, a fitting speed of the obstacle to be detected is obtained based on the plurality of detection information.

[0067] When the fitting conditions are determined to be met, a straight line fit is performed based on the multiple detection information that meets the fitting conditions to obtain the fitted speed of the obstacle to be detected. In some embodiments, the obstacle state estimation device can obtain the fitted speed of the obstacle based on a RANSAC algorithm. In practical applications, other fitting algorithms may also be used for fitting, which is not limited in the presently disclosed embodiments.

[0068] In the embodiment of the present disclosure, by preliminarily verifying the fitting conditions of the detection information in the detection information table, and screening the detection information corresponding to several frames of driving environment images that meet the fitting conditions for fitting, a fitting speed with smaller error can be obtained.

[0069] Step S204 : adjusting the filter used to estimate the obstacle state based on the fitting speed to obtain a target filter.

[0070] After obtaining the fitted speed of the obstacle to be detected, the obstacle state estimation device adjusts the filter speed based on the fitted speed, so that the filter speed increasingly approaches the actual value. When the deviation between the fitted speed and the filter speed is less than or equal to a preset difference threshold, the filter is determined to have reached convergence, and the filter that has reached convergence is determined as the target filter. In the disclosed embodiment, adjusting the filter based on the fitted speed with a smaller error can accelerate the filter convergence speed.

[0071] Step S205 : estimating the state of the obstacle to be detected based on the target filter.

[0072] The onboard camera continues to capture real-time images of the driving environment, estimating the state of obstacles based on these images and the target filter, thereby predicting the obstacle's motion. By achieving a fast fitting speed with minimal error and accelerating filter convergence, the system can quickly and accurately estimate the obstacle's motion, ensuring safe and stable vehicle operation.

[0073] The obstacle state estimation method provided by the disclosed embodiments first determines a detection information table for obstacles to be detected based on multiple frames of driving environment images collected in real time by a vehicle. Then, based on the multiple detection information in the detection information table, the relationship between the multiple detection information and fitting conditions is determined. When the multiple detection information meet the fitting conditions, a fitting speed of the obstacle to be detected is obtained based on the multiple detection information. Based on the fitting speed, a filter used to estimate the obstacle state is adjusted to obtain a target filter. Finally, the state of the obstacle to be detected is estimated based on the target filter. This method preliminarily verifies the fitting conditions for the detection information in the detection information table, selects the detection information corresponding to several frames of driving environment images that meet the fitting conditions, and performs fitting. This method can obtain a fitting speed with a small error, thereby accelerating the filter convergence speed and enabling rapid and accurate estimation of the obstacle's motion state, ensuring safe and stable vehicle driving.

[0074] In some embodiments, based on the above embodiment, step S201 in the above embodiment, "determining the detection information table of the obstacle to be detected based on the multiple frames of driving environment images collected by the vehicle in real time", can be achieved by Figure 3 Steps S2011 to S2014 shown are implemented as follows:

[0075] Step S2011 : determining, based on multiple frames of driving environment images collected in real time by the vehicle, multiple frames of images to be detected that include obstacles to be detected.

[0076] During vehicle travel, the system collects images of the driving environment in real time. If the current frame contains an obstacle to be detected, that frame is used as the image to be detected. If the current frame does not contain an obstacle to be detected, that frame is discarded and the next frame is captured. This process continues until multiple frames of images containing obstacles to be detected are obtained.

[0077] In some embodiments, multiple frames of images to be detected can be Figure 4 Steps S20111 to S20114 shown are implemented as follows:

[0078] Step S20111: Determine a first driving environment image acquired by the vehicle at a first moment and a previous driving environment image acquired at a moment before the first moment.

[0079] The obstacle state estimation device obtains two frames of driving environment images collected continuously.

[0080] Step S20112: In response to detecting a new obstacle in the first driving environment image that does not exist in the previous driving environment image, determining an obstacle to be detected based on the new obstacle.

[0081] Identify the two consecutive frames of driving environment images and determine whether a new obstacle is detected in the image captured at the first moment. If a new obstacle is detected in the first driving environment image that was not present in the image captured at the previous moment, the new obstacle is identified as the obstacle to be detected. The first moment is considered the zeroth moment when the obstacle appeared in the vehicle's field of view, and the first driving environment image is the zeroth frame in which the obstacle was detected. Continue with step S20113.

[0082] If no new obstacle is detected in the first driving environment image, it means that no new obstacle appears in the field of view of the vehicle at the current moment, and the process returns to step S20111 to continue determining whether a new obstacle appears in the field of view of the vehicle at the next moment.

[0083] Step S20113: Determine a first number of frames of driving environment images including obstacles to be detected that are continuously collected by the vehicle after the first moment.

[0084] Step S20114: Determine a plurality of frames of images to be detected based on the first driving environment image and the first number of frames of driving environment images.

[0085] The vehicle-mounted camera captures the next frame of the driving environment image and determines whether the next frame contains the obstacle to be detected. If the obstacle to be detected is detected in the next frame, the next frame is used as the next frame of the driving environment image. If the obstacle to be detected is not detected in the next frame of the driving environment image, it indicates that the next frame of the driving environment image does not contain the obstacle to be detected, and the next frame of the driving environment image is captured. This step is repeated to obtain multiple frames of images to be detected.

[0086] The first number can be recorded as n, and the number of frames of the image to be detected is (n+1), where n is a positive integer.

[0087] Step S2012: Determine the position information and time stamp information of the obstacle to be detected corresponding to each image to be detected based on each image to be detected.

[0088] Perform recognition processing on each image to be detected to determine the pixel position of the image of the obstacle to be detected in each image to be detected. According to the camera parameters of the vehicle-mounted camera, the pixel position of the obstacle to be detected is converted into a coordinate system to obtain the position coordinates of the obstacle in the vehicle coordinate system, which is recorded as P c (x c ,y c ), where x c is the longitudinal coordinate of the obstacle in the vehicle coordinate system, y c is the horizontal coordinate of the obstacle in the vehicle coordinate system. Combined with the position of the vehicle, the position coordinate of the obstacle in the vehicle coordinate system is P c (x c ,y c ) is converted to the world coordinate system to obtain the position information of the obstacle, which is recorded as P w (x w ,y w ), where x w is the vertical coordinate of the obstacle in the world coordinate system, y w It is the horizontal coordinate of the obstacle in the world coordinate system.

[0089] In the disclosed embodiments, the vehicle coordinate system is established with the midpoint of the vehicle's rear axle as the origin, the vehicle's forward direction as the positive x-axis, and the left side of the vehicle as the positive y-axis. For example, the timestamp information for the detected obstacle can be the acquisition time of the corresponding image to be detected, denoted as t.

[0090] Step S2013 : determining detection information corresponding to each image to be detected based on the position information and timestamp information corresponding to each image to be detected.

[0091] The number of frames of the image to be detected is n+1, and the position information corresponding to the i-th frame of the image to be detected can be recorded as The timestamp information can be recorded as ti , then the detection information corresponding to the i-th frame of the image to be detected can be recorded as Where i = 0, 1, 2,…, n.

[0092] Step S2014: determining a detection information table of obstacles to be detected based on the detection information corresponding to each image to be detected.

[0093] The detection information of each obstacle to be detected is stored in the form of a data table to obtain a detection information table of the obstacle to be detected. In some embodiments, the detection information corresponding to the i-th frame of the image to be detected may also include the position coordinates P of the obstacle to be detected in the vehicle coordinate system. c (x c ,y c ), position coordinates in the image coordinate system, type (whether the obstacle is a vehicle or a pedestrian, etc.) and other information.

[0094] In the disclosed embodiment, multiple frames of images to be detected, including obstacles to be detected, are screened from each frame of driving environment image acquired in real time. Detection information corresponding to each frame of the images to be detected is determined, thereby obtaining a detection information table of the obstacles to be detected, providing data support for the subsequent determination of the fitting speed of the obstacles.

[0095] In some embodiments, based on the above embodiment, step S202 in the above embodiment, "determining the relationship between the plurality of detection information and the fitting condition based on the plurality of detection information in the detection information table", can be achieved by Figure 5 Steps S2021 to S2023 shown are implemented as follows:

[0096] Step S2021: Determine the linear relationship between the position and time of the obstacle to be detected based on the multiple detection information in the detection information table.

[0097] The state of stationary obstacles should remain constant, while the state of non-stationary obstacles should change continuously. The obstacles in multiple frames of detected images change continuously in time and space. Therefore, in the absence of interference, the position of the detected obstacle, determined based on the detection information corresponding to the captured driving environment images, should change linearly with time. Based on this, the disclosed embodiments can verify whether there are any detection information with significant errors by using the linear relationship between the position of the detected obstacle and time.

[0098] For example, (t,y w ) and the Pearson correlation coefficient p1 of (t,x w) and the Pearson correlation coefficient p2 of the detected obstacle is used to determine the size of p1 and p2 relative to the linearity check threshold. If both p1 and p2 are greater than the linearity check threshold, the linearity check passes, and the linear relationship is determined to satisfy the linearity condition for the position and time of the obstacle to be detected. Otherwise, the linearity check fails, and the linear relationship is determined to not satisfy the linearity condition for the position and time of the obstacle to be detected.

[0099] Step S2022: Determine the consistency relationship of the movement of the obstacle to be detected in multiple time periods based on the multiple detection information in the detection information table.

[0100] Over a short period of time, the movement of an obstacle can be considered uniform, so the speed of the obstacle should be consistent over different time periods. Based on this, the disclosed embodiments can verify whether there is significant error in the detection information by comparing the consistency of the obstacle's movement over different time periods.

[0101] For example, a small amount of detection information from different time periods can be used to fit the interval speed within the time period, and then the interval speeds of different time periods can be compared. If the deviation of the interval speeds of different time periods is small, a consistency check is performed, and the consistency relationship is determined to be consistent with the movement of the obstacle to be detected in multiple time periods. Otherwise, the consistency check fails, and the consistency relationship is determined to be inconsistent with the movement of the obstacle to be detected in multiple time periods.

[0102] In step S2023 , in response to the linear relationship being that the position and time of the obstacle to be detected satisfy the linearity condition, and the consistency relationship being that the obstacle to be detected moves consistently in multiple time periods, the satisfying relationship is determined as the multiple detection information satisfying the fitting condition.

[0103] When the linear relationship is that the position and time of the obstacle to be detected meet the linearity condition and the consistency relationship is that the obstacle to be detected moves consistently in multiple time periods, that is, the multiple detection information in the detection information table passes the linearity check and the consistency check, it is determined that the multiple detection information meet the fitting condition.

[0104] In other embodiments, step S2023 can be replaced by step S2023a "In response to the linear relationship that the position and time of the obstacle to be detected satisfy the linearity condition, determining the satisfied relationship as multiple detection information satisfying the fitting condition"; in still other embodiments, step S2023 can also be replaced by step S2023b "In response to the consistency relationship that the obstacle to be detected moves consistently in multiple time periods, determining the satisfied relationship as multiple detection information satisfying the fitting condition".

[0105] When judging whether the multiple detection information in the detection information table meets the fitting conditions, it is possible to perform only linearity verification on the multiple detection information in the detection information table, or only consistency verification, or both linearity verification and consistency verification, or other verification methods. When performing the verification, the more verification methods are performed, the more interference detection information can be eliminated, and the closer the final fitting speed is to the true value, but the time taken to obtain the fitting speed will also increase; the fewer verification methods are performed, the faster the verification is passed, and the faster the fitting speed is obtained accordingly, but due to the small number of verification methods, the interference detection information may not be eliminated, affecting the accuracy of the fitting speed. In actual applications, the appropriate verification method can be selected for verification based on actual conditions, and this is not limited in the embodiments of the present disclosure.

[0106] In some embodiments, based on the above embodiment, step S2021 in the above embodiment, "determining the linear relationship between the position and time of the obstacle to be detected according to the multiple detection information in the detection information table", can be achieved by Figure 6 Steps S20211 to S20214 shown are implemented as follows:

[0107] Step S20211: Determine the horizontal coordinate and vertical coordinate corresponding to each piece of detection information according to the position information included in each piece of detection information in the detection information table.

[0108] The position information corresponding to the i-th frame of the image to be detected is Then the horizontal coordinate corresponding to the i-th frame of the image to be detected is The vertical coordinate is Wherein, i=0,1,2,…,n, and n is a positive integer.

[0109] Step S20212: Determine a first linear relationship between the horizontal coordinate of the obstacle to be detected and time based on the timestamp information of each detection information in the detection information table and the horizontal coordinate corresponding to each detection information.

[0110] The timestamp information corresponding to the i-th frame of the image to be detected is t i , the horizontal coordinate is Determine t i and The linear correlation between t and t is obtained to obtain the first linear relationship. i and Calculate (t,y w ), and the Pearson correlation coefficient p1 is determined as the first linear relationship between the lateral coordinate of the obstacle to be detected and time.

[0111] Step S20213: Determine a second linear relationship between the longitudinal coordinate of the obstacle to be detected and time based on the timestamp information of each detection information in the detection information table and the longitudinal coordinate corresponding to each detection information.

[0112] The timestamp information corresponding to the i-th frame of the image to be detected is t i , the vertical coordinate is Determine t i and The linear correlation between t and t is used to obtain the second linear relationship. i and Calculate (t,x w ), and the Pearson correlation coefficient p2 is determined as the second linear relationship between the longitudinal coordinate of the obstacle to be detected and time.

[0113] Step S20214: Determine the linear relationship between the position of the obstacle to be detected and time based on the first linear relationship and the second linear relationship.

[0114] For example, when the first linear relationship is (t,y w ), and the second linear relationship is the Pearson correlation coefficient p1. The magnitudes of p1 and p2 relative to the linearity verification threshold can be determined. If both p1 and p2 are greater than the linearity verification threshold, the linearity verification passes, and the linear relationship is determined to satisfy the linearity condition for the position and time of the obstacle to be detected. If p1 or p2 is less than or equal to the linearity verification threshold, the linearity verification fails, and the linear relationship is determined to fail to satisfy the linearity condition for the position and time of the obstacle to be detected.

[0115] In the embodiment of the present disclosure, the linear relationship between the position of the obstacle to be detected and the time can be used to determine whether there is detection information with a large error in the detection information table. Speed ​​fitting is performed only when the linear relationship between the position of the obstacle to be detected and the time satisfies the linearity condition, which can improve the accuracy of the fitting speed.

[0116] In some embodiments, based on the above embodiment, step S2022 in the above embodiment, "determining the consistency relationship of the movement of the obstacle to be detected in multiple time periods according to the multiple detection information in the detection information table", can be achieved by Figure 7 Steps S20221 to S20224 shown are implemented as follows:

[0117] Step S20221: Determine, based on the plurality of detection information, a plurality of first detection information corresponding to the first time period and a plurality of second detection information corresponding to the second time period.

[0118] To reduce the error in fitting speed caused by inconsistent obstacle motion during different time periods, you can use multiple detection information from different time periods to fit the interval speed. The consistency of the obstacle's performance can be determined based on the fitted interval speed for each time period. The number of intervals for consistency comparison can be two or more. The duration of each time period can be the same or different. The detection information corresponding to different time periods can overlap or not.

[0119] In one implementation, two intervals may be selected for comparison: a first interval corresponding to the first time period and a second interval corresponding to the second time period. Multiple detection information corresponding to the first time period and multiple detection information corresponding to the second time period are determined from the plurality of detection information. For ease of distinction, the detection information corresponding to the first time period is referred to as first detection information, and the detection information corresponding to the second time period is referred to as second detection information.

[0120] Step S20222: Determine a first interval fitting speed of the obstacle to be detected in the first time period according to the plurality of first detection information.

[0121] Since the displacement mainly changes in the longitudinal direction during vehicle driving, the fitting speed in the lateral direction can be ignored when comparing the consistency of motion performance, and only the fitting speed in the longitudinal direction can be calculated. In one implementation, the first interval fitting speed can be determined according to the following steps: according to the position information included in each first detection information, the longitudinal coordinate corresponding to each first detection information is determined; according to the longitudinal coordinate and timestamp information corresponding to each first detection information, a straight line is fitted to obtain the first interval fitting speed of the obstacle to be detected in the first time period. The first interval fitting speed (i.e., the first interval longitudinal speed) of the obstacle to be detected in the first time period T1 can be recorded as v T1 .

[0122] Step S20223: Determine a second interval fitting speed of the obstacle to be detected in the second time period according to the plurality of second detection information.

[0123] The method for determining the fitting speed of the first interval is the same as that for determining the fitting speed of the second interval. The fitting speed of the second interval can be determined according to the following steps: according to the position information included in each second detection information, the longitudinal coordinate corresponding to each second detection information is determined; according to the longitudinal coordinate and time stamp information corresponding to each second detection information, a straight line is fitted to obtain the fitting speed of the obstacle in the second time period. The fitting speed of the obstacle in the second time period T2 (i.e., the longitudinal speed in the second interval) can be expressed as

[0124] Step S20224: Determine the consistency relationship of the movement of the obstacle to be detected in multiple time periods based on the first interval fitting speed and the second interval fitting speed.

[0125] Compare the fitting speed of the first interval and the second interval fitting speed Determine whether the fitting speeds of the two intervals are equal, or determine whether the fitting speeds of the two intervals are similar (i.e., determine whether the difference in the fitting speeds of the two intervals is less than a preset threshold). If the fitting speeds of the two intervals are equal (or similar), it means that the fitting speeds of the two intervals are consistent, and the consistency relationship is determined as the consistent movement of the obstacle to be detected in multiple time periods; if the fitting speeds of the two intervals are not equal (or not similar), it means that the fitting speeds of the two intervals are inconsistent, indicating that there is significantly abnormal detection information, and the consistency relationship is determined as the inconsistent movement of the obstacle to be detected in multiple time periods.

[0126] In the embodiment of the present disclosure, the consistency relationship of the movement of the obstacle to be detected in multiple time periods can be used to determine whether there is detection information with large errors in the detection information table. Speed ​​fitting is performed only when the consistency relationship is determined to be consistent movement of the obstacle to be detected in multiple time periods, which can improve the accuracy of the fitting speed.

[0127] In some embodiments, based on the above embodiment, step S203 of the above embodiment, "obtaining the fitting speed of the obstacle to be detected based on the fitting of multiple detection information", can be performed by Figure 8 Steps S2031 to S2033 shown are implemented as follows:

[0128] Step S2031 , performing straight line fitting based on the position information and timestamp information included in each detection information to obtain a fitting result.

[0129] Detection information corresponding to the i-th frame of the image to be detected According to the position information included in the i-th detection information and timestamp information t i Perform straight line fitting to obtain a fitted straight line, and determine the slope of the fitted straight line as the fitting result. When performing fitting, fitting can be performed separately for the horizontal coordinate and the vertical coordinate, or displacement can be determined based on position information and fitting can be performed for the displacement.

[0130] It should be noted that in the embodiments of this disclosure, the detection information used for fitting is only verified detection information that meets the fitting conditions, and does not include abnormal detection information with large errors. Therefore, compared with the results obtained by directly fitting the detection information corresponding to the previous frames of images in the prior art, the fitting results obtained in this disclosure based on the detection information that meets the fitting conditions are closer to the true value.

[0131] Step S2032: Verify the fitting result to obtain a verification result.

[0132] In the embodiment of the present disclosure, after obtaining the fitting result, the fitting result is verified to determine whether it is accurate to ensure the accuracy of the fitting speed. One implementation method of verifying the fitting result can be referred to below. Figure 10 Details are described in the illustrated embodiments.

[0133] Step S2033: In response to the verification result being a passed verification, a fitting speed of the obstacle to be detected is determined according to the fitting result.

[0134] When the verification result obtained by verifying the fitting result is verification passed, it indicates that the fitting result is accurate and stable, and the obstacle state estimation device determines the fitting speed based on the fitting result.

[0135] In the disclosed embodiment, fitting is performed based on detection information that meets the fitting conditions, and the fitting results are verified, thereby ensuring the accuracy of the fitting speed.

[0136] In one implementation, when fitting is performed based on the detection information, fitting can be performed directly based on the original data. Based on the above embodiment, step S2031 in the above embodiment, "performing linear fitting based on the position information and timestamp information included in each detection information to obtain a fitting result", can be performed by Figure 9 Steps S20311a to S20314a shown are implemented as follows:

[0137] Step S20311a: Determine the data group to be fitted corresponding to each piece of detection information according to the position information and timestamp information included in each piece of detection information.

[0138] For example, when fitting, the original data of position information and timestamp information can be directly used for fitting. The data set to be fitted is recorded as (t, x w ,y w ), the data set to be fitted corresponding to the i-th detection information can be expressed as i=0,1,2,…,n.

[0139] Step S20312a: construct an initial motion model of the obstacle to be detected.

[0140] The initial motion model is a linear model. Based on the assumption that the obstacle moves linearly in the longitudinal and lateral directions, the initial motion model of the obstacle position time is established as follows (1):

[0141]

[0142] Where a x 、b x 、a y and b y are the model parameters that need to be determined through fitting.

[0143] Step S20313a: Optimize the initial motion model according to the to-be-fitted data set corresponding to each detection information to obtain a target motion model that meets the iteration stopping condition.

[0144] For example, the data set to be fitted (t, x w ,y w ) selects some data groups as the initial data groups, optimizes the initial motion model based on the initial data groups to obtain the intermediate motion model, and then iteratively optimizes and adjusts the intermediate motion model based on the other data groups in the data groups to be fitted except the initial data group, until the distance between each data group to be fitted and the intermediate motion model is minimized, or the number of iterations reaches a preset number threshold, the intermediate motion model is considered to be optimal. The optimal intermediate motion model is determined as the target motion model. The iterative stopping conditions may include but are not limited to the error being less than a preset error threshold and the number of iterations reaching a preset number threshold.

[0145] Step S20314a: Determine the fitting result based on the parameters of the target motion model.

[0146] For example, the target motion model obtained through iterative optimization is shown in the following formula (2):

[0147]

[0148] In the above formula, and are the parameters of the target motion model determined by fitting. The fitting result in the longitudinal direction Fitting results in the transverse direction The fitting result

[0149] In the embodiment of the present disclosure, by establishing and optimizing a mathematical model to obtain a fitting result, the accuracy of the fitting result can be improved.

[0150] In order to prevent data of different units and orders of magnitude from affecting the accuracy of the fitting results, in another implementation, when fitting based on the detection information, the raw data of the detection information can be normalized first. Specifically, based on the above embodiment, step S2031 in the above embodiment, "performing a straight line fitting based on the position information and timestamp information included in each detection information to obtain a fitting result", can be implemented by the following steps S20311b to S20319b:

[0151] Step S20311b: Determine the original data group corresponding to each detection information based on the position information and timestamp information in each detection information.

[0152] For example, the original data group corresponding to each detection information can be the data group to be fitted corresponding to each detection information in the first implementation manner above, that is, the original data group corresponding to the i-th detection information can be expressed as i=0,1,2,…,n.

[0153] Step S20312b: normalize the original data group corresponding to each detection information to obtain the data group to be fitted corresponding to each detection information.

[0154] For the t and x in the original data set corresponding to (n+1) detection information w and y w Normalize them separately and scale them to the [0,1] interval to get the normalized timestamps Normalized vertical coordinate Normalized horizontal coordinate The data set to be fitted is denoted as Among them, the data set to be fitted corresponding to the i-th detection information can be expressed as i=0,1,2,…,n.

[0155] Step S20313b: construct an initial motion model of the obstacle to be detected.

[0156] The initial motion model is a linear model. Based on the assumption that the obstacle moves linearly in the longitudinal and lateral directions, the initial motion model of the obstacle position time is established as follows (3):

[0157]

[0158] a x 、b x 、a y and b y are the model parameters that need to be determined through fitting.

[0159] Step S20314b: Optimize the initial motion model according to the data group to be fitted corresponding to each detection information to obtain a target motion model that meets the iteration stop condition.

[0160] For example, the data set to be fitted can be The initial motion model is optimized based on the selected data sets to obtain an intermediate motion model. The intermediate motion model is then iteratively optimized and adjusted based on the remaining data sets in the to-be-fitted data set, excluding the initial data set, until the distance between each to-be-fitted data set and the intermediate motion model is minimized, or the number of iterations reaches a preset threshold. The intermediate motion model is then considered optimal. The optimal intermediate motion model is then determined as the target motion model.

[0161] Step S20315b: Determine the horizontal scaling ratio, the vertical scaling ratio, and the time scaling ratio according to the original data group and the data group to be fitted corresponding to each detection information.

[0162] The original data before normalization are t, x w and y w , scaled to the [0,1] interval, the corresponding normalized data is and Calculate the original data t and x corresponding to each detection information respectively w and y w With the normalized data and The quotient of time scaling (denoted as scale t ), vertical scaling (denoted as scale x ) and horizontal scaling (denoted as scale y ).

[0163] Step S20316b: Determine the horizontal scaling component and the vertical scaling component according to the parameters of the target motion model.

[0164] For example, the target motion model obtained through iterative optimization is shown in the following formula (4):

[0165]

[0166] and is the parameter of the target motion model determined by fitting, then the vertical scaling component Horizontal scaling component

[0167] Step S20317b: Determine the lateral fitting result of the obstacle to be detected according to the lateral scaling component, the lateral scaling ratio, and the time scaling ratio.

[0168] According to the horizontal scaling scale y and time scaling t , the horizontal scaling component Perform inverse normalization to obtain the lateral fitting result of the obstacle to be detected

[0169] Step S20318b: Determine the longitudinal fitting result of the obstacle to be detected according to the longitudinal scaling component, the longitudinal scaling ratio, and the time scaling ratio.

[0170] Same as above, according to the vertical scaling scale x and time scaling t , the vertical scaling component Perform inverse normalization processing to obtain the longitudinal fitting result of the obstacle to be detected

[0171] Step S20319b: Determine the fitting result based on the horizontal fitting result and the vertical fitting result.

[0172] Determine the lateral fitting result v y And the longitudinal fitting result v x Afterwards, according to The fitting results are calculated.

[0173] In the embodiment of the present disclosure, normalization processing is performed and calculations are performed at an appropriate scale, which is conducive to more accurate comparison and analysis. Then, inverse normalization processing is performed to obtain the fitting result, which can improve the accuracy of the fitting result.

[0174] In some embodiments, the steps S20313a and S20314b in the above embodiment of "optimizing the initial motion model according to the data group to be fitted corresponding to each detection information to obtain a target motion model that meets the iteration stop condition" can be optimized according to the following steps S3131 to S3134:

[0175] Step S3131 : Filter a second number of data groups to be fitted from the data groups to be fitted corresponding to each detection information to obtain an initial inlier set.

[0176] For example, when performing model optimization, a second number (denoted as m) of data sets to be fitted are first selected from the (n+1) data sets to be fitted corresponding to the (n+1) detection information as initial inliers, and the m initial inliers constitute an initial inlier set. Here, m is a positive integer not greater than (n+1).

[0177] Step S3132: Optimize the initial motion model according to the initial interior point set to obtain a local optimal motion model.

[0178] The local optimal motion model is a local optimal model obtained by optimizing the initial motion model based on the data set to be fitted, that is, the optimal motion model at the current moment during the iterative optimization process.

[0179] Step S3133: In response to the local optimal motion model not satisfying the iteration stop condition, a third number of data groups to be fitted are re-screened from the data groups to be fitted corresponding to each detection information to obtain a new initial inlier set, and the local optimal motion model is continuously optimized according to the new initial inlier set until a local optimal motion model that satisfies the iteration stop condition is obtained.

[0180] In some embodiments, whether the local optimal motion model satisfies an iteration stop condition can be determined based on the number of iterations of the local optimal motion model. The iteration stop condition can include the number of iterations to stop. The number of iterations to stop can be determined based on the size of the data set to be fitted or based on a preset value.

[0181] For example, the number of iteration stops is denoted as q max ,q max It can be calculated according to the following formula (5):

[0182]

[0183] Where η0 is the probability parameter of the inlier distribution, ε represents the ratio of the number of inliers to the total number of points, and m represents the number of inliers. Q is a preset value, such as a positive integer such as 5, 10, or 20.

[0184] When the number of iterations of the local optimal motion model (denoted as q) reaches the number of iterations q max When , it is determined that the local optimal motion model meets the iteration stop condition, and the iteration is stopped at this time, and the process goes to step S3134. When the number of iteration rounds of the local optimal motion model does not reach the iteration stop number, the number of iteration rounds of the local optimal motion model is increased by 1, and a third number of data groups to be fitted are re-screened from the data groups to be fitted corresponding to each detection information, and the local optimal motion model is continuously optimized as a new initial interior point set. The above steps are repeated until the number of iteration rounds of the local optimal motion model reaches the iteration stop number q max When performing multiple rounds of iterations, the number of data sets to be fitted selected from the data sets to be fitted corresponding to each piece of detection information in each round of iteration can be the same (e.g., m sets are selected each time) or different (i.e., the second number ≠ the third number), which is not limited in the present embodiment.

[0185] Step S3134: Determine the target motion model based on the local optimal motion model that meets the iteration stop condition.

[0186] The number of iterations q reaches the number of iterations to stop q max When , the current local optimal motion model is used as the target motion model.

[0187] In the disclosed embodiment, model fitting is performed based on detection information that meets the fitting conditions. During the iteration process, the number of iteration stops can be flexibly set according to the size of the data group to be fitted, ensuring the coverage of the data group to be fitted participating in the iteration, which can improve the accuracy of the target motion model and thus help improve the accuracy of the fitting speed.

[0188] In some embodiments, based on the above embodiment, step S3132 in the above embodiment, “optimizing the initial motion model according to the initial interior point set to obtain a local optimal motion model”, can be implemented by following steps S31321 to S31323:

[0189] Step S31321: Adjust the initial motion model according to the initial interior point set to obtain an adjusted initial motion model.

[0190] For example, the initial interior point is the normalized data set to be fitted. For example, the m initial interior points can be recorded as The model parameters (a′) can be calculated based on the least squares method according to the m initial interior points and the initial motion model. x ,b x ′,a′ y ,b′ y ), the adjusted initial motion model is obtained as shown in the following formula (6):

[0191]

[0192] Wherein, j=1,2,…,m, m is a positive integer representing the number of initial interior points in the initial interior point set.

[0193] Determining the parameters of the model (i.e., the coefficients of the fitted straight line) based on the least squares method is a common technical means and will not be described in detail in the embodiments of the present disclosure.

[0194] Step S31322, in response to the adjusted initial motion model not satisfying the local optimization stopping condition, the initial inlier set is updated according to the data group to be fitted corresponding to each detection information and the adjusted initial motion model to obtain an updated inlier set, and the adjusted initial motion model is continued to be adjusted according to the updated inlier set until the adjusted initial motion model that satisfies the local optimization stopping condition is obtained.

[0195] After obtaining the adjusted initial motion model, it is determined whether it meets the local optimization stopping conditions. The local optimization stopping conditions are preset conditions for determining whether to stop the local optimization, such as the maximum number of inliers, the minimum model error, the optimal model quality, etc.

[0196] When the adjusted initial motion model does not meet the local optimization stop condition, the (n+1) data set to be fitted is used. With the adjusted initial motion model, determine the new inliers (for example, a total of p inliers are screened out The updated inlier point set is formed. The parameters of the adjusted initial motion model are adjusted again based on the updated inlier point set. The above steps are repeated until the adjusted initial motion model meets the local optimization stopping condition.

[0197] Step S31323: Determine a local optimal motion model based on the adjusted initial motion model that meets the local optimization stop condition.

[0198] When the number of inliers is the largest, or the model error is the smallest, or the model quality is the best, it is determined that the local optimization stop condition is met, and the current adjusted initial motion model is used as the local optimal motion model. For example, the specific adjustment process is as follows:

[0199] Step 1: Construct the initial motion model as shown in formula (3).

[0200] As shown in formula (3), the parameters of the initial motion model are (a x ,b x ,a y ,b y ).

[0201] Step 2: Randomly select m data sets to be fitted from the (n+1) data sets to be fitted corresponding to the (n+1) detection information as m initial inliers, denoted as Get the initial interior point set. Where j = 1, 2, ..., m, m is a positive integer, representing the number of initial interior points in the initial interior point set.

[0202] Step 3: Optimize the initial motion model based on the m initial interior points selected in this round to obtain the local optimal motion model.

[0203] 1), the initial motion model is adjusted according to the initial interior point set to obtain the adjusted initial motion model. For example, the model parameters (a′) can be calculated based on the least squares method according to the m initial interior points in the initial interior point set and the initial motion model. x ,b x ′,a′ y ,b′ y ), and obtain the adjusted initial motion model as shown in the above formula (6).

[0204] 2) Determine whether the adjusted initial motion model meets the local optimization stop condition.

[0205] If the adjusted initial motion model meets the local optimization stop condition, the current adjusted initial motion model is used as the local optimal motion model. If the adjusted initial motion model does not meet the local optimization stop condition, the following steps are performed:

[0206] ① Update the interior points according to the L2 paradigm to obtain the updated interior point set:

[0207] First, the i-th data set to be fitted is calculated according to the following formula (7): The distance d to the adjusted initial motion model i :

[0208]

[0209] Where i = 0, 1, 2, ..., n. Determine the i-th data set to be fitted The distance d to the adjusted initial motion model i Is it less than the preset inlier distance threshold d? thr , when d i <d thr , the i-th data set to be fitted Divided into interior points; when d i ≥d thr , continue with the i+1th data set to be fitted The judgment process is continued until the (n+1) data groups to be fitted are divided and the updated inlier set is obtained. Among them, the kth inlier in the updated inlier set can be recorded as k=1,2,…,p, where p is the number of inliers in the updated inlier set.

[0210] ②Calculate the average error of the adjusted initial motion model

[0211] Calculate the average error of the adjusted initial motion model based on the updated inliers The calculation formula is as follows (8):

[0212]

[0213] Where p is the number of interior points in the updated interior point set, that is, the number of d in the (n+1) data set to be fitted. i Less than d thr The number of data sets to be fitted. k is the kth interior point in the updated interior point set The distance to the adjusted initial motion model (see formula (7) above). Calculate the average of the p distances from each inlier point in the updated inlier point set to the adjusted initial motion model to obtain the average error of the adjusted initial motion model.

[0214] ③ Determine the average error of the adjusted initial motion model Is it less than the average error of the current optimal model recorded during the iteration process?

[0215] If the average error of the adjusted initial motion model is less than the average error of the current optimal model, it indicates that the adjusted initial motion model is better, and the adjusted initial motion model is therefore used as the new current optimal model. Return to step 1 above to continue the interior point model optimization process. If the average error of the adjusted initial motion model is greater than or equal to the average error of the current optimal model, it indicates that the adjusted initial motion model is not better than the current optimal model, and there is no need to update the current optimal model. This round of optimization is completed, and the recorded current optimal model is used as the local optimal motion model, and the fourth step is entered.

[0216] The above ② and ③ are used to determine whether the local optimization stopping condition is met based on the model error. In some embodiments, whether the local optimization stopping condition is met can also be determined based on the number of internal points. In this case, the above ② and ③ are replaced with ②':

[0217] ②', judge whether the number of inliers in the updated inlier set is greater than the number of inliers corresponding to the current optimal model recorded during the iteration process.

[0218] If the number of inliers in the updated inlier set (i.e., p) is greater than the number of inliers corresponding to the current optimal model recorded during the iteration, it indicates that the adjusted initial motion model is better, and the adjusted initial motion model is therefore used as the new current optimal model. Return to step ① above to continue the inlier model optimization process. If the number of inliers in the updated inlier set (i.e., p) is less than or equal to the number of inliers corresponding to the current optimal model recorded during the iteration, it indicates that the adjusted initial motion model is not better than the current optimal model, and there is no need to update the current optimal model. This round of optimization is complete, and the recorded current optimal model is used as the local optimal motion model, and the fourth step is entered.

[0219] In addition to the above two methods, in some embodiments, whether the local optimization stopping condition is met can also be determined based on the model quality. In this case, the above ② and ③ are replaced with ②" and ③":

[0220] ②”, calculate the quality Q of the adjusted initial motion model mod :

[0221] Calculate the quality Q of the adjusted initial motion model based on the updated inliers mod , the calculation formula is as follows (9):

[0222]

[0223] ③”, judge the quality Q of the adjusted initial motion model mod Is it greater than the quality of the current optimal model recorded during the iteration process?

[0224] If the quality Q of the adjusted initial motion model modIf the quality of the adjusted initial motion model is greater than that of the current optimal model, it indicates that the adjusted initial motion model is better, so the adjusted initial motion model is used as the new current optimal model. Return to step ① above to continue the interior point model optimization process. If the quality of the adjusted initial motion model Q mod If the quality of the initial motion model is less than or equal to the quality of the current optimal model, it indicates that the adjusted initial motion model is not better than the current optimal model, so there is no need to update the current optimal model. This round of optimization is completed, and the recorded current optimal model is used as the local optimal motion model, and the fourth step is entered.

[0225] Step 4: Determine whether the local optimal motion model meets the iteration stop condition.

[0226] Exemplarily, whether the local optimal motion model satisfies the iteration stop condition may be determined by judging whether the number of iteration rounds of the local optimal motion model reaches the iteration stop number.

[0227] The number of iterations is denoted as q, and the number of iteration stops is denoted as q max Each time an iteration is performed, the number of iterations q increases by 1, that is, the number of iterations q is equal to the number of times m data sets to be fitted are randomly selected from n+1 data sets to be fitted as the internal point set.

[0228] Determine whether the number of iterations q reaches q max , if q reaches q max , that is, q≥q max , the iteration stops, and the current local optimal motion model is used as the target motion model that meets the iteration stop condition. max , that is, q<q max , return to re-execute the second step, repeat the above steps until the iteration stops, and obtain the target motion model. In actual application, the target motion model can also be determined by other optimization methods, which are not limited in the embodiment of the present disclosure.

[0229] In some embodiments, based on the above embodiment, step S2032 of the above embodiment "verifying the fitting result to obtain the verification result" can be performed by Figure 10 Steps S20321 to S20325 shown are implemented as follows:

[0230] Step S20321: Determine the proportion of the number of inliers according to the to-be-fitted data set and the target motion model corresponding to each detection information.

[0231] The (n+1) data sets to be fitted corresponding to the (n+1) detection information are judged in turn for internal and external points: Based on the above formula (7), the i-th data set to be fitted is calculated Distance d to the target motion model i , judge d iIs it less than the preset inlier distance threshold d? thr , when d i <d thr , indicating that the distance between this point and the fitting line is relatively close, the i-th data set to be fitted Divide into interior points and add them to the interior point set of the target motion model; when d i ≥d thr , indicating that the distance between the point and the fitting line is far, the i-th data set to be fitted Divided into external points.

[0232] Determine the number of inliers in the inlier set of the target motion model, calculate the quotient of the number of inliers and the total number of points (i.e., n+1, the number of data sets to be fitted), and obtain the proportion of the number of inliers, which is recorded as γ.

[0233] Determine whether the proportion of the number of internal points is greater than the preset proportion threshold (denoted as γ thr ), when γ>γ thr , indicating that the target motion model has passed the internal point number check, and now proceeds to step S20322. thr , indicating that the number of internal points of the target motion model has not passed the verification, and the process goes to step S20325.

[0234] Step S20322: Determine a new fitting result based on multiple frames of new driving environment images collected by the vehicle in real time.

[0235] Step S20323: Determine the fitting error based on the new fitting result and the fitting result.

[0236] In some embodiments, multiple speed fitting can be performed on the obstacle based on the method provided in the embodiments of the present disclosure to obtain multiple fitting results, and it is determined whether the multiple fitting results are consistent. If they are consistent, it means that the fitting results are stable and the stability check of the target motion model passes; if they are inconsistent, it means that the fitting results are unstable and the stability check of the target motion model fails.

[0237] For example, the obstacle can be subjected to two velocity fittings. In order to make the fitting results of the two fittings independent, the above steps can be performed again after a certain number of frames based on multiple frames of new driving environment images to obtain a new fitting result. The difference between the fitting results obtained by the two fittings is calculated to obtain a fitting error. It is determined whether the fitting error is less than a preset error threshold. If the fitting error is less than the preset error threshold, it means that the fitting results obtained by the two fittings are consistent, and it is determined that the fitting results are stable. The stability check of the target motion model has passed, and the process proceeds to step S20324. If the fitting error is greater than or equal to the preset error threshold, it means that the fitting results obtained by the two fittings are inconsistent, and it is determined that the fitting results are unstable. The stability check of the target motion model has failed, and the process proceeds to step S20325.

[0238] Step S20324: In response to the fact that the proportion of the number of inliers is greater than a preset proportion threshold and / or the fitting error is less than a preset error threshold, the verification result of the fitting result is determined as passed.

[0239] When the proportion of the number of inliers is greater than the preset proportion threshold and / or the fitting error is less than the preset error threshold, that is, the number of inliers and / or stability of the target motion model are checked, the check result of the fitting result is determined to be checked.

[0240] Step S20325: In response to the fact that the proportion of the number of inliers is less than or equal to the preset proportion threshold, or the fitting error is greater than or equal to the preset error threshold, the verification result of the fitting result is determined to be verification failure.

[0241] When the proportion of inliers is less than or equal to a preset proportion threshold, or the fitting error is greater than or equal to a preset error threshold, that is, the number of inliers or stability check of the target motion model fails, the fitting result is determined to have failed the check. In practical applications, the fitting result can be verified based solely on the number of inliers, solely on the fitting error, based on both the number of inliers and the fitting error, or based on other verification methods, which are not limited in the present embodiment.

[0242] In the embodiment of the present disclosure, after obtaining the fitting result, the fitting result is verified, and the fitting speed is determined based on the fitting result that passes the verification, which can further ensure the accuracy of the fitting speed.

[0243] In some embodiments, based on the above embodiment, step S204 in the above embodiment, "adjusting the filter for estimating the obstacle state based on the fitting speed to obtain the target filter", can be achieved by Figure 11 Steps S2041 to S2044 shown are implemented as follows:

[0244] Step S2041 : In response to a first difference between the fitting speed and the first speed of the filter being greater than a preset difference threshold, determining a jump speed based on the first difference.

[0245] The fitted velocity is recorded as v, and the first velocity estimated by the filter is recorded as v filter , calculate the first difference v delta =vv filter . Determine the first difference v delta Is it greater than the preset difference threshold (denoted as v thr ), when v delta >v thr , indicating that the filter needs to be adjusted based on the first difference v deltaDetermine the adjustment step size of the filter speed, that is, the jump speed, recorded as v jump For example, the jump speed v jump , we can calculate the first difference v delta For example, for the first difference v delta Make a certain number of splits and get the jump speed v jump In practical applications, the jump speed may also be a preset fixed value.

[0246] Step S2042 : adjusting the process noise covariance matrix of the filter based on the jump speed to obtain an adjusted filter.

[0247] For example, the sum of the jump speed and the first speed of the filter can be updated to the speed of the adjusted filter, that is, the speed of the adjusted filter = v filter +v jump . Based on the speed of the adjusted filter, a new process noise covariance matrix is ​​calculated to obtain the adjusted filter. Exemplarily, the filter in the embodiment of the present disclosure may be an adaptive Kalman filter. In a traditional Kalman filter, the process noise covariance matrix is ​​fixed. However, in an intelligent driving scenario, the state of obstacles and the ambient noise change over time. In the embodiment of the present disclosure, an adaptive Kalman filter is used to dynamically adjust the process noise covariance matrix through real-time collected data, which can improve the accuracy of state estimation.

[0248] Step S2043: estimating a second speed of the filter after the preset time length based on the adjusted filter.

[0249] After the adjusted filter runs for a preset time (i.e., a number of frames), the second speed estimated by the filter is determined and recorded as v filter2 According to the fitting speed v and the second speed v of the filter filter2 , calculate the second difference v delta2 =vv filter2 . Determine the second difference v delta2 Is it greater than the preset difference threshold v thr When v delta2 >v thr , indicating that the filter needs to be adjusted further, and the process returns to step S2042. delta2 ≤v thr , indicating that the filter has converged, and the process proceeds to step S2044. For example, whether the filter has converged can be determined after each number of filter runs. The preset duration can be a multiple of the acquisition interval, for example, once every 10 frames. In practical applications, whether the filter has converged can also be determined after each frame of the filter runs. In this case, the preset duration is equal to the acquisition interval.

[0250] Step S2044 : In response to a second difference between the fitting speed and the second speed of the filter being less than or equal to a preset difference threshold, determining a target filter according to the adjusted filter.

[0251] When the filter speed is adjusted to a value less than or equal to the preset difference threshold value with the fitting speed, that is, v delta2 ≤v thr When the filter is adjusted, it is determined that the current adjusted filter has converged, and the adjusted filter is determined as the target filter. In the disclosed embodiment, when adjusting the filter, the adjustment can be performed gradually, and the jump speed used during the adjustment process has a direction, which can ensure that the adjustment direction is consistent with the filter convergence direction, ensure the adjustment direction is correct, and accelerate filter convergence. Using a converged target filter can quickly and accurately estimate the state of the obstacle.

[0252] In the embodiment of the present disclosure, when adjusting the filter, the speed of the filter is adjusted each time, and the process noise covariance matrix is ​​calculated based on the adjusted speed to achieve the purpose of adjusting the filter.

[0253] In some embodiments, when adjusting the filter, the adjustment amount for adjusting the process noise covariance matrix can be determined based on the difference between the fitting speed and the estimated speed of the filter, that is, the process noise covariance matrix can also be adjusted each time to obtain the adjusted filter. Compare the fitting speed with the estimated speed of the filter before and after adjustment to determine whether the adjustment direction is the convergence direction. If the difference between the fitting speed and the estimated speed of the adjusted filter is less than the difference between the fitting speed and the estimated speed of the filter before adjustment, it means that the adjustment direction is the convergence direction, otherwise the adjustment direction is changed. Based on this, the process noise covariance matrix is ​​continuously adjusted so that the difference between the fitting speed and the estimated speed of the adjusted filter becomes smaller and smaller until it is less than or equal to the preset difference threshold position, thereby obtaining a converged filter, that is, a target filter.

[0254] Based on the above embodiments, the linear relationship and / or the consistency of the obstacle's motion performance in multiple time periods can be used to verify whether the detection information meets the fitting conditions. Taking the linear relationship as an example to verify whether the detection information meets the fitting conditions, the present disclosure further provides an obstacle state estimation method. Figure 12 FIG. 1 is a flow chart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure. Figure 12 As shown, the obstacle state estimation method includes the following steps:

[0255] Step S1201 : determining a detection information table of obstacles to be detected based on multiple frames of driving environment images collected by the vehicle in real time.

[0256] The specific implementation of steps S1201 to S1205 in the embodiment of the present disclosure is similar to the corresponding steps in the aforementioned embodiment and will not be repeated here.

[0257] Step S1202: Determine the linear relationship between the position of the obstacle to be detected and time according to the multiple detection information in the detection information table.

[0258] If the linear relationship between the position and time of the obstacle to be detected satisfies the linearity condition, it is determined that the multiple detection information meets the fitting condition, and the process proceeds to step S1203. If the linear relationship between the position and time of the obstacle to be detected does not satisfy the linearity condition, it is determined that the multiple detection information does not meet the fitting condition, and the process proceeds to step S1206.

[0259] Step S1203 : In response to the linear relationship between the position of the obstacle to be detected and the time satisfying the linearity condition, a fitting speed of the obstacle to be detected is obtained based on the plurality of detection information.

[0260] Step S1204 : adjusting the filter used to estimate the obstacle state based on the fitting speed to obtain a target filter.

[0261] Step S1205 : Estimate the state of the obstacle to be detected based on the target filter.

[0262] Step S1206 : In response to the linear relationship between the position of the obstacle to be detected and the time not satisfying the linearity condition, determining a second driving environment image including the obstacle to be detected that is continuously collected by the vehicle.

[0263] Step S1207: Determine detection information corresponding to the second driving environment image according to the second driving environment image.

[0264] Step S1208: Update the detection information table according to the detection information corresponding to the second driving environment image.

[0265] If it is determined that the detection information does not meet the fitting condition, a subsequent second driving environment image is captured, the second driving environment image is recognized, the corresponding detection information is determined, and the new detection information is inserted into the detection information table to update the detection information table. After the detection information table is updated, the process returns to step S1202 and repeats the above steps until the detection information meets the fitting condition.

[0266] In the embodiment of the present disclosure, the fitting conditions are preliminarily verified for the detection information in the detection information table. If the fitting conditions are not met, more driving environment images are continuously collected until the detection information in the detection information table meets the fitting conditions. Then, fitting is performed based on the detection information that meets the fitting conditions. A fitting speed with a smaller error can be obtained, thereby accelerating the filter convergence speed, and further enabling the motion state of the obstacle to be estimated quickly and accurately, ensuring safe and stable driving of the vehicle.

[0267] Based on the above embodiment, the present disclosure further provides an obstacle state estimation method, taking the example of verifying whether the detection information meets the fitting condition based on the linear relationship and the consistency of the obstacle's motion performance in multiple time periods. Figure 13 FIG. 1 is a flow chart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure, such as Figure 13 As shown, the obstacle state estimation method includes the following steps:

[0268] Step S1301 : determining a detection information table of obstacles to be detected based on multiple frames of driving environment images collected by the vehicle in real time.

[0269] The specific implementation of step S1301, step S1302, step S1304 to step S1306, and step S1310 and step S1311 in the embodiment of the present disclosure is similar to the corresponding steps in the aforementioned embodiment and will not be repeated here.

[0270] Step S1302: Determine the linear relationship between the position of the obstacle to be detected and time according to the multiple detection information in the detection information table.

[0271] If the linear relationship between the position and time of the obstacle to be detected meets the linearity condition, the process proceeds to step S1303. If the linear relationship between the position and time of the obstacle to be detected does not meet the linearity condition, the process proceeds to step S1309.

[0272] Step S1303 : In response to the linear relationship between the position and time of the obstacle to be detected satisfying the linearity condition, a consistency relationship of the movement of the obstacle to be detected in multiple time periods is determined according to the multiple detection information in the detection information table.

[0273] On the premise that the linear relationship is that the position and time of the obstacle to be detected meet the linearity condition, when the consistency relationship is that the obstacle to be detected moves consistently in multiple time periods, it is determined that the multiple detection information meets the fitting condition, and the process proceeds to step S1304. When the consistency relationship is that the obstacle to be detected moves inconsistently in multiple time periods, it is determined that the multiple detection information does not meet the fitting condition, and the process proceeds to step S1307.

[0274] Step S1304 : In response to the consistency relationship indicating that the obstacle to be detected moves in a consistent manner in multiple time periods, a fitting speed of the obstacle to be detected is obtained based on the multiple detection information.

[0275] Step S1305 : Adjust the filter used to estimate the obstacle state based on the fitting speed to obtain a target filter.

[0276] Step S1306: Estimate the state of the obstacle to be detected based on the target filter.

[0277] Step S1307 : In response to the consistency relationship indicating that the obstacle to be detected has inconsistent motions in multiple time periods, determining the third detection information with the smallest timestamp in the detection information table.

[0278] Step S1308: Update the detection information table according to the third detection information.

[0279] Generally, detection information closer to the current moment is more accurate in determining the obstacle's real-time motion state, while detection information farther from the current moment is less accurate. Based on this, if the motion of an obstacle to be detected is inconsistent at different time periods based on multiple detection information stored in the detection information table, the detection information frame farthest from the current moment can be deleted from the detection information table to update the detection information table. The detection information frame farthest from the current moment is the one with the smallest timestamp in the detection information table.

[0280] Step S1309 : determining the second driving environment image that the vehicle continues to collect and includes the obstacle to be detected.

[0281] When the linear relationship is that the position and time of the obstacle to be detected do not meet the linearity condition, or when the consistency relationship is that the obstacle to be detected moves inconsistently in multiple time periods, after deleting the earliest detection information stored in the detection information table, continue to collect new driving environment images to determine the new detection information to be inserted into the detection information table.

[0282] Step S1310: Determine detection information corresponding to the second driving environment image according to the second driving environment image.

[0283] Step S1311: Update the detection information table according to the detection information corresponding to the second driving environment image.

[0284] After updating the detection information table, the process returns to step S1302.

[0285] In the embodiment of the present disclosure, the fitting conditions are preliminarily verified for the detection information in the detection information table. If the fitting conditions are not met, more driving environment images are continuously collected until the detection information in the detection information table meets the fitting conditions. Then, fitting is performed based on the detection information that meets the fitting conditions. A fitting speed with a smaller error can be obtained, thereby accelerating the filter convergence speed, and further enabling the motion state of the obstacle to be estimated quickly and accurately, ensuring safe and stable driving of the vehicle.

[0286] Based on the above embodiments, the present disclosure further provides an obstacle state estimation method. Figure 14 FIG. 1 is a flow chart of an obstacle state estimation method provided by another exemplary embodiment of the present disclosure, such as Figure 14 As shown, the obstacle state estimation method includes the following steps:

[0287] Step S1401 : determining a current driving environment image collected by the vehicle at the current moment and a previous driving environment image collected at the previous moment.

[0288] When the vehicle is driving, the on-board camera collects images of the vehicle's surrounding environment in real time, obtaining the current driving environment image collected by the vehicle at the current moment and the previous driving environment image collected at the previous moment.

[0289] Step S1402 : Determine whether a new obstacle that does not exist in the previous driving environment image is detected in the current driving environment image.

[0290] The current driving environment image and the previous driving environment image are recognized separately, and the recognition results of the two images are compared to determine whether a new obstacle appears in the current driving environment image. If a new obstacle is detected in the current driving environment image that was not present in the previous driving environment image, the new obstacle is determined as an obstacle to be detected, and the process continues with step S1043. If no new obstacle is detected in the current driving environment image, the process returns to step S1401 and continues to capture a new driving environment image.

[0291] Step S1403 : determining detection information corresponding to the current driving environment image according to the current driving environment image, and storing the detection information in a detection information table.

[0292] Based on the recognition results of the current driving environment image, the obstacle's location information and timestamp information are determined. The obstacle's location information and timestamp information determined for that frame of driving environment image are determined as the detection information corresponding to that frame of driving environment image. The timestamp information includes the acquisition time of the current driving environment image, and the location information includes the obstacle's location at the time of acquisition. After the detection information corresponding to the current driving environment image is determined, it is stored in the detection information table.

[0293] Step S1404: determine whether the number of detection information in the detection information table reaches a preset upper limit.

[0294] If the number of detection information in the detection information table reaches the preset upper limit value, it means that the data stored in the detection information table has reached the upper limit, and the earliest detection information stored in the detection information table needs to be deleted before the latest detection information can be stored. At this time, step S1405 is entered; if the number of detection information in the detection information table does not reach the preset upper limit value, it means that more detection information can be stored in the detection information table. At this time, step S1406 is entered.

[0295] Step S1405: Delete the detection information with the smallest timestamp from the detection information table.

[0296] The detection information with the smallest timestamp is the detection information stored earliest in the detection information table, that is, the detection information with the timestamp farthest from the current time.

[0297] Step S1406 , determining the next driving environment image that the vehicle continues to collect and includes the obstacle to be detected; determining detection information corresponding to the next driving environment image based on the next driving environment image, and storing the information in the detection information table.

[0298] Step S1407: determine whether the number of detection information in the detection information table reaches a preset number threshold.

[0299] The number of detection information in the detection information table represents the number of frames in which the obstacle appears. A determination is made as to whether the number of detection information in the detection information table reaches a preset threshold. If the number of detection information exceeds the preset threshold, the number of frames in which the obstacle appears meets the fitting frame requirement, and the process proceeds to step S1408. If the number of detection information is less than the preset threshold, the number of frames in which the obstacle appears is insufficient to determine a more accurate fitting speed, and the process returns to step S1406 to continue collecting more frames of driving environment images.

[0300] Step S1408: judging whether the position and time of the obstacle to be detected meet the linearity condition according to the multiple detection information in the detection information table.

[0301] Based on the multiple detection information in the detection information table, a linear relationship between the position and time of the obstacle to be detected is determined. If the linear relationship between the position and time of the obstacle to be detected meets the linearity condition, the process proceeds to step S1409 to continue checking for motion consistency. If the linear relationship between the position and time of the obstacle to be detected does not meet the linearity condition, it is determined that the multiple detection information does not meet the fitting condition, and the process returns to step S1404 to continue collecting more frames of driving environment images.

[0302] Step S1409: judging whether the movement of the obstacle to be detected is consistent in multiple time periods based on the multiple detection information in the detection information table.

[0303] Based on the multiple detection information in the detection information table, a consistency relationship is determined for the movement of the obstacle to be detected during the multiple time periods. If the consistency relationship indicates that the movement of the obstacle to be detected is consistent during the multiple time periods, the multiple detection information is determined to meet the fitting condition, and the process proceeds to step S1410. If the consistency relationship indicates that the movement of the obstacle to be detected is inconsistent during the multiple time periods, the multiple detection information is determined to not meet the fitting condition, and the process returns to step S1405, deletes the oldest stored detection information, and then continues the above verification.

[0304] Step S1410 , performing straight line fitting based on the position information and timestamp information included in each detection information to obtain a fitting result.

[0305] Step S1411 : determining the proportion of the number of inliers according to the to-be-fitted data set and the target motion model corresponding to each detection information.

[0306] Step S1412: determine whether the proportion of the number of inliers is greater than a preset proportion threshold.

[0307] The higher the inlier ratio, the more accurate the fitting result. When the inlier ratio is greater than the preset ratio threshold, the fitting result is determined to meet the fitting accuracy requirements, and the process proceeds to step S1413. When the inlier ratio is less than or equal to the preset ratio threshold, the fitting result is determined to not meet the accuracy requirements, i.e., the verification result of the fitting result is determined to have failed. The process returns to step S1405, deletes the oldest stored detection information, and then continues the verification process.

[0308] Step S1413: Determine a new fitting result.

[0309] Repeat the above steps to obtain new fitting results.

[0310] Step S1414: determine whether the new fitting result and the fitting error between the fitting results are less than a preset error threshold.

[0311] The obstacle's motion over a short period of time can be approximated as a uniform velocity state, and the error between multiple fitting results is generally small. A determination is made as to whether the fitting error between the two fitting results is less than a preset error threshold. If the fitting error is less than the preset error threshold, the fitting result is determined to meet the stability requirements, i.e., the fitting result is determined to have passed verification, and the process proceeds to step S1415. If the fitting error is greater than or equal to the preset error threshold, the fitting result is determined to have failed verification, and the process returns to step S1405, deleting the oldest stored detection information and continuing the verification process.

[0312] Step S1415: Determine the fitting speed of the obstacle to be detected according to the fitting result.

[0313] Step S1416: determine whether the difference between the fitting speed and the filter speed is greater than a preset difference threshold.

[0314] If the difference between the fitting speed and the filter speed is greater than the preset difference threshold, it indicates that the filter speed deviates significantly from the fitting speed and needs to be adjusted, and the process proceeds to step S1417. If the difference between the fitting speed and the filter speed is less than or equal to the preset difference threshold, it indicates that the filter speed has converged, and the process proceeds to step S1420.

[0315] Step S1417, determine the jump speed.

[0316] For example, the transition speed can be determined based on the difference between the fitting speed and the filter speed, for example, by dividing the difference into a certain number of parts to obtain the transition speed. The transition speed can also be set to a fixed small amount, for example, by adjusting the step size each time. In actual applications, the transition speed can also be determined using other methods, which are not limited in the present embodiment.

[0317] Step S1418: Adjust the process noise covariance matrix of the filter based on the jump speed to obtain an adjusted filter.

[0318] Step S1419: estimating the speed of the filter after the preset time length based on the adjusted filter.

[0319] After the new speed of the filter is determined, the process returns to step S1416 to continue the process of determining whether convergence has occurred.

[0320] Step S1420 : determining a target filter according to the adjusted filter, and estimating the state of the obstacle to be detected based on the target filter.

[0321] The method provided by the embodiment of the present disclosure selects multiple detection information that meet the fitting conditions from the detection information table for fitting, which can obtain a fitting speed with a smaller error, thereby accelerating the filter convergence speed, and further enabling rapid and accurate estimation of the motion state of the obstacle, ensuring safe and stable driving of the vehicle.

[0322] Exemplary devices

[0323] Figure 15 FIG. 1 is a schematic diagram showing the structure of an obstacle state estimation device provided by an exemplary embodiment of the present disclosure. Figure 15 As shown, the obstacle state estimation device 1500 may include:

[0324] The first determining module 1501 is used to determine a detection information table of obstacles to be detected based on multiple frames of driving environment images collected in real time by the vehicle;

[0325] A second determining module 1502 is configured to determine, based on the plurality of detection information in the detection information table, whether the plurality of detection information satisfies a fitting condition;

[0326] A fitting module 1503 is configured to, in response to the satisfying relationship being that the plurality of detection information satisfies the fitting condition, obtain a fitting speed of the obstacle to be detected based on the plurality of detection information;

[0327] An adjustment module 1504 is configured to adjust a filter for estimating obstacle status based on the fitting speed to obtain a target filter;

[0328] The estimation module 1505 is configured to estimate the state of the obstacle to be detected based on the target filter.

[0329] See also Figure 16 In some embodiments, the first determining module 1501 may include:

[0330] The first determining unit 15011 is configured to determine, based on the multiple frames of driving environment images acquired in real time by the vehicle, multiple frames of images to be detected that include obstacles to be detected;

[0331] The second determining unit 15012 is configured to determine, based on each of the images to be detected, the position information and timestamp information of the obstacle to be detected corresponding to each of the images to be detected;

[0332] A third determining unit 15013 is configured to determine detection information corresponding to each of the images to be detected based on the position information and the timestamp information corresponding to each of the images to be detected;

[0333] The fourth determining unit 15014 is configured to determine a detection information table of the obstacles to be detected based on the detection information corresponding to each of the images to be detected.

[0334] In some embodiments, the first determination unit 15011 can also be used to: determine a first driving environment image captured by the vehicle at a first moment and a previous driving environment image captured at a moment before the first moment; in response to detecting a new obstacle in the first driving environment image that does not exist in the previous driving environment image, determine an obstacle to be detected based on the new obstacle; determine a first number of frames of driving environment images that the vehicle continues to capture after the first moment and include the obstacle to be detected; and determine the multiple frames of images to be detected based on the first driving environment image and the first number of frames of driving environment images.

[0335] See also Figure 16 In some embodiments, the second determining module 1502 may include:

[0336] a fifth determining unit 15021, configured to determine a linear relationship between a position and time of the obstacle to be detected based on the plurality of detection information in the detection information table;

[0337] a sixth determining unit 15022, configured to determine, based on the plurality of detection information in the detection information table, a consistency relationship between the movement of the obstacle to be detected in the plurality of time periods;

[0338] The seventh determining unit 15023 is configured to, in response to the linear relationship being that the position and time of the obstacle to be detected satisfy a linearity condition and / or the consistency relationship being that the obstacle to be detected moves consistently in multiple time periods, determine the satisfying relationship as the multiple pieces of detection information satisfying the fitting condition.

[0339] In some embodiments, the fifth determination unit 15021 can also be used to: determine the horizontal coordinates and vertical coordinates corresponding to each detection information according to the position information included in each detection information in the detection information table; determine the first linear relationship between the horizontal coordinates of the obstacle to be detected and time according to the timestamp information included in each detection information in the detection information table and the horizontal coordinates corresponding to each detection information; determine the second linear relationship between the vertical coordinates of the obstacle to be detected and time according to the timestamp information included in each detection information in the detection information table and the vertical coordinates corresponding to each detection information; determine the linear relationship between the position of the obstacle to be detected and time according to the first linear relationship and the second linear relationship.

[0340] In some embodiments, the sixth determination unit 15022 can also be used to: determine, based on the multiple detection information, multiple first detection information corresponding to the first time period and multiple second detection information corresponding to the second time period; determine, based on the multiple first detection information, a first interval fitting speed of the obstacle to be detected in the first time period; determine, based on the multiple second detection information, a second interval fitting speed of the obstacle to be detected in the second time period; and determine, based on the first interval fitting speed and the second interval fitting speed, a consistency relationship between the movements of the obstacle to be detected in multiple time periods.

[0341] In some embodiments, the sixth determination unit 15022 can also be used to: determine the longitudinal coordinate corresponding to each first detection information based on the position information included in each first detection information; perform straight line fitting based on the longitudinal coordinate and timestamp information corresponding to each first detection information to obtain the first interval fitting speed of the obstacle to be detected in the first time period.

[0342] In some embodiments, the obstacle state estimation device 1500 may further include:

[0343] a third determining module, configured to, in response to the linear relationship between the position and time of the obstacle to be detected not satisfying a linearity condition, determine a second driving environment image including the obstacle to be detected that the vehicle continues to collect;

[0344] a fourth determining module, configured to determine detection information corresponding to the second driving environment image based on the second driving environment image;

[0345] The first updating module is configured to update the detection information table according to the detection information corresponding to the second driving environment image.

[0346] See also Figure 16 In some embodiments, the fitting module 1503 may include:

[0347] A fitting unit 15031 is configured to perform straight line fitting based on the position information and timestamp information included in each detection information to obtain a fitting result;

[0348] A verification unit 15032 is used to verify the fitting result and obtain a verification result;

[0349] The eighth determining unit 15033 is configured to determine, in response to the verification result being a passed verification, a fitting speed of the obstacle to be detected according to the fitting result.

[0350] In some embodiments, the fitting unit 15031 can also be used to: determine the data group to be fitted corresponding to each detection information based on the position information and timestamp information included in each detection information; construct an initial motion model of the obstacle to be detected; optimize the initial motion model based on the data group to be fitted corresponding to each detection information to obtain a target motion model that meets the iteration stop condition; and determine the fitting result based on the parameters of the target motion model.

[0351] In some embodiments, the fitting unit 15031 can also be used to: determine the original data group corresponding to each detection information based on the position information and timestamp information in each detection information; normalize the original data group corresponding to each detection information to obtain the data group to be fitted corresponding to each detection information.

[0352] In some embodiments, the fitting unit 15031 can also be used to: screen a second number of data groups to be fitted from the data groups to be fitted corresponding to each of the detection information to obtain an initial inlier set; optimize the initial motion model based on the initial inlier set to obtain a local optimal motion model; in response to the local optimal motion model not meeting the iteration stop condition, re-screen a third number of data groups to be fitted from the data groups to be fitted corresponding to each of the detection information to obtain a new initial inlier set, and continue to optimize the local optimal motion model based on the new initial inlier set until a local optimal motion model that meets the iteration stop condition is obtained; and determine the target motion model based on the local optimal motion model that meets the iteration stop condition.

[0353] In some embodiments, the fitting unit 15031 can also be used to: adjust the initial motion model according to the initial inlier set to obtain an adjusted initial motion model; in response to the adjusted initial motion model not satisfying the local optimization stop condition, update the initial inlier set according to the data group to be fitted corresponding to each detection information and the adjusted initial motion model to obtain an updated inlier set, and continue to adjust the adjusted initial motion model according to the updated inlier set until the adjusted initial motion model satisfying the local optimization stop condition is obtained; determine the local optimal motion model based on the adjusted initial motion model satisfying the local optimization stop condition.

[0354] In some embodiments, the fitting unit 15031 can also be used to: determine the horizontal scaling ratio, the vertical scaling ratio and the time scaling ratio based on the original data group and the data group to be fitted corresponding to each detection information; determine the horizontal scaling component and the vertical scaling component based on the parameters of the target motion model; determine the horizontal fitting result of the obstacle to be detected based on the horizontal scaling component, the horizontal scaling ratio and the time scaling ratio; determine the vertical fitting result of the obstacle to be detected based on the vertical scaling component, the vertical scaling ratio and the time scaling ratio; and determine the fitting result based on the horizontal fitting result and the vertical fitting result.

[0355] In some embodiments, the verification unit 15032 can also be used to: determine the proportion of the number of inliers based on the data group to be fitted corresponding to each detection information and the target motion model; determine a new fitting result based on multiple frames of new driving environment images collected by the vehicle in real time; determine the fitting error based on the new fitting result and the fitting result; in response to the proportion of the number of inliers being greater than a preset proportion threshold and / or the fitting error being less than a preset error threshold, determine the verification result of the fitting result as passed verification; in response to the proportion of the number of inliers being less than or equal to the preset proportion threshold or the fitting error being greater than or equal to the preset error threshold, determine the verification result of the fitting result as failed verification.

[0356] In some embodiments, the obstacle state estimation device 1500 may further include:

[0357] a fifth determining module, configured to, in response to the consistency relationship indicating inconsistent movement of the obstacle to be detected in multiple time periods, or the verification result indicating verification failure, or the number of the detection information in the detection information table reaching a preset threshold, determine the third detection information having the smallest timestamp in the detection information table;

[0358] a sixth determining module, configured to determine a third driving environment image continuously collected by the vehicle including the obstacle to be detected;

[0359] a seventh determining module, configured to determine detection information corresponding to the third driving environment image based on the third driving environment image;

[0360] The second updating module is configured to update the detection information table according to the third detection information and the detection information corresponding to the third driving environment image.

[0361] See also Figure 16 In some embodiments, the adjustment module 1504 may include:

[0362] a ninth determining unit 15041, configured to determine a jump speed based on a first difference between the fitting speed and the first speed of the filter in response to the first difference being greater than a preset difference threshold;

[0363] An adjusting unit 15042 is configured to adjust a process noise covariance matrix of the filter based on the transition rate to obtain an adjusted filter;

[0364] An estimating unit 15043 is configured to estimate a second speed of the filter after a preset time period based on the adjusted filter;

[0365] The tenth determining unit 15044 is configured to determine the target filter according to the adjusted filter in response to a second difference between the fitting speed and the second speed of the filter being less than or equal to the preset difference threshold.

[0366] The beneficial technical effects corresponding to the exemplary embodiments of the device disclosed herein can be found in the corresponding beneficial technical effects of the exemplary method section above, which will not be repeated here.

[0367] Exemplary electronic devices

[0368] Figure 17 FIG. 1 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 17 As shown, the electronic device 1700 includes at least one processor 1701 and a memory 1702 .

[0369] The processor 1701 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1700 to perform desired functions.

[0370] Memory 1702 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and processor 1701 may execute one or more computer program instructions to implement the obstacle state estimation method and / or other desired functions of the various embodiments of the present disclosure described above.

[0371] In one example, the electronic device 10 may further include an input device 1703 and an output device 1704 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0372] The input device 1703 may also include, for example, a keyboard, a mouse, and the like.

[0373] The output device 1704 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output devices, etc.

[0374] Of course, to simplify, Figure 17Only some of the components related to the present disclosure in the electronic device 1700 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 1700 may further include any other appropriate components.

[0375] Exemplary computer program products and computer-readable storage media

[0376] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the obstacle state estimation method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0377] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0378] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the steps of the obstacle state estimation method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0379] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or component comprising electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0380] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be considered as essential to each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0381] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A method for estimating an obstacle state, comprising: Determine the detection information table of obstacles to be detected according to the multiple frames of driving environment images collected by the vehicle in real time; Based on the multiple detection information in the detection information table, determining a satisfying relationship between the multiple detection information and the fitting condition; In response to the satisfying relationship that the plurality of detection information satisfies the fitting condition, obtaining a fitting speed of the obstacle to be detected based on the plurality of detection information; Adjusting a filter for estimating the obstacle state based on the fitting speed to obtain a target filter; Based on the target filter, the state of the obstacle to be detected is estimated.

2. The method according to claim 1, wherein: The step of determining the detection information table of obstacles to be detected based on multiple frames of driving environment images collected in real time by the vehicle includes: Determine, based on the multiple frames of driving environment images acquired in real time by the vehicle, multiple frames of images to be detected including obstacles to be detected; Determine, according to each of the images to be detected, the position information and timestamp information of the obstacle to be detected corresponding to each of the images to be detected; Determine the detection information corresponding to each of the images to be detected according to the position information and the timestamp information corresponding to each of the images to be detected; A detection information table of the obstacles to be detected is determined according to the detection information corresponding to each of the images to be detected.

3. The method according to claim 2, wherein: The step of determining, based on the multiple frames of driving environment images collected in real time by the vehicle, multiple frames of images to be detected including obstacles to be detected includes: Determining a first driving environment image acquired by the vehicle at a first moment and a previous driving environment image acquired at a moment before the first moment; In response to detecting a new obstacle in the first driving environment image that does not exist in the previous driving environment image, determining an obstacle to be detected according to the new obstacle; Determining a first number of frames of driving environment images including the obstacle to be detected that the vehicle continues to collect after the first moment; The plurality of frames of images to be detected are determined according to the first driving environment image and the first number of frames of driving environment images.

4. The method according to claim 1, wherein: The determining, based on the plurality of detection information in the detection information table, a satisfying relationship between the plurality of detection information and the fitting condition comprises: Determine the linear relationship between the position and time of the obstacle to be detected according to the plurality of detection information in the detection information table; Determining the consistency relationship of the movement of the obstacle to be detected in multiple time periods according to the multiple detection information in the detection information table; In response to the linear relationship being that the position and time of the obstacle to be detected satisfy a linearity condition and / or the consistency relationship being that the obstacle to be detected moves consistently in multiple time periods, determining the satisfying relationship as that the multiple detection information satisfy the fitting condition; In response to the linear relationship that the position and time of the obstacle to be detected do not satisfy a linearity condition, determine a second driving environment image including the obstacle to be detected that the vehicle continues to collect; determine detection information corresponding to the second driving environment image based on the second driving environment image; and update the detection information table based on the detection information corresponding to the second driving environment image.

5. The method according to claim 4, wherein: Determining the linear relationship between the position and time of the obstacle to be detected according to the plurality of detection information in the detection information table includes: Determine the horizontal coordinate and the vertical coordinate corresponding to each piece of detection information according to the position information included in each piece of detection information in the detection information table; Determine a first linear relationship between the horizontal coordinate of the obstacle to be detected and time according to the timestamp information of each piece of detection information in the detection information table and the horizontal coordinate corresponding to each piece of detection information; Determine a second linear relationship between the longitudinal coordinate of the obstacle to be detected and time according to the timestamp information of each piece of detection information in the detection information table and the longitudinal coordinate corresponding to each piece of detection information; A linear relationship between the position and time of the obstacle to be detected is determined according to the first linear relationship and the second linear relationship.

6. The method according to claim 4, wherein: The determining, based on the plurality of detection information in the detection information table, the consistency relationship of the movement of the obstacle to be detected in the plurality of time periods comprises: Determine, according to the plurality of detection information, a plurality of first detection information corresponding to the first time period and a plurality of second detection information corresponding to the second time period; Determine, according to the plurality of first detection information, a first interval fitting speed of the obstacle to be detected in the first time period; Determining a second interval fitting speed of the obstacle to be detected in the second time period according to the plurality of second detection information; According to the first interval fitting speed and the second interval fitting speed, a consistency relationship between the movements of the obstacle to be detected in multiple time periods is determined.

7. The method according to claim 4, wherein: The fitting speed of the obstacle to be detected obtained based on the plurality of detection information includes: Perform straight line fitting according to the position information and timestamp information included in each of the detection information to obtain a fitting result; Verifying the fitting result to obtain a verification result; In response to the verification result being a passed verification, a fitting speed of the obstacle to be detected is determined according to the fitting result.

8. The method according to claim 7, wherein: The performing straight line fitting according to the position information and the timestamp information included in each of the detection information to obtain a fitting result includes: Determine the original data group corresponding to each of the detection information according to the position information and the timestamp information in each of the detection information; Normalizing the original data groups corresponding to the detection information to obtain the data groups to be fitted corresponding to the detection information; Constructing an initial motion model of the obstacle to be detected; Optimizing the initial motion model according to the to-be-fitted data group corresponding to each of the detection information to obtain a target motion model that meets the iteration stop condition; The fitting result is determined according to the parameters of the target motion model.

9. The method according to claim 8, wherein: The step of optimizing the initial motion model according to the to-be-fitted data group corresponding to each of the detection information to obtain a target motion model that satisfies an iteration stop condition includes: Selecting a second number of data groups to be fitted from the data groups to be fitted corresponding to the detection information to obtain an initial interior point set; Optimizing the initial motion model according to the initial interior point set to obtain a local optimal motion model; In response to the local optimal motion model not satisfying the iteration stop condition, re-screening a third number of data groups to be fitted from the data groups to be fitted corresponding to each of the detection information to obtain a new initial interior point set, and continuing to optimize the local optimal motion model according to the new initial interior point set until a local optimal motion model satisfying the iteration stop condition is obtained; The target motion model is determined according to the local optimal motion model that meets the iteration stop condition.

10. The method according to claim 9, wherein: The step of optimizing the initial motion model according to the initial interior point set to obtain a local optimal motion model includes: Adjusting the initial motion model according to the initial interior point set to obtain an adjusted initial motion model; In response to the adjusted initial motion model not satisfying the local optimization stop condition, the initial interior point set is updated according to the to-be-fitted data set corresponding to each of the detection information and the adjusted initial motion model to obtain an updated interior point set, and the adjusted initial motion model is continuously adjusted according to the updated interior point set until the adjusted initial motion model satisfying the local optimization stop condition is obtained; The local optimal motion model is determined according to the adjusted initial motion model that meets the local optimization stop condition.

11. The method according to claim 8, wherein: Determining the fitting result according to the parameters of the target motion model includes: Determine a horizontal scaling ratio, a vertical scaling ratio and a time scaling ratio according to the original data group and the data group to be fitted corresponding to each of the detection information; Determining a horizontal scaling component and a vertical scaling component according to the parameters of the target motion model; Determining a lateral fitting result of the obstacle to be detected according to the lateral scaling component, the lateral scaling ratio and the time scaling ratio; Determining a longitudinal fitting result of the obstacle to be detected according to the longitudinal scaling component, the longitudinal scaling ratio and the time scaling ratio; The fitting result is determined according to the transverse fitting result and the longitudinal fitting result.

12. The method according to claim 8, wherein: The step of verifying the fitting result to obtain a verification result includes: Determining the proportion of the number of inliers according to the to-be-fitted data group and the target motion model corresponding to each of the detection information; Determining a new fitting result according to a plurality of frames of new driving environment images collected in real time by the vehicle; Determining a fitting error according to the new fitting result and the fitting result; In response to the proportion of the number of inliers being greater than a preset proportion threshold and / or the fitting error being less than a preset error threshold, determining the verification result of the fitting result as verification passed; In response to the fact that the proportion of the number of inliers is less than or equal to the preset proportion threshold or the fitting error is greater than or equal to the preset error threshold, the verification result of the fitting result is determined as verification failure.

13. The method according to claim 7, further comprising: In response to the consistency relationship being that the obstacle to be detected moves inconsistently in multiple time periods, or the verification result being determined as verification failure, or the number of the detection information in the detection information table reaches a preset number threshold, determining the third detection information with the smallest timestamp in the detection information table; Determining a third driving environment image that the vehicle continues to collect and includes the obstacle to be detected; Determining detection information corresponding to the third driving environment image according to the third driving environment image; The detection information table is updated according to the third detection information and the detection information corresponding to the third driving environment image.

14. The method according to any one of claims 1 to 13, wherein: The step of adjusting the filter for estimating the obstacle state based on the fitting speed to obtain a target filter includes: In response to a first difference between the fitting speed and the first speed of the filter being greater than a preset difference threshold, determining a jump speed based on the first difference; Adjusting the process noise covariance matrix of the filter based on the jump speed to obtain an adjusted filter; estimating a second speed of the filter after a preset time period based on the adjusted filter; In response to a second difference between the fitting speed and a second speed of the filter being less than or equal to the preset difference threshold, determining the target filter according to the adjusted filter.

15. An obstacle state estimation device, comprising: A first determination module is used to determine a detection information table of obstacles to be detected based on multiple frames of driving environment images collected in real time by the vehicle; A second determination module, configured to determine, based on the plurality of detection information in the detection information table, a satisfying relationship between the plurality of detection information and the fitting condition; A fitting module, configured to obtain a fitting speed of the obstacle to be detected based on the fitting of the plurality of detection information in response to the satisfying relationship being that the plurality of detection information satisfies the fitting condition; An adjustment module, used for adjusting a filter for estimating an obstacle state based on the fitting speed to obtain a target filter; An estimation module is used to estimate the state of the obstacle to be detected based on the target filter.

16. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the obstacle state estimation method according to any one of claims 1 to 14.

17. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the instruction from the memory and execute the instruction to implement the obstacle state estimation method described in any one of claims 1-14 above.

Images