Target vehicle distance determination method and device, electronic equipment and storage medium

Abstract

The method comprises: determining current frame lane line information corresponding to a current frame image and current frame detection information of a target vehicle based on the current frame image containing the target vehicle; determining a current frame physical size of the target vehicle based on the current frame detection information and the current frame lane line information; determining a current frame observation based on the current frame detection information and the current frame physical size; and determining a target distance of the target vehicle based on the current frame observation and a previous frame observation by using a Kalman filtering algorithm with adaptive observation noise. The method disclosed in the embodiments of the present disclosure realizes adaptive updating of observation noise in the filtering process, effectively improves processing efficiency, reduces artificial cost of noise parameter adjustment, and makes the obtained distance of the target vehicle more robust.

Description

Technical Field

[0001] This disclosure relates to computer vision technology, and in particular to a method, apparatus, electronic device, and storage medium for determining the distance to a target vehicle. Background Technology

[0002] In autonomous driving scenarios, advanced driver assistance systems (ADAS) typically include forward collision warning (FCW) functionality. FCW requires acquiring the state of the target ahead, including distance, speed, and acceleration. This state can usually be determined using methods such as lidar ranging, millimeter-wave radar ranging, and binocular vision ranging. However, lidar and millimeter-wave radar are expensive and lack semantic information, while binocular vision ranging is computationally intensive, resulting in high computational costs. To address this issue, Kalman filtering-based distance estimation has been proposed. However, Kalman filtering is susceptible to observation noise, leading to low ranging accuracy and even fluctuations in the filtered state. Related technologies often require manual noise tuning, but the complex interactions between different dimensions of the observed and state quantities during the filtering process make noise tuning labor-intensive and time-consuming, resulting in low efficiency and high labor costs. Summary of the Invention

[0003] To address the technical problems of low efficiency and high labor costs in designing empirical functions, this disclosure is proposed. Embodiments of this disclosure provide a method, apparatus, electronic device, and storage medium for determining the distance to a target vehicle.

[0004] According to one aspect of the present disclosure, a method for determining the distance to a target vehicle is provided, comprising: acquiring a current frame image containing the target vehicle; determining current frame lane line information corresponding to the current frame image and current frame detection information of the target vehicle based on the current frame image; determining the current frame physical size of the target vehicle based on the current frame detection information and the current frame lane line information; determining current frame observations based on the current frame detection information and the current frame physical size; and determining the target distance to the target vehicle using an adaptive observation noise Kalman filter algorithm based on the current frame observations and previous frame observations, wherein the previous frame observations include observations corresponding to a first number of historical frames.

[0005] According to another aspect of the present disclosure, a device for determining the distance to a target vehicle is provided, comprising: a first acquisition module, configured to acquire a current frame image containing a target vehicle; a first processing module, configured to determine current frame lane line information corresponding to the current frame image and current frame detection information of the target vehicle based on the current frame image; a second processing module, configured to determine the current frame physical size of the target vehicle based on the current frame detection information and the current frame lane line information; a third processing module, configured to determine current frame observations based on the current frame detection information and the current frame physical size; and a fourth processing module, configured to determine the target distance between the target vehicle and the current vehicle using an adaptive observation noise Kalman filter algorithm based on the current frame observations and previous frame observations, wherein the previous frame observations include observations corresponding to a first number of historical frames.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the target vehicle distance determination method described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the target vehicle distance determination method according to any of the above embodiments of the present disclosure.

[0008] Based on the target vehicle distance determination method, apparatus, electronic device, and storage medium provided in the above embodiments of this disclosure, lane line information and target vehicle detection information of the current frame can be determined based on the current frame image containing the target vehicle. Based on the current frame detection information and the current frame lane line information, the physical size of the target vehicle in the current frame can be determined. Based on the current frame detection information and the current frame physical size, the observations of the current frame can be determined. Then, based on the current frame observations and the previous frame observations, an adaptive observation noise Kalman filter algorithm is used to determine the target distance of the target vehicle. This achieves adaptive updating of observation noise in the filtering process, effectively improving processing efficiency, reducing the manual cost of noise parameter tuning, and reducing the workload of noise parameter tuning caused by image processing model updates, iterations, sensor upgrades, environmental changes, etc., making the obtained target vehicle distance more robust.

[0009] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the method for determining the distance to the target vehicle provided in this disclosure;

[0012] Figure 2 This is a flowchart illustrating a method for determining the distance to a target vehicle provided in an exemplary embodiment of this disclosure;

[0013] Figure 3 This is a flowchart illustrating a method for determining the distance to a target vehicle provided in another exemplary embodiment of this disclosure;

[0014] Figure 4 This is a flowchart illustrating step 2032 provided in an exemplary embodiment of this disclosure;

[0015] Figure 5 This is a schematic diagram illustrating the principle of determining vehicle width based on lane information, provided in an exemplary embodiment of this disclosure.

[0016] Figure 6 This is a schematic diagram illustrating the principle of vehicle width measurement provided in an exemplary embodiment of this disclosure;

[0017] Figure 7 This is a schematic diagram of the structure of a target vehicle distance determination device provided in an exemplary embodiment of this disclosure;

[0018] Figure 8 This is a schematic diagram of the structure of a target vehicle distance determination device provided in another exemplary embodiment of this disclosure;

[0019] Figure 9 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed herein. Detailed Implementation

[0020] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0021] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0022] This disclosure outlines

[0023] In the process of realizing this disclosure, the inventors discovered that in autonomous driving scenarios, autonomous driving assistance systems typically have a forward collision warning (FCW) function. FCW requires obtaining the state of the target ahead, including distance, speed, and acceleration. The state of the target ahead can usually be determined through methods such as lidar ranging, millimeter-wave radar ranging, and binocular vision ranging. However, lidar and millimeter-wave radar are expensive and difficult to obtain semantic information, while binocular vision ranging has a large computational load, resulting in high computational costs. To solve this problem, a method based on Kalman filtering to estimate the target distance was proposed. However, the Kalman filtering estimation method is easily affected by observation noise, leading to low ranging accuracy and even jitter in the filtered state. In related technologies, manual noise parameter tuning is usually required. However, due to the complex interaction between different dimensions of the observed and state quantities during the filtering process, noise parameter tuning requires a large amount of manpower and time, resulting in low efficiency and high labor costs.

[0024] Exemplary Overview

[0025] Figure 1 This is an exemplary application scenario of the method for determining the distance to the target vehicle provided in this disclosure.

[0026] In autonomous driving scenarios, vehicles around the vehicle (such as target vehicles in front of the vehicle or target vehicles behind the vehicle) can be considered as target vehicles. Using the target vehicle distance determination method provided in this embodiment, the current frame image containing the target vehicle can be acquired by the camera on the vehicle. Based on the current frame image, the current frame lane line information and the current frame detection information of the target vehicle are determined. The current frame lane line information may include the set of pixels belonging to the lane line on the current frame image. The current frame detection information of the target vehicle may include the current frame full vehicle frame information, the current frame rear frame information (or the current frame front frame information) of the target vehicle, and may also include the type of the target vehicle (called the target type). The vehicle type can be set according to actual needs. Based on the current frame detection information and lane line information of the target vehicle, the physical size of the target vehicle in the current frame can be determined. The physical size of the current frame can include the physical width and height of the target vehicle. Based on the current frame detection information and physical size of the target vehicle, the observations of the current frame can be determined. These observations are used for Kalman filtering of the current frame. Then, based on the current frame observations and the observations of previous frames, an adaptive observation noise Kalman filter algorithm can be used to determine the target distance between the target vehicle and the own vehicle. The observations of previous frames include the observations corresponding to a first number of historical frames. The first number can be set according to actual needs. The adaptive observation noise Kalman filter algorithm means that the observation noise can be adaptively updated with the iteration of the filter during the Kalman filtering process, effectively improving processing efficiency, reducing the manual cost of noise parameter tuning, and reducing the workload of noise parameter tuning caused by image processing model updates, iterations, sensor upgrades, environmental changes, etc., making the obtained target vehicle distance more robust.

[0027] The method provided in this disclosure is not limited to autonomous driving scenarios, but can also be applied to assisted driving scenarios and other possible scenarios according to actual needs. The specific settings can be configured according to actual needs.

[0028] Exemplary methods

[0029] Figure 2 This is a flowchart illustrating a method for determining the distance to a target vehicle provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically, for example, on an in-vehicle computing platform. Figure 2 As shown, it includes the following steps:

[0030] Step 201: Obtain the current frame image containing the target vehicle.

[0031] The current frame image containing the target vehicle can be acquired using a camera with a corresponding viewpoint. For example, for a target vehicle in front of the vehicle, a forward-looking camera can be used to acquire an image with a forward-looking view. The vehicle included in the forward-looking view image is considered the target vehicle, and thus the forward-looking view image is the current frame image containing the target vehicle. Whether the image contains the target vehicle can be determined based on a target detection model.

[0032] Step 202: Based on the current frame image, determine the current frame lane line information and the current frame detection information of the target vehicle corresponding to the current frame image.

[0033] The current frame lane line information may include straight or curved lines describing lane lines in the current frame image, or it may include pixel information belonging to lane lines in the current frame image. The specific settings can be configured according to actual needs, and this disclosure does not impose limitations. The current frame lane line information can be obtained based on a pre-trained lane line detection model or lane line semantic segmentation model. The current frame detection information of the target vehicle may include the current frame full vehicle bounding box information, the current frame rear bounding box information (for target vehicles behind the vehicle, the current frame front bounding box information), and the type of the target vehicle (referred to as the target type). The specific settings can be configured according to actual needs. The current frame detection information of the target vehicle can be obtained based on a pre-trained vehicle detection model.

[0034] In one optional embodiment, the lane line information of the current frame and the detection information of the target vehicle of the current frame can also be obtained simultaneously by the same detection model obtained through pre-training, without any specific limitation.

[0035] Step 203: Determine the physical size of the target vehicle in the current frame based on the detection information and lane line information of the current frame.

[0036] The physical size of the target vehicle in the current frame is determined based on observations of the current frame. This physical size can include at least one of the target vehicle's physical width and height. The physical size can be determined based on at least one of the following: the target vehicle's full-frame information, the rear-frame information (or front-frame information), and the target type. For example, the target vehicle's pixel width can be determined based on the rear-frame information. Combined with the lane line pixel width at the rear determined by the lane line information in the current frame and the actual physical width of the lane, the target vehicle's physical width can be determined according to the ratio of the target vehicle's width to the lane width. Alternatively, the target vehicle's pixel height can be determined by combining the full-frame information of the current frame, and the physical height can be determined based on the pixel height. The physical width and height can also be determined based on the target type and a preset correspondence between type and physical size. Multiple methods can be combined to determine the target vehicle's physical size in the current frame; the specific method is not limited.

[0037] In an optional embodiment, a target tracking algorithm can be used to establish the association between the target vehicle in the current frame and the target vehicle in the previous frame, which facilitates the filtering and optimization of the target vehicle distance in subsequent frames.

[0038] Step 204: Determine the current frame observations based on the current frame detection information and the current frame physical size.

[0039] The current frame observation measurement can include the distance to the target vehicle determined based on the current frame observation. Specifically, it can include distance determined by at least one observation method, which can include both vehicle width distance measurement and vehicle height distance measurement. Vehicle width distance measurement refers to determining the distance between the target vehicle and the vehicle itself by combining the physical width in the current frame's physical dimensions and the pixel width of the target vehicle, according to the principle of similar triangulation. The principle of vehicle height distance measurement is similar to that of vehicle width distance measurement.

[0040] For example, the current frame observation z k It can be represented as:

[0041] z k =[d wk ,d hk ] T

[0042] Where k represents the current frame as the k-th frame, d wk d represents the distance to the target vehicle in the current frame obtained from vehicle width ranging. hk This represents the distance to the target vehicle in the current frame obtained from vehicle height ranging, and T represents transpose.

[0043] Step 205: Based on the observations of the current frame and the observations of the previous frame, the Kalman filter algorithm with adaptive observation noise is used to determine the target distance of the target vehicle.

[0044] The observations in the previous frame include the observations corresponding to a first number of historical frames. These previous frame observations can be those determined and stored during the processing of historical frames, and their determination method is consistent with that of the current frame, so it will not be elaborated further here. The Kalman filter algorithm for adaptive observation noise refers to an algorithm in which the observation noise can be adaptively updated during the filtering process.

[0045] For example, the observations in the previous frame can be represented as:

[0046] z j =[d wj ,d hj ] T

[0047] Where j = kN, ..., k-1, N represents the first quantity.

[0048] In an optional embodiment, the target vehicle's state variables (previous frame posterior state variables), the previous frame posterior error covariance matrix, and the previous frame system noise matrix obtained from the previous frame filtering can be iteratively optimized according to the iterative process of the Kalman filter algorithm to obtain optimized state variables. Then, the target distance of the target vehicle in the current frame can be determined based on the optimized state variables. The target vehicle's state variable x can include at least one of the target vehicle's distance d, velocity v, and acceleration a, for example, x = [d, v, a]. T The state variables before and after filtering in each frame can be called prior state variables and posterior state variables, respectively. Prior and posterior can be represented differently; for example, "-" represents prior and "+" represents posterior. The specific representation can be set according to actual needs. For example, the posterior state variable of the target vehicle in the k-th frame... It can be represented as:

[0049]

[0050] in, This represents the posterior distance of the k-th frame. Represents the posterior velocity of the k-th frame. This represents the posterior acceleration of the k-th frame.

[0051] Prior state of the target vehicle in the kth frame It can be represented as:

[0052]

[0053] in, This represents the prior distance in the k-th frame. Represents the prior velocity of the k-th frame. This represents the prior acceleration of the k-th frame.

[0054] The target vehicle distance determination method provided in this embodiment can determine the lane line information and target vehicle detection information of the current frame based on the current frame image containing the target vehicle. Based on the current frame detection information and the current frame lane line information, the physical size of the target vehicle in the current frame is determined. Based on the current frame detection information and the current frame physical size, the observations of the current frame are determined. Then, based on the current frame observations and the previous frame observations, an adaptive observation noise Kalman filter algorithm is used to determine the target distance of the target vehicle. This achieves adaptive updating of observation noise in the filtering process, effectively improving processing efficiency, reducing the manual cost of noise parameter tuning, and reducing the workload of noise parameter tuning caused by image processing model updates, iterations, sensor upgrades, environmental changes, etc., making the obtained target vehicle distance more robust.

[0055] In an optional embodiment, the current frame lane line information and the current frame detection information of the target vehicle in step 202 can be obtained based on a pre-trained object detection model and / or semantic segmentation model. For example, the current frame image can be processed based on a pre-trained object detection model to obtain the current frame lane line information and the current frame detection information of the target vehicle. Alternatively, the current frame image can be processed based on a pre-trained lane line detection model or lane line semantic segmentation model to obtain the current frame lane line information; and the current frame image can be processed based on a pre-trained vehicle detection model or vehicle semantic segmentation model to obtain the current frame detection information of the target vehicle. The detection model and semantic segmentation model can be any implementable model, and this disclosure does not limit them.

[0056] Figure 3 This is a flowchart illustrating a method for determining the distance to a target vehicle provided in another exemplary embodiment of this disclosure.

[0057] In an optional embodiment, the current frame detection information includes the current frame full vehicle frame information, the current frame rear vehicle frame information, and the target type of the target vehicle; step 203, based on the current frame detection information and the current frame lane line information, determines the current frame physical size of the target vehicle, including:

[0058] Step 2031: Based on the lane line information and the vehicle rear frame information of the current frame, determine the lane pixel width at the vehicle rear frame of the current frame.

[0059] The width of the lane pixels in the current frame can be determined based on the relationship between the rear frame of the vehicle and the lane lines in the current frame image. For example, when the target vehicle is a vehicle traveling directly in front of the current vehicle, it can be determined based on the distance between the intersection points of the straight line containing the lower border of the rear frame information of the current frame and the lane lines on both sides. Alternatively, it can be determined in other possible ways, depending on the actual needs.

[0060] Step 2032: Based on the target type, the current frame lane pixel width, the lane physical width, the current frame full vehicle frame information, and the current frame rear vehicle frame information, determine the first physical width and the first physical height of the target vehicle.

[0061] The lane physical width can be a preset width or a lane physical width determined based on lane line information, which can be set according to actual needs. A set of physical dimensions can be determined based on the target type, and another set can be determined based on the current frame lane pixel width, lane physical width, current frame full vehicle frame information, and current frame rear vehicle frame information. These two sets of physical dimensions can be combined to determine the first physical width and first physical height of the target vehicle, which can also be set according to actual needs.

[0062] Step 2033: Use the first physical width and the first physical height as the current frame physical size of the target vehicle.

[0063] This embodiment combines the target vehicle's target type, full vehicle frame information, rear vehicle frame information, lane pixel width, and lane physical width to determine the target vehicle's physical dimensions, thereby improving the accuracy of the physical dimensions.

[0064] Figure 4 This is a flowchart illustrating step 2032 provided in an exemplary embodiment of this disclosure.

[0065] In an optional embodiment, step 2032, which determines the first physical width and first physical height of the target vehicle based on the target type, the current frame lane pixel width, the lane physical width, the current frame full vehicle frame information, and the current frame rear vehicle frame information, includes:

[0066] Step 20321: Determine the current frame pixel height of the target vehicle based on the current frame full vehicle frame information.

[0067] The current frame pixel height of the target vehicle can be determined by the pixel height of the entire vehicle frame in the current frame.

[0068] Step 20322: Determine the current frame pixel width of the target vehicle based on the current frame rear frame information.

[0069] The current frame pixel width of the target vehicle can be determined by the pixel width of the vehicle's rear frame in the current frame.

[0070] Step 20323: Determine the second physical width of the target vehicle based on the current frame pixel width, the current frame lane pixel width, and the lane physical width.

[0071] The second physical width can be determined based on the principle that the ratio of lane pixel width to lane physical width is consistent with the ratio of target vehicle pixel width to physical width.

[0072] For example, Figure 5 This is a schematic diagram illustrating the principle of determining vehicle width based on lane information, provided in an exemplary embodiment of this disclosure. At the same distance d from the vehicle, the current frame lane pixel width is represented as w. l The physical width of the lane is represented by W. l The current frame pixel width of the target vehicle is represented as w. v The physical width of the target vehicle is represented by W. v It satisfies the following relationship:

[0073]

[0074] Where f represents the focal length of the camera. Based on this, the second physical width W of the target vehicle can be obtained. v , means as follows:

[0075]

[0076] Step 20324: Determine the second physical height of the target vehicle based on the current frame pixel height, the current frame pixel width, and the second physical width.

[0077] Among them, the second physical height W h The principle for determining the second physical width is similar to that for the second physical width. For example, it can be represented as:

[0078]

[0079] Among them, w h Indicates the pixel height of the current frame.

[0080] In an alternative embodiment, the second physical height can also be determined by combining lane line information, for example, as follows:

[0081]

[0082] Step 20325: Based on the target type, determine the third physical width and third physical height of the target vehicle.

[0083] Among them, the correspondence between vehicle type and physical size can be pre-configured. Once the target type of the target vehicle is determined, the third physical width and third physical height of the target vehicle can be determined based on the target type and the correspondence between vehicle type and physical size.

[0084] Step 20326: Determine the first physical width and first physical height of the target vehicle based on the second physical width, second physical height, third physical width, and third physical height.

[0085] Among them, the two sets of physical dimensions can be merged based on preset fusion rules to obtain the first physical width and the first physical height of the target vehicle. The specific fusion rules can be set according to actual needs, such as taking the average value, and there is no specific limitation.

[0086] This embodiment can determine the physical size of the target vehicle based on the observations of the current frame, which is used to determine the observations of the current frame and provide effective observation information for filtering optimization of the target vehicle's distance.

[0087] In an optional embodiment, step 204, determining the current frame observations based on the current frame detection information and the current frame physical size, includes:

[0088] Step 2041: Determine the first distance to the target vehicle based on the first physical width, the current frame pixel width, and the camera focal length corresponding to the current frame image.

[0089] The first distance is the distance between the target vehicle and the vehicle itself, determined based on the vehicle width distance measurement principle.

[0090] For example, Figure 6 This is a schematic diagram illustrating the principle of vehicle width distance measurement provided in an exemplary embodiment of this disclosure. Based on the relationship of similar triangles, the distance d of the target vehicle, pixel width, physical width, and focal length f have the following relationship:

[0091]

[0092] Based on this, the first distance d wk It can be represented as follows:

[0093]

[0094] Where f represents the camera (camera) focal length corresponding to the current frame image, W vk w represents the first physical width. vk Indicates the pixel width of the current frame.

[0095] Step 2042: Determine the second distance to the target vehicle based on the first physical height, the current frame pixel height, and the camera focal length.

[0096] The second distance is the distance between the target vehicle and the vehicle itself, determined based on vehicle height measurement. The determination principle of the second distance is similar to that of the first distance, based on the relationship between similar triangles.

[0097] For example, the second distance can be represented as follows:

[0098]

[0099] Among them, H vkh represents the first physical height. vk Indicates the pixel height of the current frame.

[0100] Step 2043: Determine the current frame observation based on the first distance and / or the second distance.

[0101] The current frame observation can be determined based on at least one of the first distance and the second distance. For example, the current frame observation can be represented as z. k =[d wk ] or z k =[d hk To further improve the accuracy of the target vehicle's distance, the current frame observation can be determined based on a first distance and a second distance. For example, the current frame observation can be represented as z. k =[d wk ,d hk ] T The specific settings can be customized according to actual needs.

[0102] This embodiment uses the distance observed in the current frame as the observation quantity to filter and update the state of the target vehicle in order to obtain a more accurate distance.

[0103] In an optional embodiment, step 205 may specifically include the following steps:

[0104] Step 2051: Determine the prior state of the current frame based on the posterior state of the previous frame and the state transition matrix.

[0105] Here, the posterior state quantity of the previous frame is the state quantity obtained after filtering based on the prior state quantity of the previous frame. The state transition matrix is ​​the state change matrix of the target vehicle in adjacent frames determined based on the motion model.

[0106] For example, based on the motion model, the following state transition function exists:

[0107] x k =f(x) k-1 )

[0108] Where, x k x represents the state variable of the current frame. k =[d k ,v k ,a k ] T x k-1 This represents the state variables of the previous frame. Expanding the state variables, the state transition function is as follows:

[0109]

[0110] Where t represents the time interval between adjacent frames. Based on this state transition function, the state transition matrix F is determined such that:

[0111] x k =Fx k-1

[0112] Based on this, the prior state quantity of the current frame It can be represented as:

[0113]

[0114] in, This represents the posterior state of the target vehicle in the (k-1)th frame. x0 represents the initial state variable, which can be obtained based on initialization.

[0115] Step 2052: Determine the prior error covariance matrix of the current frame based on the posterior error covariance matrix, state transition matrix, and system noise matrix of the previous frame.

[0116] The error covariance matrix is ​​a covariance matrix that characterizes the uncertainty of the state. It is iteratively updated during the filtering process. For each frame, the error covariance matrix before filtering is called the prior error covariance matrix, and the one after filtering is called the posterior error covariance matrix. The prior error covariance matrix of each frame is determined based on the posterior error covariance matrix of the previous frame. The system noise matrix is ​​a matrix that characterizes the system noise. It can be obtained through initialization, or it can be continuously adjusted and updated based on the initial system noise matrix during the filtering process; the specific method is not limited.

[0117] For example, the prior error covariance matrix of the current frame It can be represented as follows:

[0118]

[0119] Where F represents the state transition matrix, Let Q represent the posterior error covariance matrix of the previous frame. k-1 Q represents the system noise matrix of the previous frame. k-1 =Q0, where Q0 represents the initialization of the system noise matrix, or Q k-1 This is the matrix adjusted based on experience. P0 represents the initial error covariance matrix.

[0120] Step 2053: Determine the current frame information based on the prior state of the current frame, the prior state of the previous frame, the observation of the current frame, and the observation of the previous frame.

[0121] The prior state quantities in the previous frame include the prior state quantities corresponding to the first number of historical frames. The determination method for the prior state quantities in the previous frame is similar to that for the current frame. The observations in the previous frame are the observations obtained from observations in each historical frame. The current frame information represents the difference between the observations and the predicted observations based on the prior state quantities.

[0122] In an optional embodiment, the predicted observations of each frame (including the current frame and a first number of historical frames) can be predicted based on the prior state quantities of the current frame and the prior state quantities of each historical frame, and then the current frame information can be determined based on the observations and predicted observations of each frame.

[0123] Step 2054: Determine the observation noise of the current frame based on the current frame information, the observation matrix, and the current frame prior error covariance matrix.

[0124] The observation matrix is ​​determined by the measurement equation z = h(x) based on the state variables to predict the observations, and can be represented as H. The observation noise of the current frame can be determined based on the adaptive update rule.

[0125] For example, the current frame observation noise R k It can be represented as:

[0126]

[0127] Among them, C vk Indicates the current frame information. This represents the prior error covariance matrix of the current frame.

[0128] Step 2055: Determine the target distance based on the current frame observation noise, the current frame prior error covariance matrix, the observation matrix, the current frame observations, and the current frame prior state variables.

[0129] Specifically, the posterior state of the current frame can be determined first based on the state update rule, and the target distance can be determined based on the posterior state of the current frame. Representation based on the posterior state of the current frame. visible That is, the target distance.

[0130] This embodiment achieves adaptive updating of observation noise based on the innovation and prior error covariance matrix, which greatly reduces the manual cost of noise parameter tuning, improves efficiency, and enhances the robustness of ranging results.

[0131] In an optional embodiment, step 2053, determining the current frame information based on the current frame prior state quantity, the previous frame prior state quantity, the current frame observation, and the previous frame observation, includes:

[0132] Based on the prior state variables and observation matrix of the current frame, determine the predicted observations of the current frame; based on the prior state variables and observation matrix of the previous frame, determine the predicted observations of the previous frame; based on the predicted observations of the current frame, the current frame observations, the predicted observations of the previous frame, and the previous frame observations, determine the information of the current frame.

[0133] Among them, the predicted observations for the current frame are predicted observations obtained based on the prior state variables and the observation matrix of the current frame. It can be represented as follows:

[0134]

[0135] Where H represents the observation matrix, This represents the prior state quantity of the current frame.

[0136] Similarly, the predicted observations in the previous frame can be expressed as:

[0137]

[0138] Where j = kN, ..., k-1, N represents the first quantity.

[0139] Current frame information C vk It can be represented as follows:

[0140]

[0141] in, z j Let N represent the observations in the j-th frame, and let N represent the first quantity. The first quantity can vary depending on the specific situation. For example, when k is small, the first quantity is k; when k reaches a certain threshold (e.g., M), the first quantity becomes the threshold M.

[0142] In an optional embodiment, step 2055, which determines the target distance based on the current frame observation noise, the current frame prior error covariance matrix, the observation matrix, the current frame observations, and the current frame prior state quantities, includes:

[0143] Based on the current frame observation noise, the current frame prior error covariance matrix, and the observation matrix, determine the current frame Kalman gain; based on the current frame Kalman gain, the current frame observations, the current frame predicted observations, and the current frame prior state variables, determine the current frame posterior state variables; based on the current frame posterior state variables, determine the target distance.

[0144] The Kalman gain for the current frame can be determined based on the Kalman gain update formula. The Kalman gain update formula is expressed as follows:

[0145]

[0146] Among them, K k This indicates the Kalman gain for the current frame. For the meanings of other symbols, please refer to the previous content.

[0147] The update of the state variables needs to be implemented based on Kalman gain. The posterior state variable of the current frame can be represented as:

[0148]

[0149] After obtaining the posterior state of the current frame, the distance state can be extracted from it as the target distance.

[0150] This embodiment updates the state variables based on adaptively updated observation noise to obtain the posterior state variables of the current frame, thereby improving the robustness of the posterior state variables.

[0151] In an optional embodiment, after determining the Kalman gain of the current frame based on the current frame observation noise, the current frame prior error covariance matrix, and the observation matrix, the method further includes:

[0152] The posterior error covariance matrix of the current frame is determined based on the prior error covariance matrix, the Kalman gain of the current frame, and the observation matrix.

[0153] The formula for updating the error covariance matrix is ​​as follows:

[0154]

[0155] in, Let I represent the posterior error covariance matrix of the current frame, and let I represent the identity matrix.

[0156] This embodiment updates the error covariance matrix based on the Kalman gain of the current frame, providing accurate and effective data for filtering in the next frame.

[0157] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0158] Any of the target vehicle distance determination methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the target vehicle distance determination methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the target vehicle distance determination methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0159] Exemplary device

[0160] Figure 7This is a schematic diagram of the structure of a target vehicle distance determination device provided in an exemplary embodiment of this disclosure. The device of this embodiment can be used to implement corresponding method embodiments of this disclosure, such as... Figure 7 The device shown includes: a first acquisition module 501, a first processing module 502, a second processing module 503, a third processing module 504, and a fourth processing module 505.

[0161] The first acquisition module 501 is used to acquire the current frame image containing the target vehicle.

[0162] The first processing module 502 is used to determine the lane line information and the target vehicle detection information of the current frame corresponding to the current frame image based on the current frame image.

[0163] The second processing module 503 is used to determine the physical size of the target vehicle in the current frame based on the detection information and lane line information of the current frame.

[0164] The third processing module 504 is used to determine the current frame observations based on the current frame detection information and the current frame physical size.

[0165] The fourth processing module 505 is used to determine the target distance between the target vehicle and the current vehicle based on the current frame observations and the previous frame observations, using an adaptive observation noise Kalman filter algorithm. The previous frame observations include the observations corresponding to a first number of historical frames.

[0166] Figure 8 This is a schematic diagram of the structure of a target vehicle distance determination device provided in another exemplary embodiment of this disclosure.

[0167] In an optional embodiment, the current frame detection information includes the current frame full vehicle frame information, the current frame rear vehicle frame information, and the target type of the target vehicle; the second processing module 503 includes: a first determining unit 5031, a second determining unit 5032, and a third determining unit 5033.

[0168] The first determining unit 5031 is used to determine the current frame lane pixel width at the rear frame based on the current frame lane line information and the current frame rear frame information.

[0169] The second determining unit 5032 is used to determine the first physical width and the first physical height of the target vehicle based on the target type, the current frame lane pixel width, the lane physical width, the current frame full vehicle frame information, and the current frame vehicle rear frame information.

[0170] The third determining unit 5033 is used to use the first physical width and the first physical height as the current frame physical size of the target vehicle.

[0171] In an optional embodiment, the second determining unit 5032 is specifically used for:

[0172] Based on the current frame full vehicle frame information, determine the current frame pixel height of the target vehicle; based on the current frame rear frame information, determine the current frame pixel width of the target vehicle; based on the current frame pixel width, the current frame lane pixel width, and the lane physical width, determine the second physical width of the target vehicle; based on the current frame pixel height, the current frame pixel width, and the second physical width, determine the second physical height of the target vehicle; based on the target type, determine the third physical width and third physical height of the target vehicle; based on the second physical width, second physical height, third physical width, and third physical height, determine the first physical width and first physical height of the target vehicle.

[0173] In an optional embodiment, the third processing module 504 includes a fourth determining unit 5041, a fifth determining unit 5042, and a sixth determining unit 5043.

[0174] The fourth determining unit 5041 is used to determine the first distance of the target vehicle based on the first physical width, the current frame pixel width and the camera focal length corresponding to the current frame image.

[0175] The fifth determining unit 5042 is used to determine the second distance of the target vehicle based on the first physical height, the current frame pixel height and the camera focal length.

[0176] The sixth determining unit 5043 is used to determine the current frame observation based on the first distance and / or the second distance.

[0177] In an optional embodiment, the fourth processing module 505 includes: a first processing unit 5051, a second processing unit 5052, a third processing unit 5053, a fourth processing unit 5054, and a fifth processing unit 5055.

[0178] The first processing unit 5051 is used to determine the prior state of the current frame based on the posterior state of the previous frame and the state transition matrix.

[0179] Among them, the posterior state quantity of the previous frame is the state quantity obtained after filtering based on the prior state quantity of the previous frame.

[0180] The second processing unit 5052 is used to determine the prior error covariance matrix of the current frame based on the posterior error covariance matrix, the state transition matrix, and the system noise matrix of the previous frame.

[0181] The third processing unit 5053 is used to determine the current frame information based on the prior state quantity of the current frame, the prior state quantity of the previous frame, the observation of the current frame, and the observation of the previous frame.

[0182] Among them, the prior state quantity of the previous frame includes the prior state quantity corresponding to the first number of historical frames.

[0183] The fourth processing unit 5054 is used to determine the observation noise of the current frame based on the current frame information, the observation matrix, and the current frame prior error covariance matrix.

[0184] The fifth processing unit 5055 is used to determine the target distance based on the current frame observation noise, the current frame prior error covariance matrix, the observation matrix, the current frame observations, and the current frame prior state variables.

[0185] In an optional embodiment, the third processing unit 5053 is specifically used for:

[0186] Based on the prior state variables and observation matrix of the current frame, determine the predicted observations of the current frame; based on the prior state variables and observation matrix of the previous frame, determine the predicted observations of the previous frame; based on the predicted observations of the current frame, the current frame observations, the predicted observations of the previous frame, and the previous frame observations, determine the information of the current frame.

[0187] In an optional embodiment, the fifth processing unit 5055 is specifically used for:

[0188] Based on the current frame observation noise, the current frame prior error covariance matrix, and the observation matrix, determine the current frame Kalman gain; based on the current frame Kalman gain, the current frame observations, the current frame predicted observations, and the current frame prior state variables, determine the current frame posterior state variables; based on the current frame posterior state variables, determine the target distance.

[0189] In an optional embodiment, the fifth processing unit 5055 is further configured to:

[0190] The posterior error covariance matrix of the current frame is determined based on the prior error covariance matrix, the Kalman gain of the current frame, and the observation matrix.

[0191] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0192] Exemplary electronic devices

[0193] Figure 9 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure, including at least one processor 11 and a memory 12.

[0194] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0195] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute one or more computer program instructions to implement the methods and / or other desired functions of the various embodiments of this disclosure described above.

[0196] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0197] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0198] The output device 14 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0199] Of course, for the sake of simplicity, Figure 9 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0200] Exemplary computer program products and computer-readable storage media

[0201] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0202] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0203] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0204] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0205] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0206] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for determining the distance to a target vehicle, comprising: Obtain the current frame image containing the target vehicle; Based on the current frame image, determine the current frame lane line information corresponding to the current frame image and the current frame detection information of the target vehicle; Based on the current frame detection information and the current frame lane line information, the current frame physical size of the target vehicle is determined; Based on the current frame detection information and the current frame physical size, determine the current frame observations; Based on the current frame observations and previous frame observations, an adaptive observation noise Kalman filter algorithm is used to determine the target distance of the target vehicle. The previous frame observations include observations corresponding to a first number of historical frames. The adaptive observation noise Kalman filter algorithm is an algorithm that adaptively updates the observation noise during the Kalman filtering process. The step of determining the target distance of the target vehicle using an adaptive observation noise Kalman filter algorithm based on the current frame observations and the previous frame observations includes: Based on the posterior state quantity and state transition matrix of the previous frame, the prior state quantity of the current frame is determined. The posterior state quantity of the previous frame is the state quantity obtained after filtering based on the prior state quantity of the previous frame. Based on the posterior error covariance matrix of the previous frame, the state transition matrix, and the system noise matrix of the previous frame, the prior error covariance matrix of the current frame is determined. Based on the prior state of the current frame, the prior state of the previous frame, the observation of the current frame, and the observation of the previous frame, the current frame information is determined, wherein the prior state of the previous frame includes the prior state of the first number of historical frames respectively. Based on the current frame information, the observation matrix, and the current frame prior error covariance matrix, the current frame observation noise is determined; The target distance is determined based on the current frame observation noise, the current frame prior error covariance matrix, the observation matrix, the current frame observations, and the current frame prior state variables.

2. The method according to claim 1, wherein, The step of determining the current frame information based on the current frame prior state quantity, the previous frame prior state quantity, the current frame observation, and the previous frame observation includes: Based on the prior state variables of the current frame and the observation matrix, determine the predicted observations of the current frame; Based on the prior state variables in the previous frame and the observation matrix, determine the predicted observations in the previous frame; The current frame information is determined based on the current frame prediction observation, the current frame observation, the previous frame prediction observation, and the previous frame observation.

3. The method according to claim 2, wherein, The step of determining the target distance based on the current frame observation noise, the current frame prior error covariance matrix, the observation matrix, the current frame observations, and the current frame prior state quantities includes: The Kalman gain of the current frame is determined based on the current frame observation noise, the current frame prior error covariance matrix, and the observation matrix. Based on the current frame Kalman gain, the current frame observations, the current frame prediction observations, and the current frame prior state, determine the current frame posterior state. The target distance is determined based on the current frame posterior state value.

4. The method according to claim 3, wherein, After determining the Kalman gain of the current frame based on the current frame observation noise, the current frame prior error covariance matrix, and the observation matrix, the method further includes: The posterior error covariance matrix of the current frame is determined based on the prior error covariance matrix of the current frame, the Kalman gain of the current frame, and the observation matrix.

5. The method according to claim 1, wherein, The current frame detection information includes the current frame full vehicle frame information, the current frame rear vehicle frame information, and the target type of the target vehicle; Determining the physical size of the target vehicle in the current frame based on the current frame detection information and the current frame lane line information includes: Based on the current frame lane line information and the current frame vehicle rear frame information, determine the current frame lane pixel width at the vehicle rear frame; Based on the target type, the current frame lane pixel width, the lane physical width, the current frame full vehicle frame information, and the current frame rear vehicle frame information, the first physical width and the first physical height of the target vehicle are determined; The first physical width and the first physical height are used as the current frame physical size of the target vehicle.

6. The method according to claim 5, wherein, The step of determining the first physical width and first physical height of the target vehicle based on the target type, the current frame lane pixel width, the lane physical width, the current frame full vehicle frame information, and the current frame rear vehicle frame information includes: Based on the current frame full vehicle frame information, determine the current frame pixel height of the target vehicle; Based on the current frame rear frame information, determine the current frame pixel width of the target vehicle; The second physical width of the target vehicle is determined based on the current frame pixel width, the current frame lane pixel width, and the lane physical width; The second physical height of the target vehicle is determined based on the current frame pixel height, the current frame pixel width, and the second physical width. Based on the target type, determine the third physical width and third physical height of the target vehicle; The first physical width and the first physical height of the target vehicle are determined based on the second physical width, the second physical height, the third physical width, and the third physical height.

7. The method according to claim 6, wherein, The step of determining the current frame observations based on the current frame detection information and the current frame physical size includes: Based on the first physical width, the current frame pixel width, and the camera focal length corresponding to the current frame image, the first distance to the target vehicle is determined; Based on the first physical height, the current frame pixel height, and the camera focal length, the second distance to the target vehicle is determined; The current frame observation is determined based on the first distance and / or the second distance.

8. A device for determining the distance to a target vehicle, comprising: The first acquisition module is used to acquire the current frame image containing the target vehicle; The first processing module is used to determine, based on the current frame image, the current frame lane line information corresponding to the current frame image and the current frame detection information of the target vehicle; The second processing module is used to determine the physical size of the target vehicle in the current frame based on the current frame detection information and the current frame lane line information. The third processing module is used to determine the current frame observation based on the current frame detection information and the current frame physical size; The fourth processing module is used to determine the target distance between the target vehicle and the current vehicle based on the current frame observations and the previous frame observations, using an adaptive observation noise Kalman filter algorithm. The previous frame observations include observations corresponding to a first number of historical frames. The adaptive observation noise Kalman filter algorithm is an algorithm that adaptively updates the observation noise during the Kalman filtering process. The fourth processing module includes: The first processing unit is used to determine the prior state of the current frame based on the posterior state of the previous frame and the state transition matrix. The posterior state of the previous frame is a state obtained by filtering the prior state of the previous frame. The second processing unit is used to determine the prior error covariance matrix of the current frame based on the posterior error covariance matrix of the previous frame, the state transition matrix, and the system noise matrix of the previous frame. The third processing unit is used to determine the current frame information based on the current frame prior state quantity, the previous frame prior state quantity, the current frame observation quantity and the previous frame observation quantity, wherein the previous frame prior state quantity includes the prior state quantities corresponding to the first number of historical frames respectively. The fourth processing unit is used to determine the observation noise of the current frame based on the current frame information, the observation matrix, and the current frame prior error covariance matrix; The fifth processing unit is used to determine the target distance based on the current frame observation noise, the current frame prior error covariance matrix, the observation matrix, the current frame observations, and the current frame prior state quantities.

9. A computer-readable storage medium storing a computer program for performing the method for determining the distance to a target vehicle as described in any one of claims 1-7.

10. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for determining the distance to the target vehicle as described in any one of claims 1-7.

Images