A target ranging method, apparatus, electronic device and storage medium

Abstract

This application relates to the field of vehicle ranging technology, and provides a target ranging method, device, electronic device, and storage medium. The method determines the current state of the target to be ranging and selects the method for calculating the distance between the target and the vehicle based on the current state of the target, thereby accurately determining the distance between the target and the vehicle. This improves the accuracy of monocular ranging and avoids the problem of poor ranging performance when using a single imaging device to measure the distance to the target in related technologies.

Description

Technical Field

[0001] This application relates to the field of vehicle ranging technology, and in particular to a target ranging method, device, electronic device and storage medium. Background Technology

[0002] With the growth of material culture, driving has become the preferred mode of transportation for most people, and vehicles are becoming increasingly intelligent. Electronic rearview mirrors are a type of automotive technology that replaces traditional optical rearview mirrors with electronic displays. They capture images of the rear using a camera and display them in real time on a screen, providing the driver with a rear view. This system often also integrates distance measurement functions to help drivers understand the distance to objects behind them, improving driving safety.

[0003] Distance measurement can improve vehicle safety, such as collision avoidance: it helps drivers avoid collisions with objects behind them when reversing, changing lanes, and merging. Parking assistance: when parking in tight spaces, distance measurement provides distance information to prevent the rear of the vehicle from colliding with obstacles. It can also improve the driving experience, such as reducing blind spots.

[0004] In existing technologies, when using a single imaging device to measure the distance to a target, the ranging effect is poor. Summary of the Invention

[0005] In view of this, embodiments of this application provide a target ranging method, apparatus, electronic device, and storage medium to solve the problem of poor ranging effect when using an imaging device to measure the distance of a target in the prior art.

[0006] A first aspect of this application provides a target ranging method, the method comprising: acquiring the current state of a vehicle; if the current state of the vehicle is in motion, acquiring a first observation image containing a target to be measured, and identifying the current state of the target based on the observation coordinates corresponding to the target in the first observation image; when the current state of the target is in motion, calculating the distance between the target and the vehicle based on the detection box corresponding to the target in the first observation image; when the current state of the target is stationary, acquiring a second observation image containing the target, and calculating the distance between the target and the vehicle based on the observation coordinates corresponding to the target in the first observation image and the observation coordinates corresponding to the target in the second observation image.

[0007] A second aspect of this application provides a target ranging device, comprising: an identification module, configured to acquire the current state of a vehicle; if the current state of the vehicle is in motion, acquiring a first observation image containing a target to be measured, and identifying the current state of the target based on the observation coordinates corresponding to the target in the first observation image; a calculation module, configured to calculate the distance between the target and the vehicle based on a detection box corresponding to the target in the first observation image when the current state of the target is in motion; and the calculation module further configured to acquire a second observation image containing the target when the current state of the target is stationary, and calculate the distance between the target and the vehicle based on the observation coordinates corresponding to the target in the first observation image and the observation coordinates corresponding to the target in the second observation image.

[0008] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0009] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0010] The beneficial effects of this application embodiment compared with the prior art are as follows: The method in this application embodiment obtains the current state of the vehicle. If the current state of the vehicle is in motion, a first observation image containing the target to be measured is obtained, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target to be measured in the first observation image. If the current state of the target to be measured is in motion, the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image. If the current state of the target to be measured is stationary, a second observation image containing the target to be measured is obtained, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image. This method determines the current state of the target to be measured and selects the method for calculating the distance between the target to be measured and the vehicle based on the current state of the target to be measured, thereby accurately determining the distance between the target to be measured and the vehicle, improving the accuracy of monocular ranging, and avoiding the problem of poor ranging effect when using a single shooting device to measure the distance of the target to be measured in related technologies. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of a target ranging method provided in an embodiment of this application;

[0013] Figure 2 This is a schematic flowchart of another target ranging method provided in an embodiment of this application;

[0014] Figure 3 This is a flowchart illustrating another target ranging method provided in an embodiment of this application;

[0015] Figure 4 This is a flowchart illustrating another target ranging method provided in the embodiments of this application;

[0016] Figure 5 This is an observation constraint diagram of an imaging device for observing a target to be ranged, provided in an embodiment of this application;

[0017] Figure 6 This is a triangulation diagram provided in an embodiment of this application;

[0018] Figure 7 This is a schematic flowchart of another target ranging method provided in the embodiments of this application;

[0019] Figure 8 This is a flowchart illustrating another optional target ranging method provided in the embodiments of this application;

[0020] Figure 9 This is a basic schematic diagram of a coordinate transformation provided in an embodiment of this application;

[0021] Figure 10 This is a flowchart illustrating another optional target ranging method provided in the embodiments of this application;

[0022] Figure 11 This is a flowchart illustrating another optional target ranging method provided in the embodiments of this application;

[0023] Figure 12 This is a schematic diagram of the structure of a target ranging device provided in an embodiment of this application;

[0024] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0026] A target ranging method and apparatus according to embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0027] Figure 1 This application provides a target ranging method, such as... Figure 1 As shown, the method includes:

[0028] S101. Obtain the current state of the vehicle. If the current state of the vehicle is in motion, obtain a first observation image containing the target to be measured, and identify the current state of the target to be measured based on the observation coordinates corresponding to the target in the first observation image.

[0029] S102. If the current state of the target to be measured is in motion, then calculate the distance between the target to be measured and the vehicle based on the detection box corresponding to the target in the first observation image.

[0030] S103. If the current state of the target to be measured is stationary, then acquire a second observation image containing the target to be measured, and calculate the distance between the target to be measured and the vehicle based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image.

[0031] It is understood that the target ranging method provided in this example is applied to vehicles, including vehicles with autonomous or intelligent driving capabilities (including passenger vehicles (e.g., cars, buses, coaches, minibuses, etc.), cargo vehicles (e.g., ordinary trucks, box trucks, trailer trucks, enclosed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, special structure trucks), special vehicles (e.g., logistics delivery vehicles, automated guided vehicles (AGVs), patrol vehicles, cranes, excavators, bulldozers, loaders, road rollers, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor scrubbers, water sprinkler trucks, sweeping robots, food delivery robots, shopping guide robots, lawnmowers, golf carts, etc.), recreational vehicles (e.g., amusement vehicles, amusement park autonomous driving devices, balance bikes, etc.), and rescue vehicles (e.g., fire trucks, ambulances, power repair vehicles, engineering emergency rescue vehicles, etc.)).

[0032] Specifically, the vehicle is equipped with at least one camera device capable of taking pictures, and the target to be measured is the target captured by the camera device.

[0033] In step S101, obtaining the current state of the vehicle includes: obtaining the current state of the vehicle through sensors installed on the vehicle, including but not limited to an inertial measurement unit (IMU) and a wheel speed odometer; by using the data output by the IMU and the wheel speed odometer, it can be determined whether the current state of the vehicle is in motion or stationary.

[0034] If the vehicle is currently in motion, a first observation image containing the target to be measured is acquired, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target in the first observation image. The first observation image is the initial image of the target to be measured captured by the imaging device. After acquiring the initial image, this example further calculates the observation coordinates of the target to be detected in the first observation image, and finally identifies the current state of the target to be measured based on the observation coordinates.

[0035] For example, after acquiring the first observation image, target recognition is performed on the first observation image to obtain the target to be measured and the bounding box corresponding to the target to be measured is determined. Then, a coordinate system is constructed with the boundary of the first observation image (for example, with the upper left corner of the first observation image as the origin, and the width and height of the first observation image as coordinate axes, respectively, to realize the construction of a coordinate system based on the first observation image). The position of the bounding box corresponding to the target to be measured in the coordinate system is calculated to obtain the observation coordinates of the target to be measured in the first observation image. Finally, the current state of the target to be measured is identified based on the observation coordinates corresponding to the target to be measured in the first observation image.

[0036] It is understandable that the specific method for identifying the current state of the target based on the observation coordinates corresponding to the target in the first observation image will be explained in detail later, and will not be repeated here.

[0037] It is understandable that the current state of the target to be measured is either in motion or stationary. This example determines the current state of the target and the specific ranging method based on that state, thereby accurately determining the distance between the target and the vehicle. Specifically, if the target is in motion, the distance between the target and the vehicle is calculated based on the detection box corresponding to the target in the first observation image. If the target is stationary, a second observation image containing the target is obtained, and the distance between the target and the vehicle is calculated based on the observation coordinates of the target in the first and second observation images.

[0038] It is understandable that when the vehicle is in motion and the target to be measured is stationary, the camera on the vehicle will move with the vehicle, while the absolute position of the target will not change. Therefore, the camera on the vehicle will be in different positions to take pictures of the target, thereby obtaining the second observation image. The positional changes of the target in the first and second observation images are related to the movement of the vehicle. Therefore, this example calculates the distance between the target and the vehicle based on the observation coordinates of the target in the first and second observation images, thus accurately determining the distance between the target and the vehicle.

[0039] When the vehicle is in motion and the target to be measured is also in motion, the camera on the vehicle moves with it, and the position of the target changes. Therefore, the observation image captured in one frame is not correlated with the target in subsequent observation images. It is impossible to correlate the first and second observation images to calculate the distance between the target and the vehicle. Thus, this example directly calculates the distance between the target and the vehicle based on the detection box corresponding to the target in the first observation image, thereby accurately determining the distance between the target and the vehicle.

[0040] It is understood that the second observation image is another image containing the target to be measured after the shooting device takes the first observation image. It is understood that the shooting interval between the second observation image and the first observation image does not exceed a preset time threshold. For example, if the time threshold is 10 frames, then after the shooting device takes the first observation image, the second observation image is another image containing the target to be measured taken by the shooting device within 10 frames.

[0041] According to the technical solution provided in the embodiments of this application, the current state of the vehicle is obtained. If the current state of the vehicle is in motion, a first observation image containing the target to be measured is obtained, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target in the first observation image. If the current state of the target to be measured is in motion, the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target in the first observation image. If the current state of the target to be measured is stationary, a second observation image containing the target to be measured is obtained, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target in the first observation image and the observation coordinates corresponding to the target in the second observation image. This method determines the current state of the target to be measured and selects the method for calculating the distance between the target to be measured and the vehicle based on the current state of the target to be measured, thereby accurately determining the distance between the target to be measured and the vehicle, improving the accuracy of monocular ranging, and avoiding the problem of poor ranging effect when using a single shooting device to measure the distance of the target in related technologies.

[0042] In some embodiments, such as Figure 2 As shown, based on the observation coordinates corresponding to the target to be ranged in the first observation image, the current state of the target to be ranged is identified, including:

[0043] S201. Obtain the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter of the shooting device when capturing the first observation image.

[0044] S202. Construct a first feature matrix based on the observation coordinates, shooting parameters and first pose parameters of the target to be ranged in the first observation image;

[0045] S203. Identify the current state of the target to be measured based on the first feature matrix.

[0046] The shooting parameters corresponding to the aforementioned shooting device characterize the internal geometry and optical properties of the shooting device. These shooting parameters include, but are not limited to, the focal length of the shooting device on the x-axis and y-axis, and the coordinates of the principal point of the shooting device on the image plane.

[0047] The first pose parameter when the above-mentioned imaging device captures the first observation image represents the pose transformation of the imaging device; the first pose parameter includes, but is not limited to, the rotation matrix and translation vector of the imaging device. It is understood that the first pose parameter is obtained by the wheel speed odometer of the vehicle.

[0048] After obtaining the above-mentioned shooting parameters and first pose parameters, this example will further construct a first feature matrix based on the observation coordinates, shooting parameters and first pose parameters of the target to be ranged in the first observation image; wherein, the first feature matrix is ​​associated with the coordinates of the target to be ranged in the vehicle coordinate system.

[0049] It is understandable that the coordinates of the target to be ranged in the vehicle coordinate system are related to the observed coordinates of the target to be ranged in the first observation image. Specifically, the observed coordinates of the target to be ranged in the first observation image are obtained by mapping the coordinates of the target to be ranged in the vehicle coordinate system.

[0050] Continuing the previous example, let the shooting parameters be denoted as K, the first pose parameter as [R t], and the X-axis coordinate of the target to be ranged in the first observation image as u1 and the Y-axis coordinate as v1. By subtracting the scalar from the homogeneous coordinate representation of the above observation coordinates, we obtain the vector representation of the observation coordinates of the target to be ranged in the first observation image. The target to be ranged is denoted as X (X-axis), Y (Y-axis), and Z (Z-axis) in the vehicle coordinate system. After decomposing the scalar values ​​from the homogeneous coordinate representation of the target in the vehicle coordinate system, the resulting vector representation of the target's coordinates in the vehicle coordinate system is: The observed coordinates of the target to be measured in the first observation image can be represented by the following equation:

[0051]

[0052] Where K is the shooting parameter, [R t] is the first pose parameter, and s is the scaling factor (obtained according to the camera parameters). The above equation can be used to map the coordinates of the target to be ranged in the vehicle coordinate system and obtain the observed coordinates of the target in the first observation image.

[0053] The first projection matrix M is calculated based on the above shooting parameters and the first pose parameter. 3×4 =K*[R t], then the above equation (1-1) can be simplified to the following equation:

[0054]

[0055] Among them, the above M 3×4 If represented as a 3x4 matrix, then the first projection matrix can be represented as: The above Let P′ be the denoted P′. Then the above equation (1-2) can be simplified to the following equation:

[0056]

[0057] It is understandable that in equation (1-1), K represents the imaging device, and [Rt] represents the first attitude parameter obtained through the wheel speed odometer. If the two vectors can be written as... This can be represented as two parallel vectors. When two vectors are parallel, their cross product is 0. Therefore, equation 1-3 can be transformed to obtain the following equation:

[0058] u1*M2*P′-M0*P′=0;

[0059] v1*M2*P′-M1*P′=0; (1-4).

[0060] Further simplification of the above equation yields:

[0061]

[0062] The above That is, the first feature matrix is ​​constructed based on the observation coordinates, shooting parameters and first pose parameters of the target to be ranged in the first observation image.

[0063] Subsequently, the current state of the target to be measured can be identified using the first feature matrix mentioned above, thereby accurately obtaining the current state of the target to be measured.

[0064] According to the technical solution provided in the embodiments of this application, the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter of the shooting device when capturing the first observation image are obtained; a first feature matrix is ​​constructed based on the observation coordinates, shooting parameters and the first pose parameter of the target to be measured in the first observation image; the current state of the target to be measured is identified based on the first feature matrix. In this way, the current state of the target to be measured is accurately obtained based on the observation coordinates, shooting parameters and the first pose parameter of the target to be measured in the first observation image.

[0065] In some embodiments, such as Figure 3 As shown, based on the first feature matrix, the current state of the target to be ranged is identified, including:

[0066] S301. Decompose the first characteristic matrix to obtain a diagonal matrix;

[0067] S302. Obtain the diagonal elements in the diagonal matrix and compare the values ​​of the diagonal elements with a preset element threshold.

[0068] S303. If the value of the diagonal element is greater than the element threshold, then the current state of the target to be measured is determined to be a motion state.

[0069] S304. If the value of the diagonal element is less than or equal to the element threshold, then the current state of the target to be measured is determined to be a stationary state.

[0070] It is understandable that the target to be ranged is currently stationary. Therefore, the vector P′ of the target's coordinates in the vehicle coordinate system can be calculated using the above equation (1-5). It can be solved; conversely, if the current state of the target to be measured is in motion, then the vector P′ of the target's coordinates in the vehicle coordinate system cannot be calculated by the above equation (1-5).

[0071] Continuing from the previous example, this example takes the first characteristic matrix as A. Then, the above equation (1-5) can be simplified to A*P′=0. According to equations (1-1) to (1-5) above, A is an M-row, four-column matrix. Using singular value decomposition (SVD), the above matrix A is decomposed into the product of three matrices, as follows:

[0072] A=UΣV T ;

[0073] Where U is an m×m orthogonal matrix, Σ is an m×4 diagonal matrix, and V is an n×n orthogonal matrix. In SVD decomposition, the diagonal elements of the diagonal matrix Σ are singular values. If the singular value is less than or equal to the element threshold, then the equation A*P′=0 has a solution, and the current state of the target to be measured is determined to be a moving state. It can be understood that the above singular value tends to zero, but is not equal to 0, and is greater than 0; conversely, if the singular value is greater than the element threshold, then the equation A*P′=0 is a solution, and the current state of the target to be measured is determined to be a stationary state.

[0074] It is understood that the above-mentioned element thresholds are determined based on historical data, and the above-mentioned element thresholds can also be a value set by relevant personnel according to actual needs; preferably, the value range of the above-mentioned element thresholds is 10. -6 Or 10 -9 .

[0075] According to the technical solution provided in the embodiments of this application, the first feature matrix is ​​decomposed to obtain a diagonal matrix; the diagonal elements in the diagonal matrix are obtained, and the values ​​of the diagonal elements are compared with a preset element threshold; if the value of the diagonal element is greater than the element threshold, the current state of the target to be measured is determined to be a moving state; if the value of the diagonal element is less than or equal to the element threshold, the current state of the target to be measured is determined to be a stationary state. This achieves accurate determination of the current state of the target to be measured based on the first feature matrix, avoiding the problem in related technologies where the current state of the target to be measured cannot be accurately determined.

[0076] In some embodiments, such as Figure 4As shown, based on the observation coordinates of the target in the first observation image and the observation coordinates of the target in the second observation image, the distance between the target and the vehicle is calculated, including:

[0077] S401. Obtain the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter when the shooting device captures the first observation image, and construct the first feature matrix based on the observation coordinates, shooting parameters and first pose parameter corresponding to the target to be measured in the first observation image.

[0078] S402. Obtain the second pose parameters when the shooting device captures the second observation image, and construct the second feature matrix based on the observation coordinates, shooting parameters and first pose parameters corresponding to the target to be ranged in the second observation image.

[0079] S403. Calculate the coordinates of the target under distance measurement in the vehicle coordinate system based on the first feature matrix and the second feature matrix, and determine the distance between the target under distance measurement and the vehicle based on the coordinates of the target under distance measurement in the vehicle coordinate system.

[0080] The principle of constructing the first feature matrix in step S401 is the same as that of constructing the first feature matrix in steps S201 and S202, and will not be repeated here; similarly, the principle of obtaining the second feature matrix in step S402 is the same as that of constructing the first feature matrix in steps S201 and S202, and will not be repeated here.

[0081] To better understand this method, this embodiment provides a more specific example for illustration.

[0082] When the vehicle is currently in motion and the target to be ranged is currently stationary, the camera will move with the vehicle, thus placing it in different positions to capture images of the target. Figure 5 As shown, Figure 5 The dashed lines above the shooting devices in the diagram represent the constraints between shooting devices at different positions. These constraints are inter-frame constraints obtained from wheel odometer measurements. Figure 5 The pentagram shown represents a feature point of the target to be measured within the monitoring frame (this feature point characterizes the target to be measured), such as... Figure 6 As shown, Figure 6 In this diagram, p1 represents the feature point captured by the camera in one frame, and p2 represents the feature point captured by the camera in another frame. The origin of the coordinate system in P1 is denoted as O1, and the origin of the coordinate system in P2 is denoted as O2. When the feature point on the same target to be measured is seen in two or more frames of images, triangulation can be performed to estimate the feature point depth, or a reprojection residual can be constructed, which can be used as a least squares problem to jointly optimize the feature points and thus determine the coordinates of the feature points.

[0083] Continuing the previous example, let the shooting parameters be denoted as K, the first pose parameter as [Rt], and the second pose parameter as [R2t]. Then, the first projection matrix obtained based on the shooting parameters and the first pose parameter is denoted as... The second projection matrix obtained based on the shooting parameters and the second pose parameters is denoted as... Let the vector corresponding to the observation coordinates in the first observation image be denoted as Represent the vector of the corresponding observation coordinates in the second observation image. The vector representation of the coordinates of the target to be ranged in the vehicle coordinate system is as follows: The first feature matrix, constructed from the observation coordinates, shooting parameters, and first pose parameter of the target in the first observation image, is as follows:

[0084]

[0085] The second feature matrix, constructed based on the observation coordinates, shooting parameters, and second pose parameters of the target in the second observation image, is as follows:

[0086]

[0087] Based on the same principle, substituting the first and second characteristic matrices into equation (1-5) yields:

[0088]

[0089] By solving the two equations above, the value of P′ can be obtained, thereby obtaining the coordinates of the target to be measured in the vehicle coordinate system. Subsequently, the distance between the target to be measured and the vehicle can be determined based on the coordinates of the target to be measured in the vehicle coordinate system, which will not be elaborated here.

[0090] Understandably, the above method represents the target to be observed through feature points. After obtaining the coordinates of the target in the vehicle coordinate system using the above method, although most feature points are on the target, in order to avoid mistaking background feature points as feature points of the target, this example can also use depth information to perform the RANSAC algorithm on the feature points within a detection box to remove feature points that are not on the target to be detected. Then, the nearest distance among the clustered feature points within the same target box is selected as the distance between the target and the current vehicle. By removing erroneous feature points within the detection box and selecting the nearest feature point as the distance between the target and the current vehicle, the accuracy of the calculation results can be more effectively guaranteed, thereby further ensuring vehicle safety.

[0091] It is understandable that if the imaging device captures the target to be measured in N frames, each frame of the observed image can construct one of the above equations. Since there are three unknowns (X, Y, Z) in P′, P′ can be obtained through at least two equations.

[0092] According to the technical solution provided in the embodiments of this application, the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter of the shooting device when capturing the first observation image are obtained, and a first feature matrix is ​​constructed together with the observation coordinates, shooting parameters and the first pose parameter of the target to be measured in the first observation image; the second pose parameter of the shooting device when capturing the second observation image is obtained, and a second feature matrix is ​​constructed together with the observation coordinates, shooting parameters and the first pose parameter of the target to be measured in the second observation image; the coordinates of the target to be measured in the vehicle coordinate system are calculated according to the first feature matrix and the second feature matrix, and the distance between the target to be measured and the vehicle is determined according to the coordinates of the target to be measured in the vehicle coordinate system. In this method, when the target to be measured is observed by two or more image frames, the pose of the image frame is obtained by recursion using the wheel speed trajectory, thereby determining whether the current state of the target to be measured is stationary. When the current state of the target to be measured is stationary, the coordinates of the target to be measured in the vehicle coordinate system are determined by jointly using multiple observation images, thereby achieving accurate acquisition of the distance between the target to be measured and the vehicle, avoiding the problem of not being able to accurately acquire the distance between the target to be measured and the vehicle.

[0093] In some embodiments, such as Figure 7 As shown, the distance between the target and the vehicle is calculated based on the detection box corresponding to the target in the first observation image, including:

[0094] S701. Obtain the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter of the shooting device when capturing the first observation image.

[0095] S702. Calculate the distance between the target and the vehicle based on the coordinates of the detection box corresponding to the target in the first observation image, the shooting parameters, and the first pose parameter.

[0096] It is understood that, in performing steps S601 and S602, this example assumes that the target to be measured and the vehicle are on the same plane.

[0097] Specifically, this example uses a target detection method to detect the target to be measured in the first observation image, thereby enabling the target to be measured to be bounded in the first observation image by a detection box, and subsequently using the coordinates of the detection box in the first observation image as the coordinates of the target to be measured in the first observation image.

[0098] It is understood that the principle of step S601 in this example is the same as that of step S201, and will not be repeated here.

[0099] After obtaining the above-mentioned shooting parameters and first pose parameters, this example further calculates the distance between the target and the vehicle based on the coordinates of the detection box corresponding to the target in the first observation image, the shooting parameters, and the first pose parameters.

[0100] For example, if there is a target to be measured around the vehicle, after capturing the first observation image, a 2D image object detection model, such as YOLOv5, is used to frame the target in the first observation image using a detection box. The detection box will completely frame the target. Since the target is on the ground, the lower edge of the detection box is used as the ground line of the object. The coordinates of the pixel at the midpoint of the lower edge of the detection box are used as the coordinates of the target. Subsequently, the position of the target in the vehicle coordinate system is calculated using the coordinates of the pixel at the midpoint of the lower edge of the detection box, thereby determining the distance between the target and the vehicle.

[0101] According to the technical solution provided in the embodiments of this application, the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter of the shooting device when capturing the first observation image are obtained; based on the coordinates of the detection box corresponding to the target to be measured in the first observation image, the shooting parameters, and the first pose parameter, the distance between the target to be measured and the vehicle is calculated, thereby realizing the determination of the distance between the target to be measured and the vehicle based on the shooting parameters, the first pose parameter, and the coordinates of the detection box corresponding to the target to be measured in the first observation image, ensuring the accuracy of the distance between the target to be measured and the vehicle.

[0102] In some examples, such as Figure 8 As shown, based on the coordinates of the detection box corresponding to the target in the first observation image, the shooting parameters, and the first pose parameter, the distance between the target and the vehicle is calculated, including:

[0103] S801. Construct the projection matrix based on the first pose parameter;

[0104] S802. Based on the coordinates of the detection box corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix, calculate the coordinates of the target to be measured in the vehicle coordinate system, and determine the distance between the target to be measured and the vehicle based on the coordinates of the target to be measured in the vehicle coordinate system.

[0105] First, construct a vehicle coordinate system based on the vehicle body. Specifically, for example... Figure 9 As shown, the origin of the coordinate system is the vehicle's contact point. v Then set the X-axis, Y-axis, and Z-axis respectively (denoted as x). v y v zv This allows for the construction of a vehicle coordinate system. The grounding point of the target to be measured (the midpoint of the lower edge of the detection frame) is taken as point P, and point P is denoted as P in the vehicle coordinate system. v Let the coordinates of point P in the vehicle coordinate system be (X, Y, Z), then P v The vector representation of P v =[XYZ] T Since the above example uses the vehicle's ground contact point as the coordinate system's origin, and this example assumes the target to be measured and the vehicle are on the same plane, then the above P... v =[XYZ] T In this example, Z can be denoted as 0. This example also constructs a coordinate system based on the first observed image (for example, using the top-left corner of the first observed image as the origin, and then using the width and height of the first observed image as coordinate axes to construct a coordinate system based on the first observed image). Let the origin of the coordinate system corresponding to the first observed image be 0. c Then set the X-axis, Y-axis, and Z-axis respectively (denoted as x). c y c z c ), where the coordinates of point P in the coordinate system corresponding to the first observation image are denoted as P. c And the coordinates of point P in the coordinate system corresponding to the first observed image are (uv1), then P c The vector representation of P v =[uv1] T .

[0106] Specifically, a projection matrix is ​​constructed based on the shooting parameters and the first pose parameter; for example, the projection matrix is ​​constructed based on the shooting parameters and the first pose parameter. This is denoted as the first pose parameter (this first pose parameter represents the rotation and translation transformation between the coordinate system corresponding to the first observed image and the vehicle coordinate system, obtained through calibration). After vectorization, the projection matrix T is obtained. 3×4 .

[0107] Based on the coordinates of the detection box corresponding to the target in the first observation image, the shooting parameters, and the projection matrix, the coordinates of the target in the vehicle coordinate system are calculated. Specifically, the projection matrix maps the coordinates of the target in the vehicle coordinate system to the coordinates of the target in the coordinate system corresponding to the first observation image. In this example, the coordinates of the detection box corresponding to the target in the first observation image are P. c The shooting parameters are denoted as K, and the coordinates of the target to be ranged in the vehicle coordinate system are denoted as P. v ,but Where s is the normalized system, and P is the normalized system. c The z-coordinate value can be obtained by further simplifying the above equation.

[0108] Based on the above equations, we can obtain an effective equation containing two unknowns, X and Y. Solving the above equations will yield P. v The value of is obtained, that is, the accurate coordinates of the target to be measured in the vehicle coordinate system.

[0109] After determining the coordinates of the target in the vehicle coordinate system, this example can accurately obtain the distance between the target and the vehicle. Specifically, as shown below... Figure 9 As shown, the Y value of the target to be measured in the vehicle coordinate system is taken as the distance between the target and the vehicle.

[0110] According to the technical solution provided in the embodiments of this application, a projection matrix is ​​constructed based on the first pose parameter; the coordinates of the target to be measured in the vehicle coordinate system are calculated based on the coordinates of the detection box corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix; and the distance between the target to be measured and the vehicle is determined based on the coordinates of the target to be measured in the vehicle coordinate system. This achieves the determination of the distance between the target to be measured and the vehicle based on the shooting parameters, the first pose parameter, and the coordinates of the detection box corresponding to the target to be measured in the first observation image, thus ensuring the accuracy of the distance between the target to be measured and the vehicle.

[0111] In some examples, such as Figure 10 As shown, the method also includes:

[0112] S1001. If the vehicle is currently stationary, acquire the first observation image containing the target to be ranged.

[0113] S1002. Calculate the distance between the target and the vehicle based on the observation coordinates corresponding to the target in the first observation image.

[0114] Specifically, in step S101, if it is determined that the current state of the vehicle is stationary, then since the vehicle is stationary, the camera device installed on the vehicle is also stationary. The camera device on the vehicle cannot be in different positions to take pictures of the target to be measured, that is, it cannot obtain the second observation image mentioned above. Therefore, at this time, this example will directly obtain the first observation image containing the target to be measured, and calculate the distance between the target to be measured and the vehicle based on the detection box corresponding to the target to be measured in the first observation image.

[0115] It is understood that the principle of "calculating the distance between the target to be measured and the vehicle based on the detection box corresponding to the target to be measured in the first observation image" in step S1002 provided in this example can be found in S701 to S702 and S801 to S802, and will not be repeated here.

[0116] To better understand this method, this embodiment provides a more specific example for illustration;

[0117] Please see Figure 11 This example first acquires a first observation image captured by the imaging device, parameters monitored by the IMU, and parameters detected by the wheel speed odometer. After the first observation image is used to identify the target to be measured, a detection box is obtained. The parameters input from the IMU and wheel speed are combined to determine whether the vehicle is currently in motion or stationary. If the vehicle is stationary, the distance between the vehicle and the target is calculated using the grounding point of the detection box combined with the first pose parameter. If the vehicle is in motion, a triangulation equation is constructed to determine if a valid solution exists for the target, thus determining whether the target is currently stationary. If the target is in motion, the distance between the target and the vehicle is calculated using the grounding point of the target's detection box. If the target is stationary, a second observation image is acquired, and the distance between the target and the vehicle is calculated using the triangulation of the first and second observation images.

[0118] According to the technical solution provided in the embodiments of this application, if the current state of the vehicle is stationary, a first observation image containing the target to be measured is obtained; the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image. The above method determines that the current state of the vehicle is stationary and directly calculates the distance between the target to be measured and the vehicle based on the first observation image, avoiding the invalid operation of identifying the current state of the target to be measured, saving system resources, and thus improving the user experience.

[0119] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0120] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0121] This embodiment also provides a target ranging device, such as... Figure 12 As shown, the device includes:

[0122] The recognition module 1201 is used to obtain the current state of the vehicle. If the current state of the vehicle is in motion, it obtains a first observation image containing the target to be measured and recognizes the current state of the target to be measured based on the observation coordinates corresponding to the target to be measured in the first observation image.

[0123] The calculation module 1202 is used to calculate the distance between the target to be measured and the vehicle based on the detection box corresponding to the target to be measured in the first observation image when the current state of the target to be measured is in motion.

[0124] The calculation module 1202 is also used to acquire a second observation image containing the target when the target to be measured is currently stationary, and to calculate the distance between the target to be measured and the vehicle based on the observation coordinates corresponding to the target in the first observation image and the observation coordinates corresponding to the target in the second observation image.

[0125] In some examples, the recognition module 1201 is also used to obtain the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter when the shooting device captures the first observation image; to construct a first feature matrix based on the observation coordinates, shooting parameters and first pose parameter corresponding to the target to be ranged in the first observation image; and to recognize the current state of the target to be ranged based on the first feature matrix.

[0126] In some examples, the recognition module 1201 is also used to decompose the first feature matrix to obtain a diagonal matrix; obtain the diagonal elements in the diagonal matrix and compare the values ​​of the diagonal elements with a preset element threshold; if the value of the diagonal element is greater than the element threshold, the current state of the target to be measured is determined to be a moving state; if the value of the diagonal element is less than or equal to the element threshold, the current state of the target to be measured is determined to be a stationary state.

[0127] In some examples, the calculation module 1202 is also used to obtain the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter of the shooting device when capturing the first observation image, and to jointly construct a first feature matrix based on the observation coordinates, shooting parameters and the first pose parameter of the target to be measured in the first observation image; to obtain the second pose parameter of the shooting device when capturing the second observation image, and to jointly construct a second feature matrix based on the observation coordinates, shooting parameters and the first pose parameter of the target to be measured in the second observation image; to calculate the coordinates of the target to be measured in the vehicle coordinate system based on the first feature matrix and the second feature matrix, and to determine the distance between the target to be measured and the vehicle based on the coordinates of the target to be measured in the vehicle coordinate system.

[0128] In some examples, the calculation module 1202 is also used to obtain the shooting parameters corresponding to the shooting device that captures the first observation image and the first pose parameter when the shooting device captures the first observation image; and to calculate the distance between the target to be measured and the vehicle based on the coordinates of the detection box corresponding to the target to be measured in the first observation image, the shooting parameters and the first pose parameter.

[0129] In some examples, the calculation module 1202 is also used to construct a projection matrix based on the first pose parameter; calculate the coordinates of the target in the vehicle coordinate system based on the coordinates of the detection box corresponding to the target in the first observation image, the shooting parameters, and the projection matrix, and determine the distance between the target and the vehicle based on the coordinates of the target in the vehicle coordinate system.

[0130] In some examples, the calculation module 1202 is also used to acquire a first observation image containing the target to be measured if the current state of the vehicle is stationary; and to calculate the distance between the target to be measured and the vehicle based on the detection box corresponding to the target to be measured in the first observation image.

[0131] According to the technical solution provided in the embodiments of this application, the target ranging device provided in this embodiment acquires the current state of the vehicle. If the current state of the vehicle is in motion, a first observation image containing the target to be measured is acquired, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target in the first observation image. If the current state of the target to be measured is in motion, the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target in the first observation image. If the current state of the target to be measured is stationary, a second observation image containing the target to be measured is acquired, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target in the first observation image and the observation coordinates corresponding to the target in the second observation image. This method determines the current state of the target to be measured and selects the method for calculating the distance between the target to be measured and the vehicle based on the current state of the target to be measured, thereby accurately determining the distance between the target to be measured and the vehicle, improving the accuracy of monocular ranging, and avoiding the problem of poor ranging effect when using a single imaging device to measure the distance of the target in related technologies.

[0132] Figure 13 This is a schematic diagram of the electronic device 13 provided in an embodiment of this application. Figure 13 As shown, the electronic device 13 of this embodiment includes: a processor 1301, a memory 1302, and a computer program 1303 stored in the memory 1302 and executable on the processor 1301. When the processor 1301 executes the computer program 1303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 1301 executes the computer program 1303, it implements the functions of each module / unit in the various device embodiments described above.

[0133] Electronic device 13 may be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 13 may include, but is not limited to, processor 1301 and memory 1302. Those skilled in the art will understand that... Figure 13 This is merely an example of electronic device 13 and does not constitute a limitation on electronic device 13. It may include more or fewer components than shown, or different components.

[0134] The processor 1301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0135] The memory 1302 can be an internal storage unit of the electronic device 13, such as a hard disk or RAM of the electronic device 13. The memory 1302 can also be an external storage device of the electronic device 13, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, FlashCard, etc., equipped on the electronic device 13. The memory 1302 can also include both internal and external storage units of the electronic device 13. The memory 1302 is used to store computer programs and other programs and data required by the electronic device.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0137] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium can be appropriately added or removed according to regional requirements and patent practice requirements. For example, in some regions, according to regional requirements and patent practice, a computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0138] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A target ranging method, characterized in that, The method includes: The current state of the vehicle is obtained. If the current state of the vehicle is in motion, a first observation image containing the target to be measured is obtained, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target to be measured in the first observation image. If the current state of the target to be measured is in motion, the distance between the target to be measured and the vehicle is calculated based on the detection box corresponding to the target in the first observation image; If the current state of the target to be measured is stationary, a second observation image containing the target to be measured is acquired, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image. Based on the observation coordinates corresponding to the target to be measured in the first observation image, the current state of the target to be measured is identified, including: acquiring the shooting parameters corresponding to the shooting device that captured the first observation image and the first pose parameter of the shooting device when capturing the first observation image; constructing a first feature matrix based on the observation coordinates corresponding to the target to be measured in the first observation image, the shooting parameters, and the first pose parameter; decomposing the first feature matrix to obtain a diagonal matrix; acquiring the diagonal elements in the diagonal matrix and comparing the values ​​of the diagonal elements with a preset element threshold; if the value of the diagonal element is greater than the element threshold, the current state of the target to be measured is determined to be a moving state; if the value of the diagonal element is less than or equal to the element threshold, the current state of the target to be measured is determined to be a stationary state.

2. The method according to claim 1, characterized in that, Based on the observation coordinates of the target to be measured in the first observation image and the observation coordinates of the target to be measured in the second observation image, the distance between the target to be measured and the vehicle is calculated, including: The shooting parameters corresponding to the shooting device that captured the first observation image and the first pose parameter of the shooting device when capturing the first observation image are obtained, and a first feature matrix is ​​constructed together based on the observation coordinates of the target to be measured in the first observation image, the shooting parameters and the first pose parameter. The second pose parameters are obtained when the shooting device captures the second observation image, and a second feature matrix is ​​constructed based on the observation coordinates corresponding to the target to be measured in the second observation image, the shooting parameters, and the first pose parameters. The coordinates of the target under distance measurement in the vehicle coordinate system are calculated based on the first feature matrix and the second feature matrix, and the distance between the target under distance measurement and the vehicle is determined based on the coordinates of the target under distance measurement in the vehicle coordinate system.

3. The method according to claim 1, characterized in that, Calculating the distance between the target and the vehicle based on the detection box corresponding to the target in the first observed image includes: Acquire the shooting parameters corresponding to the shooting device that captured the first observation image and the first pose parameter of the shooting device when capturing the first observation image; The distance between the target and the vehicle is calculated based on the coordinates of the detection box corresponding to the target in the first observation image, the shooting parameters, and the first pose parameters.

4. The method according to claim 3, characterized in that, Based on the coordinates of the detection box corresponding to the target to be measured in the first observation image, the shooting parameters, and the first pose parameters, the distance between the target to be measured and the vehicle is calculated, including: Construct a projection matrix based on the first pose parameters; Based on the coordinates of the detection box corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix, the coordinates of the target to be measured in the vehicle coordinate system are calculated, and the distance between the target to be measured and the vehicle is determined based on the coordinates of the target to be measured in the vehicle coordinate system.

5. The method according to claim 1, characterized in that, The method further includes: If the vehicle is currently stationary, a first observation image containing the target to be ranged is acquired. The distance between the target and the vehicle is calculated based on the detection box corresponding to the target in the first observation image.

6. A target ranging device, characterized in that, The device includes: The identification module is used to obtain the current state of the vehicle. If the current state of the vehicle is in motion, it obtains a first observation image containing the target to be measured and identifies the current state of the target to be measured based on the observation coordinates corresponding to the target to be measured in the first observation image. The calculation module is used to calculate the distance between the target to be measured and the vehicle based on the detection box corresponding to the target to be measured in the first observation image when the current state of the target to be measured is in motion. The calculation module is further configured to: acquire a second observation image containing the target when the current state of the target to be measured is stationary; calculate the distance between the target and the vehicle based on the observation coordinates corresponding to the target in the first observation image and the observation coordinates corresponding to the target in the second observation image; and identify the current state of the target based on the observation coordinates corresponding to the target in the first observation image, including: acquiring the shooting parameters corresponding to the shooting device that captured the first observation image and the first pose parameter when the shooting device captured the first observation image; constructing a first feature matrix based on the observation coordinates corresponding to the target in the first observation image, the shooting parameters, and the first pose parameter; decomposing the first feature matrix to obtain a diagonal matrix; acquiring the diagonal elements in the diagonal matrix and comparing the values ​​of the diagonal elements with a preset element threshold; if the value of the diagonal element is greater than the element threshold, determining that the current state of the target to be measured is a moving state; if the value of the diagonal element is less than or equal to the element threshold, determining that the current state of the target to be measured is a stationary state.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images