Target ranging method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of vehicle distance measurement, and provides a target distance measurement method and device, electronic equipment and a storage medium. According to the method, the current state of the to-be-ranged target is determined, and the mode of calculating the distance between the to-be-ranged target and the vehicle is selected according to the current state of the to-be-ranged target, so that the distance between the to-be-ranged target and the vehicle is accurately determined, the monocular distance measurement accuracy is improved, and the problem that in related technologies, the distance measurement efficiency is high is solved. And the distance measurement effect is poor when a to-be-measured target is subjected to distance measurement through a shooting device.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle ranging, and in particular to a target ranging method, device, electronic equipment and storage medium. Background Art

[0002] With the growth of material culture, driving has become the first choice for people to go out, and vehicles are becoming more and more intelligent. Electronic rearview mirror is a kind of automotive technology that replaces traditional optical rearview mirrors with electronic displays. It collects the image behind through the camera device and displays it in real time on the display screen to provide the driver with a rear view image. This system usually also integrates the distance measurement function to help the driver understand the distance of the target behind and improve driving safety.

[0003] The distance measurement function can improve vehicle safety, such as collision avoidance: The distance measurement function can help the driver avoid collisions with objects behind when reversing, changing lanes and merging. Parking assistance: When parking in a narrow space, the distance measurement function can provide distance information to avoid collisions between the rear of the vehicle and obstacles. It can also improve the driving experience, such as reducing blind spots.

[0004] In the prior art, when a photographing device is used to measure the distance of a target, the distance measurement effect is poor. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a target ranging method, device, electronic device and storage medium to solve the problem of poor ranging effect when a shooting device is used to measure the distance of a target to be measured in the prior art.

[0006] According to a first aspect of an embodiment of the present application, a target ranging method is provided, the method comprising: obtaining a current state of a vehicle, and if the current state of the vehicle is a moving state, obtaining a first observation image containing a target to be ranged, and identifying the current state of the target to be ranged based on observation coordinates corresponding to the target to be ranged in the first observation image; when the current state of the target to be ranged is a moving state, calculating the distance between the target to be ranged and the vehicle based on a detection frame corresponding to the target to be ranged in the first observation image; when the current state of the target to be ranged is a stationary state, obtaining a second observation image containing the target to be ranged, and calculating the distance between the target to be ranged and the vehicle based on the observation coordinates corresponding to the target to be ranged in the first observation image and the observation coordinates corresponding to the target to be ranged in the second observation image.

[0007] According to a second aspect of an embodiment of the present application, a target ranging device is provided, the device comprising: an identification module, for acquiring a current state of a vehicle, and if the current state of the vehicle is a moving state, acquiring a first observation image containing a target to be ranged, and identifying the current state of the target to be ranged based on observation coordinates corresponding to the target to be ranged in the first observation image; a calculation module, for calculating the distance between the target to be ranged and the vehicle based on a detection frame corresponding to the target to be ranged in the first observation image when the current state of the target to be ranged is a moving state; and the calculation module, for obtaining a second observation image containing the target to be ranged, and calculating the distance between the target to be ranged and the vehicle based on observation coordinates corresponding to the target to be ranged in the first observation image and observation coordinates corresponding to the target to be ranged in the second observation image when the current state of the target to be ranged is a stationary state.

[0008] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0009] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0010] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the method in the embodiments of the present application obtains the current state of the vehicle. If the current state of the vehicle is a moving state, a first observation image containing the target to be measured is obtained, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target to be measured in the first observation image; if the current state of the target to be measured is a moving state, the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image; if the current state of the target to be measured is a stationary state, a second observation image containing the target to be measured is obtained, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image. The method determines the current state of the target to be measured and selects a method for calculating the distance between the target to be measured and the vehicle according to the current state of the target to be measured, thereby realizing accurate determination of the distance between the target to be measured and the vehicle, improving the accuracy of monocular ranging, and avoiding the problem of poor ranging effect when the target to be measured is measured by a shooting device in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 It is a flowchart of a target ranging method provided in an embodiment of the present application;

[0013] Figure 2 It is a flowchart of another target ranging method provided in an embodiment of the present application;

[0014] Figure 3 is a flow chart of another target ranging method provided in an embodiment of the present application;

[0015] Figure 4 It is a flowchart of another target ranging method provided in an embodiment of the present application;

[0016] Figure 5 An observation constraint diagram of a shooting device observing a target to be measured provided by an embodiment of the present application;

[0017] Figure 6 It is a triangulated schematic diagram provided in an embodiment of the present application;

[0018] Figure 7 It is a flowchart of another target ranging method provided in an embodiment of the present application;

[0019] Figure 8 is a flow chart of another optional target ranging method provided in an embodiment of the present application;

[0020] Fig. 9 It is a basic schematic diagram of a coordinate transformation provided by an embodiment of the present application;

[0021] Fig.10 It is a flowchart of another optional target ranging method provided in an embodiment of the present application;

[0022] Fig.11 It is a flowchart of another optional target ranging method provided in an embodiment of the present application;

[0023] Fig.12 is a structural schematic diagram of a target distance measuring device provided in an embodiment of the present application;

[0024] Fig.13 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] A target ranging method and device according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 This is a target distance measurement method provided by an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0028] S101, obtaining a current state of the vehicle. If the current state of the vehicle is a moving state, obtaining a first observation image containing a target to be measured, and identifying the current state of the target to be measured based on observation coordinates corresponding to the target to be measured in the first observation image;

[0029] S102, if the current state of the target to be measured is a moving state, calculating the distance between the target to be measured and the vehicle based on the detection frame corresponding to the target to be measured in the first observation image;

[0030] S103: If the current state of the target to be measured is a stationary state, a second observation image containing the target to be measured is obtained, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image.

[0031] It can be understood that the target ranging method provided in this example is applied to vehicles, including vehicles with automatic driving or intelligent driving (including vehicles with manned functions (such as cars, buses, large buses, minibuses, etc.), vehicles with cargo functions (such as ordinary trucks, vans, trailers, closed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, trucks with special structures), special vehicles (such as logistics distribution vehicles, automatic guided transport vehicles AGV, patrol cars, cranes, cranes, excavators, bulldozers, forklifts, rollers, loaders, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor washing vehicles, sprinkler trucks, sweeping robots, food delivery robots, shopping guide robots, lawn mowers, golf carts, etc.), vehicles with entertainment functions (such as entertainment vehicles, amusement park automatic driving devices, balance cars, etc.), rescue vehicles (such as fire trucks, ambulances, power repair vehicles, engineering rescue vehicles, etc.)), etc.

[0032] Specifically, the vehicle is provided with at least one photographing device capable of taking photos, and the target to be measured is a target photographed by the photographing device.

[0033] In step S101, obtaining the current state of the vehicle includes: obtaining the current state of the vehicle through sensors installed on the vehicle, and the above sensors include but are not limited to inertial measurement unit (IMU) and wheel speed odometer; through the data output by the above IMU and wheel speed odometer, it can be determined whether the current state of the vehicle is a moving state or a stationary state.

[0034] If the current state of the vehicle is a moving state, a first observation image containing the target to be measured is obtained, and based on the observation coordinates corresponding to the target to be measured in the first observation image, the current state of the target to be measured is identified; wherein the first observation image is an initial image of the target to be measured taken by the shooting device. After obtaining the above initial image, this example will further calculate the observation coordinates of the target to be detected in the first observation image, and finally identify the current state of the target to be measured according to the observation coordinates.

[0035] Exemplarily, after acquiring the first observation image, target recognition is performed on the first observation image to obtain the target to be measured, and a bounding box corresponding to the target to be measured is determined, and then a coordinate system is constructed with the boundary of the first observation image (for example, the upper left corner of the first observation image is taken as the origin, and then the width and height of the first observation image are used as coordinate axes respectively to realize the construction of a coordinate system based on the first observation image), and the position of the bounding box corresponding to the target to be measured in the coordinate system is calculated to obtain the observation coordinates of the target to be measured in the first observation image, and finally, according to the observation coordinates corresponding to the target to be measured in the first observation image, the current state of the target to be measured is identified.

[0036] It is understandable that how to identify the current state of the target to be measured based on the observation coordinates corresponding to the target to be measured in the first observation image will be described in detail later and will not be repeated here.

[0037] It can be understood that the current state of the target to be measured is a moving state or a stationary state. This example determines the current state of the target to be measured and determines the specific ranging method according to the current state of the target to be measured, thereby realizing accurate determination of the distance between the target to be measured and the vehicle; specifically, if the current state of the target to be measured is a moving state, the distance between the target to be measured and the vehicle is calculated based on the detection frame corresponding to the target to be measured in the first observation image; if the current state of the target to be measured is a stationary state, a second observation image containing the target to be measured is obtained, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image.

[0038] It can be understood that when the vehicle is in motion and the target to be measured is stationary, the shooting device installed on the vehicle will move with the movement of the vehicle, while the absolute position of the target to be measured will not change. Therefore, the shooting device on the vehicle will be at different positions to shoot the target to be measured, thereby obtaining the above-mentioned second observation image, and the position change of the target to be measured in the first observation image and the second observation image is related to the movement of the vehicle. Therefore, this example calculates the distance between the target to be measured and the vehicle based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image, thereby accurately determining the distance between the target to be measured and the vehicle.

[0039] When the vehicle is in motion and the target to be measured is in motion, since the shooting device installed on the vehicle will move with the movement of the vehicle and the position of the target to be measured is changing, the observation image captured by the shooting device in the previous frame has no correlation with the target to be measured in the observation image captured later, and the first observation image and the second observation image cannot be associated to calculate the distance between the target to be measured and the vehicle. Therefore, this example directly calculates the distance between the target to be measured and the vehicle based on the detection frame corresponding to the target to be measured in the first observation image, thereby accurately determining the distance between the target to be measured and the vehicle.

[0040] It can be understood that the above-mentioned second observation image is another image containing the target to be measured after the shooting device shoots the first observation image, and it can be understood that the shooting interval between the second observation image and the first observation image does not exceed the preset time threshold; illustratively, taking the above-mentioned time threshold as 10 frames as an example, after the shooting device shoots the above-mentioned first observation image, the above-mentioned second observation image is another image containing the target to be measured taken by the shooting device within 10 frames.

[0041] According to the technical solution provided by the embodiment of the present application, the current state of the vehicle is obtained. If the current state of the vehicle is a moving state, a first observation image containing a target to be measured is obtained, and based on the observation coordinates corresponding to the target to be measured in the first observation image, the current state of the target to be measured is identified; if the current state of the target to be measured is a moving state, the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image; if the current state of the target to be measured is a stationary state, a second observation image containing the target to be measured is obtained, and based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image, the distance between the target to be measured and the vehicle is calculated. The method determines the current state of the target to be measured and selects a method for calculating the distance between the target to be measured and the vehicle according to the current state of the target to be measured, thereby realizing accurate determination of the distance between the target to be measured and the vehicle, improving the accuracy of monocular ranging, and avoiding the problem of poor ranging effect when the target to be measured is measured by a single shooting device in the related art.

[0042] In some embodiments, Figure 2 As shown, based on the observation coordinates corresponding to the target to be measured in the first observation image, the current state of the target to be measured is identified, including:

[0043] S201, obtaining shooting parameters corresponding to a shooting device for shooting a first observation image and a first pose parameter when the shooting device shoots the first observation image;

[0044] S202, constructing a first feature matrix according to the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image;

[0045] S203: Identify the current state of the target to be measured according to the first characteristic matrix.

[0046] The shooting parameters corresponding to the above-mentioned shooting device characterize the internal geometric and optical characteristics of the shooting device, and the above-mentioned shooting parameters include but are not limited to the focal length of the shooting device on the x-axis and the y-axis, and the coordinates of the optical center (principal point) of the shooting device on the image plane.

[0047] The first position parameter when the above-mentioned shooting device shoots the first observation image represents the position transformation of the shooting device; the above-mentioned first position parameter includes but is not limited to the rotation matrix and translation vector of the shooting device. It can be understood that the first position parameter is obtained through the wheel speed odometer of the vehicle.

[0048] After obtaining the above-mentioned shooting parameters and the first pose parameters, this example will further construct a first feature matrix based on the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image; wherein the first feature matrix is ​​associated with the coordinates of the target to be measured in the vehicle body coordinate system.

[0049] It can be understood that the coordinates of the target to be measured in the vehicle coordinate system are associated with the observed coordinates of the target to be measured in the first observation image. Specifically, the observed coordinates of the target to be measured in the first observation image are obtained by mapping the coordinates of the target to be measured in the vehicle coordinate system.

[0050] Continuing with the above example, the shooting parameter is recorded as K, the first pose parameter is recorded as [R t], the X-axis coordinate of the target to be measured in the first observation image is u1, and the Y-axis coordinate is v1. After the scalar in the homogeneous coordinate representation of the above observation coordinates, the vector representation of the observation coordinates corresponding to the target to be measured in the first observation image is obtained The X-axis coordinate of the target to be measured in the vehicle body coordinate system is marked as X, the Y-axis coordinate is marked as Y, and the Z-axis coordinate is marked as Z. After the scalar in the homogeneous coordinate representation of the coordinates of the target to be measured in the vehicle body coordinate system is obtained, the vector representation of the coordinates of the target to be measured in the vehicle body coordinate system is obtained as Then the observation coordinates of the above-mentioned target to be measured in the first observation image can be represented by the following equation:

[0051]

[0052] Among them, K is the shooting parameter, [R t] is the first pose parameter, and s is the scaling factor (obtained according to the camera parameters). The above equation can be used to map the coordinates of the target to be measured in the vehicle coordinate system to obtain the observed coordinates of the target to be measured in the first observation image.

[0053] Among them, the first projection matrix M is calculated based on the above shooting parameters and the first pose parameters 3×4 =K*[R t], then the above equation (1-1) can be simplified to the following equation:

[0054]

[0055] Among them, the above M 3×4 Expressed as a 3x4 matrix, the first projection matrix can be expressed as The above Expressed as P', the above equation (1-2) can be simplified to the following equation:

[0056]

[0057] It can be understood that in equation (1-1), K is the camera of the camera, and [R t] is the first position parameter obtained by the wheel speed odometer. If the two vectors can be written as It means two vectors are parallel. When two vectors are parallel, the cross product of the vectors is 0. Then we can transform equation 1-3 to get the following equation:

[0058] u1*M2*P′-M0*P′=0;

[0059] v1*M2*P′-M1*P′=0; (1-4).

[0060] Further simplifying the above equation, we can get:

[0061]

[0062] Above That is, it is the first feature matrix constructed based on the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image.

[0063] Subsequently, the current state of the target to be measured can be identified through the first characteristic matrix, thereby accurately acquiring the current state of the target to be measured.

[0064] According to the technical solution provided in the embodiment of the present application, the shooting parameters corresponding to the shooting device for shooting the first observation image and the first pose parameters when the shooting device shoots the first observation image are obtained; the first feature matrix is ​​jointly constructed according to the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image; according to the first feature matrix, the current state of the target to be measured is identified, wherein the above method realizes the joint determination based on the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image, thereby realizing accurate acquisition of the current state of the target to be measured.

[0065] In some embodiments, Figure 3 As shown, according to the first characteristic matrix, the current state of the target to be measured is identified, including:

[0066] S301, decomposing the first characteristic matrix to obtain a diagonal matrix;

[0067] S302, obtaining diagonal elements in the diagonal matrix, and comparing the values ​​of the diagonal elements with a preset element threshold;

[0068] S303: If the value of the diagonal element is greater than the element threshold, it is determined that the current state of the target to be measured is a moving state;

[0069] S304: If the value of the diagonal element is less than or equal to the element threshold, it is determined that the current state of the target to be measured is a stationary state.

[0070] It can be understood that the current state of the target to be measured is a stationary state, and the coordinate vector P′ of the target to be measured in the vehicle coordinate system can be calculated by the above equation (1-5), that is, the above equation can be solved; on the contrary, if the current state of the target to be measured is a moving state, the vector P′ of the coordinates of the target to be measured in the vehicle coordinate system cannot be calculated through the above equation (1-5).

[0071] Continuing from the above example, this example takes the first characteristic matrix as A, then the above equation (1-5) can be simplified to A*P′=0, where, according to the above equations (1-1) to (1-5), A is a matrix with M rows and four columns. The singular value decomposition (SVD) method is used to decompose the above A matrix into the product of three matrices, as follows:

[0072] A=UΣV T ;

[0073] Where U is an m×m orthogonal matrix, Σ is an m×4 diagonal matrix, and V is an n×n orthogonal matrix. In the SVD decomposition, the diagonal elements of the diagonal matrix Σ are singular values. If the singular value is less than or equal to the element threshold, then the equation A*P′=0 has a solution, and at this time, it is determined that the current state of the target to be measured is a moving state. It can be understood that the above singular value tends to zero, but is not equal to 0, and is greater than 0; conversely, if the singular value is greater than the element threshold, then the equation A*P′=0 is misunderstood, and at this time, it is determined that the current state of the target to be measured is a stationary state.

[0074] It can be understood that the above element threshold is determined based on historical data, and the above element threshold can also be a value set by relevant personnel according to actual needs; preferably, the value range of the above element threshold is 10 -6 or 10 -9 .

[0075] According to the technical solution provided in the embodiment of the present application, the first characteristic matrix is ​​decomposed to obtain a diagonal matrix; the diagonal elements in the diagonal matrix are obtained, and the values ​​of the diagonal elements are compared with a preset element threshold; if the value of the diagonal element is greater than the element threshold, it is determined that the current state of the target to be measured is a moving state; if the value of the diagonal element is less than or equal to the element threshold, it is determined that the current state of the target to be measured is a stationary state, thereby achieving accurate determination of the current state of the target to be measured according to the above-mentioned first characteristic matrix, avoiding the problem in the related art that the current state of the target to be measured cannot be accurately determined.

[0076] In some embodiments, Figure 4As shown, based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image, the distance between the target to be measured and the vehicle is calculated, including:

[0077] S401, obtaining shooting parameters corresponding to a shooting device that shoots a first observation image and a first pose parameter when the shooting device shoots the first observation image, and jointly constructing a first feature matrix according to the observation coordinates, shooting parameters and first pose parameter corresponding to the target to be measured in the first observation image;

[0078] S402, obtaining a second posture parameter when the shooting device shoots a second observation image, and constructing a second feature matrix according to the observation coordinates, shooting parameters and first posture parameters corresponding to the target to be measured in the second observation image;

[0079] S403, calculating the coordinates of the target to be measured in the vehicle body coordinate system according to the first characteristic matrix and the second characteristic matrix, and determining the distance between the target to be measured and the vehicle according to the coordinates of the target to be measured in the vehicle body coordinate system.

[0080] The principle of constructing the first characteristic matrix in step S401 is the same as the principle of constructing the first characteristic matrix in steps S201 and S202, and will not be repeated here; similarly, the principle of obtaining the second characteristic matrix in step S402 is the same as the principle of constructing the first characteristic matrix in steps S201 and S202, and will not be repeated here.

[0081] In order to better understand the present method, this embodiment provides a more specific example for illustration.

[0082] When the current state of the vehicle is a moving state and the current state of the target to be measured is a stationary state, since the shooting device moves with the movement of the vehicle, the shooting device on the vehicle will be at different positions to shoot the target to be measured, such as Figure 5 As shown, Figure 5 The dotted line above the camera in the figure represents the constraint between cameras at different positions, which is the inter-frame constraint obtained by the round odometer. Figure 5 The five-pointed star shown in is the characteristic point of the target to be measured in the monitoring frame (the characteristic point can characterize the target to be measured), as shown in FIG. Figure 6 As shown, Figure 6 Where p1 is the feature point captured by the camera in one frame, p2 is the feature point captured by the camera in another frame, the origin of the coordinate system in P1 is denoted as O1, and the origin of the coordinate system in P2 is denoted as O2. When the feature point on the same target to be measured is seen by two or more frames of images, the depth of the feature point can be estimated by triangulation, or the reprojection residual can be constructed as a least squares problem, and the feature points can be jointly optimized to determine the coordinates of the feature points.

[0083] Continuing with the above example, let the shooting parameter be K, the first pose parameter be [R t], and the second pose parameter be [R2t2]. Then the first projection matrix obtained based on the shooting parameter and the first pose parameter is The second projection matrix obtained based on the shooting parameters and the second pose parameters is recorded as The vector of the corresponding observation coordinates in the first observation image is recorded as The vector representation of the corresponding observation coordinates in the second observation image is The vector of the coordinates of the target to be measured in the vehicle coordinate system is expressed as The first feature matrix constructed based on the observation coordinates, shooting parameters and first pose parameters corresponding to the target to be measured in the first observation image is as follows:

[0084]

[0085] The second feature matrix constructed based on the observation coordinates, shooting parameters and second posture parameters corresponding to the target to be measured in the second observation image is as follows:

[0086]

[0087] Based on the same principle, the first characteristic matrix and the second characteristic matrix are respectively substituted into the above equations (1-5) to obtain:

[0088]

[0089] By solving the above two equations, the value of P′ can be obtained, and then the coordinates of the target to be measured in the vehicle body coordinate system can be obtained. Subsequently, the distance between the target to be measured and the vehicle can be determined according to the coordinates of the target to be measured in the vehicle body coordinate system, which will not be repeated here.

[0090] It can be understood that the above method uses feature points to characterize the target to be observed. After the coordinates of the target to be measured in the vehicle body coordinate system are obtained by the above method, although most of the feature points are on the target to be measured, in order to avoid mistaking background feature points as feature points of the target to be measured, this example can also use depth information to perform RANSAC algorithm on feature points in a detection frame, eliminate feature points that are not on the target to be detected, and then select the nearest distance in the set of feature points clustered in the same target frame as the distance between the target to be measured and the current vehicle. Among them, by eliminating erroneous feature points in the detection frame and selecting the feature point with the closest distance as the distance between the target to be measured and the current vehicle, the accuracy of the calculation results can be more effectively guaranteed, thereby further ensuring the safety of the vehicle.

[0091] It can be understood that if the shooting device captures the above-mentioned target to be measured in N frames, each frame of observation image can construct one of the above-mentioned equations. Since there are three unknowns (X, Y, Z) in P′, P′ can be obtained through at least two equations.

[0092] According to the technical solution provided by the embodiment of the present application, the shooting parameters corresponding to the shooting device for shooting the first observation image and the first pose parameters when the shooting device shoots the first observation image are obtained, and the first feature matrix is ​​jointly constructed according to the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image; the second pose parameters when the shooting device shoots the second observation image are obtained, and the second feature matrix is ​​jointly constructed according to the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the second observation image; the coordinates of the target to be measured in the vehicle body coordinate system are calculated according to the first feature matrix and the second feature matrix, and the distance between the target to be measured and the vehicle is determined according to the coordinates of the target to be measured in the vehicle body coordinate system. In the above method, when the target to be measured is observed by two or more image frames, the image frame pose is recursively obtained by using the wheel speed trajectory, and then whether the current state of the target to be measured is a stationary state is determined. When the current state of the target to be measured is a stationary state, the coordinates of the target to be measured in the vehicle body coordinate system are jointly determined by multiple observation images, thereby realizing accurate acquisition of the distance between the target to be measured and the vehicle, and avoiding the problem of being unable to accurately obtain the distance between the target to be measured and the vehicle.

[0093] In some embodiments, Figure 7 As shown, the distance between the target to be measured and the vehicle is calculated based on the detection frame corresponding to the target to be measured in the first observation image, including:

[0094] S701, obtaining shooting parameters corresponding to a shooting device for shooting a first observation image and a first pose parameter when the shooting device shoots the first observation image;

[0095] S702: Calculate the distance between the target to be measured and the vehicle according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters and the first pose parameters.

[0096] It can be understood that when executing step S601 and step S602, this example assumes that the target to be measured and the vehicle are in the same plane.

[0097] Specifically, this example detects the target to be measured in the first observation image through a target detection method, and then frames the target to be measured in the first observation image through a detection frame, and subsequently uses the coordinates of the detection frame in the first observation image as the coordinates of the target to be measured in the first observation image.

[0098] It can be understood that the principle of step S601 in this example is the same as that of step S201, and will not be repeated here.

[0099] After obtaining the above-mentioned shooting parameters and the first pose parameters, this example further calculates the distance between the target to be measured and the vehicle according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters and the first pose parameters.

[0100] Exemplarily, there is a target to be measured around the vehicle. After the first observation image is captured, a 2D image object detection model, such as yolov5, is used to frame the target to be measured in the first observation image through a detection frame, and the detection frame completely frames the target to be measured. The target to be measured is on the ground, so this example uses the lower edge of the detection frame as the object grounding wire. This example uses the coordinates of the pixels at the middle point of the lower edge of the detection frame as the coordinates of the target to be detected. Subsequently, the coordinates of the pixels at the middle point of the lower edge of the detection frame are used to calculate the position of the target to be detected in the vehicle body coordinate system, thereby determining the distance between the target to be measured and the vehicle.

[0101] According to the technical solution provided in the embodiment of the present application, the shooting parameters corresponding to the shooting device that shoots the first observation image and the first pose parameters when the shooting device shoots the first observation image are obtained; the distance between the target to be measured and the vehicle is calculated according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters and the first pose parameters, thereby achieving the distance between the target to be measured and the vehicle being determined jointly according to the shooting parameters, the first pose parameters and the coordinates of the detection frame corresponding to the target to be measured in the first observation image, thereby ensuring the accuracy of the distance between the target to be measured and the vehicle.

[0102] In some examples, such as Figure 8 As shown, according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters and the first pose parameters, the distance between the target to be measured and the vehicle is calculated, including:

[0103] S801, constructing a projection matrix according to the first pose parameter;

[0104] S802, calculating the coordinates of the target to be measured in the vehicle body coordinate system according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix, and determining the distance between the target to be measured and the vehicle according to the coordinates of the target to be measured in the vehicle body coordinate system.

[0105] First, the vehicle body coordinate system is constructed according to the vehicle body. Specifically, Fig. 9 As shown, the vehicle grounding point is taken as the origin of the coordinate system 0 v , and then set the X-axis, Y-axis, and Z-axis (respectively denoted as x v ,y v 、zv ), and then the vehicle coordinate system is constructed, and the grounding point of the target to be measured (the middle point of the lower edge of the detection frame) is taken as point P, and point P is recorded as P in the vehicle coordinate system. v , let the coordinates of point P in the vehicle coordinate system be (X, Y, Z), then P v The vector is represented as P v =[XYZ] T Since the above example takes the vehicle grounding point as the coordinate system origin, and this example assumes that the target to be measured and the vehicle are in the same plane, the above P v =[XYZ] T In this example, Z can be recorded as 0; this example also constructs a coordinate system based on the first observation image (for example, the upper left corner of the first observation image is taken as the origin, and the width and height of the first observation image are used as coordinate axes respectively to realize the construction of the coordinate system based on the first observation image), and the origin of the coordinate system corresponding to the first observation image is set to 0 c , and then set the X-axis, Y-axis, and Z-axis (respectively denoted as x c ,y c 、z c ), where the coordinates of point P in the coordinate system corresponding to the first observation image are marked as P c , and the coordinates of point P in the coordinate system corresponding to the first observation image are (uv1), then P c The vector is represented as P v =[uv1] T .

[0106] Among them, the projection matrix is ​​constructed according to the shooting parameters and the first pose parameters; exemplarily, Denoted as the first pose parameter (the first pose parameter can represent the rotation and translation transformation between the coordinate system corresponding to the first observation image and the vehicle body coordinate system, obtained by calibration), the first pose parameter After vectorization, the projection matrix T is obtained 3×4 .

[0107] According to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix, the coordinates of the target to be measured in the vehicle coordinate system are calculated. Specifically, the coordinates of the target to be measured in the vehicle coordinate system can be mapped to the coordinates of the target to be measured in the coordinate system corresponding to the first observation image through the above projection matrix. In this example, the coordinates of the detection frame corresponding to the target to be measured in the first observation image are P c , the shooting parameter is recorded as K, and the coordinates of the target to be measured in the vehicle coordinate system are recorded as P v ,but Among them, the above s is the normalized system, which is P c The z coordinate value of

[0108] According to the above equation, we can get an effective equation containing two unknowns X and Y. Solving the above equation can get P v The value of is obtained accurately, that is, the coordinates of the target to be measured in the vehicle coordinate system.

[0109] After determining the coordinates of the target to be measured in the vehicle coordinate system, this example can accurately obtain the distance between the target to be measured and the vehicle. Specifically, Fig. 9 As shown, the Y value of the coordinates of the above-mentioned target to be measured in the vehicle body coordinate system is used as the distance between the target to be measured and the vehicle.

[0110] According to the technical solution provided in the embodiment of the present application, a projection matrix is ​​constructed according to the first pose parameter; the coordinates of the target to be measured in the vehicle body coordinate system are calculated according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix, and the distance between the target to be measured and the vehicle is determined according to the coordinates of the target to be measured in the vehicle body coordinate system, thereby achieving the distance between the target to be measured and the vehicle being determined according to the shooting parameters, the first pose parameter, and the coordinates of the detection frame corresponding to the target to be measured in the first observation image, thereby ensuring the accuracy of the distance between the target to be measured and the vehicle.

[0111] In some examples, such as Fig.10 As shown, the method also includes:

[0112] S1001, if the current state of the vehicle is a stationary state, obtaining a first observation image containing a target to be measured;

[0113] S1002: Calculate the distance between the target to be measured and the vehicle based on the observation coordinates corresponding to the target to be measured in the first observation image.

[0114] Specifically, in step S101, if it is determined that the current state of the vehicle is a stationary state, then since the vehicle is stationary, the shooting device installed on the vehicle is also stationary, and the shooting device on the vehicle cannot be at different positions to shoot the target to be measured, that is, the above-mentioned second observation image cannot be obtained. Therefore, at this time, this example will directly obtain the first observation image containing the target to be measured, and calculate the distance between the target to be measured and the vehicle based on the detection frame corresponding to the target to be measured in the first observation image.

[0115] It can be understood that the principle of "calculating the distance between the target to be measured and the vehicle based on the detection frame corresponding to the target to be measured in the first observation image" in step S1002 provided in this example can be found in S701 to S702, and S801 to S802, and will not be repeated here.

[0116] In order to better understand the present method, this embodiment provides a more specific example for illustration;

[0117] See also Fig.11 , this example first obtains the first observation image taken by the shooting device, the parameters monitored by the IMU, and the parameters detected by the wheel speed odometer. The first observation image obtains a detection frame after the target to be measured is identified. The parameters of the IMU and wheel speed input are combined to determine whether the current state of the vehicle is a moving state or a stationary state. If the vehicle is in a stationary state, the ground point of the detection frame is used in combination with the first posture parameter to calculate the distance between the vehicle and the target to be measured. When the current state of the vehicle is a moving state, a triangulated equation is constructed to determine whether there is a valid solution for the target to be measured, and then to determine whether the current state of the target to be measured is a stationary state. If the target to be measured is in a moving state, the distance between the target to be measured and the vehicle is calculated by the ground point of the detection frame of the target to be measured. If the current state of the target to be measured is a stationary state, a second observation image is obtained, and the distance between the target to be measured and the vehicle is determined by triangulating the first observation image and the second observation image.

[0118] According to the technical solution provided in the embodiment of the present application, if the current state of the vehicle is a stationary state, a first observation image containing the target to be measured is obtained; the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image. The above method directly calculates the distance between the target to be measured and the vehicle based on the first observation image when determining that the current state of the vehicle is a stationary state, thereby avoiding invalid operations for identifying the current state of the target to be measured, saving system resources, and thus improving user experience.

[0119] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0120] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0121] This embodiment also provides a target distance measuring device, such as Fig.12 As shown, the device comprises:

[0122] The identification module 1201 is used to obtain the current state of the vehicle. If the current state of the vehicle is a moving state, a first observation image containing the target to be measured is obtained, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target to be measured in the first observation image;

[0123] A calculation module 1202 is used to calculate the distance between the target to be measured and the vehicle based on the detection frame corresponding to the target to be measured in the first observation image when the current state of the target to be measured is a moving state;

[0124] The calculation module 1202 is further used to obtain a second observation image containing the target to be measured when the current state of the target to be measured is a stationary state, and calculate the distance between the target to be measured and the vehicle based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image.

[0125] In some examples, the identification module 1201 is also used to obtain shooting parameters corresponding to the shooting device that shoots the first observation image and the first pose parameters when the shooting device shoots the first observation image; jointly construct a first feature matrix based on the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image; and identify the current state of the target to be measured based on the first feature matrix.

[0126] In some examples, the identification module 1201 is also used to decompose the first feature matrix to obtain a diagonal matrix; obtain the diagonal elements in the diagonal matrix, and compare the value of the diagonal elements with a preset element threshold; if the value of the diagonal element is greater than the element threshold, it is determined that the current state of the target to be measured is a moving state; if the value of the diagonal element is less than or equal to the element threshold, it is determined that the current state of the target to be measured is a stationary state.

[0127] In some examples, the calculation module 1202 is also used to obtain shooting parameters corresponding to the shooting device that shoots the first observation image and the first pose parameters when the shooting device shoots the first observation image, and jointly construct a first feature matrix based on the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the first observation image; obtain the second pose parameters when the shooting device shoots the second observation image, and jointly construct a second feature matrix based on the observation coordinates, shooting parameters and the first pose parameters corresponding to the target to be measured in the second observation image; calculate the coordinates of the target to be measured in the vehicle body coordinate system based on the first feature matrix and the second feature matrix, and determine the distance between the target to be measured and the vehicle based on the coordinates of the target to be measured in the vehicle body coordinate system.

[0128] In some examples, the calculation module 1202 is also used to obtain shooting parameters corresponding to the shooting device that shoots the first observation image and the first pose parameters of the shooting device when shooting the first observation image; calculate the distance between the target to be measured and the vehicle according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters and the first pose parameters.

[0129] In some examples, the calculation module 1202 is also used to construct a projection matrix based on the first pose parameter; calculate the coordinates of the target to be measured in the vehicle body coordinate system according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix, and determine the distance between the target to be measured and the vehicle based on the coordinates of the target to be measured in the vehicle body coordinate system.

[0130] In some examples, the calculation module 1202 is further used to obtain a first observation image containing the target to be measured if the current state of the vehicle is a stationary state; and calculate the distance between the target to be measured and the vehicle based on the detection frame corresponding to the target to be measured in the first observation image.

[0131] According to the technical solution provided by the embodiment of the present application, the target ranging device provided by the embodiment obtains the current state of the vehicle. If the current state of the vehicle is a moving state, a first observation image containing the target to be measured is obtained, and the current state of the target to be measured is identified based on the observation coordinates corresponding to the target to be measured in the first observation image; if the current state of the target to be measured is a moving state, the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image; if the current state of the target to be measured is a stationary state, a second observation image containing the target to be measured is obtained, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image. The method determines the current state of the target to be measured and selects a method for calculating the distance between the target to be measured and the vehicle according to the current state of the target to be measured, thereby realizing accurate determination of the distance between the target to be measured and the vehicle, improving the accuracy of monocular ranging, and avoiding the problem of poor ranging effect when the target to be measured is measured by a shooting device in the related art.

[0132] Fig.13 Schematic diagram of an electronic device 13 provided in an embodiment of the present application. Fig.13 As shown, the electronic device 13 of this embodiment includes: a processor 1301, a memory 1302, and a computer program 1303 stored in the memory 1302 and executable on the processor 1301. When the processor 1301 executes the computer program 1303, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 1301 executes the computer program 1303, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0133] The electronic device 13 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 13 may include, but is not limited to, a processor 1301 and a memory 1302. Those skilled in the art will appreciate that Fig.13 The electronic device 13 is merely an example and does not limit the electronic device 13 , and may include more or fewer components than those shown in the figure, or different components.

[0134] The processor 1301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0135] The memory 1302 may be an internal storage unit of the electronic device 13, for example, a hard disk or memory of the electronic device 13. The memory 1302 may also be an external storage device of the electronic device 13, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 13. The memory 1302 may also include both an internal storage unit of the electronic device 13 and an external storage device. The memory 1302 is used to store computer programs and other programs and data required by the electronic device.

[0136] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of regional requirements and patent practice. For example, in some areas, according to regional requirements and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0138] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A target ranging method, characterized in that: The method comprises: Acquire a current state of the vehicle, and if the current state of the vehicle is a moving state, acquire a first observation image containing a target to be measured, and identify the current state of the target to be measured based on observation coordinates corresponding to the target to be measured in the first observation image; If the current state of the target to be measured is a moving state, calculating the distance between the target to be measured and the vehicle based on the detection frame corresponding to the target to be measured in the first observation image; If the current state of the target to be measured is a stationary state, a second observation image containing the target to be measured is obtained, and the distance between the target to be measured and the vehicle is calculated based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image.

2. The method according to claim 1, characterized in that Based on the observation coordinates corresponding to the target to be measured in the first observation image, identifying the current state of the target to be measured, including: Acquire shooting parameters corresponding to a shooting device that shoots the first observation image and a first pose parameter when the shooting device shoots the first observation image; Constructing a first feature matrix based on the observation coordinates corresponding to the target to be measured in the first observation image, the shooting parameters and the first posture parameters; According to the first characteristic matrix, the current state of the target to be measured is identified.

3. The method according to claim 1, characterized in that Identifying a current state of the target to be measured according to the first characteristic matrix includes: Decomposing the first characteristic matrix to obtain a diagonal matrix; Obtaining diagonal elements in the diagonal matrix, and comparing the values ​​of the diagonal elements with a preset element threshold; If the value of the diagonal element is greater than the element threshold, it is determined that the current state of the target to be measured is a moving state; If the value of the diagonal element is less than or equal to the element threshold, it is determined that the current state of the target to be measured is a stationary state.

4. The method according to claim 1, characterized in that: Calculating the distance between the target to be measured and the vehicle based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image, including: Acquire shooting parameters corresponding to a shooting device that shoots the first observation image and a first pose parameter when the shooting device shoots the first observation image, and jointly construct a first feature matrix according to the observation coordinates corresponding to the target to be measured in the first observation image, the shooting parameters and the first pose parameter; Acquire a second posture parameter when the shooting device shoots the second observation image, and construct a second feature matrix according to the observation coordinates corresponding to the target to be measured in the second observation image, the shooting parameters and the first posture parameter; The coordinates of the target to be measured in the vehicle body coordinate system are calculated according to the first characteristic matrix and the second characteristic matrix, and the distance between the target to be measured and the vehicle is determined according to the coordinates of the target to be measured in the vehicle body coordinate system.

5. The method according to claim 1, characterized in that Calculating the distance between the target to be measured and the vehicle based on a detection frame corresponding to the target to be measured in the first observation image includes: Acquire shooting parameters corresponding to a shooting device that shoots the first observation image and a first pose parameter when the shooting device shoots the first observation image; The distance between the target to be measured and the vehicle is calculated according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters and the first posture parameters.

6. The method according to claim 5, characterized in that Calculating the distance between the target to be measured and the vehicle according to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters and the first posture parameters, comprising: Constructing a projection matrix according to the first posture parameters; According to the coordinates of the detection frame corresponding to the target to be measured in the first observation image, the shooting parameters, and the projection matrix, the coordinates of the target to be measured in the vehicle body coordinate system are calculated, and the distance between the target to be measured and the vehicle is determined according to the coordinates of the target to be measured in the vehicle body coordinate system.

7. The method according to claim 1, characterized in that The method further comprises: If the current state of the vehicle is a stationary state, acquiring a first observation image containing the target to be measured; The distance between the target to be measured and the vehicle is calculated based on a detection frame corresponding to the target to be measured in the first observation image.

8. A target distance measuring device, characterized in that: The device comprises: an identification module, configured to obtain a current state of the vehicle, and if the current state of the vehicle is a moving state, obtain a first observation image containing a target to be measured, and identify the current state of the target to be measured based on the observation coordinates corresponding to the target to be measured in the first observation image; a calculation module, configured to calculate the distance between the target to be measured and the vehicle based on a detection frame corresponding to the target to be measured in the first observation image when the current state of the target to be measured is a moving state; The calculation module is further used to obtain a second observation image containing the target to be measured when the current state of the target to be measured is a stationary state, and calculate the distance between the target to be measured and the vehicle based on the observation coordinates corresponding to the target to be measured in the first observation image and the observation coordinates corresponding to the target to be measured in the second observation image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images