Distance measurement method, device and electronic device

Abstract

Embodiments of the present invention provide a ranging method, apparatus, and electronic device. The ranging method includes: obtaining category information of a target to be measured and parameter information of the target to be measured in a first image; determining, according to the category information, an envelope corresponding to the target to be measured in a spatial coordinate system and size information of the envelope; performing a first iterative process, where the first iterative process includes: determining a distance value for the i-th iteration, and determining first coordinate information of the envelope in the spatial coordinate system according to the distance value for the i-th iteration, a preset viewing angle, and the size information; projecting the envelope onto the first image according to the first coordinate information to obtain a first projection region; i is a positive integer; and when specific conditions are met, using the distance value for the i-th iteration as the ranging value. Embodiments of the present invention can effectively reduce the computational amount in the process of calculating the ranging value and improve the ranging efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a ranging method, apparatus, and electronic device. Background Art

[0002] In applications such as driverless driving, there is usually a need to measure the distance to targets such as obstacles. Ranging based on a vision sensor is one of the relatively common ranging methods currently. Vision sensors that can be used for ranging are mainly classified into a monocular camera and a binocular camera according to the number of lenses. Among them, the monocular camera has been widely used in ranging applications due to its advantages such as low cost and strong adaptability.

[0003] Since the monocular camera itself lacks depth information, it is usually difficult to directly and accurately obtain the distance from the target to the vehicle through the monocular camera. Instead, it is necessary to calculate the above distance based on the pixel coordinates of the target in the image captured by the monocular camera. In the prior art, in order to ensure the accuracy of the calculated distance, a relatively complex conversion model between the pixel distance and the actual distance is often required, which leads to a large amount of calculation in the distance calculation process and low ranging efficiency. Summary of the Invention

[0004] Embodiments of the present invention provide a ranging method, apparatus, and electronic device to solve the problem that when the prior art uses a monocular camera for ranging, a relatively complex conversion model between the pixel distance and the actual distance is often required, which leads to a large amount of calculation in the distance calculation process and low ranging efficiency.

[0005] To solve the above technical problems, the present invention is implemented as follows:

[0006] In a first aspect, an embodiment of the present invention provides a ranging method, including:

[0007] Obtain the category information of the target to be measured, and the parameter information of the target to be measured in a first image, where the target to be measured is any one of at least one target included in the first image, and the parameter information includes first height information;

[0008] Determine the envelope corresponding to the target to be measured in a spatial coordinate system according to the category information, and the size information of the envelope;

[0009] Execute a first iteration process, where the first iteration process includes: determining the distance value of the i-th iteration, and determining the first coordinate information of the envelope in the spatial coordinate system according to the distance value of the i-th iteration, a preset viewing angle, and the size information; projecting the envelope onto the first image according to the first coordinate information to obtain a first projection area; i is a positive integer;

[0010] When the first preset relationship is satisfied between the second height information corresponding to the first projection area and the first height information, and / or when i is equal to the first iteration threshold, the distance value of the i-th iteration is used as the ranging value.

[0011] In a second aspect, an embodiment of the present invention further provides a ranging device, including:

[0012] An acquisition module, configured to acquire the category information of a target to be measured, and the parameter information of the target to be measured in a first image, where the target to be measured is any one of at least one target included in the first image, and the parameter information includes first height information;

[0013] A first determination module, configured to determine, according to the category information, an envelope corresponding to the target to be measured in a space coordinate system, and the size information of the envelope;

[0014] A first execution module, configured to execute a first iteration process, where the first iteration process includes: determining a distance value of the i-th iteration, and determining first coordinate information of the envelope in the space coordinate system according to the distance value of the i-th iteration, a preset viewing angle, and the size information; projecting the envelope onto the first image according to the first coordinate information to obtain a first projection area; i is a positive integer;

[0015] A second determination module, configured to use the distance value of the i-th iteration as the ranging value when the first preset relationship is satisfied between the second height information corresponding to the first projection area and the first height information, and / or when i is equal to the first iteration threshold.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the above method when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0018] The ranging method provided by the embodiment of the present invention obtains the category information of the target to be measured and the first height information of the target to be measured in the first image. According to the category information, the size information of the corresponding envelope of the target to be measured in the space coordinate system is determined, and further, the first iteration process is executed. In the i-th iteration of the first iteration process, the distance value of the i-th iteration is determined. According to the distance value of the i-th iteration, the preset viewing angle, and the size information, the first coordinate information of the envelope in the space coordinate system is determined; according to the first coordinate information, the envelope is projected onto the first image to obtain a first projection area; in the case where the second height information corresponding to the first projection area satisfies a first preset relationship with the first height information, and / or in the case where i is equal to the first iteration threshold, the distance value of the i-th iteration is used as the ranging value. The embodiment of the present invention only needs to perform projection and comparison calculations on the first image, without using a relatively complex conversion model between pixel distance and actual distance, which can effectively reduce the calculation amount in the process of calculating the ranging value and improve the ranging efficiency. Description of the Drawings

[0019] Figure 1 It is a flowchart of the ranging method provided by the embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of the principle of the ranging method provided by the embodiment of the present invention;

[0021] Figure 3 It is a flowchart of an application example of the ranging method provided by the embodiment of the present invention;

[0022] Figure 4 It is a schematic structural diagram of the ranging device provided by the embodiment of the present invention. Detailed Embodiments

[0023] To make the technical problems, technical solutions, and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help a comprehensive understanding of the embodiments of the present invention. Therefore, those skilled in the art should clearly understand that various changes and modifications can be made to the embodiments described here without departing from the scope and spirit of the present invention. In addition, descriptions of known functions and structures are omitted for clarity and conciseness.

[0024] Unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meaning understood by those of ordinary skill in the field to which the present invention belongs. The "first", "second", and similar terms used in the present invention do not indicate any order, quantity, or importance, but are only used to distinguish different components. Similarly, terms such as "a" or "one" do not indicate a quantity limitation, but indicate the existence of at least one.

[0025] As Figure 1 shown, the ranging method provided by the embodiment of the present invention includes:

[0026] Step 101: Obtain the category information of the target to be measured, and the parameter information of the target to be measured in the first image. The target to be measured is any one of at least one target included in the first image, and the parameter information includes the first height information;

[0027] Step 102: Determine the envelope corresponding to the target to be measured in the spatial coordinate system according to the category information, and the size information of the envelope;

[0028] Step 103: Execute the first iterative process. The first iterative process includes: determining the distance value of the i-th iteration, and determining the first coordinate information of the envelope in the spatial coordinate system according to the distance value of the i-th iteration, the preset viewing angle, and the size information; projecting the envelope onto the first image according to the first coordinate information to obtain a first projection area; i is a positive integer;

[0029] Step 104: In the case where the second height information corresponding to the first projection area satisfies a first preset relationship with the first height information, and / or, in the case where i is equal to the first iteration threshold, use the distance value of the i-th iteration as the ranging value.

[0030] In this embodiment, the first image can be understood as an image obtained by framing targets such as obstacles in the original image collected by the vision sensor. In other words, in the first image, each target has corresponding parameter information. For example, if the target is framed by a rectangular box, then the target will have parameter information such as position coordinates, height, and width in the first image.

[0031] It is easy to understand that the first image can be considered as an image obtained by detecting the targets in the original image. Therefore, a rectangular box or the like used to frame the target can be referred to as a detection box hereinafter, and the manifestation form of the detection box is the parameter information of the corresponding target in the first image.

[0032] The first image may include at least one target. The embodiment of the present invention will mainly take any one of the at least one target, that is, the above-mentioned target to be measured, as an example to illustrate the ranging method.

[0033] For the target to be measured, its category information may be used to reflect what kind of obstacle the target to be measured is specifically, such as a vehicle, a road sign, or others. In this embodiment, the category information of the target to be measured and the parameter information in the first image may be obtained automatically based on a model or algorithm such as a deep learning model, or obtained based on manual input, and no specific limitation is made here.

[0034] It is easy to understand that for the manual input method, it can refer to directly marking and inputting or selecting the target type on the above-mentioned original image manually, etc., which can be used in some test scenarios.

[0035] As described above, the category information can be used to reflect what specific kind of obstacle the target to be measured is. For each kind of obstacle, it can be simplified into an envelope with specific size information according to empirical data. In this way, according to the category information, the envelope corresponding to the target to be measured in the space coordinate system and its size information can be determined. It should be noted that the space coordinate system here is relative to the pixel coordinate system. For the vehicle ranging occasion, it is usually understood as the vehicle body coordinate system.

[0036] In this embodiment, the finally obtained ranging value is mainly obtained by performing iterative operations on the distance value. The iterative operation here corresponds to the above-mentioned first iterative process. The following mainly takes the i-th iteration in the first iterative process as an example for illustration:

[0037] In the i-th iteration, first, the corresponding distance value needs to be determined, that is, the distance value of the i-th iteration. For the sake of simplicity in description, hereinafter, it is represented by d i . For d i , it can be an initial preset value, can be determined based on the distance value d i-1 of the previous iteration, or can be determined based on a preset distance value sequence, etc. No specific limitation is made here.

[0038] After determination, the first coordinate information of the envelope in the space coordinate system can be determined according to d i , the preset viewing angle θ, and the size information of the envelope. In practical applications, according to the first coordinate information, combined with the internal parameters and external parameters of the relevant vision sensor, the envelope can be projected onto the first image to obtain the first projection area; the specific implementation process of the projection here belongs to the existing conventional technology and will not be elaborated here.

[0039] The first projection area corresponds to the second height information in the first image. When performing the first iterative process, if in a certain iteration, the second height information and the first height information satisfy the first preset relationship, for example, the absolute value of the difference between the first height information and the second height information is less than a preset value, or when they are the same, the distance value used in this iteration can be used as the ranging value, that is, the distance output value obtained based on the above-mentioned ranging method. Of course, to avoid excessive iteration times resulting in too long a time consumption, the distance value used in this iteration can also be used as the ranging value when the iteration times is equal to the first iteration threshold.

[0040] The ranging method provided by the embodiment of the present invention obtains the category information of the target to be measured and the first height information of the target to be measured in the first image. According to the category information, the size information of the corresponding envelope in the spatial coordinate system of the target to be measured is determined, and the first iterative process is further executed. In the i-th iteration of the first iterative process, the distance value of the i-th iteration is determined. According to the distance value of the i-th iteration, the preset viewing angle, and the size information, the first coordinate information of the envelope in the spatial coordinate system is determined. According to the first coordinate information, the envelope is projected onto the first image to obtain a first projection area. When the second height information corresponding to the first projection area satisfies a first preset relationship with the first height information, and / or when i is equal to the first iteration threshold, the distance value of the i-th iteration is used as the ranging value. In the embodiment of the present invention, only projection and comparison calculations need to be performed on the first image, and there is no need to use a relatively complex conversion model between pixel distance and actual distance, which can effectively reduce the amount of calculation in the process of calculating the ranging value and improve the ranging efficiency.

[0041] To improve the acquisition efficiency of the category information of the target to be measured and the parameter information of the target to be measured in the first image, in a preferred embodiment, step 101, obtaining the category information of the target to be measured and the parameter information of the target to be measured in the first image, includes:

[0042] Obtain the original image from the vision sensor;

[0043] Based on the target detector, detect the target in the original image and the category information of the target. The target detector is obtained by training an initial target detection network built with sample targets and sample target category pairs;

[0044] Project the target onto the original image to obtain the first image and the parameter information of the target in the original image.

[0045] In this embodiment, the deep learning model of the target detector is used to obtain the above category information and parameter information.

[0046] It is easy to understand that the target detector can be a trained neural network model. Specifically, an initial target detection network can be first built. The specific network type is not limited here and can be, for example, a Convolutional Neural Networks (CNN), The Single Shot Detector (SSD), a YOLO model, etc. Then, training samples can be used to train the initial target detection network. The training samples here can be sample targets and sample target categories. The sample targets are, for example, training samples of targets such as vehicles and roadblocks, and the sample target categories are the specific categories of the sample targets. After the initial target detection network is trained, a target detector can be obtained. At this time, when the original image is input into the target detector, the target detector can automatically detect the target and identify the category of the target.

[0047] As shown above, when the target is detected, it can be marked in the original image in the form of a detection box to obtain the first image. The detection box here specifically reflects the parameter information of the target in the original image.

[0048] Optionally, to improve the acquisition efficiency of the first projection area, the step of projecting the envelope onto the first image according to the first coordinate information to obtain the first projection area includes:

[0049] Project all N corner points included in the envelope onto the first image to obtain N projection points, where N is an integer greater than 1;

[0050] According to the maximum and minimum values of the coordinates of the N projection points in the width direction of the first image, and the maximum and minimum values of the coordinates of the N projection points in the height direction of the first image, determine the first projection area of the envelope on the first image; the second height information corresponds to the height of the first projection area in the first image.

[0051] The following uses a specific application example to illustrate this embodiment:

[0052] In this specific application example, the envelope is in the shape of a cuboid, and its length, width, and height are l, w, and h respectively. The size information of the envelope can be expressed as (l, w, h); d i Corresponding to the distance from the vehicle body to the geometric center of the envelope, the preset viewing angle is θ and can be set to 0. Assume that the height of the Inertial Measurement Unit (IMU) carried on the vehicle body from the ground is z imu ; then there is:

[0053] d iThe projection distances on the x-axis and y-axis of the spatial coordinate system are d x = d i ·cos(θ), d y = d i ·sin(θ);

[0054] The envelope body has a total of eight corner points, and the coordinate values p w0 , p w1 , ……, p w7 of the eight corner points in the spatial coordinate system are respectively:

[0055]

[0056] Assume that the vision sensor is a camera, and the external parameters of the camera usually include the rotation matrix R and the translation matrix T. Then the coordinates of the above eight corner points in the camera coordinate system can be expressed as:

[0057]

[0058] Among them, P c represents the coordinate of any one of the above eight corner points in the camera coordinate system, and P w represents the coordinate of this corner point in the spatial coordinate system.

[0059] The internal parameters of the camera include the focal lengths f x and f y on the x-axis and y-axis of the camera coordinate system; if the coordinate of a certain corner point among the eight corner points in the spatial coordinate system is (x c , y c , z c ), then the coordinates (u, v) of the projection point of this corner point on the first image in the image coordinate system can be calculated by the following formula:

[0060]

[0061] Assume that the coordinates of the eight projection points corresponding to the above eight corner points in the image coordinate system are respectively (u 0 , v 0 ), (u 1 , v 1 ), ……, (u 7 , v 7 ). Then the maximum value u max and the minimum value u min of the coordinates of the above N projection points in the width direction of the first image, as well as the maximum value v max and the minimum value v min of the coordinates of the above N projection points in the height direction of the first image, can be calculated by the following formula:

[0062]

[0063] For the above first projection area, the shape is rectangular, and the coordinates of the upper left corner point on the first image are (u min , v min ), and the coordinates of the lower right corner point on the first image are (u max , v max ); the height corresponding to the second height information is v max - v min .

[0064] Of course, in some feasible embodiments, if only the projection height of the envelope in the first image is to be obtained, the coordinates of each projection point on the v-axis in the image coordinate system can be obtained only.

[0065] Optionally, determining the distance value of the i-th iteration includes:

[0066] In the case of i = 1, determining the distance value of the i-th iteration according to a preset initial distance value range;

[0067] In the case of i > 1, according to the second height information of the first projection area obtained in the (i - 1)-th iteration, the first height information, the distance value of the (i - 1)-th iteration, and the distance value range used to determine the distance value of the (i - 1)-th iteration, determining the distance value range of the i-th iteration, and determining the distance value of the i-th iteration according to the distance value range of the i-th iteration.

[0068] In this embodiment, the distance value of the first iteration is determined according to a preset initial distance value range, and the distance values of the second and subsequent iterations are all determined based on the distance value of the previous iteration.

[0069] In a specific application example, the binary search method can be used to determine the distance value of the i-th iteration. Specifically:

[0070] Let the preset initial distance value range be [d min , d max , then the distance value d 1 of the first iteration is determined to be d 1 = (d min + d max ) / 2;

[0071] If the second height information of the first projection area obtained in the first iteration is greater than the first height information, and the two do not satisfy the first preset relationship, it means that the value of d 1 is too small, resulting in too large a height of the projection of the envelope on the first image. Therefore, in the second iteration, the distance value of the iteration needs to be increased. Thus, according to the binary search method, the distance value range of the second iteration is determined to be [d 1, d max , set the distance value d of the second iteration 2 as d 2 =(d 1 +d max ) / 2;

[0072] If the second height information of the first projection area obtained in the first iteration is less than the first height information, and the first preset relationship is not satisfied between the two, it means that the value of d 1 is too large, resulting in too small a height of the projection of the envelope on the first image. Therefore, it is necessary to reduce the iteration distance value in the second iteration. Thus, according to the binary search method, the distance value range of the second iteration is determined as [d min , d 1 , and the distance value d of the second iteration 2 is set as d 2 =(d min +d 1 ) / 2;

[0073] Subsequent iterations are similar to the second iteration and will not be elaborated here.

[0074] Of course, in some feasible implementation manners, to determine the distance value of the i-th iteration according to the distance value range of the i-th iteration, other methods can also be used. For example: perform weighted averaging on the upper and lower limit values of the distance value range of the i-th iteration according to a certain weighting coefficient to obtain the distance value of the i-th iteration; or add or subtract a preset value to the upper limit value or the lower limit value of the distance value range of the i-th iteration, and then take the average value to obtain the distance value of the i-th iteration, etc.

[0075] In an example, when i>1, the determination of the distance value range of the i-th iteration includes:

[0076] When the difference between the second height information of the first projection area obtained in the (i - 1)-th iteration and the first height information is greater than the first preset value, the upper limit value of the distance value range of the i-th iteration is determined as the sum value of the upper limit value of the distance value range used to determine the distance value of the (i - 1)-th iteration and the preset adjustment parameter; the lower limit value of the distance value range of the i-th iteration is determined as the distance value of the (i - 1)-th iteration, and the first preset value is a positive number;

[0077] When the difference between the second height information of the first projection area obtained in the (i - 1)-th iteration and the first height information is less than a second preset value (the second preset value is negative), the upper limit value of the distance value range in the i-th iteration is determined as the distance value in the (i - 1)-th iteration; the lower limit value of the distance value range in the i-th iteration is determined as the difference between the lower limit value of the distance value range used to determine the distance value in the (i - 1)-th iteration and a preset adjustment parameter.

[0078] In this embodiment, by adjusting the upper limit value or the lower limit value of the distance value range in the i-th iteration through a preset adjustment parameter, fine-tuning can be performed during the process of dichotomy iteration of the distance, thereby accelerating the iteration convergence speed.

[0079] As for the above-mentioned preset adjustment parameter, it can be a fixed value or determined based on the distance value in the i-th iteration. For the first preset value and the second preset value, they can correspond to the first preset relationship in the above text. For example, if the first preset relationship means that the difference between the second height information and the first height information is within the interval of [-0.5, 0.5], the first preset value can be 0.5 and the second preset value can be -0.5.

[0080] Further, in a preferred embodiment, to further accelerate the iteration convergence speed, the parameter information includes the first coordinate of the geometric center of the target to be measured in the first image width direction;

[0081] The preset adjustment parameter is obtained based on the first coordinate and the second coordinate obtained in the (i - 1)-th iteration. The second coordinate is the coordinate of the geometric center of the first projection area of the envelope on the first image in the first image width direction.

[0082] Let the above-mentioned preset adjustment parameter be represented by offset, and the first coordinate and the second coordinate be represented by (center u , center v ) and (centerP u , centerP v ) respectively. Then, in an actual application scenario, it can be made that:

[0083] offset = |center u - centerP u |·0.001 + 0.0001

[0084] Of course, the coefficients in this calculation formula can be adjusted according to actual needs.

[0085] When the determination is obtained, the calculation methods of the upper limit value and the lower limit value of the distance value range in the i-th iteration can be further expressed as:

[0086]

[0087]

[0088] Among them, The lower limit value of the distance value range in the i-th iteration, is the upper limit value of the distance value range in the i-th iteration, d i-1 is the distance value in the (i - 1)-th iteration, is the upper limit value of the distance value range of the distance value in the (i - 1)-th iteration, is the lower limit value of the distance value range of the distance value in the (i - 1)-th iteration, heightP is the height corresponding to the second height information of the first projection area obtained in the (i - 1)-th iteration, height is the height corresponding to the first height information, n 1 and n 2 are the first preset value and the second preset value respectively, if is the judgment process, and the specific meaning is when.

[0089] For the convenience of more clearly understanding the implementation composition of the embodiments of the present invention, for the first coordinate and the second coordinate, and related parameters, they can be obtained in the following manner:

[0090] Assume that the coordinate of the upper left corner point of the detection frame corresponding to the above-mentioned target to be detected in the first image is (u lu , v lu ), and the coordinate of the lower right corner is (u lr , v lr ), then the geometric center coordinates (center u , center v ), width width, and height height of the detection frame are respectively:

[0091]

[0092] The geometric center coordinates (centerP u , centerP v ), width widthP, and height heightP of the first projection area obtained in the (i - 1)-th iteration are respectively:

[0093]

[0094] See Figure 2 , Figure 2The general implementation principle of the ranging method provided by the embodiments of the present invention is shown. Specifically, on the first image 210, there is a detection frame 220 with a width×height. According to the category information of the target in the detection frame 220, an envelope 230 is determined. A distance for iteration is set, and then the coordinates of the corner points of the envelope in the vehicle body coordinate system (corresponding to the above-mentioned space coordinate system) are obtained. Combining the external parameters and internal parameters of the camera, the coordinates in the vehicle body coordinate system are converted into the coordinates in the pixel coordinate system of the camera screen (i.e., the first image 210), and a first projection area 240 with a widthP×heightP is obtained; and according to the difference between height and heightP, the iteration distance is continuously adjusted. After the iteration is completed, a ranging value reflecting the distance between the vehicle body 250 and the obstacle is obtained.

[0095] In practical applications, since the viewing angle mainly affects the width of the first projection area of the envelope on the first image and has less impact on the height of the first projection area, continuously adjusting the iteration distance according to the difference between height and heightP can, to a certain extent, avoid the ranging value error caused by the viewing angle factor.

[0096] See Figure 3 , Figure 3 shows the flowchart of an application example of the ranging method provided by the embodiments of the present invention, which includes:

[0097] Step 301, obtain a two-dimensional detection frame;

[0098] That is, obtain the detection frame corresponding to the target to be measured on the first image;

[0099] Step 302, generate a three-dimensional envelope;

[0100] The target to be measured has category information. Combining the preset corresponding relationship between the category information and the envelope, an envelope with corresponding dimensions can be obtained;

[0101] Step 303, set the distance value range;

[0102] The distance value range can be a preset initial distance value range, or a distance value range adjusted according to data such as the distance value of the previous iteration;

[0103] For the preset initial distance value range, it can be adjusted according to the viewing angles of different cameras. For example, the preset initial distance value range can be set to [0, 200].

[0104] Step 304, determine whether the number of iterations is less than or equal to a preset number of iterations, such as 30. If so, execute step 305; if not, execute step 313;

[0105] Step 305, determine the iteration distance value;

[0106] The iteration distance value can be determined according to the corresponding distance value range;

[0107] Step 306, adjust the envelope;

[0108] It mainly refers to adjusting the coordinate information of the envelope in the space coordinate system, such as the vehicle body coordinate system; when the iteration distance value changes, the coordinates of the corner points of the envelope in the space coordinate system also change accordingly;

[0109] Step 307, determine whether all the corner points of the envelope have been traversed. If not, execute Step 308; if so, execute Step 310;

[0110] Step 308, convert the coordinates of the corner point in the space coordinate system to the coordinates in the camera coordinate system;

[0111] The coordinate system conversion in this step can be achieved based on the external parameters of the camera;

[0112] Step 309, project the corner point onto the image plane;

[0113] That is, based on the internal parameters of the camera, convert the coordinates of the corner point in the camera coordinate system to the coordinates in the image coordinate system of the first image; then return to execute Step 307;

[0114] Step 310, find the two-dimensional projection area of the envelope;

[0115] That is, determine the projection area of the envelope in the first image. The coordinates of the corner points of the projection area here can be determined according to the corner point projection results in Steps 307 to 309;

[0116] Step 311, obtain the height of the projection area;

[0117] Step 312, determine whether the error between the height of the projection area and the height of the two-dimensional detection frame is less than the threshold. If so, execute Step 313; if not, return to execute Step 303;

[0118] Step 313, output the distance;

[0119] That is, output the final ranging value, which is used to reflect the distance from the vehicle body to the corresponding obstacle of the target to be measured.

[0120] Optionally, after the step 104, taking the distance value of the i-th iteration as the ranging value, the method further includes:

[0121] Determine whether the first width information included in the parameter information and the second width information corresponding to the first projection area obtained in the i-th iteration satisfy the second preset relationship;

[0122] If the second preset relationship is not satisfied between the first width information and the second width information, a second iteration process is executed. The second iteration process includes: determining a viewing angle value of the j-th iteration, and determining second coordinate information of the envelope in the spatial coordinate system according to the viewing angle value of the j-th iteration, the ranging value, and the dimension information; projecting the envelope onto the first image according to the second coordinate information to obtain a second projection area; j is a positive integer;

[0123] When the second width information corresponding to the second projection area satisfies the first preset relationship with the first width information, and / or when j is equal to the second iteration threshold, the viewing angle value of the j-th iteration is used as the angle measurement value.

[0124] In this embodiment, on the basis of obtaining the ranging value, the angle measurement value, that is, the direction of the obstacle corresponding to the target to be measured relative to the vehicle body, can be obtained through further iteration of the viewing angle value. By simultaneously measuring the distance and angle of the obstacle relative to the vehicle body, it helps to improve the reliability of the vehicle to avoid obstacles subsequently.

[0125] For the iteration process of the viewing angle value, it is similar to the iteration process of the distance value. The main difference is that the iteration process of the viewing angle value uses the width information instead of the height information as the basis for determining whether the iteration is completed.

[0126] In addition, since the widths of the projections of the envelope in the first image may be equal at two symmetric angles, such as +30° and -30°, during the selection process of the viewing angle value, it may be necessary to adjust according to the area of the target to be measured in the first image. For example, when the target to be measured is located in the left half area of the first image, the value of the iteration viewing angle value can be negative, and when the target to be measured is located in the right half area of the first image, the value of the iteration viewing angle value can be positive.

[0127] For the iteration process of the viewing angle value, the binary search method can also be used to adjust the viewing angle value for iteration. For example:

[0128] Let the preset initial viewing angle value range be [θ min , θ max , then the viewing angle value θ 1 of the first iteration is determined as θ 1 =(θ min +θ max ) / 2;

[0129] If the second width information of the first projection area obtained in the first iteration of the second iteration process is less than the first width information and the second preset relationship is not satisfied between them, it indicates that θ 1Taking a value that is too small results in an overly large width of the projection of the envelope on the first image. Therefore, it is necessary to increase the viewing angle value during the second iteration. Thus, according to the binary search method, the viewing angle value range for the second iteration is determined to be [θ 1 , θ max , and the viewing angle value θ 2 for the second iteration is determined to be θ 2 = (θ 1 + θ max ) / 2;

[0130] If the second width information of the first projection area obtained in the first iteration of the second iteration process is greater than the first width information and the two do not satisfy the second preset relationship, it indicates that the value of θ 1 is too large, resulting in an overly small width of the projection of the envelope on the first image. Therefore, it is necessary to decrease the viewing angle value during the second iteration. Thus, according to the binary search method, the viewing angle value range for the second iteration is determined to be [θ min , θ 1 , and the viewing angle value θ 2 for the second iteration is determined to be θ 2 = (θ min + θ 1 ) / 2;

[0131] As for the above-mentioned second preset relationship and the optional implementation manners when determining the viewing angle value range and viewing angle value for the j-th iteration, etc., they are similar to the specific implementation manners of the embodiment for distance value iteration, and will not be elaborated one by one here.

[0132] As Figure 4 shown, an embodiment of the present invention further provides a ranging device, including:

[0133] An acquisition module 401, configured to acquire the category information of the target to be measured and the parameter information of the target to be measured in the first image, where the target to be measured is any one of at least one target included in the first image, and the parameter information includes first height information;

[0134] A first determination module 402, configured to determine the envelope corresponding to the target to be measured in the spatial coordinate system and the size information of the envelope according to the category information;

[0135] A first execution module 403, configured to execute a first iteration process, where the first iteration process includes: determining the distance value for the i-th iteration, and determining the first coordinate information of the envelope in the spatial coordinate system according to the distance value for the i-th iteration, the preset viewing angle, and the size information; projecting the envelope onto the first image according to the first coordinate information to obtain a first projection area; i is a positive integer;

[0136] A second determination module 404, configured to use the distance value of the i-th iteration as the ranging value when the second height information corresponding to the first projection area satisfies a first preset relationship with the first height information, and / or when i is equal to the first iteration threshold.

[0137] Optionally, the first execution module 403 includes:

[0138] A first determination unit, configured to determine the distance value of the i-th iteration according to a preset initial distance value range when i = 1;

[0139] A second determination unit, configured to determine the distance value range of the i-th iteration according to the second height information of the first projection area obtained from the (i - 1)-th iteration, the first height information, the distance value of the (i - 1)-th iteration, and the distance value range used to determine the distance value of the (i - 1)-th iteration when i > 1, and determine the distance value of the i-th iteration according to the distance value range of the i-th iteration.

[0140] Optionally, the second determination unit includes:

[0141] A first determination subunit, configured to, when the difference between the second height information of the first projection area obtained from the (i - 1)-th iteration and the first height information is greater than a first preset value, determine the upper limit value of the distance value range of the i-th iteration as the sum value of the upper limit value of the distance value range used to determine the distance value of the (i - 1)-th iteration and a preset adjustment parameter; determine the lower limit value of the distance value range of the i-th iteration as the distance value of the (i - 1)-th iteration, where the first preset value is a positive number;

[0142] A second determination subunit, configured to, when the difference between the second height information of the first projection area obtained from the (i - 1)-th iteration and the first height information is less than a second preset value, determine the upper limit value of the distance value range of the i-th iteration as the distance value of the (i - 1)-th iteration; determine the lower limit value of the distance value range of the i-th iteration as the difference value between the lower limit value of the distance value range used to determine the distance value of the (i - 1)-th iteration and a preset adjustment parameter, where the second preset value is a negative number.

[0143] Optionally, the parameter information includes a first coordinate of the geometric center of the target to be measured in the first image width direction;

[0144] The preset adjustment parameter is obtained based on the first coordinate and a second coordinate obtained from the (i - 1)-th iteration, where the second coordinate is the coordinate of the geometric center of the first projection area of the envelope on the first image in the first image width direction.

[0145] Optionally, the first execution module 403 further includes:

[0146] A first projection unit, configured to project all N corner points included in the envelope body onto the first image to obtain N projection points, where N is an integer greater than 1;

[0147] A third determination unit, configured to determine a first projection area of the envelope body on the first image according to the maximum value and the minimum value of the coordinates of the N projection points in the width direction of the first image, and the maximum value and the minimum value of the coordinates of the N projection points in the height direction of the first image; the second height information corresponds to the height of the first projection area in the first image.

[0148] Optionally, the acquisition module 401 includes:

[0149] An acquisition unit, configured to acquire a raw image from a vision sensor;

[0150] A detection unit, configured to detect a target and category information of the target in the raw image based on a target detector, where the target detector is obtained by training an initial target detection network built with a sample target and a sample target category pair;

[0151] A second projection unit, configured to project the target onto the raw image to obtain a first image and parameter information of the target in the raw image.

[0152] Optionally, the apparatus further includes:

[0153] A judgment module, configured to judge whether a second preset relationship is satisfied between a first width information included in the parameter information and a second width information corresponding to a first projection area obtained in the i-th iteration;

[0154] A second execution module, configured to, if the second preset relationship is not satisfied between the first width information and the second width information, execute a second iteration process, where the second iteration process includes: determining a viewing angle value of the j-th iteration, and determining second coordinate information of the envelope body in a space coordinate system according to the viewing angle value of the j-th iteration, the ranging value, and the size information; projecting the envelope body onto the first image according to the second coordinate information to obtain a second projection area; j is a positive integer;

[0155] A third determination module, configured to, when the second width information corresponding to the second projection area satisfies a first preset relationship with the first width information, and / or when j is equal to a second iteration threshold, use the viewing angle value of the j-th iteration as the angle measurement value.

[0156] It should be noted that the ranging device is a device corresponding to the above ranging method. All implementation manners in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effects.

[0157] Optionally, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above ranging method is implemented.

[0158] Optionally, an embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above ranging method is implemented.

[0159] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A ranging method, characterized in that, it includes: obtaining the category information of the target to be measured, and the parameter information of the target to be measured in the first image, where the target to be measured is any one of at least one target included in the first image, and the parameter information includes first height information; determining, according to the category information, the envelope corresponding to the target to be measured in the space coordinate system, and the size information of the envelope; performing a first iterative process, where the first iterative process includes: determining the distance value of the i-th iteration, and determining the first coordinate information of the envelope in the space coordinate system according to the distance value of the i-th iteration, the preset viewing angle, and the size information; projecting the envelope onto the first image according to the first coordinate information to obtain a first projection area; i is a positive integer; when the second height information corresponding to the first projection area satisfies a first preset relationship with the first height information, and / or when i is equal to the first iteration threshold, using the distance value of the i-th iteration as the ranging value.

2. The method according to claim 1, characterized in that, the determining the distance value of the i-th iteration includes: when i = 1, determining the distance value of the i-th iteration according to a preset initial distance value range; when i > 1, determining the distance value range of the i-th iteration according to the second height information of the first projection area obtained in the (i - 1)-th iteration, the first height information, the distance value of the (i - 1)-th iteration, and the distance value range used to determine the distance value of the (i - 1)-th iteration, and determining the distance value of the i-th iteration according to the distance value range of the i-th iteration.

3. The method according to claim 2, characterized in that, when i > 1, the determining the distance value range of the i-th iteration includes: when the difference between the second height information of the first projection area obtained in the (i - 1)-th iteration and the first height information is greater than a first preset value, determining the upper limit value of the distance value range of the i-th iteration as the sum value of the upper limit value of the distance value range used to determine the distance value of the (i - 1)-th iteration and a preset adjustment parameter; determining the lower limit value of the distance value range of the i-th iteration as the distance value of the (i - 1)-th iteration, and the first preset value is a positive number; when the difference between the second height information of the first projection area obtained in the (i - 1)-th iteration and the first height information is less than a second preset value, determining the upper limit value of the distance value range of the i-th iteration as the distance value of the (i - 1)-th iteration; determining the lower limit value of the distance value range of the i-th iteration as the difference value between the lower limit value of the distance value range used to determine the distance value of the (i - 1)-th iteration and a preset adjustment parameter, and the second preset value is a negative number.

4. The method according to claim 3, characterized in that, the parameter information includes the first coordinate of the geometric center of the target to be measured in the width direction of the first image; The preset adjustment parameter is obtained based on the first coordinate and the second coordinate obtained in the (i - 1)-th iteration. The second coordinate is the coordinate of the geometric center of the first projection area of the envelope body on the first image in the width direction of the first image.

5. The method according to claim 1, wherein, the step of projecting the envelope body into the first image according to the first coordinate information to obtain a first projection area includes: projecting all N corner points included in the envelope body into the first image to obtain N projection points, where N is an integer greater than 1; determining the first projection area of the envelope body on the first image according to the maximum and minimum values of the coordinates of the N projection points in the width direction of the first image and the maximum and minimum values of the coordinates of the N projection points in the height direction of the first image; the second height information corresponds to the height of the first projection area in the first image.

6. The method according to claim 1, wherein, the step of obtaining the category information of the target to be measured and the parameter information of the target to be measured in the first image includes: obtaining an original image from a vision sensor; detecting the target and the category information of the target in the original image based on a target detector, where the target detector is obtained by training an initial target detection network built with sample targets and sample target categories; projecting the target into the original image to obtain a first image and the parameter information of the target in the original image.

7. The method according to claim 1, wherein, after using the distance value of the i-th iteration as the ranging value, the method further includes: judging whether a second preset relationship is satisfied between the first width information included in the parameter information and the second width information corresponding to the first projection area obtained in the i-th iteration; if the second preset relationship is not satisfied between the first width information and the second width information, then performing a second iteration process, where the second iteration process includes: determining the viewing angle value of the j-th iteration, and determining the second coordinate information of the envelope body in the space coordinate system according to the viewing angle value of the j-th iteration, the ranging value, and the size information; projecting the envelope body into the first image according to the second coordinate information to obtain a second projection area; j is a positive integer; when the second width information corresponding to the second projection area satisfies the first preset relationship with the first width information, and / or when j is equal to the second iteration threshold, using the viewing angle value of the j-th iteration as the angle measurement value.

8. A ranging device, wherein, it includes: an acquisition module, configured to acquire the category information of a target to be measured and the parameter information of the target to be measured in a first image, where the target to be measured is any one of at least one target included in the first image, and the parameter information includes first height information; a first determination module, configured to determine the envelope body corresponding to the target to be measured in the space coordinate system and the size information of the envelope body according to the category information; The first execution module is used to execute the first iteration process, and the first iteration process includes: determining the distance value of the i-th iteration, and determining the first coordinate information of the envelope in the space coordinate system according to the distance value of the i-th iteration, the preset viewing angle, and the size information; projecting the envelope onto the first image according to the first coordinate information to obtain a first projection area; i is a positive integer; The second determination module is used to use the distance value of the i-th iteration as the ranging value when the second height information corresponding to the first projection area satisfies a first preset relationship with the first height information, and / or when i is equal to the first iteration threshold.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, wherein, when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images