Method and device for determining distance between target object and vehicle, medium and electronic equipment

Abstract

The embodiment of the invention discloses a method and device for determining the distance between a target object and a vehicle, a medium and electronic equipment. The method comprises the steps that the current vehicle mode of the vehicle is determined; determining a distance measurement mode corresponding to the current vehicle mode; determining a target object from the external environment image of the vehicle; determining a first image plane coordinate corresponding to a to-be-measured pixel point of the target object in the external environment image; and determining the distance between the target object and the vehicle in the space based on the distance measurement mode and the first image plane coordinate. According to the embodiment of the invention, the method can meet the requirements of distance measurement under different scenes and different precisions, does not need to add more sensors, avoids the use of more data to determine the distance between the target object and the vehicle, and reduces the cost of software and hardware.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of computer vision and image processing, and in particular to a method, device, medium and electronic device for determining the distance of a target object relative to a vehicle. Background Art

[0002] In the image processing process, it is usually necessary to obtain the distance between the target object and the camera and complete the corresponding downstream tasks based on the distance. For example, in the field of intelligent driving, vehicle braking and avoidance are achieved by measuring the distance between the target object and the vehicle. In the field of industrial automation, the distance between the target object and the robot is measured to achieve robot object grasping.

[0003] For different application tasks, the accuracy requirements for distance estimation of the target object are usually different. For example, the sentry mode of the car has high requirements for the accuracy of distance estimation, while the camping mode only needs to judge whether the target object is approaching from a distance, and the accuracy requirements for distance estimation are not high. In the related art, distance estimation can be achieved by using the features of the target object in the image. For example, the distance of the target object (target vehicle) is estimated based on the size of the license plate in the image, but due to the differences in the features of the target object, the accuracy of estimating the distance using the features of the target object is low, which is difficult to meet the needs of some application tasks. Summary of the invention

[0004] Embodiments of the present disclosure provide a method, device, medium, and electronic device for determining the distance of a target object relative to a vehicle.

[0005] An embodiment of the present disclosure provides a method for determining the distance of a target object relative to a vehicle, the method comprising: determining a current vehicle mode of a host vehicle; determining a ranging mode corresponding to the current vehicle mode; determining a target object from an external environment image of the host vehicle; determining a first image plane coordinate corresponding to a pixel point to be measured of the target object in the external environment image; and determining the distance between the target object and the host vehicle in space based on the above ranging mode and the first image plane coordinate.

[0006] According to another aspect of an embodiment of the present disclosure, a device for determining the distance of a target object relative to a vehicle is provided, the device comprising: a first determination module, used to determine a current vehicle mode of the vehicle; a second determination module, used to determine a ranging mode corresponding to the current vehicle mode; a target determination module, used to determine a target object from an external environment image of the vehicle; a point determination module, used to determine a first image plane coordinate corresponding to a pixel point to be measured of the target object in the external environment image; and a distance determination module, used to determine the distance between the target object and the vehicle in space based on the above ranging mode and the first image plane coordinate.

[0007] According to another aspect of an embodiment of the present disclosure, a system for determining the relative distance of a target object to a vehicle is provided, comprising: at least one camera device, a processing unit, and a storage unit; at least one camera device is used to obtain an image of the external environment of a vehicle; a storage unit is used to store ranging modes corresponding to various vehicle modes of the vehicle; a processing unit is used to determine a current vehicle mode of the vehicle; determine the ranging mode corresponding to the current vehicle mode; determine a target object from the external environment image of the vehicle; determine a first image plane coordinate corresponding to a pixel point to be measured of the target object in the external environment image; and determine the distance between the target object and the vehicle in space based on the above ranging mode and the first image plane coordinate.

[0008] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned method for determining the distance of a target object relative to a vehicle.

[0009] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; and a processor for reading executable instructions from the memory and executing the instructions to implement the above-mentioned method for determining the distance of a target object relative to a vehicle.

[0010] Based on the method, device, medium and electronic device for determining the distance of a target object relative to a vehicle provided by the above-mentioned embodiments of the present disclosure, the current vehicle mode of the vehicle is determined, the current distance measurement mode is determined, and then the first image plane coordinates corresponding to the pixel point to be measured of the target object in the external environment image are determined. Based on the determined distance measurement mode and the first image plane coordinates, the distance between the target object and the vehicle in space can be determined. As a result, the present disclosure realizes the determination of the distance of the target object relative to the vehicle based on the distance measurement mode matching the current vehicle mode, meets the distance measurement requirements under different scenarios and different precisions, and does not need to add more sensors, avoiding the use of more data to determine the distance of the target object relative to the vehicle, thereby reducing software and hardware costs on the basis of improving the distance measurement accuracy.

[0011] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0013] Figure 1 This is an exemplary application scenario of determining the distance of a target object relative to a vehicle provided by the present disclosure;

[0014] Figure 2 is a schematic diagram of a process for determining the distance of a target object relative to a vehicle provided by an exemplary embodiment of the present disclosure;

[0015] Figure 3 is a schematic diagram of a process for determining the distance of a target object relative to a vehicle in a close-range mode provided by an exemplary embodiment of the present disclosure;

[0016] Figure 4 A schematic diagram of a first image plane coordinate of a pixel point to be measured provided by an exemplary embodiment of the present disclosure;

[0017] Figure 5 A schematic diagram of a second image plane coordinate of a pixel point to be measured provided by an exemplary embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram of a third image plane coordinate of a pixel point to be measured provided by an exemplary embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram of a process for determining the distance of a target object relative to a vehicle in a long-distance mode provided by an exemplary embodiment of the present disclosure;

[0020] Figure 8 is a schematic diagram of equidistant lines in an image provided by an exemplary embodiment of the present disclosure;

[0021] Fig. 9 is a schematic diagram of a flow chart of determining the position relationship between a target object and an equidistant line in a long-distance mode provided by an exemplary embodiment of the present disclosure;

[0022] Fig.10 is a schematic diagram of a flow chart of determining a positional relationship between a target object and an equidistant line in a long-distance mode provided by another exemplary embodiment of the present disclosure;

[0023] Fig.11 is a structural schematic diagram of a device for determining the distance of a target object relative to a vehicle provided by an exemplary embodiment of the present disclosure;

[0024] Fig.12 is a schematic structural diagram of a device for determining the distance of a target object relative to a vehicle provided by another exemplary embodiment of the present disclosure;

[0025] Fig.13 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0027] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0028] SUMMARY OF THE DISCLOSURE

[0029] In the process of realizing the present disclosure, the inventors have found through research that in different vehicle modes, the distance measurement accuracy requirements for the distance between the target objects in the surrounding environment and the vehicle are also different. For example, in sentinel mode, the target objects are often very close to the vehicle, and the accuracy requirements for distance measurement are high; while in camping mode, the target objects are often far away from the vehicle, and the accuracy requirements for distance measurement are low, but the range requirements are large. In order to meet the requirements of high-precision and wide-range distance measurement at the same time. In the related art, distance estimation is achieved based on the features of the target objects in the image, but due to the differences in the features of the target objects, the accuracy of estimating the distance using the features of the target objects is low, which is difficult to meet the requirements of some application tasks. In order to improve the accuracy of distance estimation, it may be necessary to use high-precision sensors, which in turn leads to an increase in hardware costs and an increase in the amount of software algorithm calculations.

[0030] The sentinel mode is used to indicate that the vehicle is in a parking state, and the vehicle and its surrounding environment are determined by the vehicle's camera device. The distance estimation accuracy of surrounding targets is required to be high. The camping mode is used to indicate that the vehicle is in a long-term power supply support mode to meet the long-term power demand of passengers in the car. In this mode, passengers are usually in a camping state such as watching movies, lighting, sleeping, etc. in the car. The target object is far away from the vehicle, so the accuracy of distance estimation is required to be low.

[0031] In addition to the above-mentioned sentinel mode and camping mode, the vehicle may also have other modes, such as rainy night mode, pet mode, normal driving mode, etc. For different vehicle modes, the accuracy and range requirements for distance estimation of surrounding targets are often different.

[0032] Exemplary Systems

[0033] Figure 1 This is an exemplary application scenario of determining the distance of a target object relative to a vehicle provided by the present disclosure, such as Figure 1As shown, the vehicle 11 can be in different vehicle modes, such as sentry mode, camping mode, etc., and the external environment image of the vehicle 11 is collected based on the camera device 111 set on the vehicle 11. Using the method for determining the relative distance of the target object to the vehicle disclosed in the present invention, a matching ranging mode can be determined based on different vehicle modes, and the target object 12 can be identified on the collected external environment image to determine the pixel points to be measured in the external environment image of the target object 12, thereby determining the distance between the target object 12 and the vehicle 11 in space based on the pixel points to be measured and using the ranging mode corresponding to the current vehicle mode. The technical solution for determining the relative distance of the target object 12 to the vehicle disclosed in the present invention combines the ranging mode corresponding to the vehicle mode to determine the distance between the target object 12 and the vehicle 11, which is conducive to improving the accuracy of the determined distance between the vehicle 11 and the target object 12.

[0034] The target object 12 may be a pedestrian, a vehicle, a pet, etc.

[0035] In the present disclosure, the ranging mode is used to indicate the working mode that can meet different distance measurement requirements. The ranging mode in the present disclosure includes at least a short-distance mode and a long-distance mode. Among them, the short-distance mode is used to indicate the working mode that requires a higher accuracy of distance measurement of the target object, such as the distance measurement accuracy error is limited to 0.01 meters; the long-distance mode is used to indicate the working mode that requires a lower accuracy of distance measurement of the target object, such as the distance accuracy error is within 1 meter. The present disclosure is not limited to only these two ranging modes. According to actual needs, ranging modes that can meet other distance measurement needs can also be added.

[0036] The camera device provided on the vehicle 11 indicates a camera device provided on the vehicle for collecting images of the surrounding environment, which may include a camera of a front view angle, a camera of a left front view angle, a camera of a right front view angle, a camera of a rear view angle, a camera of a left rear view angle, a camera of a right rear view angle, etc. The present disclosure does not limit the number, installation position, and viewing angle range of the camera device.

[0037] Exemplary Methods

[0038] Figure 2 is a flow chart of a method for determining the distance of a target object relative to a vehicle provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to an electronic device, and the electronic device can be arranged on the above Figure 1 On the vehicle 11 shown in the figure, Figure 2 As shown, the following steps are included:

[0039] Step 201, determining the current vehicle mode of the vehicle.

[0040] In the present disclosure, the current vehicle mode of the vehicle is used to indicate the vehicle mode the vehicle is currently in, such as the current vehicle mode is the sentry mode.

[0041] In some implementations, the user can set the vehicle mode of the vehicle through the display panel or button of the vehicle-mounted terminal or mobile terminal. For example, the vehicle mode can be selected through the application function setting item on the vehicle-mounted terminal. Specifically, the vehicle mode displayed on the display panel of the vehicle-mounted terminal can be selected. The operation form can be a click operation, a long press operation, etc. The embodiments of the present disclosure do not specifically limit the above operation forms.

[0042] In other implementations, the vehicle mode of the vehicle may also be set by voice, gesture, etc., for example, by emitting a voice of "sentry mode".

[0043] Among them, the vehicle mode may include sentinel mode, camping mode, etc. Among them, the sentinel mode is used to indicate that when the vehicle is in a parking state, a mode in which the vehicle and the surrounding environment are determined by various sensors and cameras on the vehicle body. The above-mentioned camping mode is used to indicate that the vehicle is in a mode of long-term power supply support to meet the long-term power needs of passengers in the car. In this mode, passengers are usually in a camping state such as watching movies, lighting, sleeping, etc. in the car. In addition to the above-mentioned sentinel mode and camping mode, the vehicle may also have other modes, such as rainy night mode, pet mode, normal driving mode, etc. For different vehicle modes, the accuracy and range requirements of the distance estimation of surrounding targets are often different.

[0044] Step 202, determining the distance measurement mode corresponding to the current vehicle mode.

[0045] In the present disclosure, the ranging mode is used to indicate a working mode that can meet different distance measurement requirements. The ranging mode in the present disclosure includes at least a short-distance mode and a long-distance mode. Among them, the short-distance mode is used to indicate a working mode that requires a higher accuracy of distance measurement of the target object, such as the distance measurement accuracy error is limited to 0.01 meters, and the long-distance mode is used to indicate a working mode that requires a lower accuracy of distance measurement of the target object, such as the distance accuracy error is within 1 meter. The present disclosure is not limited to only these two ranging modes. According to actual needs, ranging modes that can meet other distance requirements can also be added.

[0046] In the present disclosure, the ranging mode corresponding to each vehicle mode can be pre-set according to the accuracy requirements of the distance measurement of the surrounding target objects by various vehicle modes, and the mapping relationship between the vehicle mode and the ranging mode is stored in a ranging mode mapping table. Therefore, after the current vehicle mode is determined, the ranging mode corresponding to the current vehicle mode can be directly obtained in the ranging mode mapping table.

[0047] Further, after executing step 202, execute step 205.

[0048] Step 203: determine the target object from the external environment image of the vehicle.

[0049] The external environment image is used to indicate the image of the vehicle's surrounding environment acquired by a camera device installed on the vehicle. The target object may be a pedestrian, a bicycle, a motor vehicle, etc.

[0050] In the present disclosure, the target object in the external environment image can be detected by a target detection model or a target detection algorithm. The target detection model can be a region-based convolutional neural network (R-CNN) model, a fast R-CNN model, a YOLO (You only look once) target detection model, etc.

[0051] It should be noted that the present disclosure does not limit the execution order of step 203 and step 201. Step 203 may be executed first, and then step 201; step 201 may be executed first, and then step 203; or step 201 and step 203 may be executed simultaneously.

[0052] Step 204: determine the first image plane coordinates corresponding to the pixel point to be measured of the target object in the external environment image.

[0053] The pixel point to be measured is used to indicate the pixel point that is closest to the target object in the external environment image and the vehicle, and is the pixel point used to measure the distance between the target object and the vehicle. The first image plane coordinate is used to indicate the pixel coordinate of the pixel point to be measured in the image coordinate system of the external environment image.

[0054] In some implementations, after detecting the target object in the image through the target detection model, the pixel points to be measured of the target object in the external environment image can be further acquired through the deep learning model.

[0055] In other implementations, by further training and optimizing the target detection model, an improved target detection model is obtained. The improved target detection model can classify and identify targets in the external environment image, and output the 2D box where the target is located and the pixel points to be measured on the target.

[0056] In the specific implementation, a distance point estimation subnetwork is added to the target detection model. Thus, the target detection subnetwork in the target detection model can classify and identify the target objects in the image, and obtain the 2D box where the target objects are located. The distance point estimation subnetwork can estimate the distance between each point on the target object and the vehicle, and identify the point closest to the vehicle as the pixel point to be measured.

[0057] Step 205 : determining the distance between the target object and the host vehicle in space based on the distance measurement mode and the first image plane coordinates.

[0058] The distance between the target object and the vehicle in space refers to the absolute distance between the target object and the vehicle in the real world.

[0059] In the present disclosure, if the ranging mode is the short-distance mode, the distance between the target object and the vehicle in space can be accurately calculated according to the first image plane coordinates. For details, see Figure 3 If the distance measurement mode is the long-distance mode, the distance between the target object and the vehicle in space can be estimated according to the first image plane coordinates. For details, see Figure 7 The embodiment shown.

[0060] The method for determining the distance between a target object and a vehicle provided in this embodiment determines the current vehicle mode of the vehicle, determines the current distance measurement mode, and then determines the first image plane coordinates corresponding to the pixel point to be measured of the target object from the external environment image. Based on the determined distance measurement mode and the first image plane coordinates, the distance between the target object and the vehicle in space can be determined. Thus, the present disclosure realizes the determination of the distance of the target object relative to the vehicle based on the distance measurement mode matching the current vehicle mode, meets the distance measurement requirements under different scenes and different precisions, and does not need to add more sensors, avoiding the use of more data to determine the distance of the target object relative to the vehicle, thereby reducing software and hardware costs on the basis of improving the distance measurement accuracy.

[0061] like Figure 3 As shown in the above Figure 2 Based on the illustrated embodiment, step 205 includes the following steps:

[0062] Step 251 : determining the second image plane coordinates of the pixel point to be measured in the dedistorted coordinate system based on the intrinsic parameter matrix and distortion parameters of the camera device that collects the external environment image.

[0063] Among them, the camera device for collecting external environment images can be a camera device with a fisheye lens. The fisheye lens has a large field of view (FOV) and can collect images with a super-large viewing angle, such as a 180-degree viewing angle.

[0064] Among them, the intrinsic parameter matrix of the camera device is used to describe the parameters of the internal properties of the camera device, wherein the intrinsic parameter matrix includes focal length, principal point (the position of the optical axis relative to the image plane), etc. The distortion parameters are used to describe the distortion phenomenon caused by the non-ideal optical system during the imaging process of the camera device, including radial distortion parameters and tangential distortion parameters.

[0065] In this embodiment, the intrinsic parameter matrix and the distortion parameters of the camera device may be determined by a pre-calibration method, for example, the intrinsic parameter matrix and the distortion parameters may be determined by a checkerboard calibration method.

[0066] In some implementations, the first image plane coordinates of the pixel point to be measured in the collected original external environment image are converted based on the above-mentioned intrinsic parameter matrix and distortion parameters to obtain the second image plane coordinates in the dedistorted coordinate system.

[0067] The dedistorted coordinate system is an image coordinate system for the external environment image, which is an image coordinate system obtained by removing radial distortion and tangential distortion using distortion parameters.

[0068] For example, see Figure 4 , which illustrates the external environment image and the pixel point P0 to be measured in the external environment image, whose first image plane coordinate is (494, 285). After dedistorting the external environment image using the intrinsic parameter matrix and distortion parameters, we get Figure 5 Schematic dedistorted image, Figure 4 The point P0 in is mapped to Figure 5 Point P1 in (corresponding to Figure 4 The corresponding second image plane coordinates are (585, 358).

[0069] Step 252: Determine the third image plane coordinates of the pixel point to be measured in the bird's-eye view coordinate system based on the transformation matrix and the second image plane coordinates.

[0070] The transformation matrix is ​​used to indicate a matrix for converting the dedistorted image into a bird's-eye view.

[0071] In this embodiment, the transformation matrix for converting the dedistorted image to the bird's-eye view can be determined by a perspective transformation algorithm. For example, four points are selected from the dedistorted image, and then four points in the bird's-eye view corresponding to the four points are determined from the bird's-eye view. The transformation matrix is ​​calculated using mathematical theories in related technologies.

[0072] In some implementations, the third image plane coordinates in the bird's-eye view coordinate system can be obtained based on the above transformation matrix by obtaining the second image plane coordinates of the pixel point to be measured in the dedistorted image.

[0073] The bird's-eye view coordinate system is used to indicate the coordinate system where the bird's-eye view is located, and is a coordinate system converted from the dedistorted coordinate system using a transformation matrix.

[0074] For example, Figure 4 The first image plane coordinates (494, 285) of point P0 in is mapped to Figure 5 Point P1 in (corresponding to Figure 4 The second image plane coordinates of P0 in the image are (585, 358), and after transforming the second image plane coordinates through the transformation matrix, we can get Figure 6 The third image plane coordinates in the illustrated bird's-eye view are, for example, P2 (534, 436).

[0075] Step 253, based on the pixel scale and the third image plane coordinates in the bird's-eye view coordinate system, determine the distance between the target object and the vehicle in space.

[0076] The pixel scale is the ratio between the pixel distance between two points in the bird's-eye view and the absolute distance between the two points in space. The pixel scale can be determined by offline calibration. For example, if the distance of 10 pixels in the bird's-eye view is 10 pixels and the absolute distance in space is 10 centimeters, the pixel scale s = 1 centimeter / pixel.

[0077] In some implementations, the pixel distance between the pixel point to be measured in the bird's-eye view and the vehicle in the bird's-eye view is first determined, and then the distance in space corresponding to the pixel distance is determined according to the pixel scale. In specific implementation, when the position of the camera device in the vehicle is fixed, the position of the outer contour of the vehicle in the bird's-eye view is also fixed. A vertical line is drawn from the pixel point to be measured in the bird's-eye view to the outer contour of the vehicle, and the pixel point on the vehicle closest to the pixel point to be measured can be determined.

[0078] For example, see Figure 6 A bird's-eye view, where the pixel point to be measured on the target object is P2. A perpendicular line is drawn from point P2 to the contour of the vehicle to obtain point P3 (534, 526). Therefore, the pixel distance between points P2 and P3 in the bird's-eye view is 90 pixels. According to the pixel scale s=1 cm / pixel shown above, the actual distance between the target object and the vehicle in space is 90 cm.

[0079] Through the above steps 251-253, this embodiment uses the pre-calibrated internal parameter matrix, distortion parameters, transformation matrix and pixel scale of the camera device to achieve the accurate distance between the target object and the vehicle in the close-range mode based only on the external environment image, thereby meeting the high-precision distance measurement requirements, and there is no need to add more sensors to the hardware, avoiding the use of more data to determine the distance of the target object relative to the vehicle, thereby reducing software and hardware costs while improving the accuracy and efficiency of distance measurement.

[0080] In another optional implementation, in the close-range mode, the distance corresponding to the first image plane coordinate can also be obtained from a distance mapping table, which records the mapping relationship between each first image plane coordinate and the distance to the vehicle in space.

[0081] Specifically, the mapping relationship between the first image plane coordinates recorded in the distance mapping table and the distance to the vehicle in space can be determined by offline calibration. When the installation position and installation angle of the camera device are determined, the first image plane coordinates corresponding to each point in the different images collected by the camera device and the corresponding distance to the vehicle in space are determined. Therefore, after pre-calibrating the actual spatial distance corresponding to each pixel point offline and storing it in the distance mapping table, the distance mapping table can be queried to obtain the distance between the target object and the vehicle in space based on the first image plane coordinates of each pixel point to be measured.

[0082] In this implementation, after obtaining the pixel point to be measured and the corresponding first image plane coordinates through the target detection model, the distance between the target object and the vehicle in space can be obtained by querying the distance mapping table. The implementation is simple, the algorithm complexity is low, the calculation amount is small, and it is conducive to reducing software costs.

[0083] like Figure 7 As shown in the above Figure 2 Based on the illustrated embodiment, step 205 includes the following steps:

[0084] Step 251 ′: determining the positional relationship between the pixel point to be measured and at least one equidistant line based on the first image plane coordinates and at least one pre-calibrated equidistant line.

[0085] Among them, at least one equidistant line is an equidistant line in the image coordinate system where the external environment image is located. Each equidistant line can be determined by an offline calibration method. When the installation position and installation angle of the camera device of the vehicle are determined, the distance can be directly measured in the field of view of the camera device, and each equidistant point from the vehicle can be calibrated, and the pixel coordinates of the equidistant points on each equidistant line in the image can be recorded.

[0086] Furthermore, after recording the pixel coordinates of each equidistant point of each equidistant line in the image, the equidistant line equations of each equidistant line can be fitted by curve fitting.

[0087] For example, see Figure 8 , which intuitively shows the 2m-7m equidistant lines in the image captured by the fisheye camera. Figure 2 After step 204 in the illustrated embodiment obtains the first image plane coordinates corresponding to the pixel point to be measured, if the first image plane coordinates of point P0 in 8 are (185, 494), and the coordinates of a pixel point on the 2-meter equidistant line are obtained as (185, 454) and the coordinates of a pixel point on the 3-meter equidistant line are obtained as (185, 527), then it can be determined that the positional relationship between the pixel point to be measured and at least one equidistant line is that the pixel point to be measured is located between the 2-meter equidistant line and the 3-meter equidistant line.

[0088] Step 252 ′: determine the distance between the target object and the vehicle in space based on the positional relationship between the pixel point to be measured and at least one equidistant line.

[0089] Among them, since high distance accuracy is often not required in the long-distance ranging mode, the distance between the target object and the vehicle in space can be estimated based on the positional relationship between the pixel point to be measured and at least one equidistant line, for example, Figure 8 If the pixel point to be measured is between the 2-meter equidistant line and the 3-meter equidistant line, the distance between the target object and the vehicle in space can be estimated to be any value such as 2.3 meters, 2.5 meters, or 2.7 meters, or the distance between the target object and the vehicle in space can be estimated to be in a range of 2 meters to 3 meters.

[0090] In this embodiment, after estimating the distance between the target object and the vehicle, you can also choose to perform a warning operation based on the distance range. For example, if the distance between the target object and the vehicle is less than 3 meters, the warning prompt information can be directly output through the vehicle-mounted terminal or the user's mobile terminal.

[0091] Through the above steps 251'-252', this embodiment can estimate the approximate distance between the target object and the vehicle in the long-distance mode based on the equidistant lines in the external environment image and the first image plane coordinates of the pixel points to be measured through the pre-calibrated equidistant lines, thereby meeting the needs of a wide range of distance measurements. In addition, the distance is estimated directly based on the positional relationship between the pixel points to be measured and the equidistant lines, which reduces the amount of calculation and further improves the efficiency of determining the distance.

[0092] like Fig. 9 As shown in the above Figure 7 Based on the illustrated embodiment, step 251' includes the following steps:

[0093] Step 2511, obtaining the equidistant line equation corresponding to at least one equidistant line in the first equidistant line table.

[0094] The first equidistant line table records the mapping relationship between the equidistant line equations of each equidistant line in the image coordinate system and the distance from the vehicle in space.

[0095] In this embodiment, when calibrating the equidistant lines offline, the distance is actually measured in the field of view of the camera device, and then the equidistant points at various distances from the vehicle are calibrated, and the pixel coordinates of the equidistant points on each equidistant line in the image are recorded.

[0096] For example, when performing offline calibration, 20 equidistant points with an actual distance of 2 meters from the vehicle may be calibrated first, and then the equidistant line equation of the 2-meter equidistant line may be fitted based on the 20 equidistant points by curve fitting; and by analogy, the equidistant line equations of the 3-meter equidistant line, the 4-meter equidistant line, the 5-meter equidistant line, etc. may be fitted. After fitting the equidistant line equations, the mapping relationship between the equidistant line equations and the distance from the vehicle in space may be stored in the first equidistant line table.

[0097] Step 2512: Determine the positional relationship between the pixel point to be measured and at least one equidistant line based on the first image plane coordinates and the equidistant line equation of at least one equidistant line.

[0098] In some embodiments, the horizontal coordinate in the first image plane coordinate can be input into each equidistant line equation to obtain the vertical coordinate corresponding to the horizontal coordinate on each equidistant line, and then the vertical coordinate corresponding to the horizontal coordinate on each equidistant line can be compared with the vertical coordinate in the first image plane coordinate to determine the positional relationship between the pixel point to be measured and at least one equidistant line.

[0099] For example, the first image plane coordinate is (185, 494). When the horizontal coordinate 185 in the first image plane coordinate is input into the equidistant line equation of the 2-meter equidistant line, the corresponding vertical coordinate is 442. When the coordinate 185 in the first image plane coordinate is input into the equidistant line equation of the 3-meter equidistant line, the corresponding vertical coordinate is 502. 442<494<502. Therefore, it can be determined that the positional relationship between the pixel point to be measured and at least one equidistant line is between the 2-meter equidistant line and the 3-meter equidistant line.

[0100] This embodiment discloses, through the above steps 2511-2512, an implementation method for determining the positional relationship between the pixel point to be measured and the equidistant line based on the equidistant line equations of each equidistant line and the first image plane coordinates. Since the number of equidistant lines is small, the corresponding vertical coordinate on each equidistant line is determined based on the horizontal coordinate in the first image plane coordinate of the pixel point to be measured, and the amount of calculation for determining the positional relationship between the pixel point to be measured and the equidistant line is very small, which helps to improve the efficiency of determining the distance.

[0101] like Fig.10 As shown in the above Figure 7 Based on the illustrated embodiment, step 251' includes the following steps:

[0102] Step 2511 ′, obtaining in the second equidistant line table each vertical coordinate corresponding to the horizontal coordinate in the first image plane coordinate.

[0103] The second equidistant line table records the mapping relationship between the pixel coordinates of the points on each equidistant line and the distance from the vehicle in space.

[0104] In this embodiment, when calibrating equidistant lines offline, the distance is actually measured in the field of view of the camera device, and then the equidistant points at each distance from the vehicle are calibrated, and the pixel coordinates of each equidistant point on each equidistant line in the image are recorded, and the mapping relationship between the pixel coordinates of the points on each equidistant line and the distance to the vehicle in space is stored in a second equidistant line table.

[0105] Step 2512', compare the acquired vertical coordinates with the vertical coordinates in the first image plane coordinates, and determine the positional relationship between the pixel point to be measured and at least one equidistant line according to the comparison result.

[0106] In some embodiments, the horizontal coordinate in the first image plane coordinate can be used as a retrieval condition, and the vertical coordinate of each equidistant point in the second equidistant line table or corresponding to the horizontal coordinate can then be compared with the vertical coordinate in the first image plane coordinate to determine the positional relationship between the pixel point to be measured and at least one equidistant line.

[0107] For example, the coordinates of the first image plane are (185, 494). Based on 185, the second equidistant line table is searched and the vertical coordinate of the equidistant point with the horizontal coordinate of 185 on the 2-meter equidistant line is 442, while the vertical coordinate of the equidistant point with the horizontal coordinate of 185 on the 3-meter equidistant line is 502. 442<494<502. Therefore, it can be determined that the positional relationship between the pixel point to be measured and at least one equidistant line is between the 2-meter equidistant line and the 3-meter equidistant line.

[0108] This embodiment discloses, through the above steps 2511'-2512', an implementation method for determining the positional relationship between the pixel point to be measured and the equidistant line based on the pixel coordinates of the equidistant points on each equidistant line and the first image plane coordinates. Since the pixel coordinates of the equidistant points are calibrated offline with high accuracy, the accuracy of determining the positional relationship based on the pixel coordinates of the equidistant points on the equidistant line can be greatly improved, which helps to improve the accuracy and efficiency of determining the distance.

[0109] Exemplary Devices

[0110] Fig.11is a schematic diagram of a device for determining the distance of a target object relative to a vehicle provided by an exemplary embodiment of the present disclosure. The device can be arranged in the above Figure 1 On the vehicle 11 shown in the figure, Fig.11 As shown, the device comprises:

[0111] A first determination module 111 is used to determine the current vehicle mode of the vehicle;

[0112] A second determination module 112, used to determine the distance measurement mode corresponding to the current vehicle mode;

[0113] A target determination module 113 is used to determine a target object from an external environment image of the vehicle;

[0114] The point determination module 114 is used to determine the first image plane coordinates corresponding to the pixel point to be measured in the external environment image of the target object;

[0115] The distance determination module 115 is used to determine the distance between the target object and the vehicle in space based on the distance measurement mode and the first image plane coordinates.

[0116] Fig.12 is a schematic diagram of a structure of a device for determining the distance of a target object relative to a vehicle provided by another exemplary embodiment of the present disclosure, such as Fig.12 As shown, in Fig.11 Based on the illustrated embodiment, in some implementations, in response to the ranging mode being the short-distance mode, the distance determination module 115 includes:

[0117] The first determination submodule 1151 is used to determine the second image plane coordinates of the pixel point to be measured in the dedistorted coordinate system based on the intrinsic parameter matrix and distortion parameters of the camera device that collects the external environment image;

[0118] The second determining submodule 1152 is used to determine the third image plane coordinates of the pixel point to be measured in the bird's-eye view coordinate system based on the transformation matrix and the second image plane coordinates;

[0119] The third determination submodule 1153 is used to determine the distance between the target object and the vehicle in space based on the pixel scale and the third image plane coordinates in the bird's-eye view coordinate system. In some embodiments, in response to the ranging mode being the long-distance mode, the distance determination module 115 includes:

[0120] The position relationship determination submodule 1154 determines the position relationship between the pixel point to be measured and the at least one equidistant line based on the first image plane coordinates and at least one pre-calibrated equidistant line, wherein the at least one equidistant line is an equidistant line in the image coordinate system where the external environment image is located;

[0121] The distance determination submodule 1155 is used to determine the distance between the target object and the vehicle in space based on the positional relationship between the pixel point to be measured and at least one equidistant line.

[0122] In some implementations, in response to the ranging mode being the short-distance mode, the distance determination module 115 includes:

[0123] The query submodule 1156 is used to obtain the distance corresponding to the first image plane coordinate in the distance mapping table, and the distance mapping table records the mapping relationship between each first image plane coordinate and the distance to the vehicle in space.

[0124] In some implementations, the position relationship determination submodule 1154 includes:

[0125] A first acquiring unit 11541 is used to acquire an equidistant line equation corresponding to at least one equidistant line in a first equidistant line table, wherein the first equidistant line table records a mapping relationship between the equidistant line equation of each equidistant line in the image coordinate system and the distance from the vehicle in space;

[0126] The first determining unit 11542 is used to determine the positional relationship between the pixel point to be measured and at least one equidistant line based on the first image plane coordinates and the equidistant line equation of at least one equidistant line.

[0127] In some implementations, the position relationship determination submodule 1154 includes:

[0128] A second acquisition unit 11543 is used to acquire each ordinate corresponding to the abscissa in the first image plane coordinate in a second equidistant line table, wherein the second equidistant line table records a mapping relationship between the pixel coordinates of each point on each equidistant line and the distance from the vehicle in space;

[0129] The second determining unit 11544 is used to compare the acquired vertical coordinates with the vertical coordinates in the first image plane coordinates, and determine the positional relationship between the pixel point to be measured and at least one equidistant line according to the comparison result.

[0130] The beneficial technical effects corresponding to the exemplary embodiment of the present device can be found in the corresponding beneficial technical effects of the above exemplary method section, which will not be repeated here.

[0131] Exemplary Electronic Devices

[0132] Fig.13 A structural diagram of an electronic device provided in an embodiment of the present disclosure includes at least one processor 131 and a memory 132.

[0133] The processor 131 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 13 to perform desired functions.

[0134] The memory 132 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 131 may execute one or more computer program instructions to implement the vehicle posture detection method and / or other desired functions of the various embodiments of the present disclosure described above.

[0135] In one example, the electronic device 13 may further include: an input device 133 and an output device 134 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0136] The input device 133 may also include, for example, a keyboard, a mouse, etc.

[0137] The output device 134 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0138] Of course, to simplify, Fig.13 Only some of the components related to the present disclosure in the electronic device 13 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 13 may also include any other appropriate components.

[0139] Exemplary computer program products and computer-readable storage media

[0140] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for determining the distance of a target object relative to a vehicle according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0141] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0142] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of a method for determining the distance of a target object relative to a vehicle according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0143] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0144] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0145] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0146] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0147] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0148] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0149] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0150] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for determining the distance of a target object relative to a vehicle, comprising: Determining a current vehicle mode of the host vehicle; Determining a distance measurement mode corresponding to the current vehicle mode; Determine a target object from an image of the external environment of the host vehicle; Determine the first image plane coordinates corresponding to the pixel point to be measured of the target object in the external environment image; Based on the distance measurement mode and the first image plane coordinates, the distance between the target object and the host vehicle in space is determined.

2. The method according to claim 1, wherein: In response to the ranging mode being a short-distance mode, determining the distance between the target object and the host vehicle in space based on the ranging mode and the first image plane coordinates includes: Determine the second image plane coordinates of the pixel point to be measured in the dedistorted coordinate system based on the intrinsic parameter matrix and distortion parameters of the camera device that collects the external environment image; Determine the third image plane coordinates of the pixel point to be measured in the bird's-eye view coordinate system based on the transformation matrix and the second image plane coordinates; Based on the pixel scale in the bird's-eye view coordinate system and the third image plane coordinate, the distance between the target object and the host vehicle in space is determined.

3. The method according to claim 1, in response to the ranging mode being a short-distance mode, determining the distance between the target object and the host vehicle in space based on the ranging mode and the first image plane coordinates, comprises: The distance corresponding to the first image plane coordinate is obtained in a distance mapping table, wherein the distance mapping table records a mapping relationship between each first image plane coordinate and a distance from the host vehicle in space.

4. The method according to claim 1, in response to the distance measurement mode being a long-distance mode, determining the distance between the target object and the host vehicle in space based on the distance measurement mode and the first image plane coordinates, comprises: Based on the first image plane coordinates and at least one pre-calibrated equidistant line, determining a positional relationship between the pixel point to be measured and at least one equidistant line, wherein the at least one equidistant line is an equidistant line in the image coordinate system where the external environment image is located; Based on the positional relationship between the pixel point to be measured and at least one equidistant line, the distance between the target object and the host vehicle in space is determined.

5. The method according to claim 4, wherein determining the positional relationship between the pixel point to be measured and at least one equidistant line based on the first image plane coordinates and at least one pre-calibrated equidistant line comprises: Obtaining an equidistant line equation corresponding to the at least one equidistant line in a first equidistant line table, wherein the first equidistant line table records a mapping relationship between the equidistant line equation of each equidistant line in the image coordinate system and the distance from the host vehicle in space; Based on the first image plane coordinates and the equidistant line equation of the at least one equidistant line, a positional relationship between the pixel point to be measured and the at least one equidistant line is determined.

6. The method according to claim 4, wherein determining the positional relationship between the pixel point to be measured and at least one equidistant line based on the first image plane coordinates and at least one pre-calibrated equidistant line comprises: Acquire each ordinate corresponding to the abscissa in the first image plane coordinate in a second equidistant line table, wherein the second equidistant line table records a mapping relationship between the pixel coordinates of a point on each equidistant line and the distance from the vehicle in space; The acquired vertical coordinates are compared with the vertical coordinates in the first image plane coordinates, and the positional relationship between the pixel point to be measured and at least one equidistant line is determined according to the comparison result.

7. A device for determining the distance of a target object relative to a vehicle, comprising: A first determination module, used to determine the current vehicle mode of the vehicle; A second determination module, used to determine the distance measurement mode corresponding to the current vehicle mode; A target determination module, used to determine a target object from an external environment image of the host vehicle; A point determination module, used to determine a first image plane coordinate corresponding to a pixel point to be measured of the target object in the external environment image; A distance determination module is used to determine the distance between the target object and the host vehicle in space based on the ranging mode and the first image plane coordinates.

8. A system for determining the distance of a target object relative to a vehicle, comprising: At least one camera device, a processing unit, and a storage unit; At least one camera device, used to obtain an image of the external environment of the vehicle; The storage unit is used to store the distance measurement modes corresponding to each vehicle mode of the vehicle; The processing unit is used to determine the current vehicle mode of the vehicle; determine the ranging mode corresponding to the current vehicle mode; determine a target object from the external environment image of the vehicle; determine the first image plane coordinates corresponding to the pixel points to be measured of the target object in the external environment image; and determine the distance between the target object and the vehicle in space based on the ranging mode and the first image plane coordinates.

9. A computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program is used to execute the method according to any one of claims 1 to 6.

10. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 6.

11. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 6 is implemented.

Images