Ranging method, apparatus, device, and storage medium

Abstract

Embodiments of the present disclosure provide a ranging method, device, equipment and storage medium. The method comprises: acquiring original images of a driving environment in front of a vehicle through at least two cameras; performing semantic segmentation on the at least two original images to obtain at least two segmented images corresponding to the at least two original images; determining at least two pixel positions of a target according to the at least two segmented images; determining at least two spatial planes where the target is located according to the at least two pixel positions; determining an actual position of the target according to the at least two spatial planes; and determining a distance between the vehicle and the target according to the actual position. Embodiments of the present disclosure can improve the accuracy of ranging and reduce the cost of ranging.

Description

Technical Field

[0001] This disclosure relates to the field of automotive technology, and more particularly to a ranging method, apparatus, device, and storage medium. Background Technology

[0002] With the improvement of people's living standards and the development of automotive technology, more and more vehicles are equipped with intelligent driving and assisted driving functions, which can significantly improve the safety and comfort of vehicles. Advanced functions such as autonomous driving and intelligent suspension require obtaining distance information of vehicles, pedestrians, or road targets in front of the vehicle. Currently, the industry mainly uses three methods to obtain distance information: 1) Using LiDAR for ranging. This solution has high ranging accuracy and can obtain three-dimensional point cloud information, but its hardware cost is high, making it difficult to promote and use on a large scale. Furthermore, conventional LiDAR is limited by installation height, resulting in low point cloud resolution when detecting road surfaces at low angles, making it difficult to use for intelligent control algorithms targeting vehicle vertical performance; 2) Using traditional binocular cameras for ranging. Binocular cameras can perform stereo matching based on the disparity maps of the two cameras to obtain point cloud information, but its… The first approach requires the use of two identical cameras with baseline constraints, incurring additional costs and a large setup volume. Additionally, the use of two identical cameras also wastes resources. Point cloud data is also much larger than image data, requiring additional computing power to process the point cloud. Furthermore, binocular cameras have a longer focal length and a smaller field of view, resulting in a large blind spot in the image. The second approach uses a monocular camera for ranging, which is a lower-cost solution. Currently, widely used monocular ranging methods rely on prior or assumed information, such as pre-defined vehicle width or human height, or default road level. This method has low accuracy and poor robustness. If the target object or surrounding environment does not meet the prior or assumed conditions, the calculation cannot be performed. Summary of the Invention

[0003] This disclosure provides a ranging method, apparatus, device, and storage medium that can improve ranging accuracy and reduce ranging costs.

[0004] In a first aspect, embodiments of this disclosure provide a ranging method, which involves acquiring raw images of the driving environment in front of a vehicle using at least two cameras; performing semantic segmentation on the at least two raw images to obtain at least two corresponding segmented images; determining at least two pixel positions of a target based on the at least two segmented images; determining at least two spatial planes in which the target is located based on the at least two pixel positions; determining the actual position of the target based on the at least two spatial planes; and determining the distance between the vehicle and the target based on the actual position.

[0005] Secondly, embodiments of this disclosure also provide a ranging device, comprising: an image acquisition module for acquiring original images of the driving environment in front of a vehicle through at least two cameras; an image segmentation module for performing semantic segmentation on the at least two original images to obtain at least two corresponding segmented images; a pixel position determination module for determining at least two pixel positions of a target based on the at least two segmented images; a spatial plane determination module for determining at least two spatial planes in which the target is located based on the at least two pixel positions; an actual position determination module for determining the actual position of the target based on the at least two spatial planes; and a distance determination module for determining the distance between the vehicle and the target based on the actual position.

[0006] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0007] One or more processors;

[0008] Storage device for storing one or more programs.

[0009] When the one or more programs are executed by the one or more processors, the one or more processors implement the ranging method as described in the embodiments of this disclosure.

[0010] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the ranging method as described in embodiments of this disclosure.

[0011] The technical solution of this disclosure involves acquiring original images of the driving environment in front of the vehicle using at least two cameras; performing semantic segmentation on the at least two original images to obtain at least two corresponding segmented images; determining at least two pixel positions of a target based on the at least two segmented images; determining at least two spatial planes where the target is located based on the at least two pixel positions; determining the actual position of the target based on the at least two spatial planes; and determining the distance between the vehicle and the target based on the actual position. This disclosure, by acquiring original images using at least two cameras, segmenting the at least two original images to obtain at least two corresponding segmented images, determining at least two spatial planes where the target is located based on the at least two segmented images, determining the actual position of the target based on the at least two spatial planes, and performing distance measurement based on the actual position, can improve the accuracy of distance measurement and reduce distance measurement costs. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a schematic flowchart of the ranging method provided in the embodiments of this disclosure;

[0014] Figure 2 This is a schematic diagram of the segmented image provided in an embodiment of the present invention;

[0015] Figure 3 A plan view of the target location provided in an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram of the actual location of the target provided in an embodiment of the present invention;

[0017] Figure 5 This is a schematic diagram of a ranging device structure provided in an embodiment of the present disclosure;

[0018] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0021] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0022] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0023] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0024] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0025] Figure 1 This is a schematic diagram of the ranging method provided in the embodiments of this disclosure. The embodiments of this disclosure are applicable to situations where a vehicle is driving and measuring the distance to a target (such as an obstacle) on the road ahead. The method can be executed by a ranging device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, such as a mobile terminal, a PC, or a server.

[0026] like Figure 1 As shown, the method includes:

[0027] S110: Obtain raw images of the driving environment in front of the vehicle using at least two cameras.

[0028] The original image includes the target obstacle (impact object). The target obstacle can be a pedestrian, vehicle, speed bump, etc., and this embodiment is not limited to this. Any object that affects the smooth movement of a vehicle during its journey can be considered a target obstacle. At least two cameras can have different pixel counts, different field of view angles, and other different parameters.

[0029] Specifically, during vehicle operation, at least two cameras mounted on the vehicle's front view are used to capture real-time images of target obstacles that may affect the vehicle's smooth operation.

[0030] S120. Perform semantic segmentation on at least two original images to obtain at least two corresponding segmented images.

[0031] Specifically, at least two original images can be input into an image segmentation model for semantic segmentation to obtain at least two corresponding segmented images. One original image corresponds to one segmented image. Alternatively, at least two original images can be input into different image segmentation models for semantic segmentation to obtain corresponding segmented images.

[0032] Optionally, semantic segmentation is performed on at least two original images to obtain at least two corresponding segmented images, including: preprocessing each original image; wherein, the preprocessing includes adjusting the size of the original image and / or eliminating distortion of the original image; and inputting the preprocessed at least two original images into an image segmentation model to obtain at least two corresponding segmented images.

[0033] The image segmentation model can be a pre-trained deep neural network model with instance segmentation or semantic segmentation capabilities. For example... Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the segmentation effect of an embodiment of the present invention. Figure 2 As can be seen, objects of the same type (or the same category) are displayed in the same color, while different categories are displayed in different colors. For example, pedestrians are displayed in red (not shown), vehicles are displayed in purple (not shown), and traffic lights are displayed in yellow (not shown).

[0034] S130. Determine at least two pixel positions of the target based on at least two segmented images.

[0035] Specifically, a segmented image can determine the pixel position of a target in a pixel coordinate system. Depending on the camera, different original images can be obtained, and correspondingly, different segmented images and different pixel positions can be obtained.

[0036] Optionally, determining at least two pixel positions of the target based on at least two segmented images includes: for each segmented image, smoothing the segmented image to obtain a smoothed segmented image; determining the target pixel region based on the smoothed segmented image; determining the contour curve of the target pixel region; determining the lower edge tangent based on the contour curve; and using the pixel position of the lower edge tangent as the pixel position of the target in the pixel coordinate system.

[0037] Specifically, to determine the pixel position of a target from a segmented image, the process is as follows: smooth the segmented image to obtain a smoothed segmented image; filter the smoothed segmented image to obtain a filtered image; determine the target pixel region based on the smoothed segmented image or the filtered image; determine the contour curve of the target ground part of the target pixel region; determine the lower edge tangent of the target pixel region based on the contour curve; and use the pixel position of the lower edge tangent as the pixel position of the target in the pixel coordinate system.

[0038] S140. Determine at least two spatial planes where the target is located based on at least two pixel positions.

[0039] In this embodiment, the spatial plane where the target is located can be determined based on the pixel position. Specifically, the linear equation of the image coordinate system can be determined based on the pixel position, thereby determining the linear equation of the pixel position in the camera coordinate system, which in turn determines the plane bundle equation in the camera coordinate system, and the spatial plane where the target is located can be determined based on the plane bundle equation and the optical center point.

[0040] Optionally, at least two spatial planes in which the target is located are determined based on at least two pixel positions, including: for each pixel position, determining the equation of the lower edge tangent line in the image coordinate system based on the pixel position of the lower edge tangent line; determining the plane bundle equation of the lower edge tangent line in the camera coordinate system based on the equation of the line; substituting the camera optical center coordinates into the plane bundle equation to obtain the spatial plane in which the target is located, wherein the spatial plane is a plane in the camera coordinate system; different spatial planes are located in different camera coordinate systems.

[0041] Specifically, for a pixel position corresponding to a camera, the equation of the lower edge tangent line in the image coordinate system (i.e., the two-dimensional coordinate system) is determined based on the pixel position of the lower edge tangent line: a i1 x+b i1 y+c i1 =0. Where, a i b i and c i These are the coefficients of the equation of the line corresponding to the tangent line at the lower edge of the image.

[0042] Figure 3 A plan view of the target location provided in the embodiments of the present invention, such as Figure 3 As shown, the equation of the lower edge tangent line in the camera coordinate system (i.e., the three-dimensional coordinate system) can be determined based on the line equation:

[0043]

[0044] Where f is the focal length of the camera.

[0045] Therefore, the plane bundle equation of this line in the camera coordinate system can be determined:

[0046] a i1 x+b i1 y+c i1 +λyz-f)=0

[0047] Substituting the coordinates of the camera's optical center (the origin of the camera coordinate system) into the plane bundle equation, we get:

[0048] c i1 -λf=0

[0049]

[0050] The equation of the plane determined by the optical center and the tangent can be obtained, which is the spatial plane where the target is located: The spatial plane where the target corresponding to the first camera is located is denoted as S1, and the equation of plane S1 is as follows:

[0051]

[0052] Similarly, the spatial plane containing the target corresponding to the second camera is denoted as S2, and the equation of plane S2 is as follows:

[0053]

[0054] Among them, spatial planes S1 and S2 are planes in the camera coordinate system; different spatial planes are located in different camera coordinate systems.

[0055] S150. Determine the actual location of the target based on at least two spatial planes.

[0056] Specifically, the intersection line between at least two spatial planes can represent the actual position of the target.

[0057] Optionally, determining the actual position of the target based on at least two spatial planes includes: determining the intersection line between at least two spatial planes; and determining the actual position of the target based on the intersection line.

[0058] In this embodiment, the intersection line between spatial planes S1 and S2 can be used as the position of the lower edge tangent of the target in space, i.e., the actual position.

[0059] Optionally, determining the intersection line between at least two spatial planes includes: transforming the spatial planes in at least two camera coordinate systems into spatial planes in the vehicle coordinate system; and determining the intersection line between at least two spatial planes in the vehicle coordinate system.

[0060] In this embodiment, since the at least two spatial planes are obtained in their respective camera coordinate systems, and there are pose deviations between different camera coordinate systems, it is necessary to translate and rotate the coordinate systems to unify them, thereby solving for the intersection line of the two planes within the same coordinate system. In this example, to facilitate subsequent calculations, the at least two spatial planes can be transformed to the vehicle coordinate system, or the spatial planes in different camera coordinate systems can be transformed to the same camera coordinate system (i.e., a spatial plane in one camera coordinate system can be transformed to a spatial plane in another camera coordinate system), or other transformation methods can be used.

[0061] In this embodiment, we take the transformation of two spatial planes to the vehicle coordinate system as an example: the first spatial plane corresponds to the first camera coordinate system, and the second spatial plane corresponds to the second camera coordinate system; the specific method for transforming the spatial plane of the first camera coordinate system to the spatial plane of the vehicle coordinate system is as follows:

[0062]

[0063] Where: S1' is the spatial plane of the target in the first camera coordinate system after coordinate transformation, i.e., the spatial plane in the vehicle coordinate system; T1 is the coordinate transformation matrix, expressed in homogeneous coordinate form as:

[0064]

[0065] Where: R1 is the rotation matrix between the first camera coordinate system and the vehicle coordinate system, representing the angular change of the coordinate system; t1 is the translation vector between the first camera coordinate system and the vehicle coordinate system, representing the displacement change of the coordinate system. The coordinate system rotation adopts an external rotation method, rotating in the order of pitch angle-roll angle-yaw angle. The calculation method of the rotation matrix R1 is as follows:

[0066] R1 = R 1x (α)*R 1z (γ)*R 1y (β)

[0067] Where: R 1x (α) is the rotation matrix about the y-axis of the first camera coordinate system, i.e., the pitch angle direction; R 1z (γ) is the rotation matrix about the x-axis of the first camera coordinate system, i.e., the roll angle direction; R 1y (β) is the rotation matrix about the z-axis of the first camera coordinate system, i.e., the yaw angle direction.

[0068]

[0069]

[0070]

[0071] Where: α1 is the rotation angle around the x-axis; γ1 is the rotation angle around the z-axis; β1 is the rotation angle around the y-axis. Since the camera is fixed to the vehicle body, it can be assumed that there is no relative displacement between the camera and the vehicle body, that is, the rotation angle between the camera coordinate system and the vehicle coordinate system is fixed. Angles α1, γ1, and β1 can be obtained through calibration or actual measurement, thereby determining the rotation matrix R1.

[0072] The specific method for determining the translation vector t1 is as follows:

[0073]

[0074] Where: t 1x t 1y t 1zThese represent the displacements of the origin of the first camera coordinate system relative to the origin of the vehicle coordinate system in three directions. Similarly, since the camera and the vehicle are fixed, t is considered to be... 1x t 1y t 1z It is known and invariant, and its specific value can be obtained through measurement or calibration. Therefore, the translation vector t1 is determined, that is, the coordinate system transformation matrix T1 is used for calculation.

[0075] Points in the first camera coordinate system will undergo the following changes after being transformed by transformation matrix T1:

[0076]

[0077] The equation of the space plane S1 in the first camera coordinate system Rewritten in homogeneous coordinate matrix form, we have:

[0078]

[0079] Substituting the transformation matrix, the homogeneous equation of the space plane S1' after coordinate transformation is:

[0080]

[0081] The coefficients of the general equation of a plane are:

[0082]

[0083] This completes the transformation from the spatial plane in the first camera coordinate system to the vehicle coordinate system. Using the same method, the transformation matrix T2 from the second camera coordinate system to the vehicle coordinate system can also be determined.

[0084] The general equation coefficients of the plane after transforming the spatial plane in the second camera coordinate system to the vehicle coordinate system are obtained as follows:

[0085] [a i2 `b i2 `c i2 `d i2 `]

[0086] In the vehicle coordinate system, the intersection line L2 between the two spatial planes can be determined by solving the equations of the two planes simultaneously. The specific formula for the intersection line L2 is as follows;

[0087]

[0088] Optionally, determining the actual position of the target based on the intersection line includes: for each camera coordinate system, extracting the maximum and minimum points of the target in the first coordinate component of the camera coordinate system; determining a first plane passing through the maximum point and the camera optical center and parallel to the second coordinate component of the camera coordinate system; determining a second plane passing through the minimum point and the camera optical center and parallel to the second coordinate component of the camera coordinate system; transforming the first and second planes into a first plane and a second plane in the vehicle coordinate system, respectively; determining the first intersection point of the intersection line with the first plane in the vehicle coordinate system; determining the second intersection point of the intersection line with the second plane in the vehicle coordinate system; determining the spatial line segment of the target based on the first and second intersection points; and determining the actual position of the target based on at least two spatial line segments.

[0089] In this embodiment, the area of ​​the target on the intersection line L2 can be determined, that is, the position of the line segment on the intersection line (straight line) can be determined. Specifically, taking two cameras as an example, the position of the line segment on the intersection line L2 corresponding to the first camera is as follows: Figure 4 This is a schematic diagram of the actual location of the target provided in an embodiment of the present invention.

[0090] Extract the maximum and minimum points of the target on the first coordinate component (x-axis) of the camera coordinate system; determine the first plane passing through the maximum point and the camera optical center and parallel to the second coordinate component (y-axis) of the camera coordinate system; determine the second plane passing through the minimum point and the camera optical center and parallel to the second coordinate component of the camera coordinate system; transform the first and second planes in the camera coordinate system to the first and second planes in the vehicle coordinate system, respectively; determine the first intersection point of the intersection line in the vehicle coordinate system with the first plane in the vehicle coordinate system; determine the second intersection point of the intersection line in the vehicle coordinate system with the second plane in the vehicle coordinate system; determine the spatial line segment of the target, i.e., the range of the spatial line segment, based on the first and second intersection points: y 1min ,y 1max Similarly, the range y of the spatial line segment on the intersection line L2 corresponding to the second camera can be obtained. 2min ,y 2max The two intervals are merged to obtain the target space line segment, and the actual position of the target is determined based on the target space line segment.

[0091] Optionally, the actual position of the target can be determined based on at least two spatial line segments, including: merging at least two spatial line segments to obtain a target spatial line segment; if the target is a speed bump, then the position of the target spatial line segment is taken as the actual position of the target; if the target is a pedestrian or a vehicle, then the position of the center point of the target spatial line segment is taken as the actual position of the target.

[0092] In this embodiment, taking two cameras as an example, the spatial line segment ranges corresponding to the two cameras are fused to eliminate the errors present in a single image, thereby determining the spatial line segment L2' equation of the target:

[0093]

[0094] At this point, the target location in the vehicle coordinate system is complete. The appropriate ranging method can be selected according to the target type. If the target is a speed bump, the position of the target spatial line segment is taken as the actual position of the target; if the target is a pedestrian or vehicle, the position of the center point of the target spatial line segment is taken as the actual position of the target.

[0095] S160. Determine the distance between the vehicle and the target based on the actual location.

[0096] Specifically, if the target is a pedestrian, the distance between the vehicle and the target is determined based on the vehicle's front end and its actual position. If the target is a vehicle or a speed bump, the distance between the vehicle and the target is determined based on the vehicle's front tires and their actual position.

[0097] The technical solution of this disclosure involves acquiring original images of the driving environment in front of the vehicle using at least two cameras; performing semantic segmentation on the at least two original images to obtain at least two corresponding segmented images; determining at least two pixel positions of a target based on the at least two segmented images; determining at least two spatial planes where the target is located based on the at least two pixel positions; determining the actual position of the target based on the at least two spatial planes; and determining the distance between the vehicle and the target based on the actual position. This disclosure improves the accuracy and reduces the cost of distance measurement by acquiring original images using at least two cameras, segmenting the original images to obtain at least two corresponding segmented images, determining at least two spatial planes where the target is located based on the at least two segmented images, determining the actual position of the target based on the at least two spatial planes, and measuring distance based on the actual position. Furthermore, the technical solution disclosed in this embodiment does not require additional sensor hardware and can achieve target distance measurement at the software algorithm level, reducing the cost of distance measurement, making it easy to promote and apply, and eliminating the need for stereo matching of at least two images globally. It also requires low computational power for distance calculation of targets of interest.

[0098] Figure 5 This is a schematic diagram of a ranging device provided in an embodiment of the present disclosure; the device includes: an image acquisition module 210, an image segmentation module 220, a pixel position determination module 230, a spatial plane determination module 240, an actual position determination module 250, and a distance determination module 260;

[0099] Image acquisition module 210 is used to acquire raw images of the driving environment in front of the vehicle through at least two cameras;

[0100] Image segmentation module 220 is used to perform semantic segmentation on at least two original images to obtain at least two corresponding segmented images;

[0101] The pixel position determination module 230 is used to determine at least two pixel positions of the target based on the at least two segmented images;

[0102] The spatial plane determination module 240 is used to determine at least two spatial planes in which the target is located based on the at least two pixel positions.

[0103] The actual position determination module 250 is used to determine the actual position of the target based on the at least two spatial planes.

[0104] The distance determination module 260 is used to determine the distance between the vehicle and the target based on the actual location.

[0105] The technical solution of this disclosure involves: acquiring original images of the driving environment in front of the vehicle using at least two cameras via an image acquisition module; performing semantic segmentation on the at least two original images using an image segmentation module to obtain at least two corresponding segmented images; determining at least two pixel positions of the target based on the at least two segmented images using a pixel position determination module; determining at least two spatial planes where the target is located based on the at least two pixel positions using a spatial plane determination module; determining the actual position of the target based on the at least two spatial planes using an actual position determination module; and determining the distance between the vehicle and the target based on the actual position using a distance determination module. This disclosure, by acquiring original images using at least two cameras, segmenting the original images to obtain at least two corresponding segmented images, determining at least two spatial planes where the target is located based on the at least two segmented images, determining the actual position of the target based on the at least two spatial planes, and performing distance measurement based on the actual position, can improve the accuracy of distance measurement and reduce distance measurement costs.

[0106] Optionally, the image segmentation module is specifically used to: preprocess each original image; wherein the preprocessing includes adjusting the size of the original image and / or eliminating distortion of the original image; and inputting at least two preprocessed original images into the image segmentation model to obtain at least two corresponding segmented images.

[0107] Optionally, the pixel position determination module is specifically used for: for each segmented image, smoothing the segmented image to obtain a smoothed segmented image; determining a target pixel region based on the smoothed segmented image; determining the contour curve of the target pixel region; determining the lower edge tangent based on the contour curve; and using the pixel position of the lower edge tangent as the pixel position of the target in the pixel coordinate system.

[0108] Optionally, the spatial plane determination module is specifically used for: for each pixel position, determining the linear equation of the lower edge tangent in the image coordinate system based on the pixel position of the lower edge tangent; determining the plane bundle equation of the lower edge tangent in the camera coordinate system based on the linear equation; substituting the camera optical center coordinates into the plane bundle equation to obtain the spatial plane where the target is located, wherein the spatial plane is a plane in the camera coordinate system; different spatial planes are located in different camera coordinate systems.

[0109] Optionally, the actual position determination module is specifically used to: determine the intersection line between the at least two spatial planes; and determine the actual position of the target based on the intersection line.

[0110] Optionally, the actual position determination module is also used to: transform the spatial planes in at least two camera coordinate systems into spatial planes in the vehicle coordinate system; and determine the intersection line between at least two spatial planes in the vehicle coordinate system.

[0111] Optionally, the actual position determination module is further configured to: for each camera coordinate system, extract the maximum and minimum points of the target in the first coordinate component of the camera coordinate system; determine a first plane passing through the maximum point and the camera optical center and parallel to the second coordinate component of the camera coordinate system; determine a second plane passing through the minimum point and the camera optical center and parallel to the second coordinate component of the camera coordinate system; transform the first plane and the second plane into a first plane and a second plane in the vehicle coordinate system, respectively; determine the first intersection point of the intersection line with the first plane in the vehicle coordinate system; determine the second intersection point of the intersection line with the second plane in the vehicle coordinate system; determine the spatial line segment of the target based on the first intersection point and the second intersection point; and determine the actual position of the target based on at least two of the spatial line segments.

[0112] Optionally, the actual position determination module is further configured to: merge at least two of the spatial line segments to obtain a target spatial line segment; if the target is a speed bump, then the position of the target spatial line segment is taken as the actual position of the target; if the target is a pedestrian or a vehicle, then the position of the center point of the target spatial line segment is taken as the actual position of the target.

[0113] The ranging device provided in this disclosure can execute the ranging method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the method execution.

[0114] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0115] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 6 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 6 The diagram below shows the structure of the terminal device or server 300. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0116] like Figure 6 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An edit / output (I / O) interface 305 is also connected to the bus 304.

[0117] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0118] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of embodiments of this disclosure.

[0119] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0120] The electronic device provided in this embodiment and the ranging method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0121] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the ranging method provided in the above embodiments.

[0122] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0124] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0125] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire raw images of the driving environment in front of the vehicle using at least two cameras; perform semantic segmentation on the at least two raw images to obtain at least two corresponding segmented images; determine at least two pixel positions of a target based on the at least two segmented images; determine at least two spatial planes in which the target is located based on the at least two pixel positions; determine the actual position of the target based on the at least two spatial planes; and determine the distance between the vehicle and the target based on the actual position.

[0126] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0129] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0130] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0132] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0133] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A distance measurement method, characterized in that, include: Raw images of the driving environment in front of the vehicle are obtained using at least two cameras; Perform semantic segmentation on at least two original images to obtain at least two corresponding segmented images; Determine at least two pixel positions of the target based on the at least two segmented images; Determine at least two spatial planes where the target is located based on the at least two pixel positions; The actual position of the target is determined based on the at least two spatial planes; The distance between the vehicle and the target is determined based on the actual location; Determining the actual position of the target based on the at least two spatial planes includes: Determine the line of intersection between the at least two spatial planes; The actual location of the target is determined based on the intersection line; Determining the actual position of the target based on the intersection line includes: For each camera coordinate system, extract the maximum and minimum points of the target in the first coordinate component of the camera coordinate system; Determine a first plane that passes through the maximum point and the camera optical center and is parallel to the second coordinate component of the camera coordinate system; Determine a second plane that passes through the minimum point and the camera optical center and is parallel to the second coordinate component of the camera coordinate system; Transform the first plane and the second plane into the first plane and the second plane in the vehicle coordinate system, respectively. Determine the first intersection point between the line of intersection and the first plane in the vehicle coordinate system; Determine the second intersection point between the line of intersection and the second plane in the vehicle coordinate system; Determine the spatial line segment of the target based on the first intersection point and the second intersection point; The actual location of the target is determined based on at least two of the spatial line segments.

2. The method according to claim 1, characterized in that, Perform semantic segmentation on at least two original images to obtain at least two corresponding segmented images, including: Each original image is preprocessed; wherein the preprocessing includes adjusting the size of the original image and / or eliminating distortion of the original image; Input at least two preprocessed original images into the image segmentation model to obtain at least two corresponding segmented images.

3. The method according to claim 1, characterized in that, Determining at least two pixel positions of a target based on the at least two segmented images includes: For each segmented image, the segmented image is smoothed to obtain a smoothed segmented image; The target pixel region is determined based on the smoothed segmented image; Determine the contour curve of the target pixel region; Determine the lower edge tangent line based on the contour curve; The pixel position of the lower edge tangent is taken as the pixel position of the target in the pixel coordinate system.

4. The method according to claim 3, characterized in that, Determining at least two spatial planes containing the target based on the at least two pixel positions includes: For each pixel position, the equation of the lower edge tangent line in the image coordinate system is determined based on the pixel position of the lower edge tangent line. Determine the plane bundle equation of the lower edge tangent in the camera coordinate system based on the straight line equation; Substituting the camera's optical center coordinates into the plane beam equation yields the spatial plane where the target is located, where the spatial plane is a plane in the camera coordinate system; different spatial planes reside in different camera coordinate systems.

5. The method according to claim 1, characterized in that, Determining the line of intersection between the at least two spatial planes includes: The spatial planes in at least two camera coordinate systems will be transformed into spatial planes in the vehicle coordinate system; Determine the intersection line between at least two spatial planes in the vehicle coordinate system.

6. The method according to claim 1, characterized in that, Determining the actual location of the target based on at least two of the spatial line segments includes: At least two of the aforementioned spatial line segments are merged to obtain the target spatial line segment; If the target is a speed bump, then the position of the target spatial line segment shall be taken as the actual position of the target; If the target is a pedestrian or a vehicle, the position of the center point of the target spatial line segment shall be taken as the actual position of the target.

7. A ranging device, characterized in that, include: An image acquisition module is used to acquire raw images of the driving environment in front of the vehicle through at least two cameras; The image segmentation module is used to perform semantic segmentation on at least two original images to obtain at least two corresponding segmented images. A pixel position determination module is used to determine at least two pixel positions of a target based on the at least two segmented images; A spatial plane determination module is used to determine at least two spatial planes in which the target is located based on the at least two pixel positions; The actual position determination module is used to determine the actual position of the target based on the at least two spatial planes; A distance determination module is used to determine the distance between the vehicle and the target based on the actual location; The actual position determination module is specifically used to determine the intersection line between the at least two spatial planes; and determine the actual position of the target based on the intersection line. The actual position determination module is further configured to, for each camera coordinate system, extract the maximum and minimum points of the target in the first coordinate component of the camera coordinate system; determine a first plane passing through the maximum point and the camera optical center and parallel to the second coordinate component of the camera coordinate system; determine a second plane passing through the minimum point and the camera optical center and parallel to the second coordinate component of the camera coordinate system; transform the first plane and the second plane into a first plane and a second plane in the vehicle coordinate system, respectively; and determine the first intersection point of the intersection line and the first plane in the vehicle coordinate system. Determine the second intersection point between the line of intersection and the second plane in the vehicle coordinate system; Determine the spatial line segment of the target based on the first intersection point and the second intersection point; The actual location of the target is determined based on at least two of the spatial line segments.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the ranging method as described in any one of claims 1-6.

9. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the ranging method as described in any one of claims 1-6.

Images