Distance measurement system, projection method based on distance measurement and vehicle

Abstract

The invention discloses a ranging system, a projection method based on ranging and a vehicle. The ranging system on the equipment comprises a first projection module and a controller. The first projection module is used for projecting a first pattern to a target object. The controller is configured to determine a first distance based on the first pattern, the first distance being a distance between the device and the target object. The first projection module is further used for projecting a first picture to the target object based on the first distance. The first projection module is further used for projecting a second pattern to the target object. The controller is further configured to determine a distance between the device and the target object as a second distance based on the second pattern, the second distance being different from the first distance. The first projection module is further used for projecting a second picture to the target object based on the second distance. The first picture and the second picture are consistent in picture size. According to the scheme, distance measurement can be realized at low cost, the distance measurement requirement under a dark light condition is met, and the user experience under an application scene is improved.

Description

Technical Field

[0001] The present application relates to the field of distance measurement technology, and in particular to a distance measurement system, a projection method based on distance measurement, and a vehicle. Background Art

[0002] In autonomous driving perception systems, 3D perception is a fundamental capability, enabling the perception of objects such as target vehicles, lane lines, and lane widths. Depth measurement, or distance measurement, is the key to transitioning from 2D perception to 3D perception. Without ranging capabilities, low-level autonomous driving, such as Level 2 autonomous driving based on cruise control, lane keeping, and collision warning, is impossible. Existing solutions use radar for ranging, but these solutions are costly. Other solutions use monocular or binocular ranging, but these solutions fail in low-light conditions at night. Furthermore, existing binocular ranging solutions rely on the binocular baseline T. A too small T results in poor accuracy for long-range ranging, while a large T hinders vehicle deployment, complicates the design, and increases costs. Therefore, achieving low-cost ranging that meets low-light requirements is an urgent technical challenge. Summary of the Invention

[0003] The present application provides a ranging system, a projection method based on ranging, and a vehicle, which can achieve ranging at low cost and meet the ranging requirements under low-light conditions.

[0004] In a first aspect, the present application provides a ranging system on a device, the ranging system comprising a first projection module and a controller.

[0005] The first projection module is used to project a first pattern onto the target object.

[0006] The controller is configured to determine a first distance based on the first pattern, where the first distance is the distance between the device and the target object.

[0007] The first projection module is further configured to project a first image onto the target object based on the first distance.

[0008] The first projection module is further used to project a second pattern onto the target object.

[0009] The controller is further configured to determine, based on the second pattern, that the distance between the device and the target object is a second distance, where the second distance is different from the first distance.

[0010] The first projection module is further configured to project a second image onto the target object based on the second distance.

[0011] The frame sizes of the first picture and the second picture are consistent.

[0012] For example, the first projection module can be a pixel-type light-emitting module (e.g., a headlight) or a projector. For example, the first pattern or the second pattern can be a geometric figure or an array of geometric figures. The first pattern and the second pattern can be the same or different.

[0013] In the above solution, the projection module on the device can be used to project a pattern onto the target object to achieve ranging. Compared with the solution using radar ranging, this solution has a low implementation cost. In particular, for equipment that already includes a projection module (such as vehicles, etc.), the projection module can be reused to achieve ranging, without the need to add more hardware devices for ranging, which greatly saves costs. In addition, since the pattern projected by the projection module can also be perceived in a dark environment for ranging, ranging can also be achieved under dark conditions. Distance measurement can thus be achieved at a low cost and the ranging requirements under dark conditions can be met. In the above solution, the frame size of the projection screen projected by the projection module can also be corrected based on the measured distance, so that even if the distance between the projection module and the target object is different, the frame size of the screen projected onto the target object remains consistent. So as to conform to the user's usage habits and improve the user experience.

[0014] In a possible implementation, the distance measurement system further includes a camera. The first projection module includes a plurality of pixel units for emitting light.

[0015] The aforementioned first projection module is specifically used to light up the target pixel unit to project the aforementioned first pattern onto the aforementioned target object, and the aforementioned target pixel unit includes one or more of the aforementioned multiple pixel units.

[0016] The aforementioned photographing device is used to photograph a first image, wherein the first image includes the aforementioned target object projected with the aforementioned first pattern.

[0017] The controller is specifically configured to obtain first information and the first image, and determine the first distance based on the first information and the first image. The first information includes the coordinates of the target pixel unit.

[0018] In this solution, leveraging the reversible nature of the optical path, the projection model and the camera imaging model are equated. The combination of a projection module and a camera creates a ranging system equivalent to binocular vision. This is lower-cost than radar ranging solutions. Furthermore, because the pattern projected by the projection module can be captured by the camera even in low-light environments, ranging can be achieved in low-light conditions. This enables low-cost ranging and meets the needs of low-light ranging.

[0019] In one possible implementation, the controller is specifically configured to construct a first pixel plane based on the first information, and determine the first distance based on the first pixel plane and the first image. The first pixel plane includes a plurality of pixels, and the plurality of pixels correspond one-to-one to the plurality of pixel units.

[0020] In the above scheme, the constructed pixel plane can be equivalent to the pixel matrix of an image, then the pixel plane and the first image are equivalent to the image obtained by binocular photography of the binocular vision ranging system, so that ranging can be performed based on the pixel plane and the first image.

[0021] In one possible implementation, the controller is specifically configured to:

[0022] The first image is stereo-matched with the first pixel plane to determine pixel coordinates of the first pattern in the first image.

[0023] The first distance is determined based on the coordinates of the target pixel unit, the pixel coordinates of the first pattern, and a first conversion matrix, wherein the first conversion matrix is ​​a conversion matrix between the first projection module and the camera.

[0024] In the above solution, the distance can be calculated by matching the coordinates of the pattern in the image and combining the known pixel coordinates of the first pattern and the first transformation matrix. This method is simple to implement, reduces the complexity of the solution, and saves computing resource costs.

[0025] In a possible implementation, the first information further includes a lighting time of the target pixel unit. The shooting device is further configured to send the shooting time of the first image to the controller.

[0026] Before determining the first distance based on the first information and the first image, the controller is further configured to: achieve time synchronization between the first information and the first image based on the lighting time and the shooting time.

[0027] In the above scheme, the accuracy of subsequent ranging is ensured and the precision is improved through time synchronization of information.

[0028] In a possible implementation, the first information further includes a brightness value of the target pixel unit.

[0029] In the above solution, the brightness value can further assist in the construction of the pixel plane, so that the constructed pixel plane data is closer to the data of the captured image, thereby improving the ranging accuracy.

[0030] In one possible implementation, the controller is specifically configured to:

[0031] The first image and the first pixel plane are compared to obtain a similarity between the first image and the first pixel plane.

[0032] When the similarity is greater than a preset similarity threshold, the first image is subjected to the stereo matching with the first pixel plane.

[0033] When the similarity is less than the preset similarity threshold, the first projection module is controlled to project a third pattern onto the target object.

[0034] In the above scheme, the similarity between the captured image and the projection module's pixel plane is first compared. If the similarity meets a preset threshold, the captured image is clear and usable, and the subsequent ranging calculation continues. Conversely, if the similarity does not meet the preset threshold, the captured image is unusable, which will affect the accuracy and precision of subsequent ranging, so reprojection is performed. This shows that this scheme reduces unnecessary computational overhead while further ensuring the accuracy and precision of ranging.

[0035] In a possible implementation, the brightness of the third pattern is greater than that of the first pattern, and / or the shape of the third pattern is more complex than that of the first pattern.

[0036] In the above solution, the usability of the pattern captured by the camera device is increased by enhancing the brightness and / or complexity of the pattern, so as to optimize the distance measurement result.

[0037] In a second aspect, the present application provides a projection method based on ranging, which is applied to a controller in a device, and the device also includes a first projection module. The method includes: controlling the first projection module to project a first pattern onto a target object. Determining a first distance based on the first pattern, the first distance being the distance between the device and the target object. Controlling the first projection module to project a first picture onto the target object based on the first distance. When the distance between the device and the target object is a second distance, controlling the first projection module to project a second pattern onto the target object. Determining the distance between the device and the target object is a second distance based on the second pattern, the second distance being different from the first distance. Controlling the first projection module to project a second picture onto the target object based on the second distance. The frame sizes of the first picture and the second picture remain consistent.

[0038] In one possible implementation, determining the first distance based on the first pattern includes obtaining first information and a first image. The first information includes coordinates of target pixel units, and the target pixel units include one or more luminous pixel units illuminated by the first projection module projecting the first pattern onto the target object. The first image is an image of the target object including the first pattern, captured by a camera on the device. The first distance is determined based on the first information and the first image.

[0039] In one possible implementation, determining the first distance based on the first information and the first image includes: constructing a first pixel plane based on the first information. The first pixel plane includes a plurality of pixels, and the plurality of pixels correspond one-to-one to the plurality of pixel units. Determining the first distance based on the first pixel plane and the first image.

[0040] In one possible implementation, determining the first distance based on the first pixel plane and the first image includes stereo matching the first image with the first pixel plane to determine the pixel coordinates of the first pattern in the first image. The first distance is determined based on the coordinates of the target pixel unit, the pixel coordinates of the first pattern, and a first transformation matrix. The first transformation matrix is ​​a transformation matrix between the first projection module and the camera.

[0041] In one possible implementation, the first information also includes the lighting time of the target pixel unit. The method further includes receiving the capture time of the first image sent by the camera. Before determining the first distance based on the first information and the first image, the method further includes synchronizing the time between the first information and the first image based on the lighting time and the capture time.

[0042] In a possible implementation, the first information further includes a brightness value of the target pixel unit.

[0043] In a possible implementation, before performing stereo matching on the first image and the first pixel plane, the method further includes: comparing the first image and the first pixel plane to obtain a similarity between the first image and the first pixel plane.

[0044] When the similarity is greater than a preset similarity threshold, the first image is subjected to the stereo matching with the first pixel plane.

[0045] The method further includes: when the similarity is less than the preset similarity threshold, controlling the first projection module to project a third pattern onto the target object.

[0046] In a possible implementation, the brightness of the third pattern is greater than that of the first pattern, and / or the shape of the third pattern is more complex than that of the first pattern.

[0047] In a third aspect, the present application provides a controller, which is provided on a device, and the device also includes a first projection module. The controller includes:

[0048] The control unit is used to control the first projection module to project the first pattern onto the target object.

[0049] A processing unit is configured to determine a first distance based on the first pattern, where the first distance is the distance between the device and the target object.

[0050] The control unit is further configured to control the first projection module to project a first image onto the target object based on the first distance.

[0051] The control unit is further configured to control the first projection module to project a second pattern onto the target object.

[0052] The processing unit is further configured to determine, based on the second pattern, that the distance between the device and the target object is a second distance, where the second distance is different from the first distance.

[0053] The control unit is further configured to control the first projection module to project a second image onto the target object based on the second distance. The first image and the second image have the same frame size.

[0054] In one possible implementation, the processing unit is specifically configured to: obtain first information and a first image, and determine the first distance based on the first information and the first image. The first information includes coordinates of target pixel units, and the target pixel units include one or more luminous pixel units that are illuminated by the first projection module projecting the first pattern onto the target object. The first image is an image of the target object including the first pattern, captured by a camera on the device.

[0055] In one possible implementation, the processing unit is specifically configured to construct a first pixel plane based on the first information, and determine the first distance based on the first pixel plane and the first image. The first pixel plane includes a plurality of pixels, and the plurality of pixels correspond one-to-one to the plurality of pixel units.

[0056] In one possible implementation, the processing unit is specifically configured to perform stereo matching between the first image and the first pixel plane to determine pixel coordinates of the first pattern in the first image. Furthermore, the first distance is determined based on the coordinates of the target pixel unit, the pixel coordinates of the first pattern, and a first transformation matrix. The first transformation matrix is ​​a transformation matrix between the first projection module and the camera.

[0057] In a possible implementation, the first information further includes a lighting time of the target pixel unit. The controller further includes a receiving unit configured to receive a shooting time of the first image sent by the shooting device.

[0058] The processing unit is further configured to achieve time synchronization between the first information and the first image based on the lighting time and the shooting time before determining the first distance based on the first information and the first image.

[0059] Exemplarily, the aforementioned first information also includes the brightness value of the aforementioned target pixel unit.

[0060] In a possible implementation, the processing unit is further configured to compare the first image with the first pixel plane before stereo matching the first image with the first pixel plane to obtain a similarity between the first image and the first pixel plane.

[0061] The processing unit is further configured to perform the stereo matching between the first image and the first pixel plane when the similarity is greater than a preset similarity threshold.

[0062] The processing unit is further configured to control the first projection module to project a third pattern onto the target object when the similarity is less than the preset similarity threshold.

[0063] In a possible implementation, the brightness of the third pattern is greater than that of the first pattern, and / or the shape of the third pattern is more complex than that of the first pattern.

[0064] In a fourth aspect, the present application provides a controller comprising a processor and a memory. The memory is coupled to the processor, and when the processor executes a computer program or computer instruction stored in the memory, the method described in any one of the second aspects above can be implemented. The controller may also include a communication interface for communicating between the controller and other devices (such as the above-mentioned camera or first projection module). Exemplarily, the communication interface may be a transceiver, circuit, bus, module, or other type of communication interface.

[0065] In one possible implementation, the controller may include:

[0066] Memory for storing computer programs or computer instructions;

[0067] A processor is configured to: control a first projection module to project a first pattern onto a target object; determine a first distance based on the first pattern, the first distance being the distance between the device and the target object; control the first projection module to project a first image onto the target object based on the first distance; control the first projection module to project a second pattern onto the target object; determine a second distance between the device and the target object based on the second pattern, the second distance being different from the first distance; and control the first projection module to project a second image onto the target object based on the second distance. The first image and the second image maintain the same frame size.

[0068] It should be noted that the computer programs or computer instructions in the memory of this application can be pre-stored or downloaded from the Internet when the controller is used. This application does not specifically limit the source of the computer programs or computer instructions in the memory. The coupling in the embodiments of this application is an indirect coupling or connection between units or modules, which can be electrical, mechanical, or other forms, for information exchange between units or modules.

[0069] In a fifth aspect, the present application provides a computer-readable storage medium, which stores a computer program or computer instructions, and the aforementioned computer program or computer instructions are executed by a processor to implement the method described in any one of the above second aspects.

[0070] In a sixth aspect, the present application provides a computer program product. When the computer program product is executed by a processor, the method described in any one of the above second aspects will be implemented.

[0071] In a seventh aspect, the present application provides a vehicle comprising the ranging system described in any one of the first aspects above.

[0072] The solutions provided in the second to seventh aspects are used to implement or cooperate with the corresponding systems provided in the first aspect, and therefore can achieve the same or corresponding beneficial effects as the corresponding systems in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figures 1A to 1D Shown is a schematic diagram of the coordinate system involved in the embodiment of the present application.

[0074] Figure 1E Shown is a schematic diagram of a ranging system provided in an embodiment of the present application.

[0075] Figure 2 Shown is a schematic diagram of the mapping between pixel units and pixels provided in an embodiment of the present application.

[0076] Figure 3 Shown is a schematic diagram of projection and imaging provided by an embodiment of the present application.

[0077] Figure 4 Shown is a schematic diagram of a ranging system on a vehicle provided in an embodiment of the present application.

[0078] Figure 4A FIG2 is a schematic diagram showing the relationship between the projection frame size and distance provided in an embodiment of the present application.

[0079] Figures 5 to 8 Shown is a projection schematic diagram provided in an embodiment of the present application.

[0080] Figure 9 Shown is a schematic diagram of projection and imaging provided by an embodiment of the present application.

[0081] Figure 10 Shown is a schematic diagram of a ranging system provided in an embodiment of the present application.

[0082] Figure 11 and Figure 12 Shown is a schematic diagram of projection and imaging provided by an embodiment of the present application.

[0083] Figure 13 Shown is a schematic diagram of a ranging system on a vehicle provided in an embodiment of the present application.

[0084] Figure 14 The figure shows a schematic diagram of the architecture of the ranging system in a vehicle provided in an embodiment of the present application.

[0085] Figure 15 and Figure 16 Shown is a schematic diagram of the structure of the controller provided in an embodiment of the present application. DETAILED DESCRIPTION

[0086] In the embodiment of the present application, "multiple" refers to two or more. In the embodiment of the present application, "and / or" is used to describe the association relationship of associated objects, indicating three relationships that can exist independently. For example, A and / or B can be expressed as follows: A exists alone, B exists alone, or A and B exist at the same time. The description methods such as "at least one of a1, a2, ... and an" used in the embodiment of the present application include the situation where any one of a1, a2, ... and an exists alone, and also include any combination of any multiple of a1, a2, ... and an, each of which can exist alone; for example, the description method of "at least one of a, b and c" includes the situation where a is alone, b is alone, c is alone, a and b combination, a and c combination, b and c combination, or abc combination.

[0087] In this application, the terms "first," "second," and the like are used to distinguish between identical or similar items having substantially the same function or effect. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another.

[0088] In the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0089] The following first introduces the concepts of various coordinate systems involved in the embodiments of this application.

[0090] 1. Pixel coordinate system.

[0091] An image is composed of countless pixels arranged in rows and columns. a Represents) Establishing a rectangular coordinate system uv with pixels as the unit is the pixel coordinate system, for example, see Figure 1A As shown in the figure, the horizontal coordinate u and vertical coordinate v of a pixel are the column number and row number of the pixel in the image pixel array respectively.

[0092] 2. Image coordinate system.

[0093] Since the pixel coordinates (u, v) only represent the number of columns and rows of pixels, and the position of the pixel in the image is not expressed in physical units, it is also necessary to establish an image coordinate system xy expressed in physical units (such as millimeters). The intersection of the camera optical axis and the image plane (usually located at the center of the image plane, also called the principal point of the image) is defined as the origin of the coordinate system (denoted by o b For example, see Figure 1B . The x-axis is parallel to the u-axis, and the y-axis is parallel to the v-axis.

[0094] Assuming that (u0, v0) represents the coordinates of o2 in the uv coordinate system, dx and dy represent the physical size of each pixel on the horizontal axis x and vertical axis y respectively, the conversion formula from pixel coordinates to image coordinates is as follows:

[0095]

[0096] Assuming that the unit in the image coordinate system is millimeter, the unit of dx is: millimeter / pixel.

[0097] 3. Camera coordinate system.

[0098] The camera coordinate system is a three-dimensional coordinate system, for example see Figure 1C The origin Oc of the camera coordinate system is the optical center of the camera (projection center). The Xc axis and Yc axis are parallel to the x axis and y axis of the image coordinate system respectively. The Zc axis is the optical axis of the camera and is perpendicular to the image plane. The intersection of the optical axis and the image plane is the principal point o of the image. b . o b -Oc is the focal length of the camera.

[0099] 4. World coordinate system.

[0100] A reference coordinate system is also selected in the environment to describe the position of the camera and objects. This coordinate system is called the world coordinate system. Figure 1D As shown in , the world coordinate system can be composed of the origin Ow, the Xw axis, the Yw axis, and the Zw axis. Any point in space can determine its position in the world coordinate system. For example, the Xw axis, the Yw axis, and the Zw axis have translation and rotation relationships with other coordinate systems, such as Figure 1D shown.

[0101] In order to achieve ranging at low cost and meet the ranging requirements under dark light conditions, an embodiment of the present application provides a ranging system. For example, the ranging system provided in the embodiment of the present application can be applied in scenarios including but not limited to the following scenarios: positioning, projection image correction, three-dimensional reconstruction, object recognition, face recognition or three-dimensional measurement to inspect product quality, etc. In particular, in the field of intelligent transportation, the ranging system can be used to achieve ranging or positioning of traffic participants such as vehicles or pedestrians or any other objects (such as walls or ground, etc.), or to achieve lane line or lane width recognition and image projection correction, etc. It will be understood that the introduction of the application scenarios of the ranging system here is only an example and does not constitute a limitation to the embodiments of the present application.

[0102] To facilitate understanding of the ranging system provided in the embodiment of the present application, please refer to the example Figure 1E The distance measurement system includes a first projection module 110 , a camera 120 and a controller 130 .

[0103] For example, the first projection module 110 can project a light beam to form a pattern. Optionally, the first projection module 110 can also project a light beam for illumination. For example, the first projection module 110 can be a pixel lamp or a projector. The specific form of the first projection module 110 is not limited in this embodiment of the application.

[0104] For example, the pattern formed by projection in the embodiments of the present application may include, but is not limited to, a geometric figure or an array of geometric figures. The geometric figure may, for example, include a planar geometric figure or a solid geometric figure. Planar geometric figures include, for example, various geometric shapes such as dots, triangles, squares, five-pointed stars, or hearts. Solid geometric figures may include, for example, cubes, cylinders, pyramids, or prisms. It will be understood that the description of the pattern herein is merely an example and does not constitute a limitation on the embodiments of the present application.

[0105] For example, the first projection module 110 may include a plurality of pixel units for emitting light. Each pixel unit can be individually controlled to be turned on or off. A pixel unit being turned on indicates that the pixel unit is controlled to emit light, i.e., is illuminated. A pixel unit being turned off indicates that the pixel unit is controlled not to emit light, i.e., is extinguished. By controlling the lighting or extinguishing of each of the plurality of pixel units, the light emitted by the first projection module can form various patterns.

[0106] For example, the pixel units in the first projection module 110 may be turned on and off by a projection module controller ( Figure 1E Alternatively, for example, the controller 130 may be used to control the projection module, that is, the controller 130 includes the projection module controller or the controller 130 may implement the function of the projection module controller. This embodiment of the present application does not limit this.

[0107] In one possible implementation, the first projection module 110 may be, for example, a light-emitting module including a plurality of light-emitting diodes (LEDs). Each LED represents a pixel unit. By controlling the lighting or extinguishing of each of the plurality of LEDs, the light emitted by the first projection module 110 may form various patterns. For example, the plurality of LEDs may be any number between 20 and 100 LEDs, for example, an LED matrix consisting of 84 LEDs, etc. The embodiment of the present application does not limit the number of LEDs.

[0108] In another possible implementation, the first projection module 110 can be, for example, a light-emitting module based on digital light processing (DLP). The light-emitting module can include a digital micromirror device (DMD). For example, a matrix composed of a large number of small lenses is provided on the DMD, and each small lens represents a pixel unit. By controlling the angle of the small lens, light from a light source (laser or LED lamp, etc.) can be emitted to the outside, which indicates that the pixel unit is lit. Alternatively, by controlling the angle of the small lens, light can be absorbed and not emitted to the outside, which indicates that the pixel unit is extinguished. For example, millions or even more pixel units can be provided on the DMD, so that light-emitting control at the million-pixel level can be achieved.

[0109] Alternatively, for example, in another possible implementation, the first projection module 110 may be a light-emitting module based on reflective liquid crystal projection technology. The light-emitting module may include a liquid crystal on silicon (LCoS) chip. An LCoS chip may integrate millions of pixel electrodes (i.e., pixel units). By controlling these pixel units, light emission control at the million-pixel level can be achieved.

[0110] It should be understood that the above description of the first projection module 110 is merely an example and does not constitute a limitation to the embodiments of the present application.

[0111] For example, the plurality of pixel units in the first projection module 110 are fixedly arranged at a certain position, so the plurality of pixel units correspond to a pixel plane (referred to as pixel plane A for short). The pixel plane A may include a plurality of pixels, and the plurality of pixels correspond one-to-one to the plurality of pixel units in the first projection module 110. For ease of understanding, it can be exemplified by referring to Figure 2 As shown. Figure 2 As shown, each pixel unit in the first projection module 110 corresponds to a pixel in the pixel plane A. In addition, each pixel in the pixel plane A has its own coordinates, called pixel coordinates. For example, Figure 2 The pixel plane A in FIG is shown as an example with m rows and n columns of pixels, i.e., m*n pixels. Here, m and n are integers greater than 1.

[0112] For example, if a pixel unit in the first projection module 110 is illuminated, it can be said that the pixel corresponding to the pixel unit in pixel plane A is illuminated. The illuminated pixel can also have its own brightness value. For example, the brightness value of the pixel is the brightness value of the light emitted by the pixel unit corresponding to the pixel. Therefore, the pixel plane A can be equivalent to the pixel matrix of an image, or equivalent to an image.

[0113] Exemplarily, the first projection module 110 projects different patterns or projects patterns with different brightness, and the data of the pixel surface A (such as the coordinates of the pixels that are lit or unlit and / or the brightness values ​​of the lit pixels) are all different, which can be regarded as pixel matrices of different images.

[0114] For example, the camera 120 may be used to capture images including the pattern projected by the first projection module 110, or may be used to capture other images. For example, the camera 120 may be a camera or a video camera, and the specific form of the camera 120 is not limited in the present embodiment.

[0115] For example, the controller 130 may be configured to collect data of the pixel plane A from the first projection module 110 and to collect captured images from the camera 120 , so as to perform distance measurement based on the collected information.

[0116] Exemplarily, the above-mentioned ranging system is realized by utilizing the principle that the projection model is equivalent to the camera imaging model based on the reversible characteristics of the optical path. Specifically, since the optical path is reversible, each pixel unit in the above-mentioned first projection module 110 is either lit or extinguished to match the pixel surface A corresponding to the projected pattern, which is equivalent to the pixel matrix of the image obtained by photographing the pattern at the position of the first projection module 110. Based on this, the combination of the first projection module 110 and the shooting device 120 can form a ranging system equivalent to binocular vision. For ease of understanding, please refer to the exemplary embodiment. Figure 3 .

[0117] For example, in Figure 3 In the figure, O1 represents the center point of the first projection module 110, and O0 represents the center point of the camera 120. The first projection module 110 projects the pattern onto point P. For example, one or more pixel units in the first projection module 110 are illuminated to project the pattern onto point P. The one or more pixel units correspond to one or more pixels of the pixel plane A ( Figure 3 The light projecting the pattern is reflected or scattered by point P to the camera 120 and captured in the image by the camera 120. Point P is imaged at point Q2 in the image captured by the camera 120. Figure 3In the example, pixel plane A can be equivalent to the pixel matrix of the image obtained by photographing the pattern at the position of the first projection module 110. Based on this, the distance to point P can be measured using pixel plane A and the image captured by the camera 120. The specific distance measurement process can be found in the following description and is not described in detail here.

[0118] In a possible implementation, the above Figure 1E The distance measurement system shown is applied to the field of intelligent transportation as an example. The distance measurement system can be set on a vehicle. For the sake of understanding, you can refer to the example Figure 4 As shown. Figure 4 The vehicle shown in the figure includes a first headlight and a second headlight. For example, the two headlights can be headlights of the vehicle. The vehicle is also provided with a camera. For example, the camera can be provided at the junction of the top of the vehicle and the front windshield, such as Figure 4 As shown. Alternatively, for example, the camera can also be set at the position of the left or right rearview mirror, or the camera can also be set close to the first headlight or the second headlight, etc. It is understandable that the embodiment of the present application does not limit the position of the camera, as long as the camera can capture the pattern projected by the first headlight or the second headlight. Then, the first projection module 110 of the above-mentioned ranging system can be the first headlight or the second headlight, Figure 4 In the figure, the first headlight is used as the first projection module 110 as an example. The imaging device 120 of the ranging system can be the camera. In addition, the controller 130 of the ranging system can be, for example, a headlight controller, a domain controller, or a combination of the headlight controller and the domain controller. As can be seen, the vehicle's headlights and cameras can be reused to achieve ranging, eliminating the need for additional hardware devices (such as radar or other cameras) for ranging, thereby significantly saving costs.

[0119] For example, the distance measurement system provided in the embodiments of the present application is not limited to being installed in a vehicle, but can also be installed in a robot, a drone, an airplane, a ship, etc. The embodiments of the present application do not limit the device or scenario in which the distance measurement system is applied.

[0120] In a possible implementation, the practical application of the above-mentioned ranging system is introduced by taking the scenario of projection picture correction as an example. For example, the ranging system is set on a device (such as a vehicle, etc.). In a specific implementation, the picture can be projected onto the target object by the first projection module 110 of the ranging system. For example, the picture can include any pattern or content, and the embodiment of the present application does not limit this. For example, the target object can be, for example, a wall, the ground or the road surface, a traffic participant such as a vehicle or pedestrian, or any other arbitrary object, and the embodiment of the present application does not limit this.

[0121] It is understandable that the first projection module 110 projects a picture onto the target object. If other factors remain unchanged, only the distance between the first projection module 110 and the target object is changed. The farther the distance between the first projection module 110 and the target object, the larger the size of the picture projected on the target object. Conversely, the closer the distance between the first projection module 110 and the target object, the smaller the size of the picture projected on the target object. For example, see Figure 4A As shown. It can be seen that if the radiation angle of the projection of the first projection module 110 remains unchanged, it is always θ. When the distance between the first projection module 110 and the target object is d1, the frame size of the image projected onto the target object is represented by c1, for example. When the distance between the first projection module 110 and the target object is d2, the frame size of the image projected onto the target object is represented by c2, for example. d2 is greater than d1, and c2 is also greater than c1. It can be understood that Figure 4A This is only an example and does not constitute a limitation to the embodiments of this application. Figure 4A The picture size shown in is only an example. In a specific implementation, the picture size can also be expressed by area size or length and width of the picture, etc., and the embodiment of the present application does not limit this.

[0122] In order to keep the size of the image projected on the target object consistent, the embodiment of the present application can adjust the size of the image projected by the first projection module 110 according to the distance.

[0123] In a specific implementation, the first projection module 110 may project a first pattern toward the target object. This first pattern may be used as a test pattern for distance measurement, i.e., the controller 130 may determine a first distance based on the first pattern. The first distance may be the distance between the device equipped with the distance measurement system and the target object. For the specific implementation of the first distance, the projected first pattern, and the determination of the first distance based on the first pattern, reference may be made to the description of the distance measurement implementation process using the distance measurement system described later, and will not be described in detail here.

[0124] After the controller 130 obtains the first distance, it can control the first projection module 110 to project the first picture onto the target object based on the first distance. For example, in a specific implementation, the controller 130 can determine the coordinates of the pixel units that need to be lit to project the first picture onto the target object based on the first distance. Then, a control instruction can be sent to the first projection module 110 based on the coordinates of the determined pixel units to instruct the first projection module 110 to light up these pixel units. Thus, the first picture is projected onto the target object. The controller 130 can determine the coordinates of the pixel units that need to be lit to project the first picture onto the target object based on the first distance through any possible implementation method, and the embodiments of the present application are not limited to this.

[0125] Then, in a specific implementation, the distance between the above-mentioned device and the target object can change, for example, to a second distance. The second distance is different from the first distance. In this case, the above-mentioned first projection module 110 can project a second pattern onto the above-mentioned target object. Similarly, the second pattern can be used as a test pattern for ranging, that is, the controller 130 can determine the size of the second distance based on the second pattern. Regarding the specific implementation of the second distance, the projected second pattern, and the determination of the second distance based on the second pattern, you can refer to the description of the implementation process of ranging by the above-mentioned ranging system introduced later, which will not be described in detail here.

[0126] After obtaining the second distance, the controller 130 may control the first projection module 110 to project a second image toward the target object based on the second distance. The second image and the first image have the same frame size. For example, maintaining the same frame size may mean no change in the frame size, or a small change in the frame size, such as a change in size that is less than a preset size change threshold.

[0127] For example, in a specific implementation, the controller 130 can determine the coordinates of the pixel units required to light up to project the second image onto the target object based on the second distance. For example, in a possible implementation, the coordinates of the pixel units required to light up to project the first image corresponding to the above-mentioned first distance are known. Then, the ratio between the second distance and the first distance can be calculated, and then the coordinates of the pixel units required to light up can be adjusted proportionally based on the ratio. For example, assuming that the ratio is greater than 1, it indicates that the second distance is greater than the first distance. Then, the number of pixel units required to light up is reduced according to the ratio, so that the size of the projected second image and the above-mentioned first image remain consistent. Assuming that the ratio is greater than 1, it indicates that the second distance is smaller than the first distance. Then, the number of pixel units required to light up is increased according to the ratio, so that the size of the projected second image and the above-mentioned first image remain consistent. Thus, the coordinates of the pixel units required to light up to project the second image corresponding to the second distance are obtained. Alternatively, for example, instead of using the coordinates of the pixel units corresponding to the first distance to illuminate the first image as a reference, the coordinates of the pixel units required to illuminate the second image to be projected onto the target object may be determined directly based on the second distance. The controller 130 may determine the coordinates of the pixel units required to illuminate the second image to be projected onto the target object based on the second distance in any possible implementation manner, and this embodiment of the present application is not limited thereto.

[0128] After determining the coordinates of the pixel units that need to be lit to project the second image onto the target object, the controller 130 can send a control instruction to the first projection module 110 based on the determined coordinates of the pixel units to instruct the first projection module 110 to light up these pixel units, thereby projecting the second image onto the target object.

[0129] Based on the above description, it can be seen that after the distance is measured by the distance measurement system provided by the embodiment of the present application, functions such as projection image correction can be implemented based on the measured distance. It is understood that the above mainly uses the application scenario of projection image correction as an example and does not constitute a limitation on the application scenarios of the embodiment of the present application.

[0130] The following describes an exemplary implementation of distance measurement using the aforementioned distance measurement system. In a specific implementation, the projection module controller or controller 130 may send control instructions to the first projection module 110 to control the first projection module 110 to illuminate target pixel units and project a first pattern onto the target object. This first pattern may include, but is not limited to, a geometric shape or an array of geometric shapes. For details, please refer to the previous description and will not be repeated here.

[0131] For example, the target pixel unit may include one or more pixel units in the first projection module 110. For ease of understanding, please refer to Figures 5 to 8 , Figures 5 to 8 The projection pattern of the ranging system on a vehicle is taken as an example.

[0132] For example, Figure 5 and Figure 6 The example shown is a wall as the target object. Figure 7 and Figure 8 The example shown is an example where the target object is the ground. Figure 5 and Figure 7 The first pattern is shown as a dot as an example. Figure 6 and Figure 8 The first pattern is shown as a plurality of dots arranged in a certain order. It can be understood that Figures 5 to 8 What is shown is merely an example and does not constitute a limitation to the embodiments of the present application.

[0133] For example, if the projection module controller sends a control instruction to the first projection module 110 to control the lighting of the target pixel unit, then the first projection module 110 may also send first information to the controller 130. The first information may include the coordinates of the target pixel unit. Optionally, the first information may also include the brightness value and / or lighting time of the target pixel unit.

[0134] For example, if the controller 130 sends a control instruction to the first projection module 110 to control the lighting of the target pixel unit, the controller 130 can obtain the coordinates and / or brightness value of the target pixel unit through internal communication. Optionally, the first projection module 110 can also send the lighting time of the target pixel unit to the controller 130. Optionally, the first projection module 110 can also send the coordinates and / or brightness value of the target pixel unit and the lighting time of the target pixel unit to the controller 130.

[0135] It is understandable that the above description of the controller 130 obtaining the above first information is only an example and does not constitute a limitation to the present application.

[0136] After the first projection module 110 projects the first pattern onto the target object, the camera 120 may capture a first image. The first image includes the target object onto which the first pattern is projected. For example, in a specific implementation, the controller 130 may send a start-shooting instruction to the camera 120 to control the camera 120 to begin shooting. After capturing the first image, the camera 120 may send the first image to the controller 130. For example, the camera 120 may also send the time when the first image was captured to the controller 130.

[0137] After the controller 130 obtains the first information and the first image, it can calculate the first distance based on the first information and the first image. Based on the above description, the first distance is the distance between the device on which the ranging system is set and the target object. Exemplarily, in a possible implementation, the distance between the device and the target object may refer to, for example, the distance between the optical center of the shooting device in the ranging system and the first pattern projected on the target object. Or, exemplarily, the distance between the device and the target object may refer to, for example, the distance between the center of the projection module in the ranging system and the first pattern projected on the target object. Or, exemplarily, the distance between the device and the target object may refer to, for example, the distance between any point on the device (for example, taking a vehicle as an example, it may be the front of the vehicle or the center point of the vehicle, etc.) and the first pattern projected on the target object. It will be understood that the introduction of the distance between the device and the target object here is only an example and does not constitute a limitation to the embodiments of the present application.

[0138] By way of example, in one possible implementation, after the controller 130 obtains the first information, it can construct the pixel plane corresponding to the first pattern projected by the first projection module 110 (hereinafter referred to as the first pixel plane). By way of example, the controller 130 presets the size information (e.g., m rows and n columns) of the pixel plane A corresponding to the first projection module 110. After obtaining the first information, the controller can determine the coordinates of the illuminated pixels in pixel plane A based on the coordinates of the target pixel unit, and the coordinates of the remaining unlit pixels, thereby constructing the first pixel plane.

[0139] Alternatively, in another possible implementation, the first information includes, in addition to the coordinates of the target pixel unit, the coordinates of the unlit pixel units in the first projection module 110. Based on this, after obtaining the first information, the controller 130 can determine the coordinates of the lit pixels in the pixel plane A based on the coordinates of the target pixel unit, and determine the coordinates of the unlit pixels based on the coordinates of the unlit pixel units, thereby constructing the first pixel plane.

[0140] For example, each pixel in the first pixel plane may also be associated with a corresponding brightness value. The brightness value of an extinguished pixel may be 0. The brightness value of the illuminated pixel may be determined based on the brightness value of the target pixel unit in the first information. Alternatively, for example, the first information may not include the brightness value of the illuminated pixel, and the brightness value of the illuminated pixel may be pre-set and configured in the controller 130. Therefore, the controller 130 may determine the brightness value of the illuminated pixel based on the pre-configured brightness value.

[0141] It is understandable that the above description of the controller 130 constructing the above-mentioned first pixel plane is only an example and is not intended to limit the embodiments of the present application.

[0142] For example, after obtaining the first pixel plane, the controller 130 may perform stereo matching between the first image and the first pixel plane to determine the pixel coordinates of the first pattern in the first image. Specifically, the stereo matching algorithm may be used to match pixels in the first image that correspond one-to-one to the pixels lit on the first pixel plane. For example, in the above Figure 3 In the example, the stereo matching algorithm can be used to match Q2 in the image captured by the camera 120 with Q1 on the pixel plane A. The pixels in the first image that correspond one-to-one to the illuminated pixels on the first pixel plane are the pixels of the first pattern in the first image. Furthermore, the pixel coordinates of the first pattern in the first image can be determined, thus achieving pixel keypoint matching.

[0143] Exemplarily, there are many stereo matching algorithms mentioned above. For example, depending on the different optimization theory methods adopted, the stereo matching algorithm can be divided into a global stereo matching algorithm and a local stereo matching algorithm. Depending on the primitives used to represent the image (for example, including edges, corners, straight line segments, circles, holes, ellipses and other points of interest, etc.), the stereo matching algorithm can be divided into a regional stereo matching algorithm, a feature-based stereo matching algorithm and a phase-based stereo matching algorithm, etc. It will be understood that the introduction of the stereo matching algorithm here is only an example and does not constitute a limitation on the embodiments of the present application. In a specific implementation, any stereo matching algorithm can be used to achieve pixel point matching between the first pixel plane and the first image, and the embodiments of the present application do not impose any restrictions on this.

[0144] After determining the pixel coordinates of the first pattern in the first image, the first distance can be calculated based on the pixel coordinates of the first pattern in the first image, the coordinates of the illuminated pixels in the first pixel plane (which are also the coordinates of the target pixel unit), and the first transformation matrix. For example, if there is one illuminated pixel in the first pixel plane, then there is also one corresponding pixel of the first pattern in the first image. A distance can be calculated based on the coordinates of the illuminated pixel in the first pixel plane, the coordinates of the corresponding pixel of the first pattern, and the first transformation matrix. This distance is the first distance. For example, if there are multiple illuminated pixels in the first pixel plane, then there are also multiple corresponding pixels of the first pattern in the first image. Multiple distances can be calculated based on the coordinates of the multiple illuminated pixels in the first pixel plane, the coordinates of the multiple pixels of the corresponding first pattern, and the first transformation matrix. That is, the first distance includes multiple distances. For ease of description, the following example is based on the case where there is one illuminated pixel in the first pixel plane and one corresponding pixel of the first pattern in the first image.

[0145] Exemplarily, the first conversion matrix is ​​a conversion matrix between the first projection module 110 and the shooting device 120. Based on the above introduction, it can be seen that the combination of the first projection module 110 and the shooting device 120 is equivalent to the binocular in the binocular ranging system. Then, the first projection module 110 is equivalent to a camera, and a camera coordinate system (referred to as the first camera coordinate system for short) can be constructed based on the first projection module 110. In addition, a camera coordinate system (referred to as the second camera coordinate system for short) can also be constructed based on the shooting device 120. In a specific implementation, the conversion matrix between the first projection module 110 and the shooting device 120 is a conversion matrix between the first camera coordinate system and the second camera coordinate system.

[0146] In a possible implementation, the first conversion matrix is ​​a conversion matrix from the first projection module 110 to the shooting device 120, that is, a conversion matrix from the first camera coordinate system to the second camera coordinate system, which can be expressed as M 10 For example, the conversion matrix M10 can be expressed as follows:

[0147]

[0148] For example, the transformation matrix M 10 It can be pre-configured in the controller 130. 10 The various parameters in are also pre-calibrated. 10 There is no restriction on the specific values ​​of the parameters.

[0149] Then, based on the transformation matrix M 10 , the first distance is calculated based on the pixel coordinates of the first pattern in the first image and the coordinates of the pixels lit in the above-mentioned first pixel plane. In a specific implementation, the pixel coordinates of the first pattern in the first image are coordinates defined based on the pixel coordinate system of the first image. Similarly, the first pixel plane is equivalent to the pixel matrix of the image, so the coordinates of the pixels lit in the first pixel plane are also equivalent to the coordinates defined based on the pixel coordinate system of the first pixel plane. In order to describe the actual size of each pixel point in the image in the environmental information, it is necessary to first convert the pixel coordinates of the first pattern into coordinates in the image coordinate system of the first image. Similarly, it is necessary to first convert the coordinates of the pixels lit in the first pixel plane into coordinates in the image coordinate system of the first pixel plane.

[0150] Specifically, the pixel coordinates of the first pattern in the first image can be converted to coordinates in the image coordinate system of the first image using the above-mentioned conversion formula for converting pixel coordinates to image coordinates. The converted coordinates can be expressed as (X0, Y0). Similarly, the coordinates of the illuminated pixels in the first pixel plane can be converted to coordinates in the image coordinate system of the first pixel plane using the same conversion formula. The converted coordinates can be expressed as (X1, Y1).

[0151] After obtaining the above coordinates (X0, Y0) and (X1, Y1), we can use the above transformation matrix M 10 , coordinates (X0, Y0) and coordinates (X1, Y1) to calculate the above-mentioned first distance. Specifically, the above-mentioned first camera coordinate system can be defined as the world coordinate system, assuming that the world coordinate system is expressed as O1-xyz. The image coordinate system of the imaging surface corresponding to the first camera coordinate system (referred to as the first image coordinate system for short) is expressed as o1-X1Y1, and its corresponding camera focal length (i.e., the distance between O1 and o1) is expressed as f1. The above-mentioned second camera coordinate system is expressed as O0-x0y0z0. The image coordinate system of the imaging surface corresponding to the second camera coordinate system (referred to as the second image coordinate system for short) is expressed as o0-X0Y0, and its corresponding camera focal length (i.e., the distance between O0 and o0) is expressed as f0. The relationship between these coordinate systems can be exemplified by referring to Figure 9 As shown. Figure 9 In the figure, point P is the location of the first pattern projected by the first projection module 110 on the target object. In the world coordinate system, the coordinates of point P can be expressed as (x, y, z). Based on the geometric projection relationship of point P on the first pixel plane, the geometric projection relationship of point P in the first image, and the transformation relationship from the first camera coordinate system to the second camera coordinate system (through the transformation matrix M 10 The three-dimensional coordinates of point P can be solved as follows:

[0152]

[0153] The z obtained by the above calculation is the above first distance.

[0154] In another possible implementation, the first conversion matrix is ​​a conversion matrix from the shooting device 120 to the first projection module 110, that is, a conversion matrix from the second camera coordinate system to the first camera coordinate system, which can be expressed as M 01 For example, the transformation matrix M 01 It can be expressed as follows:

[0155]

[0156] For example, similarly, the transformation matrix M 01It can be pre-configured in the controller 130. 01 The various parameters in are also pre-calibrated. 01 There is no restriction on the specific values ​​of the parameters in . In this case, the second camera coordinate system can be defined as the world coordinate system. Then, based on the transformation matrix M 01 , the pixel coordinates of the first pattern in the first image and the coordinates of the illuminated pixels in the first pixel plane are used to calculate the first distance. 10 , the pixel coordinates of the first pattern in the first image and the coordinates of the illuminated pixels in the first pixel plane are used to calculate the first distance, which is not repeated here. The calculation formula for the final first distance z can be as follows:

[0157]

[0158] It should be understood that the above-described calculation process for the first distance is merely an example and does not constitute a limitation of the embodiments of the present application. In another possible implementation, the disparity between the first pixel plane and the first image can be calculated using the above-described stereo matching algorithm, and then the first distance can be calculated based on the disparity. This implementation is a mature binocular distance measurement calculation solution, and the embodiments of the present application will not elaborate on this implementation in detail.

[0159] For example, the above description is based on the example that the first pixel plane includes one illuminated pixel and the first image also includes one corresponding pixel of the first pattern. For an implementation method in which the first pixel plane includes multiple illuminated pixels and the first image also includes multiple corresponding pixels of the first pattern, multiple distances can be calculated using the above distance calculation method. These multiple distances represent the distances between different positions in the target object and the ranging system. For example, a three-dimensional model of the target object can also be constructed using these multiple distances.

[0160] In one possible implementation, during the process of projecting the first pattern by the first projection module 110, the coordinates of the target pixel unit, as well as information such as the brightness value and / or lighting time of the target pixel unit (referred to as projection information) can be continuously sent to the controller 130. Then, the controller 130 will receive a projection information stream. Similarly, after the first projection module 110 projects the first pattern, the shooting device 120 can continuously shoot multiple images including the target object projected with the first pattern. And the captured images are continuously sent to the controller 130. Then, the controller 130 will receive an image frame stream. Based on this, the controller 130 needs to perform time synchronization in the information stream and the image frame stream to match the projection information and image at the same moment or the closest moment. To ensure the accuracy of subsequent ranging and improve precision.

[0161] For example, there are many methods for time synchronization. For ease of understanding, the following examples are provided. In one possible implementation, if the time synchronization calibration between the first projection module 110 and the camera 120 has been completed, the lighting time in the projection information and the image capture time can be directly compared. If the two times are the same or the time difference is within a preset range, it can be determined that the projection information and the image match, that is, the projection information and the image are considered to be time synchronized.

[0162] Alternatively, in another possible implementation, the controller 130 may pre-transmit a time synchronization request to the first projection module 110 and the camera 120. This request instructs the first projection module 110 and the camera 120 to send a message to the controller 130 at the same time, according to their respective clock systems. This message may be the real-time time information of the first projection module 110 and the camera 120, or a pre-configured instruction. After receiving the messages from the first projection module 110 and the camera 120, the controller 130 records the time difference between the two messages (referred to as the target time difference). The target time difference is the time difference between the clock systems of the first projection module 110 and the camera 120. Based on this, the controller 130 may calculate the time difference between the lighting time in the projection information and the capture time of the image. If the calculated time difference is the same as the target time difference, or the difference between the calculated time difference and the target time difference is within a preset range, the projection information and the image may be determined to match, i.e., the projection information and the image are considered to be time synchronized.

[0163] It will be understood that the introduction to time synchronization herein is merely an example and does not constitute a limitation to the embodiments of the present application.

[0164] Based on the above description, the controller 130 may determine the time synchronization between the first image and the first pixel plane through time synchronization calculation, and then perform stereo matching based on the first image and the first pixel plane.

[0165] In one possible implementation, the controller 130 may compare the first image with the first pixel plane before performing stereo matching on the first image and the first pixel plane. After obtaining the similarity between the first image and the first pixel plane, it is determined whether to further perform stereo matching based on the similarity. For example, based on the previous introduction, it can be seen that the first pixel plane is equivalent to the pixel matrix of the image. Therefore, an image similarity calculation method can be used to calculate the similarity between the first image and the first pixel plane. For example, the similarity between the first image and the first pixel plane can be calculated by a cosine similarity algorithm, a hash algorithm, a histogram algorithm, a mutual information algorithm, a pixel matching algorithm (pixelmatch), a structural similarity (structural similarity, SSIM) algorithm or a deep learning algorithm. It will be understood that the algorithm for similarity calculation here is only an example and does not constitute a limitation to the embodiments of the present application.

[0166] After obtaining the similarity between the first image and the first pixel plane, the similarity can be compared with a preset similarity threshold. If the similarity is greater than or equal to the similarity threshold, indicating that the projected first pattern can be clearly captured by the camera 120, the stereo matching operation between the first image and the first pixel plane is further performed. If the similarity is less than the similarity threshold, indicating that the projected first pattern is blurry and cannot be clearly captured by the camera 120. In this case, the controller 130 can send a control instruction to the first projection module 110 to control the first projection module 110 to project the second pattern onto the target object. Alternatively, the controller 130 can send an instruction to the projection module controller, which in turn sends a control instruction to the first projection module 110 to control the first projection module 110 to project the second pattern onto the target object. For example, the brightness of the second pattern is greater than that of the first pattern. And / or the shape of the second pattern is more complex than that of the first pattern. Specifically, increasing the brightness of the projected pattern can enable the camera 120 to clearly capture it. Increasing the complexity of the pattern not only enables the camera 120 to capture it clearly, but also increases the accuracy of subsequent stereo matching.

[0167] After the second pattern is re-projected onto the target object, the camera 120 similarly captures the second pattern to obtain an image. The controller 130 then obtains the corresponding information and image to perform distance measurement. The specific process is described above and will not be repeated here.

[0168] As can be seen in the above scheme, the similarity between the captured image and the projection module's pixel plane is first compared. If the similarity meets the preset threshold, the captured image is clear and usable, and the subsequent ranging calculation continues. Conversely, if the similarity does not meet the preset threshold, the captured image is unusable, which will affect the accuracy and precision of subsequent ranging, so re-projection is required. This shows that this scheme reduces unnecessary computational overhead while further ensuring the accuracy and precision of ranging.

[0169] In another possible implementation, after obtaining the first information and the first image, the controller 130 may input the first information and the first image into a first distance measurement model for calculation, with the output being the first distance. For example, the first distance measurement model may be a pre-trained machine learning model or a neural network model, etc., although this embodiment of the present application is not limited thereto.

[0170] Alternatively, in another possible implementation, after obtaining the first pixel plane and the first image, the controller 130 may input the first pixel plane and the first image into a second ranging model for calculation, with the output being the first distance. For example, the second ranging model may be, for example, a pre-trained machine learning model or a neural network model, etc., which is not limited in this embodiment of the present application.

[0171] In another possible implementation, see for example Figure 10 As shown, the ranging system provided in the embodiment of the present application may further include a second projection module 140 .

[0172] For example, the second projection module 140 can project a light beam to form a pattern. Optionally, the second projection module 140 can also project a light beam for illumination. For example, the second projection module 140 can be a pixel lamp or a projector, etc. The specific form of the second projection module 140 is not limited in this embodiment of the application. For example, the introduction of the second projection module 140 can refer to the description of the first projection module above and is not repeated here.

[0173] Similarly, the plurality of pixel units in the second projection module 140 correspond to a pixel plane (abbreviated as pixel plane B). The introduction of pixel plane B can refer to the description of the first pixel plane above, which will not be repeated here.

[0174] Similarly, based on the reversible characteristics of the optical path, each pixel unit in the second projection module 140 is either lit or extinguished to match the pixel plane B corresponding to the projected pattern, which is equivalent to the pixel matrix of the image obtained by photographing the pattern at the position of the second projection module 140. Based on this, the combination of the first projection module 110, the second projection module 140 and the camera 120 can form a ranging system equivalent to multi-eye vision. For ease of understanding, please refer to the example Figure 11 .

[0175] For example, in Figure 11 In the figure, O1 represents the center point of the first projection module 110, O0 represents the center point of the camera 120, and O2 represents the center point of the second projection module 140. Figure 3 As can be seen from the introduction, the distance measurement of point P can be achieved through the first pixel plane and the image captured by the camera 120. Similarly, the second projection module 140 projects a pattern to point P (which is the same pattern projected by the first projection module 110 at the same time). For example, one or more pixel units in the second projection module 140 are illuminated to project the pattern to point P. The one or more pixel units correspond to one or more pixels of the pixel plane B ( Figure 11 (denoted by Q3 in FIG). Similarly, the pattern is captured by the camera 120 and imaged at Q2 in the image captured by the camera 120. That is, the pixel plane B can be equivalent to the pixel matrix of the image obtained by capturing the pattern at the position of the second projection module 140. Based on this, the distance to point P can be achieved through the pixel plane B and the image captured by the camera 120. The specific distance measurement implementation process can be referred to the above introduction and will not be repeated here. The two distance measurement results can be averaged or weighted averaged to obtain the final distance measurement result.

[0176] For example, the above Figure 11 In the embodiment, the first projection module 110 and the second projection module 140 may project the pattern onto the point P at the same time. Alternatively, the first projection module 110 and the second projection module 140 may project the pattern onto the point P at different times. This embodiment of the present application is not limited to this.

[0177] Alternatively, in another possible implementation, see for example Figure 12As shown. O1 represents the center point of the above-mentioned first projection module 110, O0 represents the center point of the above-mentioned camera 120, and O2 represents the center point of the above-mentioned second projection module 140. The first projection module 110 projects pattern 1 to point P1 through the first projection module, and the corresponding one or more pixels of the first pixel plane are represented by Q1. The pattern 1 is captured by the camera 120 and imaged at Q2 in the image captured by the camera 120. In addition, the second projection module 140 projects pattern 2 to point P2, and the corresponding one or more pixels of the pixel plane B are represented by Q3. The pattern 2 is captured by the camera 120 and imaged at Q4 in the image captured by the camera 120. Then, the distance to point P1 can be achieved through the first pixel plane and the pattern 1 captured by the camera 120. The distance to point P2 can be achieved through the pixel plane B and the pattern 2 captured by the camera 120. The specific distance measurement implementation process can be referred to the above introduction and will not be repeated here.

[0178] In a possible implementation, the above Figure 10 The distance measurement system shown is applied to the field of intelligent transportation as an example. The distance measurement system can be set on a vehicle. For the sake of understanding, you can refer to the example Figure 13 As shown. Figure 13 In the vehicle shown in FIG, the first headlight can be used as the above Figure 10 The ranging system shown in the figure comprises a first projection module 110, and a second headlight serving as a second projection module 140. The camera can serve as the imaging device 120 of the ranging system. Furthermore, by way of example, a headlight controller, a domain controller, or a combination of the headlight controller and the vehicle's domain controller can serve as the controller 130 of the ranging system. This shows that the vehicle's headlights and cameras can be reused to achieve ranging, eliminating the need for additional hardware devices (such as radar or other cameras) for ranging, thereby significantly saving costs.

[0179] For example, in order to more intuitively understand the direction of the data flow of the ranging system in the vehicle during the ranging process, you can refer to Figure 14 . Figure 14 In the figure, a schematic diagram of the architecture of the ranging system in a vehicle is shown as an example. Figure 14 In the embodiment, the controller may be the controller 130, the first headlight may be the first projection module 110, the second headlight may be the second projection module 140, the camera may be the shooting device 120, and the headlight pixel control unit may be the projection module controller. Figure 14In the embodiment, the controller may include a headlight pixel control unit, a data collection unit, a configuration unit, and a distance measurement calculation unit. For example, the headlight pixel control unit may control the projection pattern of the first headlight and / or the second headlight. The first headlight and / or the second headlight may also send projection information corresponding to the projection pattern to the data collection unit. The camera may capture the projected pattern to form an image and send the image to the data collection unit. The data collection unit may send the received information and image to the distance measurement calculation unit. In addition, the configuration unit is configured with a conversion matrix between the first headlight and the camera (for example, the above-mentioned conversion matrix M 10 or M 01 ) and / or the conversion matrix between the second headlight and the camera (refer to the conversion matrix M 10 or M 01 ). Optionally, the configuration unit is further configured with a conversion matrix between the first headlight and the second headlight (refer to the conversion matrix M 10 or M 01 ). For example, the configuration unit may also be configured with other information, such as the aforementioned preset time range, preset time difference, or preset threshold, etc., which is not limited in the embodiments of the present application. The ranging calculation unit performs ranging calculation based on the information and image sent by the data collection unit and the preset configuration information obtained from the configuration unit. The specific calculation process is described in the description of the operations performed by the controller 130 above and is not repeated here.

[0180] For example, in another possible implementation, the ranging calculation unit may also evaluate the ranging quality and ranging blind spots, and then provide feedback on the next ranging strategy to the headlight pixel control unit. For example, before the controller 130 stereo-matches the first image with the first pixel plane, the similarity obtained by comparing the first image with the first pixel plane may be determined by the ranging calculation unit. Then, if the similarity is less than a preset similarity threshold, the ranging calculation unit may send an instruction to the headlight pixel control unit, which in turn sends a control instruction to the first headlight to control the first headlight to project the second pattern onto the target object.

[0181] It is understandable that, in order to implement the above-mentioned corresponding functions, the above-mentioned controller includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and steps of the various examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0182] The embodiment of the present application can divide the controller into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0183] In the case of dividing each functional module into corresponding functional modules, the embodiment of the present application also provides a controller for implementing any of the above methods, for example, a controller is provided including units (or means) for implementing each step in any of the above methods. For example, please refer to Figure 15 , which is a schematic diagram of the structure of a controller 1500 provided in an embodiment of the present application. The controller is provided on a device, which also includes a first projection module. The controller 1500 may include a control unit 1501 and a processing unit 1502. Among them:

[0184] The control unit 1501 is configured to control the first projection module to project a first pattern onto a target object.

[0185] The processing unit 1502 is configured to determine a first distance based on the first pattern, where the first distance is the distance between the device and the target object.

[0186] The control unit 1501 is further configured to control the first projection module to project a first image onto the target object based on the first distance.

[0187] The control unit 1501 is further configured to control the first projection module to project a second pattern onto the target object.

[0188] The processing unit 1502 is further configured to determine, based on the second pattern, that the distance between the device and the target object is a second distance, where the second distance is different from the first distance.

[0189] The control unit 1501 is further configured to control the first projection module to project a second image onto the target object based on the second distance. The sizes of the first image and the second image are consistent.

[0190] In one possible implementation, the processing unit 1502 is specifically configured to: obtain first information and a first image, and determine the first distance based on the first information and the first image. The first information includes coordinates of target pixel units, where the target pixel units include one or more luminous pixel units that are illuminated by the first projection module projecting the first pattern onto the target object. The first image is an image of the target object including the first pattern, captured by a camera on the device.

[0191] In one possible implementation, the processing unit 1502 is specifically configured to: construct a first pixel plane based on the first information, and determine the first distance based on the first pixel plane and the first image. The first pixel plane includes a plurality of pixels, and the plurality of pixels correspond one-to-one to the plurality of pixel units.

[0192] In one possible implementation, the processing unit 1502 is specifically configured to perform stereo matching on the first image and the first pixel plane to determine pixel coordinates of the first pattern in the first image. Furthermore, the processing unit 1502 is configured to determine the first distance based on the coordinates of the target pixel unit, the pixel coordinates of the first pattern, and a first transformation matrix. The first transformation matrix is ​​a transformation matrix between the first projection module and the camera.

[0193] In a possible implementation, the first information further includes a lighting time of the target pixel unit. The controller further includes a receiving unit configured to receive a shooting time of the first image sent by the shooting device.

[0194] The processing unit 1502 is further configured to achieve time synchronization between the first information and the first image based on the lighting time and the shooting time before determining the first distance based on the first information and the first image.

[0195] Exemplarily, the first information also includes a brightness value of the target pixel unit.

[0196] In a possible implementation, the processing unit 1502 is further configured to compare the first image with the first pixel plane before stereo matching the first image with the first pixel plane to obtain a similarity between the first image and the first pixel plane.

[0197] The processing unit 1502 is further configured to perform stereo matching on the first image and the first pixel plane when the similarity is greater than a preset similarity threshold.

[0198] The processing unit 1502 is further configured to control the first projection module to project a third pattern toward the target object when the similarity is less than the preset similarity threshold.

[0199] In a possible implementation, the brightness of the third pattern is greater than that of the first pattern, and / or the shape of the third pattern is more complex than that of the first pattern.

[0200] Figure 15 The specific operations and beneficial effects of each unit in the controller 1500 can be found in the description of the aforementioned embodiments, and will not be repeated here.

[0201] For example, see Figure 16, which is a structural diagram of a possible physical entity of the controller provided in this application. Figure 16 The controller 1600 shown may be a controller for implementing any of the above embodiments. The controller 1600 includes a processor 1601, a memory 1602, and a communication interface 1603. The processor 1601, the communication interface 1603, and the memory 1602 may be interconnected or connected to each other via a bus 1604.

[0202] Exemplarily, the memory 1602 is used to store computer programs and data of the controller 1600. The memory 1602 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM).

[0203] The software or program codes required for all or part of the functions of the controller in the above method embodiment may be stored in the memory 1602 .

[0204] In one possible implementation, if the software or program code required for some functions is stored in the memory 1602, the processor 1601, in addition to calling the program code in the memory 1602 to implement some functions, can also cooperate with other components (such as the communication interface 1603) to jointly complete other functions described in the method embodiment (such as the function of receiving or sending data).

[0205] There may be multiple communication interfaces 1603 , which are used to support the controller 1600 to communicate, such as receiving or sending data or messages.

[0206] Exemplarily, the processor 1601 may be a circuit having data processing capabilities. In one implementation, the processor may be a circuit having instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. Alternatively, the processor 1601 can be a combination of at least two of these processor types, etc.

[0207] The processor 1601 may be configured to read the program stored in the memory 1602 and execute the operations performed by the controller in any of the possible embodiments described above.

[0208] Figure 16 The specific operations and beneficial effects of each unit in the controller 1600 can be found in the corresponding descriptions in the above possible embodiments, and will not be repeated here.

[0209] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program or computer instructions, and the computer program or computer instructions are executed by a processor to implement the method implemented by the controller in any of the above-mentioned possible implementation methods.

[0210] An embodiment of the present application further provides a computer program product. When the computer program product is read and executed by a computer, the method implemented by the controller in any of the above-mentioned possible implementation modes will be executed.

[0211] The embodiment of the present application further provides a device, which may include any of the possible ranging systems described above. For example, the device may be a vehicle.

[0212] In summary, in the embodiment of the present application, based on the reversible characteristics of the optical path, the projection model is equated with the camera imaging model, and a ranging system equivalent to binocular vision is formed by combining a projection module and a shooting device. Compared with the solution using radar ranging, the cost is low. In particular, for equipment (such as vehicles, etc.) that already includes a projection module and a shooting device, the projection module and the shooting device can be reused to achieve ranging, without the need to add other hardware devices for ranging, which greatly saves costs. In addition, since the pattern projected by the projection module can also be captured by the camera device in a dark light environment, ranging can also be achieved under dark light conditions.

[0213] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0214] It will also be understood that the term “comprise” (also known as “includes,” “including,” “comprises,” and / or “comprising”) when used in this specification specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0215] It should also be understood that references throughout this specification to "one embodiment," "an embodiment," or "one possible implementation" mean that specific features, structures, or characteristics associated with that embodiment or implementation are included in at least one embodiment of the present application. Therefore, the appearance of "in one embodiment," "in an embodiment," or "one possible implementation" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A distance measurement system on a device, characterized in that: The distance measurement system includes a first projection module and a controller; The first projection module is used to project a first pattern onto the target object; The controller is configured to determine a first distance based on the first pattern, the first distance being the distance between the device and the target object; The first projection module is further configured to project a first image onto the target object based on the first distance; The first projection module is further configured to project a second pattern onto the target object; The controller is further configured to determine, based on the second pattern, that the distance between the device and the target object is a second distance, where the second distance is different from the first distance; The first projection module is further configured to project a second image onto the target object based on the second distance; The frame sizes of the first picture and the second picture are consistent.

2. The system according to claim 1, wherein: The distance measurement system further includes a shooting device; the first projection module includes a plurality of pixel units for emitting light; The first projection module is specifically configured to light up a target pixel unit to project the first pattern onto the target object, wherein the target pixel unit includes one or more of the plurality of pixel units; The shooting device is configured to shoot a first image; the first image includes the target object projected with the first pattern; The controller is specifically configured to obtain first information and the first image, and determine the first distance based on the first information and the first image; the first information includes the coordinates of the target pixel unit.

3. The system according to claim 2, characterized in that The controller is specifically used for: constructing a first pixel plane based on the first information; the first pixel plane includes a plurality of pixels, and the plurality of pixels correspond one-to-one to the plurality of pixel units; The first distance is determined based on the first pixel plane and the first image.

4. The system according to claim 3, characterized in that The controller is specifically used for: Performing stereo matching on the first image and the first pixel plane to determine pixel coordinates of the first pattern in the first image; The first distance is determined based on the coordinates of the target pixel unit, the pixel coordinates of the first pattern and a first conversion matrix; the first conversion matrix is ​​a conversion matrix between the first projection module and the shooting device.

5. The system according to any one of claims 2 to 4, characterized in that: The first information also includes the lighting time of the target pixel unit; the shooting device is further used to send the shooting time of the first image to the controller; Before determining the first distance based on the first information and the first image, the controller is further configured to: Time synchronization between the first information and the first image is achieved based on the lighting time and the shooting time.

6. The system according to any one of claims 2 to 5, characterized in that: The first information also includes a brightness value of the target pixel unit.

7. The system according to claim 4, wherein: The controller is specifically used for: comparing the first image with the first pixel plane to obtain a similarity between the first image and the first pixel plane; When the similarity is greater than a preset similarity threshold, performing stereo matching on the first image and the first pixel plane; When the similarity is less than the preset similarity threshold, the first projection module is controlled to project a third pattern toward the target object.

8. The system according to claim 7, characterized in that The brightness of the third pattern is greater than that of the first pattern, and / or the shape of the third pattern is more complex than that of the first pattern.

9. The system according to any one of claims 1 to 8, characterized in that: The first pattern is a geometric figure or an array of geometric figures.

10. A projection method based on distance measurement, characterized in that: The method is applied to a controller in a device, and the device further includes a first projection module; the method includes: Controlling the first projection module to project a first pattern onto a target object; determining a first distance based on the first pattern, the first distance being the distance between the device and the target object; controlling the first projection module to project a first image toward the target object based on the first distance; controlling the first projection module to project a second pattern onto the target object; determining, based on the second pattern, a distance between the device and the target object as a second distance, where the second distance is different from the first distance; controlling the first projection module to project a second image toward the target object based on the second distance; The frame sizes of the first picture and the second picture are consistent.

11. The method according to claim 10, characterized in that The determining the first distance based on the first pattern includes: Acquire first information and a first image; the first information includes coordinates of target pixel units, the target pixel units including one or more luminous pixel units that are illuminated when the first projection module projects the first pattern onto the target object; the first image is an image of the target object including the first pattern, captured by a camera on the device; The first distance is determined based on the first information and the first image.

12. The method according to claim 11, characterized in that The determining a first distance based on the first information and the first image includes: constructing a first pixel plane based on the first information; the first pixel plane includes a plurality of pixels, and the plurality of pixels correspond one-to-one to the plurality of pixel units; The first distance is determined based on the first pixel plane and the first image.

13. The method according to claim 12, characterized in that The determining the first distance based on the first pixel plane and the first image includes: Performing stereo matching on the first image and the first pixel plane to determine pixel coordinates of the first pattern in the first image; The first distance is determined based on the coordinates of the target pixel unit, the pixel coordinates of the first pattern and a first conversion matrix; the first conversion matrix is ​​a conversion matrix between the first projection module and the shooting device.

14. The method according to any one of claims 11 to 13, characterized in that: The first information also includes the lighting time of the target pixel unit; the method further includes: receiving the shooting time of the first image sent by the shooting device; Before determining the first distance based on the first information and the first image, the method further includes: Time synchronization between the first information and the first image is achieved based on the lighting time and the shooting time.

15. The method according to any one of claims 11 to 14, characterized in that: The first information also includes a brightness value of the target pixel unit.

16. The method according to claim 13, characterized in that Before performing stereo matching on the first image and the first pixel plane, the method further includes: comparing the first image and the first pixel plane to obtain a similarity between the first image and the first pixel plane; When the similarity is greater than a preset similarity threshold, performing stereo matching on the first image and the first pixel plane; The method further includes: controlling the first projection module to project a third pattern toward the target object when the similarity is less than the preset similarity threshold.

17. The method according to claim 16, characterized in that The brightness of the third pattern is greater than that of the first pattern, and / or the shape of the third pattern is more complex than that of the first pattern.

18. The method according to any one of claims 10 to 17, characterized in that: The first pattern is a geometric figure or an array of geometric figures.

19. A means of transport, characterized in that: The device comprises a ranging system as described in any one of claims 1 to 9.

Images