Parking system using surface structured light and operation method thereof

Abstract

A parking system utilizing surface structured light and its operating method are disclosed. The parking system includes: a first lighting module configured to project surface structured light; an imaging module configured to match the spectrum of infrared light emitted by the first lighting module and the spectrum of visible light from ambient light, and configured to generate a two-dimensional image of terrain based at least in part on the visible light and a three-dimensional image of the terrain based at least in part on the surface structured light projected by the first lighting module; and a control module configured to fuse the two-dimensional image and the three-dimensional image to construct a three-dimensional topographic information map of the terrain; and to generate an obstacle avoidance route based at least in part on the three-dimensional topographic information map for use by the parking system during automated parking. Also provided are operating methods for the parking system and numerous other aspects.

Description

Technical Field

[0001] The present invention generally relates to the field of intelligent driving, and more particularly to a parking system for utilizing surface structured light and an operating method thereof. Background Art

[0002] With the development of intelligent driving, various automated parking systems have emerged. Existing parking systems typically rely solely on two-dimensional images and are unable to identify complex and difficult-to-measure three-dimensional topography, such as depressions, protrusions, or suspended obstacles. This can lead to obstacles being overlooked during automated parking, potentially causing damage to the vehicle or the obstacle. Furthermore, factors such as the complexity of the vehicle's surroundings, limited time and space, and vehicle speed make it difficult to quickly determine an obstacle avoidance route during parking, resulting in low efficiency and accuracy, limiting the application of automated parking systems.

[0003] Therefore, there is an urgent need in the art for a parking system that has improved obstacle recognition capability and accuracy and can quickly achieve recognition. Summary of the Invention

[0004] The following is a brief summary of one or more aspects to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceivable aspects and is neither intended to identify key or critical elements of all aspects nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that will be presented later.

[0005] In order to solve the above problems, the present invention proposes a parking system for utilizing surface structured light and an operating method thereof.

[0006] In one aspect, a parking system utilizing surface structured light is provided. The parking system includes: a first lighting module for projecting surface structured light; an imaging module that matches the spectrum of infrared light from the first lighting module and the spectrum of visible light from ambient light, and is configured to generate a two-dimensional image of terrain based at least in part on the visible light and a three-dimensional image of the terrain based at least in part on the surface structured light; and a control module for fusing the two-dimensional image with the three-dimensional image to construct a three-dimensional topographic information map of the terrain; and generating an obstacle avoidance route based at least in part on the three-dimensional topographic information map for use by the parking system during automatic parking.

[0007] Preferably, the imaging module generates the three-dimensional image including: using the surface structured light to perform three-dimensional scanning on the terrain and protruding, recessed or suspended obstacles on the terrain to obtain depth information of the obstacles; and generating the three-dimensional image at least partially based on the scanning results.

[0008] Preferably, the control module generates the obstacle avoidance route including: generating an instruction to bypass the obstacle or trigger an alarm when it is detected in the three-dimensional shape information map that the parameter of the obstacle exceeds a threshold.

[0009] Preferably, the control module updates the generated obstacle avoidance route at least in part based on the three-dimensional topography information map and the vehicle's moving speed and direction, and the vehicle's moving speed and direction are determined at least in part based on the relative relationship between different frames in the two-dimensional image.

[0010] Preferably, it is characterized in that the user interface device of the parking system presents at least one of the following: the three-dimensional shape information map, an update to the three-dimensional shape information map, the obstacle avoidance route, or an update to the obstacle avoidance route.

[0011] Preferably, the surface structured light utilizes at least one of LED structured light or VCSEL structured light, and the surface structured light is at least one of grid structured light, line structured light, dot matrix structured light or coded structured light.

[0012] Preferably, the parking system further comprises a second lighting module for projecting visible light; and at least one motion sensor for detecting a moving speed and a moving direction of the vehicle during movement of the vehicle.

[0013] In another aspect, a method for operating automatic parking using surface structured light is provided, the method comprising: projecting surface structured light; generating a two-dimensional image of the terrain based at least in part on visible light in the environment and generating a three-dimensional image of the terrain based at least in part on the surface structured light; fusing the two-dimensional image and the three-dimensional image to construct a three-dimensional topography information map of the terrain; and generating an obstacle avoidance route based at least in part on the three-dimensional topography information map for use by a parking system during automatic parking.

[0014] Preferably, generating the three-dimensional image includes: using the surface structured light to perform a three-dimensional scan of the terrain and protruding, recessed or suspended obstacles on the terrain to obtain depth information of the obstacles; and generating the three-dimensional image at least partially based on the scanning results.

[0015] Preferably, generating the obstacle avoidance route includes: generating an instruction to bypass the obstacle or trigger an alarm when it is detected in the three-dimensional appearance information map that the parameter of the obstacle exceeds a threshold value.

[0016] Preferably, the method includes updating the generated three-dimensional topography information map and obstacle avoidance route based at least in part on the three-dimensional topography information map and the moving speed and moving direction of the vehicle, and the moving speed and moving direction of the vehicle are determined at least in part based on the relative relationship between different frames in the two-dimensional image.

[0017] In yet another aspect, an electronic device is provided, including a processor and a memory, wherein the memory stores program instructions; the processor runs the program instructions to implement the operating method for automatic parking using surface structured light as described in the aforementioned method.

[0018] In yet another aspect, a device for executing any of the aforementioned methods for automatic parking using surface structured light is provided.

[0019] In yet another aspect, a non-transitory computer-readable storage medium storing instructions is provided. When the instructions are executed by a computer, the computer executes any one of the aforementioned methods for automatic parking using surface structured light.

[0020] This summary is provided to introduce some concepts in a simplified form, which will be further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Other aspects, features, and / or advantages of various embodiments will be set forth in part in the description that follows and will be apparent in part from the description, or may be learned through practice of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to understand in detail the manner in which the above-described features of the present invention are employed, the contents briefly summarized above may be described in more detail with reference to various embodiments, some aspects of which are illustrated in the accompanying drawings. However, it should be noted that the accompanying drawings illustrate only certain typical aspects of the present invention and should not be considered to limit its scope, as the description allows for other equally effective aspects. In the accompanying drawings, similar reference numerals are used similarly throughout. It should be noted that the drawings described are merely schematic and non-limiting. In the drawings, the dimensions of some components may be exaggerated and are not drawn to scale for illustrative purposes.

[0022] Figure 1 A diagram illustrating an example of a parking system according to an embodiment of the present invention.

[0023] Figure 2 A diagram illustrating an example of a process flow for a parking system utilizing area structured light according to an embodiment of the present invention.

[0024] Figure 3A diagram illustrating a block diagram of a device supporting the use of area structured light according to an embodiment of the present invention.

[0025] Figure 4 A diagram illustrating an example of a method for parking using surface structured light according to an embodiment of the present invention.

[0026] Figure 5 A diagram illustrating an example of a device for parking using surface structured light according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] To make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. In the following detailed description, many specific details are set forth to provide a thorough understanding of the described exemplary embodiments. However, it is apparent to those skilled in the art that the described embodiments can be practiced without some or all of these specific details. In other exemplary embodiments, well-known structures or processing steps are not described in detail to avoid unnecessarily obscuring the concepts of the present disclosure.

[0028] In this specification, unless otherwise stated, the term "A or B" used throughout this specification refers to "A and B" and "A or B" rather than to A and B being exclusive.

[0029] Figure 1 A diagram illustrating an example of a parking system according to an embodiment of the present invention. The example vehicle 100 is capable of communicating with other vehicles and / or entities.

[0030] As shown in the figure, in the exemplary depicted vehicle 100, a parking system may be installed. The parking system may be a system operable to enable the vehicle 100 to park autonomously, which may be an independent system, such as a system separate from the vehicle 100, or a system combined with an on-board computer, processor, etc. It may be utilized and / or integrated into the vehicle 100 or any other system or device to communicate wirelessly with the vehicle, as previously described. When utilized by the vehicle 100, the parking system may include or be integrated into a vehicle computer system that is used to manage one or more systems related to navigation and / or autonomous driving of the vehicle, as well as communicate with other onboard systems and / or other traffic entities. In an embodiment of the present application, the parking system may include an independent device or component of the vehicle 100 that is communicatively coupled to other components / devices of the vehicle 100.

[0031] In an embodiment of the present application, the parking system may include: a first lighting module for projecting surface structured light. Additionally, it may further include a second lighting module for projecting visible light. For example, the surface structured light projected by the first lighting module may utilize light emitting diode (LED) structured light and / or utilize vertical cavity surface emitting laser (VCSEL) structured light. In an embodiment of the present application, the surface structured light projected by the first lighting module may have a wavelength of, for example, 940nm. In an embodiment of the present application, the infrared surface structured light projected by the first lighting module may be grid structured light, line structured light, dot matrix structured light or coded structured light. As shown in the figure, generally, the above-mentioned lighting module is installed on the trunk, hatch or rear bumper of the vehicle. It should be understood that the position of the lighting module depicted in the figure is only for illustrative purposes to illustrate the technical solution of the present application, and is not intended to be limiting.

[0032] In embodiments of the present application, the parking system may additionally or alternatively include at least one motion sensor for detecting the rate and direction of movement of vehicle 100 during its motion. For example, the motion sensor can be used to determine certain real-time characteristics of the vehicle, such as its position, speed, acceleration, and the like. In embodiments of the present application, real-time parameters related to vehicle movement, such as the vehicle moving backward at a speed of 5 kilometers per hour (km / h) in a direction 5 degrees east of south, can be transmitted to the vehicle's processor for subsequent use in determining the vehicle's position and obstacle parameters (such as size, height, and obstacle material). It should be understood that the motion sensor may include, but is not limited to, one or more inertial sensors and / or other sensors (e.g., accelerometers, gyroscopes, cameras, video cameras, magnetometers, altimeters, microphones, proximity sensors, light sensors, barometers, etc.). Vehicle 100 may also include a laser radar (LIDAR) for detecting objects and measuring distances to those objects. The LIDAR is typically mounted on the vehicle's roof; however, if multiple LIDAR units are present, they may be positioned around the front, rear, and sides of the vehicle. The vehicle 100 may have various other location-related systems, various wireless communication interfaces (such as WAN, WLAN, V2X), RADAR (typically in the front bumper), and SONAR (typically located on both sides of the vehicle, if present). Various wheel sensors and driveline sensors may also be present, such as tire pressure sensors, accelerometers, gyroscopes, and wheel rotation detection and / or counters. In one embodiment, distance measurements and relative positions determined via various sensors (such as LIDAR, RADAR, cameras, GNSS, and SONAR) may be combined with car size and shape information and information about sensor locations to determine the distance and relative position between surfaces of different vehicles, so that the distance or vector from a sensor to another vehicle or between two different sensors is incrementally increased to account for the positioning of the sensor on each vehicle. It should be understood that the above description is intended to provide examples of various sensors in embodiments of a vehicle including the parking system 100, and is not intended to be limiting.

[0033] In an embodiment of the present application, the parking system may further include an imaging module (such as a camera, a motion camera, a video camera, a camera head, etc.), and the imaging module matches the spectrum of the surface structured light projected by the first lighting module and the visible light in the ambient light. For example, in one aspect, the imaging module can generate a two-dimensional image of the terrain based at least in part on the visible light in the ambient light and / or the visible light projected by the second lighting module. In an embodiment of the present application, the imaging module can identify the coordinate origin of the two-dimensional image based on the generated two-dimensional image of the terrain. For example, the point with the highest brightness in the image, the point with the highest imaging quality, the point at a specified distance from the vehicle, the closest point between the vehicle and the obstacle, or the midpoint of the line connecting the vehicle and the obstacle, the center pixel of the camera's light-sensitive imaging element (photoelectric sensor), a fixed pixel in the upper right corner of the image, etc. can be identified as the coordinate origin. It should be understood that identifying the coordinate origin of the two-dimensional image may include: comparing the brightness of each pixel in the image, and then identifying the point with the highest brightness as the coordinate origin to improve the accuracy of the imaging. Additionally or alternatively, identifying the coordinate origin of the two-dimensional image may include identifying the center pixel of the camera's light-sensitive imaging element (photoelectric sensor) in the image as the coordinate origin to reduce image processing costs. As illustrated and previously discussed, the vehicle 100 may have a camera mounted on a rearview mirror, a camera mounted on a front fender, a camera mounted on a side mirror, and a rear-mounted camera.

[0034] On the other hand, the imaging module generates a three-dimensional image of the terrain and obstacles based at least in part on the surface structured light projected by the first lighting module. In an embodiment of the present application, the imaging module generates the three-dimensional image by performing a three-dimensional scan of the terrain and obstacles on the terrain in the direction of movement of the vehicle 100 using the projected structured light to obtain depth information of the obstacles, thereby determining three-dimensional parameter data (including depth information) of the obstacles and fitting (e.g., reconstructing) a three-dimensional model of the obstacles. For example, based on the three-dimensional parameter data, obstacles can be distinguished as protruding, recessed, or suspended obstacles. For objects protruding from the ground, the three-dimensional parameter data may include the height, width, length, etc. of the object; for objects recessed from the ground, the three-dimensional parameter data may include the depth, width, length, etc. of the object; or for objects suspended from the ground, the three-dimensional parameter data may include the height, width, length, height from the horizontal plane, etc. of the object. Thus, a three-dimensional model image of the obstacle can be generated (e.g., fitted) based on the scanning results including the three-dimensional parameter data.

[0035] In an embodiment of the present application, the parking system may further include a control module configured to fuse the aforementioned two-dimensional image with the aforementioned three-dimensional image to construct a three-dimensional topographic information map of the terrain. For example, the distance of the three-dimensional obstacle from the coordinate origin in three-dimensional space can be measured and the three-dimensional obstacle can be placed in the two-dimensional image, thereby fusing the two-dimensional and three-dimensional images to construct three-dimensional topographic information of the terrain. The generated images are fused based on visible light and structured light. The high resolution of visible light imaging compensates for the low resolution of structured light; structured light provides three-dimensional information, compensating for the only two-dimensional information of visible light, thereby improving the ability, accuracy, and speed of obstacle recognition. The fusion process may include, for example, calculation / processor processing to make the field of view of the camera's infrared pixel imaging and RGB pixel imaging consistent and the pixel coordinates correspond one to one, where the RGB pixel coordinate E: (Xa, Yb) corresponds to the infrared pixel coordinate F: (Xp, Yq), and structured light processing can output the depth Zc of point F. The corresponding relationship is superimposed on the planar two-dimensional information formed by the RGB pixels to form a data matrix: (Xa, Yb, Zc), and then the planar two-dimensional information is given three-dimensional information, thereby providing the system with a basis for identifying obstacles and constructing three-dimensional shape information with depth information.

[0036] Furthermore, the parking system may generate an obstacle avoidance route based at least in part on the three-dimensional shape information map for use by the parking system during automatic parking. In an embodiment of the present application, the control module generating the obstacle avoidance route may include: when it is detected in the three-dimensional shape information map that the parameters of the obstacle exceed a threshold, determining to bypass the obstacle and generating corresponding instructions. Preferably, the three-dimensional parameter data may be searched and matched in a database including common obstacles, or a trained neural network may be used for identification to determine the type of obstacle, such as a stone, a traffic cone, a pothole, etc. In an embodiment of the present application, in terms of obstacle type recognition, the obstacle type may be identified by extracting effective features by marking the type of obstacle based on the features or shape characteristics of common objects in the parking environment, thereby helping the processing system on the vehicle to make a better decision on whether it can pass through the obstacle. Similarly, for example, parking lot ground locks and wheel stoppers can be identified by their distinctive contours as obstacles that the vehicle can pass over. Rocks or ground surfaces with less pronounced unevenness do not affect parking, and can be determined to be passable. Three-dimensional objects with distinct edge features, such as boxes and warning triangles, can be clearly detected using 2D images and structured light data, indicating that such obstacles should be bypassed and corresponding instructions generated. Preferably, the parking system can design a route around an obstacle, such as a traffic cone, if it detects a height of 30 centimeters (cm) and a height threshold of 10 cm set by the system. Preferably, the parking system can design a route around the obstacle, such as a pothole, if it detects a depth of 3 cm and a depth threshold of 5 cm set by the system. Preferably, when an obstacle is identified as a parking lot ground lock, wheel stopper, etc., the system can communicate with the obstacle or the roadside unit that manages the obstacle and send instructions to change the obstacle's shape (e.g., lower the ground lock). Preferably, the control module can also generate an alarm triggering instruction when it detects that an obstacle parameter exceeds a threshold. For example, if the measured proximity between the vehicle and the obstacle is less than a specified distance (e.g., when the proximity is less than or equal to 30 cm), an alarm can be triggered to alert the driver. Alternatively, instructions can be issued to the vehicle control device to stop the vehicle from continuing in the direction of travel, reverse the direction of travel, control the vehicle's speed, or perform other operations such as turning left or right.

[0037] Traditional two-dimensional image perception judges the distance to obstacles based on the assumption of a ground plane. It lacks depth information and relies on the characteristics of two-dimensional imaging. It has low accuracy, high requirements for image sensors, and is easily affected by ambient brightness, image complexity, etc., resulting in judgment deviations. If the obstacle is suspended, determining the distance based solely on the height of the obstacle will result in a far-fetched distance judgment, and the vehicle body is likely to collide with the obstacle. In comparison, the distance can be determined based on a comparison of a second height of the obstacle (for example, the bottom) from the horizontal plane with a threshold, thereby avoiding collisions. Indoor parking scenes are complex and prone to suspended obstacles (for example, suspended billboards, etc.). Using structured light algorithms to identify suspended, raised, and recessed obstacles can effectively reduce the chance of collisions.

[0038] In an embodiment of the present application, the parking system can issue a command to a device controlling the vehicle to perform an automatic parking operation based on the generated obstacle avoidance route. The automatic parking operation can be performed by the device controlling the vehicle in conjunction with a power device such as the vehicle engine and ignition, and will not be described in detail here.

[0039] In an embodiment of the present application, the control module may update the generated obstacle avoidance route based at least in part on the three-dimensional topography information map and the vehicle's moving speed and direction. In an embodiment of the present application, the vehicle's moving speed and direction may be determined at least in part based on the relative relationship between different frames in the two-dimensional image. Additionally or alternatively, the vehicle's moving speed and direction may be measured by a motion sensor and transmitted to a processor in the on-board system. For example, as the vehicle's speed and direction change, the three-dimensional topography information map is constantly corrected, and the obstacle avoidance route also changes accordingly. It should be understood that the vehicle can be controlled to perform automatic parking accordingly.

[0040] In an embodiment of the present application, at least one of the following can be presented on a user interface device of a parking system: a three-dimensional shape information map, an update to the three-dimensional shape information map, an obstacle avoidance route, or an update to the obstacle avoidance route. For example, a vehicle may include one or more input devices, which may include devices related to a user interface (e.g., a touch screen, a touchpad, a microphone, buttons, a dial, a switch, etc.) and / or devices related to navigation, autonomous driving, etc. Similarly, one or more output devices may involve interaction with a user (e.g., via a display, a light-emitting diode (LED), a speaker, etc.) and / or devices related to navigation, autonomous driving, etc. The three-dimensional shape information map, an update to the three-dimensional shape information map, an obstacle avoidance route, or an update to the obstacle avoidance route can be displayed in real time on an on-board display. For example, the control module may also generate two or more obstacle avoidance routes for display on the user interface device for selection by a user on the vehicle.

[0041] Figure 2A diagram illustrating an example of a process flow for a parking system utilizing area structured light according to an embodiment of the present invention.

[0042] As shown in the figure, the surface structured light illumination at 210 can be Figure 1 The first lighting module described in [2] is used to provide infrared surface structured light illumination. At 220, target obstacles on the terrain reflect the surface structured light, causing the lines or dots of the surface structured light to deform. Based on the deformation, the size parameters of the obstacle (protrusion, depression, or suspended obstacle) can be calculated. Furthermore, at 230, the reflected structured light is received by the infrared pixels of the imaging system's camera and converted into corresponding three-dimensional obstacle data.

[0043] Similarly, the ambient lighting at 240 can be ambient light (e.g., daytime environment or environment with strong street lights) or can be Figure 1 The second lighting module described in

[15] provides visible light illumination (e.g., in a night environment or a basement environment). At 250, objects on the ground and in the surrounding environment (including obstacles) reflect the visible light. Further, the reflected visible light is received by the RGB pixels of the camera of the imaging system at 260 and converted into corresponding two-dimensional terrain data. Furthermore, the vehicle's movement speed and direction can also be determined based at least in part on the relative relationship between different frames in the two-dimensional image.

[0044] At 270, images associated with the 3D obstacle data and the 2D terrain data can be fused by the imaging system to construct a refined 3D topography information map. For example, the image can be processed by a computer / processor to ensure that the field of view of the camera's infrared pixel imaging and the RGB pixel imaging are consistent and the pixel coordinates correspond one-to-one, where the RGB pixel coordinate E: (Xa, Yb) corresponds to the infrared pixel coordinate F: (Xp, Yq). After structured light processing, the depth Zc of point F can be output. This correspondence is then superimposed on the two-dimensional planar information formed by the RGB pixels to form a data matrix: (Xa, Yb, Zc). The two-dimensional planar information is then given three-dimensional information, providing the system with a basis for obstacle recognition and constructing three-dimensional topography information with depth information.

[0045] It should be noted that Figure 2 It is intended only to provide a generalized explanation of the process flow of a parking system utilizing area structured light, and any or all of the steps may be utilized as appropriate.

[0046] Figure 3 FIG is a block diagram of an embodiment of a device 300 supporting the use of surface structured light according to an embodiment of the present invention. Figure 3 It is intended only to provide a generalized description of the various components, any or all of which may be utilized as appropriate. Figure 3 The illustrated components may be localized into a single physical device and / or distributed among various networked devices, for example, which may be located at different physical locations on the vehicle 100 or other entity.

[0047] Device 300 is shown as including hardware elements that may be electrically coupled (or may otherwise be in communication, as appropriate) via bus 305. The hardware elements may include processing unit(s) 310, which may include, but is not limited to, one or more general-purpose processors, one or more special-purpose processors (such as digital signal processing (DSP) chips, graphics acceleration processors, application-specific integrated circuits (ASICs), etc.), and / or other processing structures or devices.

[0048] The device 300 may also include one or more input devices 370, which may include devices related to user interface (e.g., touch screen, touchpad, microphone, buttons, dials, switches, etc.) and / or devices related to navigation, autonomous driving, etc. Similarly, one or more output devices 315 may be related to interaction with a user (e.g., via a display, light emitting diodes (LEDs), speakers, etc.) and / or devices related to navigation, autonomous driving, etc.

[0049] The device 300 may also include a wireless communication interface 330, which may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device and / or a chipset (such as Devices such as wireless devices, Wi-Fi devices, WiMax devices, WAN devices, and / or various cellular devices, etc., can be used. The wireless communication interface 330 can enable the device 300 to communicate with other devices. This can include various forms of communication of the previously described embodiments. As such, it can be capable of transmitting direct communications, broadcast wireless signals, receiving direct and / or broadcast wireless signals, and the like. Accordingly, the wireless communication interface 330 can be capable of transmitting and / or receiving RF signals from various RF channels / bands. Communications using the wireless communication interface 330 can be performed via one or more wireless communication antennas 332 that transmit and / or receive wireless signals 334.

[0050] Device 300 may further include sensor(s) 340. Sensor 340 may include, but is not limited to, one or more inertial sensors and / or other sensors (e.g., accelerometers, gyroscopes, cameras, magnetometers, altimeters, microphones, proximity sensors, light sensors, barometers, etc.). Sensor 340 may be used, for example, to determine certain real-time characteristics of the vehicle, such as position, speed, acceleration, etc.

[0051] The device 300 may further include and / or be in communication with memory 360. The memory 360 may include, but is not limited to, local and / or network accessible storage, disk drives, drive arrays, optical storage devices, solid-state storage devices (such as random access memory (RAM) and / or read-only memory (ROM)), which may be programmable, flash-updatable, etc. Such storage devices may be configured to implement any appropriate data storage, including, but not limited to, various file systems, database structures, and / or the like.

[0052] The memory 360 of the device 300 may also include software elements ( Figure 3 300 (and / or the processing unit(s) 310 or DSP 320 within the device 300), including the following: Figure 4 In one aspect, such codes and / or instructions may be used to configure and / or adapt a general purpose computer (or other device) to perform one or more operations according to the described method.

[0053] Figure 4 A diagram illustrating an example of a method for parking using surface structured light according to an embodiment of the present invention.

[0054] Method 400 may include projecting surface structured light at operation 410. This operation may be similar to that described in detail in FIG. Figure 1 and Figure 2 As described in [ 410 ], operation 410 is performed using a first lighting module. Preferably, the infrared structured light projected by the first lighting module can utilize at least one of LED structured light or VCSEL structured light. Preferably, a second lighting module can also be used to project visible light. Preferably, operation 410 can be specifically implemented by lighting module 510, as described in more detail below.

[0055] The method 400 may further include generating a two-dimensional image of the terrain based at least in part on visible light in the environment and generating a three-dimensional image of the terrain based at least in part on the projected surface structured light at operation 420. This operation may be similar to that described in FIG. Figure 1 and Figure 2 As described in [ 4 ], the projected surface structured light can be used to perform a three-dimensional scan of the terrain and any protruding, recessed, or suspended obstacles on the terrain to obtain depth information of the obstacles; and the three-dimensional image can be generated based at least in part on the scanning results. Preferably, various onboard sensors can be used to determine certain real-time characteristics of the vehicle, such as position, speed, acceleration, etc. Preferably, operation 420 can be specifically implemented by imaging module 520, as described in more detail below.

[0056] The method 400 may further include: fusing the two-dimensional image and the three-dimensional image to construct a three-dimensional topographic information map of the terrain at operation 430. The method 400 may further include: generating an obstacle avoidance route based at least in part on the three-dimensional topographic information map at operation 440 for use by a parking system during automatic parking. These operations may be similar to those described in the example of FIG. Figure 1 and Figure 2 As described in , a three-dimensional shape information map can be generated based on the coordinate origin in the two-dimensional image and the three-dimensional parameter data of the obstacle. Preferably, when the parameters of the obstacle detected in the three-dimensional shape information map exceed a threshold, an instruction to bypass the obstacle or trigger an alarm can be generated. Preferably, the generated obstacle avoidance route can be updated based on the three-dimensional shape information map and the movement speed and direction of the vehicle. Preferably, operations 430 and 440 can be specifically implemented by the control module 530, as described in more detail below.

[0057] Figure 5 FIG. 5 is a flow chart of a method 500 for coordinating vehicle maneuvers according to an embodiment. Alternative embodiments may be combined, separated, or otherwise varied. Figure 5 The functionality described in the illustrated box can be used to change the functionality. Figure 5 The method 500 illustrates how the above-described (eg, regarding Figure 1 ) of the vehicle. Thus, for performing Figure 5 The functional means of one or more blocks illustrated in FIG. 1 may include hardware and / or software components of the vehicle 100, which (as previously mentioned) may include the Figure 3 One or more components of the device 300 illustrated in and described above.

[0058] At block 510, the functionality includes projecting surface structured light. Preferably, visible light may also be projected. The lighting module 510 for performing the functionality of block 510 may include one or more software and / or hardware components of the device, such as bus 305, processing unit(s) 310, memory 360, and / or at least one of the following: Figure 3 Other software and / or hardware components of the device 300 illustrated in FIG.

[0059] At block 520, the functionality includes generating a two-dimensional image of the terrain based at least in part on visible light in the environment and generating a three-dimensional image of the terrain based at least in part on the projected surface structured light. Preferably, various onboard sensors can be used to determine certain real-time characteristics of the vehicle, such as position, speed, acceleration, etc. The imaging module 520 for performing the functionality of block 520 includes one or more software and / or hardware components of the device, such as the bus 305, the processing unit(s) 310, the memory 360, and / or at least one of the components of the device. Figure 3 Other software and / or hardware components of the device 300 illustrated in FIG.

[0060] At block 530, the functionality includes fusing the two-dimensional image and the three-dimensional image to construct a three-dimensional topographic information map of the terrain; and generating an obstacle avoidance route based at least in part on the three-dimensional topographic information map for use by the parking system during automatic parking. The control module 530 for executing the functionality of block 530 may include one or more software and / or hardware components of the device, such as the bus 305, the processing unit(s) 310, the memory 360, and / or the like. Figure 3 Other software and / or hardware components of the device 300 illustrated in FIG.

[0061] Moreover, an embodiment of the present application further discloses a computer-readable storage medium including computer-executable instructions stored thereon, and when the computer-executable instructions are executed by a processor, the processor is caused to perform the methods of the various embodiments described herein.

[0062] In addition, an embodiment of the present application further discloses a device, which includes a processor and a memory storing computer-executable instructions. When the computer-executable instructions are executed by the processor, the processor executes the methods of the embodiments described herein.

[0063] In addition, embodiments of the present application also disclose a parking system utilizing surface structured light, which includes a device for implementing the methods of various embodiments described herein. In one aspect, the system includes: a device for projecting surface structured light; a device for generating a two-dimensional image of terrain based at least in part on visible light in the environment and a three-dimensional image of the terrain based at least in part on the projected surface structured light; a device for fusing the two-dimensional image and the three-dimensional image to construct a three-dimensional topographic information map of the terrain; and a device for generating an obstacle avoidance route based at least in part on the three-dimensional topographic information map for use by the parking system during automated parking.

[0064] The above describes the parking system and control method thereof using surface structured light according to the present invention. Compared with the prior art, the method of the present invention has at least the following advantages:

[0065] 1. Two imaging systems: one for structured light imaging and the other for visible light imaging. The imaging systems can fuse the images generated based on visible light and structured light.

[0066] 2. The high resolution of visible light imaging compensates for the low resolution of surface structured light;

[0067] 3. Surface structured light provides three-dimensional information, compensating for the situation where visible light only provides two-dimensional information, thereby improving the ability, accuracy, and speed of obstacle recognition.

[0068] Throughout this specification, reference has been made to "an embodiment" to mean that a particular described feature, structure, or characteristic is included in at least one embodiment. Thus, use of these phrases may not refer to only one embodiment. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0069] The various steps and modules of the methods and devices described above can be implemented in hardware, software, or a combination thereof. If implemented in hardware, the various illustrative steps, modules, and circuits described in conjunction with the present disclosure can be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic components, hardware components, or any combination thereof. The general-purpose processor can be a processor, a microprocessor, a controller, a microcontroller, or a state machine, etc. If implemented in software, the various illustrative steps and modules described in conjunction with the present disclosure can be stored or transmitted as one or more instructions or codes on a computer-readable medium. The software modules that implement the various operations of the present disclosure can reside in a storage medium, such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, cloud storage, etc. The storage medium can be coupled to a processor so that the processor can read and write information from / to the storage medium and execute the corresponding program modules to implement the various steps of the present disclosure. Moreover, software-based embodiments can be uploaded, downloaded, or remotely accessed through appropriate communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, an intranet, software applications, cables (including fiber optic cables), magnetic communications, electromagnetic communications (including RF microwave and infrared communications), electronic communications, or other such communication means.

[0070] The numerical values ​​given in the various embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Furthermore, as a whole, the technical solution may include other components or steps not listed in the claims or the specification. Furthermore, a single name for a component does not preclude the use of other names for that component.

[0071] It should also be noted that these embodiments may be described as processes depicted as flow charts, flow diagrams, structure diagrams, or block diagrams. Although a flow chart may describe the operations as sequential processes, many of these operations can be performed in parallel or concurrently. In addition, the order of these operations can be rearranged.

[0072] The disclosed methods, devices, and systems should not be limited in any way. On the contrary, the present disclosure covers all novel and non-obvious features and aspects of the various disclosed embodiments (alone and in various combinations and subcombinations with each other). The disclosed methods, devices, and systems are not limited to any specific aspect or feature or combination thereof, nor do any disclosed embodiments require any one or more specific advantages or solutions to specific or all technical problems.

[0073] The present invention is not limited to the above-mentioned specific embodiments. The above-mentioned specific embodiments are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many forms without departing from the scope of protection of the present invention and the claims. These all fall within the scope of protection of the present invention.

[0074] Persons skilled in the relevant art will recognize that the embodiments can be practiced without one or more of the specific details, or with other methods, resources, materials, etc. In other instances, well-known structures, resources, or operations are not shown or described in detail, merely to obscure aspects of the embodiments.

[0075] Although the embodiments and applications have been illustrated and described, it should be understood that the embodiments are not limited to the precise configuration and resources described above. Various modifications, substitutions, and improvements apparent to those skilled in the art may be made in the arrangement, operation, and details of the methods and systems disclosed herein without departing from the scope of the claimed embodiments.

[0076] As used herein, the terms "and," "or," and "and / or" may include and are intended to have various meanings that depend, at least in part, on the context in which such terms are used. Generally, "or," if used in connection with a list, such as A, B, or C, is intended to mean A, B, and C (where used in an inclusive sense) as well as A, B, or C (where used in an exclusive sense). Additionally, the term "one or more," as used herein, may be used to describe any feature, structure, or characteristic in the singular, or may be used to describe a plurality of features, structures, or characteristics, or some other combination thereof. However, it should be noted that this is merely an illustrative example, and claimed subject matter is not limited to this example.

[0077] While there has been illustrated and described what are presently considered to be example features, it will be understood by those skilled in the art that various other modifications may be made, and equivalents may be substituted, without departing from the claimed subject matter. Additionally, many modifications may be made to adapt a particular scenario to the teachings of the claimed subject matter without departing from the central concept described herein.

Claims

1. A parking system using surface structured light, the parking system comprising: a first lighting module, configured to project the surface structured light; an imaging module, the imaging module being matched to the spectrum of the planar structured light and the spectrum of the visible light of the ambient light, and being configured to generate a two-dimensional image of the terrain based at least in part on the visible light and to generate a three-dimensional image of the terrain based at least in part on the planar structured light; as well as A control module for: fusing the two-dimensional image and the three-dimensional image to construct a three-dimensional topographic information map of the terrain; and An obstacle avoidance route is generated based at least in part on the three-dimensional topographic information map for use by the parking system during automatic parking.

2. The parking system according to claim 1, wherein: The imaging module generating the three-dimensional image includes: Performing a three-dimensional scan of the terrain and protruding, concave, or suspended obstacles on the terrain using the surface structured light to obtain depth information of the obstacles; and The three-dimensional image is generated based at least in part on the three-dimensional scan results.

3. The parking system according to claim 2, wherein: The control module generates the obstacle avoidance route including: When it is detected in the three-dimensional appearance information map that the parameter of the obstacle exceeds a threshold, an instruction to bypass the obstacle or trigger an alarm is generated.

4. The parking system according to claim 1, wherein: The control module updates the generated obstacle avoidance route based at least in part on the three-dimensional topography information map and the vehicle's moving speed and direction, and the vehicle's moving speed and direction are determined at least in part based on the relative relationship between different frames in the two-dimensional image.

5. The parking system according to claim 1 or 4, characterized in that: At least one of the following is presented on a user interface device of the parking system: The three-dimensional shape information map, update of the three-dimensional shape information map, the obstacle avoidance route, and update of the obstacle avoidance route.

6. The parking system according to claim 1, wherein: The surface structured light utilizes at least one of LED structured light or VCSEL structured light, and the surface structured light is at least one of grid structured light, line structured light, dot matrix structured light or coded structured light.

7. The parking system according to claim 1, wherein: The parking system further includes a second lighting module for projecting visible light; and At least one motion sensor is used to detect the moving speed and moving direction of the vehicle during the movement of the vehicle.

8. A method for automatic parking using surface structured light, the method comprising: Projecting surface structured light; generating a two-dimensional image of terrain based at least in part on visible light in an environment and generating a three-dimensional image of the terrain based at least in part on the area structured light; fusing the two-dimensional image and the three-dimensional image to construct a three-dimensional topographic information map of the terrain; and An obstacle avoidance route is generated based at least in part on the three-dimensional topographic information map for use by a parking system during automatic parking.

9. The operating method according to claim 8, characterized in that: Generating the three-dimensional image includes: Performing a three-dimensional scan of the terrain and protruding, concave, or suspended obstacles on the terrain using the surface structured light to obtain depth information of the obstacles; and The three-dimensional image is generated based at least in part on the three-dimensional scan results.

10. The operating method according to claim 9, characterized in that: Generating the obstacle avoidance route includes: When it is detected in the three-dimensional appearance information map that the parameter of the obstacle exceeds a threshold, an instruction to bypass the obstacle or trigger an alarm is generated.

11. The operating method according to claim 8, wherein: The generated three-dimensional topography information map and obstacle avoidance route are updated at least in part based on the three-dimensional topography information map and the movement rate and movement direction of the vehicle, and the movement rate and movement direction of the vehicle are determined at least in part based on the relative relationship between different frames in the two-dimensional image.

12. An electronic device comprising a processor and a memory, wherein the memory stores program instructions; the processor executes the program instructions to implement the operating method for automatic parking using surface structured light according to any one of claims 8 to 11. 13 . A non-transitory computer-readable storage medium storing instructions, wherein when the instructions are executed by a computer, the computer is caused to perform the method for automatic parking using surface structured light according to claim 8 .

Images