Valet parking method, device, equipment and storage medium based on binocular camera

Abstract

This application discloses a valet parking method, apparatus, device, and storage medium based on a binocular camera, relating to the field of automatic parking technology. The binocular camera-based valet parking method involves: preprocessing images acquired by the binocular camera to determine whether the car is in a low-speed cruising phase or an automatic parking phase; when the car is determined to be in the automatic parking phase, using the YOLOv8 detection algorithm to identify obstacle types and parking space markings in the preprocessed image; upon identification of parking space markings, controlling the vehicle to park based on the obstacle type and the parking space markings. When the car is determined to be in the automatic parking phase, the images acquired by the binocular camera are preprocessed, and YOLOv8 is used for obstacle type and parking space marking identification. Based on this, the distances to obstacles and parking space markings are calculated in real time using the binocular camera, and the acquired data is then fused with data from other sensors to achieve stable and efficient vehicle control.

Description

Technical Field

[0001] This application relates to the field of automatic parking technology, and in particular to a valet parking method, apparatus, device and storage medium based on a binocular camera. Background Technology

[0002] Automated Valet Parking (AVP) is an advanced automated parking technology. It is defined as follows: when a user gets out of the car at the designated drop-off point, they issue a parking instruction via a mobile app. Upon receiving the instruction, the vehicle can automatically drive to the parking space in the parking lot without user operation or monitoring. When a user issues a pick-up instruction via the mobile app, the vehicle can automatically drive from the parking space to the designated pick-up point.

[0003] In valet parking, environmental perception is paramount in the technological solution. Currently, multi-sensor fusion has become a development trend in autonomous driving. By integrating perception results from multiple cameras and radars, vehicles can more accurately identify their surroundings and reconstruct the scene. Mainstream AVP solutions typically use sensors such as ultrasonic radar, surround-view cameras, and millimeter-wave radar to detect obstacles and parking space information around the vehicle.

[0004] Mainstream solutions typically use ultrasonic radar for ranging. However, this method cannot identify the type of obstacle and is often affected by interference from unnecessary obstacle detection, leading to sudden braking or a complete stop. Furthermore, when multiple ultrasonic radars operate simultaneously, co-channel interference may occur, and the propagation delay of ultrasonic signals during driving can cause a delay in the received information. These factors affect the stability and reliability of the system.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide a valet parking method, apparatus, device, and storage medium based on a binocular camera, aiming to solve the technical problems of being unable to identify the type of obstacle during valet parking and the inability to identify the type of obstacle and the occurrence of co-frequency interference when multiple ultrasonic radars emit sound.

[0007] To achieve the above objectives, this application proposes a valet parking method based on a binocular camera, the method comprising:

[0008] The images captured by the binocular cameras are preprocessed to determine whether the car is in the low-speed cruising phase or the automatic parking phase.

[0009] When the car is determined to be in the automatic parking stage, the YOLOv8 detection algorithm is called to identify obstacle types and parking space lines in the preprocessed image.

[0010] When the parking space markings are detected, the vehicle is controlled to park according to the type of obstacle and the parking space markings.

[0011] In one embodiment, after identifying the parking space markings and controlling the vehicle to park according to the type of obstacle and the parking space markings, the following steps are included:

[0012] During the automatic parking process, the binocular camera detects the distance to obstacles and the distance to the parking space markings.

[0013] Adjust the parking path based on obstacle distance information and parking space line distance information until the parking process is completed.

[0014] In one embodiment, after the step of calling the YOLOv8 detection algorithm to perform obstacle type identification and parking space identification on the binocular camera image when determining that the car is in the automatic parking stage, the method includes:

[0015] When no marked parking space is identified, the binocular camera is controlled to identify vehicles in front and behind on the same side, determine the available parking space, and generate virtual parking space lines based on the available parking space.

[0016] The vehicle is automatically parked based on the virtual parking space lines.

[0017] In one embodiment, after preprocessing the images captured by the binocular cameras to determine whether the car is in a low-speed cruising phase or an automatic parking phase, the process includes:

[0018] When it is determined that the car is in a low-speed cruising phase, the YOLOv8 detection algorithm is called to identify the types of obstacles and annotate the obstacle boxes in the preprocessed image;

[0019] Based on the obstacle type recognition results, depth information is obtained by ranging the bounding area of ​​the identified obstacle using a binocular camera.

[0020] The vehicle is controlled to travel at a low speed based on the depth information.

[0021] In one embodiment, the step of controlling the vehicle to travel at low speed based on the depth information further includes:

[0022] Acquire information from other sensors;

[0023] The other sensor information is fused with the depth information to calculate the obstacle's position information;

[0024] Acquire information about the vehicle's external environment and internal vehicle information;

[0025] The vehicle's direction and speed are controlled by combining information on obstacle type, obstacle location, external environment, and internal vehicle information.

[0026] In one embodiment, after the step of controlling the vehicle's driving direction and speed by combining obstacle type, obstacle location information, external vehicle environment information, and internal vehicle information, the process includes:

[0027] Detect whether the vehicle has entered the automatic parking phase;

[0028] When the vehicle enters the automatic parking phase, the steps of calling the YOLOv8 detection algorithm to identify obstacle types and parking spaces in the binocular camera images are executed.

[0029] In one embodiment, the step of obtaining depth information of the bounding box region containing the identified obstacles using a binocular camera based on the obstacle type identification result includes:

[0030] Based on the obstacle type identification results and the bounding map region of the obstacle, the binocular vision image is preprocessed;

[0031] The SGBM algorithm is used to match the processed binocular vision images to generate disparity maps, and depth information is calculated from the disparity maps.

[0032] Furthermore, to achieve the above objectives, this application also proposes a valet parking device based on a binocular camera, the binocular camera-based valet parking device comprising:

[0033] The status determination module is used to preprocess the images captured by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage.

[0034] The information processing module is used to call the YOLOv8 detection algorithm to identify obstacle types and marked parking spaces in the pre-processed image when the car is determined to be in the automatic parking stage.

[0035] The parking control module, upon recognizing the parking space markings, controls the vehicle to park according to the type of obstacle and the parking space markings.

[0036] In addition, to achieve the above objectives, this application also proposes a valet parking device based on a binocular camera, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the valet parking method based on a binocular camera as described above.

[0037] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the valet parking method based on a binocular camera as described above.

[0038] This application discloses a valet parking method, apparatus, device, and storage medium based on a binocular camera, relating to the field of automatic parking technology. The binocular camera-based valet parking method includes: preprocessing images acquired by the binocular camera to determine whether the vehicle is in a low-speed cruising phase or an automatic parking phase; when the vehicle is determined to be in an automatic parking phase, using the YOLOv8 detection algorithm to identify obstacle types and parking space markings in the preprocessed image; and when parking space markings are identified, controlling the vehicle to park based on the obstacle types and parking space markings. When the vehicle is determined to be in an automatic parking phase, the images acquired by the binocular camera are preprocessed, and YOLOv8 is used for obstacle type and parking space marking recognition. Based on this, the distances to obstacles and parking space markings are calculated in real time using the binocular camera, and the acquired data is then fused with data from other sensors to achieve stable and efficient vehicle control. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 This is a flowchart illustrating an embodiment of the valet parking method based on a binocular camera in this application.

[0042] Figure 2 This is a flowchart illustrating Embodiment 2 of the valet parking method based on binocular cameras in this application.

[0043] Figure 3 This is a structural diagram provided for Embodiment 2 of the valet parking method based on binocular cameras in this application;

[0044] Figure 4 This is a flowchart illustrating Embodiment 3 of the valet parking method based on binocular cameras in this application;

[0045] Figure 5This is a schematic diagram of the module structure of the valet parking device based on a binocular camera according to an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the hardware operating environment of the valet parking device based on a binocular camera, which is involved in the valet parking method based on a binocular camera in the embodiments of this application.

[0047] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] The main solution of this application embodiment is: the present invention is based on a binocular camera, and uses a YOLOv8 target detection algorithm combined with training to perform target recognition on binocular images to obtain recognition results; based on the recognition results, distance measurement is performed to obtain obstacle information and parking space information.

[0051] It's important to note that valet parking typically involves two main phases: low-speed cruising and automatic parking. During low-speed cruising, the vehicle moves from the user's drop-off point to an empty parking space, or from the parking lot to the user's pick-up point; this process usually involves the vehicle traveling at a low speed. During automatic parking, the vehicle automatically parks itself into or out of the parking space.

[0052] This application provides a solution for a valet parking method based on a binocular camera, comprising: preprocessing images captured by the binocular camera to determine whether the vehicle is in a low-speed cruising phase or an automatic parking phase; when the vehicle is determined to be in an automatic parking phase, using the YOLOv8 detection algorithm to identify obstacle types and parking space lines in the preprocessed image; and when parking space lines are identified, controlling the vehicle to park based on the obstacle type and the parking space lines. When the vehicle is determined to be in an automatic parking phase, the images captured by the binocular camera are preprocessed, and YOLOv8 is used to identify obstacle types and parking space lines. Based on this, the distances to obstacles and parking space lines are calculated in real time using the binocular camera, and the acquired data is then fused with data from other sensors to achieve stable and efficient vehicle control.

[0053] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as an in-vehicle computer, controller, or mobile phone, or a valet parking device based on a binocular camera capable of achieving the above functions. The following description uses a parking controller as an example to illustrate this embodiment and the subsequent embodiments.

[0054] Based on this, the embodiments of this application provide a valet parking method based on a binocular camera, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the valet parking method based on a binocular camera according to this application.

[0055] In this embodiment, the valet parking method based on a binocular camera includes steps S10 to S30:

[0056] Step S10: Preprocess the images captured by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage.

[0057] As we can understand, binocular cameras capture images of a scene using two cameras. Utilizing the principle of triangulation, they calculate the distance and angle between the two cameras to simulate human vision, obtaining depth information and reconstructing the surrounding 3D shape and position. Specifically, the farther an object is from the camera, the smaller its pixel size in the image. In a binocular camera, the two cameras are positioned differently, so the same object will appear in different pixel sizes in the two images. This difference is called parallax. Binocular cameras use this parallax information to calculate the distance to objects.

[0058] It should be noted that by detecting whether the vehicle has entered a parking lot, it can determine whether the vehicle is in the low-speed cruising phase or the automatic parking phase. Simultaneously, the images captured by the binocular cameras undergo preprocessing, including scaling, cropping, and color space conversion, to make them conform to the input requirements of the improved YOLOv8 detection algorithm.

[0059] Step S20: When it is determined that the car is in the automatic parking stage, the YOLOv8 detection algorithm is called to identify the types of obstacles and the parking space markings in the preprocessed image.

[0060] It's important to note that YOLOv8 (You Only Look Once version 8) is a deep learning framework used for real-time object detection. The YOLOv8 algorithm can identify the category and location of objects in an image after only one viewing. It employs supervised learning, training a deep convolutional neural network to achieve object detection. The supervisory information for each image consists of the objects it contains, and each object's information includes its center position (x, y), height (h), width (w), and object category.

[0061] Understandably, this application uses the lightweight network MobileNetV3 instead of Darknet53. MobileNetV3 is a network optimized for mobile and embedded devices, which reduces computational cost while maintaining high performance through the use of depthwise separable convolutions. The improved YOLOv8 algorithm enhances performance on mobile classification, detection, and segmentation tasks through a novel architectural design, including inverted residual structures and linear bottleneck layers, as well as a new efficient segmentation decoder, Lite Reduced Atrous Spatial PyramidPooling (LR-ASPP). These improvements achieve higher accuracy, lower latency, and better performance through a carefully designed lightweight architecture.

[0062] Specifically, the processed image is input into the MobileNetV3 backbone network to extract feature information; the network output is calculated layer by layer based on convolution, activation and pooling operations; target location and category information are obtained in the output layer, and obstacle box annotation and parking space marking recognition are performed on the processed image.

[0063] Step S30: When the parking space markings are detected, control the vehicle to park according to the type of obstacle and the parking space markings.

[0064] Understandably, obstacles need to be categorized, such as vehicles, pedestrians, bicycles, and fixed objects (e.g., bollards, trees). The location, size, and dynamic state (e.g., speed of movement) of the obstacles need to be determined. Parking space markings need to be identified, including parallel, perpendicular, and diagonal markings. The precise location and size of the parking space need to be determined.

[0065] It should be noted that a safe and feasible parking route should be planned based on parking space markings and obstacle information. The type and location of obstacles should be considered to avoid collisions. If multiple obstacles exist, multiple alternative routes may need to be planned, and the optimal route should be selected.

[0066] Step S40: During the automatic parking process, the distance to obstacles and the distance to the parking space markings are detected by the binocular camera.

[0067] Understandably, the planned parking path is translated into vehicle control commands, such as steering, acceleration, deceleration, and braking. Vehicle motion control is achieved through actuators (such as the steering motor, braking system, and accelerator pedal). During parking, the positions of obstacles and parking space markings, as well as the vehicle's driving status, are continuously monitored to ensure the accuracy and safety of the parking process.

[0068] Step S50: Adjust the parking path according to the obstacle distance information and the parking space line distance information until the parking process is completed.

[0069] Understandably, once a vehicle successfully parks in a space, sensors confirm its position and whether it meets parking requirements. If an emergency occurs during parking (such as another vehicle entering the space or a pedestrian suddenly appearing), the system should be able to respond promptly to avoid collisions or take other safety measures.

[0070] In this embodiment, a binocular camera captures images, preprocesses the captured images, and determines whether the car has entered the parking lot based on the captured images or the user's control APP, thereby determining whether the vehicle is in the low-speed cruising stage or the automatic parking stage. When it is determined that the car is in the automatic parking stage, the preprocessed image is input into the trained YOLOv8 neural network for target detection, identification of obstacle types and parking space marking recognition. When a marked parking space is identified, the car automatically parks.

[0071] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This is a flowchart illustrating Embodiment 2 of the valet parking method based on binocular cameras provided in this application. Figure 3 This is a structural diagram provided for Embodiment 2 of the valet parking method based on binocular cameras in this application.

[0072] In step S20, the valet parking method based on a binocular camera includes:

[0073] Step S201: When no marked parking space is identified, control the binocular camera to identify vehicles in front and behind on the same side, determine the space parking space, and generate virtual parking space lines based on the space parking space.

[0074] It should be noted that when the image captured by the binocular camera does not identify a marked parking space, a space is identified by judging the vehicles in front and behind on the same side, such as... Figure 3As shown, when the cars in front and behind on the same side are parallel to the car waiting to park, the parking space is a parallel parking space, and the binocular camera generates a virtual parallel parking space as shown in the figure; when the cars in front and behind on the same side are perpendicular to the car waiting to park, the parking space is a perpendicular parking space, and the binocular camera generates a virtual perpendicular parking space as shown in the figure.

[0075] Step S202: Control the vehicle to park automatically according to the virtual parking space lines.

[0076] It should be noted that after a virtual parking space is set, the parking controller will plan the optimal path to that virtual parking space, taking into account factors such as vehicle size, steering ability, speed, and obstacles in the surrounding environment.

[0077] In this embodiment, the binocular camera captures images. When it is determined that the car is in the automatic parking stage, the pre-processed image is input into the trained YOLOv8 neural network for target detection, identification of obstacle types and parking space marking recognition. If no marked parking space is identified, the space parking space is identified by judging the vehicles in front and behind on the same side, forming a virtual parking space, and then automatic parking is performed.

[0078] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 , Figure 3 This is a flowchart illustrating Embodiment 3 of the valet parking method based on binocular cameras in this application.

[0079] Prior to step S10, the valet parking method based on binocular cameras also includes...

[0080] Step S301: When it is determined that the car is in the low-speed cruising stage, the YOLOv8 detection algorithm is called to identify the types of obstacles and mark the obstacle boxes in the preprocessed image.

[0081] It should be noted that low-speed cruise refers to activating the cruise control system in a parking lot at a low speed to keep the vehicle moving steadily at the set speed and direction.

[0082] Understandably, when the car is cruising at low speed, the binocular camera inputs the pre-processed image into the trained YOLOv8 neural network to perform target detection and identify the types of obstacles.

[0083] Step S302: Based on the obstacle type recognition results, depth information is obtained by measuring the distance of the frame area where the obstacles are identified using a binocular camera.

[0084] It should be noted that step S302 includes the following steps:

[0085] Step S321: Based on the obstacle type identification results and the bounding box region of the obstacle, preprocess the binocular vision image.

[0086] It should be noted that the image preprocessing involves replacing the horizontal Sobel operator with the Scharr operator for edge detection on both the left and right images. The Scharr operator enhances the difference between the Sobel operator's edges. The Scharr operator's edge detection filter has a size of 3×3, which can increase the difference between pixel values ​​by amplifying the weight coefficients in the filter, thus compensating for the Sobel operator's poor edge extraction performance in images.

[0087] As is understandable, the Scharr operator is an edge detection operator in image processing, primarily used for calculating image gradients. It contains two 3x3 convolution kernels, used to calculate the gradients in the horizontal and vertical directions, respectively. Compared to the Sobel operator, the Scharr operator has heavier weights on the center element in the smoothing portion, equivalent to using a Gaussian function with a smaller standard deviation for smoothing. This makes the Scharr operator perform better in terms of noise sensitivity and gradient accuracy, making it particularly suitable for processing images with high noise levels.

[0088] Step S322: The SGBM algorithm is used to match the processed binocular vision image to generate a disparity map, and depth information is calculated through the disparity map.

[0089] It should be noted that the gradients of the Scharr operator's convolution kernel in the horizontal and vertical directions are as follows: Horizontal gradient Scharr convolution kernel Kx:

[0090]

[0091] Vertical gradient Scharr convolution kernel Ky:

[0092]

[0093] For each pixel, a weighted summation is performed within the local region using the corresponding convolution kernel to obtain the gradient values ​​Scharr(x) and Scharr(y) in the x and y directions, respectively, as shown in the following formulas:

[0094] Scharr(x)=10[P(x+1,y)-P(x-1,y)]+3[P(x+1,y-1)-P(x-1,y-1)]+3[P(x+1,y+1)-P(x-1,y+1)]

[0095] Scharr(y)=10[P(x,y-1)-P(x,y+1)]+3[P(x-1,y-1)-P(x-1,y+1)]+3[P(x+1,y-1)-P(x+1,y+1)]

[0096] Where P(x,y) represents the pixel value of the original image at coordinates (x,y).

[0097] The weighting factor, which reflects the edge strength of the image, is calculated using the gradient value.

[0098]

[0099] The SAD cost of each pixel under different disparities is calculated using the original image, as shown in the following formula:

[0100] SAD(p,d)=∑(x,y)∈window∣IL(x,y)-IR(xd,y)∣

[0101] Where IL and IR are the pixel values ​​at position p(x,y) in the left and right images, respectively, d is the disparity, (xd,y) represents the corresponding position of position p in the right image, and window is the SAD window, which is set to 3*3 here.

[0102] Combining SAD cost and gradient information, construct the cost function:

[0103] Ccombined(p,d)=SAD(p,d)×W(p)

[0104] Then, a disparity map is obtained through dynamic programming and data post-processing, and depth information is calculated from the disparity map.

[0105] Understandably, the SGBM (Semi-Global Block Matching) algorithm is a semi-global matching algorithm used to calculate disparity in binocular vision. It selects the disparity of each pixel to form a disparity map, and sets a global energy function associated with the disparity map. Minimizing this energy function achieves the goal of finding the optimal disparity for each pixel.

[0106] Step S303: Obtain information from other sensors.

[0107] It should be noted that other sensors include integrated LiDAR, millimeter-wave radar, ultrasonic sensors, and other sensors to acquire comprehensive and multi-layered environmental information. Each sensor has its unique advantages: for example, LiDAR can provide high-precision distance information, while millimeter-wave radar has strong penetration capabilities.

[0108] Step S304: Fuse the other sensor information with the depth information to calculate the location information of the obstacle.

[0109] It should be noted that the type of obstacle, such as vehicle, pedestrian, and tree, is identified by using a binocular camera combined with the YOLOv8 detection algorithm. The depth information is calculated by using the binocular camera combined with the SGBM algorithm, and the depth information of the obstacle is provided by LiDAR and ultrasonic sensors to estimate the state of the obstacle, including its position, speed and direction.

[0110] Step S305: Obtain information about the vehicle's external environment and internal vehicle information.

[0111] It should be noted that vehicle external environment information includes information obtained through vehicle-to-everything (V2X) technology, which allows vehicles to communicate with surrounding vehicles and traffic infrastructure to obtain real-time traffic information and road condition data, thus gaining a more comprehensive understanding of the vehicle's external environment. Vehicle external environment information can also include high-precision maps and GPS positioning information, enabling vehicles to obtain their own location, driving direction, and surrounding road information in real time. Internal vehicle information includes basic information such as vehicle speed, acceleration, steering angle, and braking status, which is crucial for path planning and obstacle avoidance decisions.

[0112] Step S306: Combine obstacle type, obstacle location information, vehicle external environment information, and internal vehicle information to control the vehicle's driving direction and speed.

[0113] It should be noted that the optimal driving route is generated based on obstacle information, combined with the binocular camera and the vehicle's current status. Path planning needs to consider the vehicle's dynamic characteristics and obstacle information. During driving, the vehicle speed and direction are controlled based on real-time information until the low-speed cruising phase ends or the vehicle enters the automatic parking phase.

[0114] In this embodiment, when the vehicle is determined to have entered the low-speed cruising phase, an improved YOLOv8 algorithm is used to identify obstacle types and draw obstacle bounding maps on the binocular images. Based on the obstacle bounding maps, an optimized SGBM algorithm is used to calculate obstacle depth information. Areas outside the bounding maps can be ignored in the calculation, reducing the computational load. By using obstacle type and location information, external environmental information and internal vehicle information are obtained. This information is then fused with information from other vehicle sensors to select different driving strategies, reducing errors during the low-speed cruising phase.

[0115] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the valet parking method based on binocular cameras in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0116] This application also provides a valet parking device based on a binocular camera; please refer to [reference needed]. Figure 5 The valet parking device based on binocular cameras includes:

[0117] The state determination module 10 is used to preprocess the images captured by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage.

[0118] Information processing module 20 is used to call the YOLOv8 detection algorithm to identify obstacle types and marked parking spaces in the preprocessed image when it is determined that the car is in the automatic parking stage.

[0119] The parking control module 30, upon recognizing the parking space markings, controls the vehicle to park according to the type of obstacle and the parking space markings.

[0120] The valet parking device based on a binocular camera provided in this application, employing the valet parking method based on a binocular camera in the above embodiments, can solve the technical problems of being unable to identify the type of obstacle during valet parking and the inability to identify the type of obstacle and the occurrence of co-channel interference when multiple ultrasonic radars emit sound. Compared with the prior art, the beneficial effects of the valet parking device based on a binocular camera provided in this application are the same as the beneficial effects of the valet parking method based on a binocular camera provided in the above embodiments, and other technical features in the valet parking device based on a binocular camera are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0121] This application provides a valet parking device based on a binocular camera. The valet parking device based on a binocular camera includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the valet parking method based on a binocular camera in the first embodiment described above.

[0122] The following is for reference. Figure 6This document illustrates a structural schematic diagram of a valet parking device based on a binocular camera, suitable for implementing embodiments of this application. The binocular camera-based valet parking device in this application embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The valet parking device based on a binocular camera shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments in this application.

[0123] like Figure 6 As shown, the valet parking device based on a binocular camera may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the binocular camera-based valet parking device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the binocular camera-based valet parking equipment to exchange data wirelessly or via wired communication with other devices. Although binocular camera-based valet parking equipment with various systems is shown in the figures, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0124] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0125] The valet parking device based on a binocular camera provided in this application, employing the valet parking method based on a binocular camera in the above embodiments, can solve the technical problem of mechanical fatigue imbalance of the dual-motor spindle. Compared with the prior art, the beneficial effects of the valet parking device based on a binocular camera provided in this application are the same as the beneficial effects of the valet parking method based on a binocular camera provided in the above embodiments, and other technical features in this valet parking device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0126] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0127] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0128] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the valet parking method based on a binocular camera in the above embodiments.

[0129] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0130] The aforementioned computer-readable storage medium may be included in a binocular camera-based valet parking device; or it may exist independently and not assembled into a binocular camera-based valet parking device.

[0131] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0132] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0133] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0134] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described valet parking method based on a binocular camera. This solves the technical problems of being unable to identify obstacle types during valet parking and the inability to identify obstacle types and co-channel interference when multiple ultrasonic radars emit sound. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the binocular camera-based valet parking method provided in the above embodiments, and will not be repeated here.

[0135] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A valet parking method based on a binocular camera, characterized in that, The aforementioned valet parking method based on binocular cameras includes: The images captured by the binocular cameras are preprocessed to determine whether the car is in the low-speed cruising phase or the automatic parking phase. When the car is determined to be in the automatic parking stage, the YOLOv8 detection algorithm is called to identify obstacle types and parking space lines in the preprocessed image. When the parking space markings are detected, the vehicle is controlled to park according to the type of obstacle and the parking space markings. After preprocessing the images captured by the binocular cameras to determine whether the car is in a low-speed cruising phase or an automatic parking phase, the following steps are included: When it is determined that the car is in a low-speed cruising phase, the YOLOv8 detection algorithm is called to identify the types of obstacles and annotate the obstacle boxes in the preprocessed image; Based on the obstacle type recognition results, depth information is obtained by ranging the bounding area of ​​the identified obstacle using a binocular camera. The vehicle is controlled to travel at a low speed based on the depth information. The step of obtaining depth information of the bounding area containing the identified obstacles using a binocular camera based on the obstacle type recognition result includes: Based on the obstacle type identification results and the bounding map region of the obstacle, the binocular vision image is preprocessed; The SGBM algorithm is used to match the processed binocular vision images to generate disparity maps, and depth information is calculated from the disparity maps.

2. The valet parking method based on a binocular camera as described in claim 1, characterized in that, Upon recognizing the parking space markings, the step of controlling the vehicle to park based on the type of obstacle and the parking space markings includes: During the automatic parking process, the binocular camera detects the distance to obstacles and the distance to the parking space markings. Adjust the parking path based on obstacle distance information and parking space line distance information until the parking process is completed.

3. The valet parking method based on a binocular camera as described in claim 1, characterized in that, After determining that the car is in the automatic parking stage, the step of calling the YOLOv8 detection algorithm to perform obstacle type identification and parking space identification on the binocular camera image includes: When no marked parking space is identified, the binocular camera is controlled to identify vehicles in front and behind on the same side, determine the available parking space, and generate virtual parking space lines based on the available parking space. The vehicle is automatically parked based on the virtual parking space lines.

4. The valet parking method based on a binocular camera as described in claim 1, characterized in that, The step of controlling the vehicle to travel at low speed based on the depth information further includes: Acquire information from other sensors; The other sensor information is fused with the depth information to calculate the obstacle's position information; Acquire information about the vehicle's external environment and internal vehicle information; The vehicle's direction and speed are controlled by combining information on obstacle type, obstacle location, external environment, and internal vehicle information.

5. The valet parking method based on a binocular camera as described in claim 4, characterized in that, After the steps of controlling the vehicle's direction and speed by combining obstacle type, obstacle location information, external vehicle environment information, and internal vehicle information, the following are included: Detect whether the vehicle has entered the automatic parking phase; When the vehicle enters the automatic parking phase, the steps of calling the YOLOv8 detection algorithm to identify obstacle types and parking spaces in the binocular camera images are executed.

6. A valet parking device based on a binocular camera, characterized in that, The device includes: The status determination module is used to preprocess the images captured by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage. The information processing module is used to call the YOLOv8 detection algorithm to identify obstacle types and marked parking spaces in the pre-processed image when the car is determined to be in the automatic parking stage. The parking control module, upon recognizing the parking space markings, controls the vehicle to park according to the type of obstacle and the parking space markings; The state determination module is also used to, when determining that the car is in a low-speed cruising phase, call the YOLOv8 detection algorithm to identify obstacle types and mark obstacle bounding boxes on the preprocessed image; based on the obstacle type identification results, obtain depth information of the bounding box area of ​​the identified obstacles by measuring distance with a binocular camera; and control the vehicle to drive at low speed according to the depth information. The state determination module is also used to preprocess the binocular vision image based on the obstacle type identification result and the obstacle bounding box region; to use the SGBM algorithm to match the processed binocular vision image to generate a disparity map, and to calculate depth information through the disparity map.

7. A valet parking device based on a binocular camera, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the valet parking method based on a binocular camera as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the valet parking method based on a binocular camera as described in any one of claims 1 to 5.

Images