Valet parking method, device and equipment based on binocular camera and storage medium

Abstract

The invention discloses a valet parking method, device and equipment based on a binocular camera and a storage medium, and relates to the technical field of automatic parking. The invention discloses a valet parking method based on a binocular camera. The valet parking method comprises the following steps: preprocessing an image acquired by the binocular camera, and judging whether an automobile is in a low-speed cruise stage or an automatic parking stage; when it is judged that the automobile is in the automatic parking stage, a YOLOv8 detection algorithm is called to conduct obstacle type recognition and parking space lineation recognition on the preprocessed picture; and when the parking space lineation is recognized, controlling the vehicle to park according to the type of the obstacle and the parking space lineation. When it is judged that the automobile is in the automatic parking stage, an image collected by the binocular camera is preprocessed, obstacle type and parking space lineation recognition is conducted by introducing YOLOv8, the obstacle and parking space lineation distance is calculated and obtained in real time through the binocular camera on the basis, and then obtained data is fused with data of other sensors; and on the basis, stable and efficient control of the vehicle is realized.

Description

Technical Field

[0001] The present application relates to the technical field of automatic parking, and in particular to a valet parking method, device, equipment and storage medium based on a binocular camera. Background Art

[0002] Automated Valet Parking (AVP) is an advanced automatic parking technology. It is defined as the user gets off the car at the drop-off point and issues a parking instruction through a mobile phone APP. After receiving the instruction, the vehicle can automatically drive to the parking space in the parking lot without the need for user operation and monitoring; the user issues a pick-up instruction through a mobile phone APP, and the vehicle can automatically drive from the parking space to the designated pick-up point after receiving the instruction.

[0003] In the process of valet parking, environmental perception is the top priority of the technical solution. At present, multi-sensor fusion has become the development trend of autonomous driving. By integrating the perception results of multiple cameras and radars, the vehicle can more accurately identify the surrounding environment and achieve scene reconstruction. The current mainstream AVP solutions usually use sensors such as ultrasonic radar, surround view camera, millimeter wave radar, etc. to detect obstacles around the vehicle and parking space information.

[0004] In mainstream solutions, ultrasonic radar is used for distance measurement. However, this solution cannot identify the type of obstacles and is often affected by interference caused by unnecessary obstacle identification, resulting in sudden braking or direct stopping. In addition, when multiple ultrasonic radars are working at the same time, co-frequency interference may occur. During driving, the propagation delay of ultrasonic signals will cause a certain delay in the received information. These factors will affect the stability and reliability of the system.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a valet parking method, device, equipment and storage medium based on a binocular camera, aiming to solve the technical problems of being unable to identify the type of obstacles during valet parking and being unable to identify the type of obstacles when multiple ultrasonic radars sound and co-frequency interference.

[0007] To achieve the above objectives, the present application proposes a valet parking method based on a binocular camera, and the valet parking method based on a binocular camera includes:

[0008] Pre-process the images collected by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage;

[0009] When judging that the car is in the automatic parking stage, the YOLOv8 detection algorithm is called to identify the obstacle type and parking space line of the pre-processed image;

[0010] When the parking space markings are identified, the vehicle is controlled to park according to the type of obstacle and the parking space markings.

[0011] In one embodiment, after the step of controlling the vehicle to park according to the obstacle type and the parking space markings when the parking space markings are identified, the method further comprises:

[0012] During the automatic parking process of the vehicle, the distance to the obstacle and the distance to the parking space marking are detected by the binocular camera;

[0013] The parking path is adjusted according to the obstacle distance information and the parking space line distance information until the parking process is completed.

[0014] In one embodiment, when determining that the automobile is in the automatic parking stage, after the step of calling the YOLOv8 detection algorithm to perform obstacle type recognition and parking space recognition on the binocular camera image, the following steps are included:

[0015] When no marked parking space is identified, the binocular camera is controlled to identify the front and rear vehicles on the same side, determine the space parking space and generate a virtual parking space line according to the space parking space;

[0016] The vehicle is controlled to perform automatic parking according to the virtual parking space line.

[0017] In one embodiment, after the step of preprocessing the image collected by the binocular camera to determine whether the vehicle is in the low-speed cruising stage or the automatic parking stage, the method includes:

[0018] When it is determined that the car is in the low-speed cruising stage, the YOLOv8 detection algorithm is called to identify the obstacle types and mark the obstacle boxes on the pre-processed images;

[0019] Based on the obstacle type recognition result, the depth information of the frame area where the obstacle is identified is obtained by using a binocular camera to measure the distance;

[0020] The vehicle is controlled to travel at a low speed according to the depth information.

[0021] In one embodiment, the step of controlling the vehicle to travel at a low speed according to the depth information further includes:

[0022] Get other sensor information;

[0023] Combining the other sensor information with the depth information to calculate the location information of the obstacle;

[0024] Obtain vehicle external environment information and internal vehicle information;

[0025] The vehicle's driving direction and speed are controlled by combining obstacle type, obstacle location information, vehicle external environment information, and internal vehicle information.

[0026] In one embodiment, after the step of controlling the driving direction and driving speed of the vehicle in combination with the obstacle type, obstacle location information, vehicle external environment information, and internal vehicle information, the following steps are included:

[0027] Detect whether the vehicle enters the automatic parking stage;

[0028] When the vehicle enters the automatic parking stage, the step of calling the YOLOv8 detection algorithm to perform obstacle type recognition and parking space recognition on the binocular camera image is executed.

[0029] In one embodiment, the step of obtaining depth information of the frame area where the obstacle is identified by using a binocular camera based on the obstacle type identification result includes:

[0030] Preprocess the binocular vision image based on the obstacle type recognition result and the obstacle block diagram area;

[0031] The SGBM algorithm is used to match the processed binocular vision images to generate a disparity map, and the depth information is calculated through the disparity map.

[0032] In addition, to achieve the above-mentioned purpose, the present application also proposes a valet parking device based on a binocular camera, and the valet parking device based on a binocular camera includes:

[0033] The state determination module is used to pre-process the images collected by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage;

[0034] The information processing module is used to call the YOLOv8 detection algorithm to identify obstacle types and marked parking spaces in the pre-processed images when judging that the car is in the automatic parking stage;

[0035] The parking control module controls the vehicle to park according to the obstacle type and the parking space markings when the parking space markings are identified.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a valet parking device based on a binocular camera, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the valet parking method based on a binocular camera as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the valet parking method based on a binocular camera as described above are implemented.

[0038] The present application discloses a valet parking method, device, equipment and storage medium based on a binocular camera, and relates to the field of automatic parking technology. The valet parking method based on a binocular camera includes: pre-processing the image collected by the binocular camera to determine whether the car is in a low-speed cruising stage or an automatic parking stage; when it is determined that the car is in the automatic parking stage, calling the YOLOv8 detection algorithm to identify the obstacle type and parking line of the pre-processed image; when the parking line is identified, the vehicle is controlled to park according to the obstacle type and the parking line. When it is determined that the car is in the automatic parking stage, the image collected by the binocular camera is pre-processed, and the obstacle type and parking line are identified by introducing YOLOv8. On this basis, the binocular camera is used to calculate and obtain the obstacle and parking line distance in real time, and then the acquired data is integrated with other sensor data, and on this basis, stable and efficient control of the vehicle is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A flowchart diagram of a first embodiment of a valet parking method based on a binocular camera of the present application;

[0042] Figure 2 A flowchart diagram of a second embodiment of a valet parking method based on a binocular camera of the present application;

[0043] Figure 3 A structural diagram of a second embodiment of a valet parking method based on a binocular camera of the present application;

[0044] Figure 4 A flowchart diagram of a third embodiment of a valet parking method based on a binocular camera of the present application;

[0045] Figure 5This is a schematic diagram of the module structure of a valet parking device based on a binocular camera according to an embodiment of the present application;

[0046] Figure 6 It is a schematic structural diagram of a valet parking device based on a binocular camera of the hardware operating environment involved in the valet parking method based on a binocular camera in an embodiment of the present application.

[0047] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] The main solution of the embodiment of the present application is: based on a binocular camera, the present invention adopts the YOLOv8 target detection algorithm combined with training to perform target recognition on the binocular image to obtain a recognition result; based on the recognition result, distance measurement is performed to obtain obstacle information and parking space information.

[0051] It should be noted that valet parking usually includes two main stages: low-speed cruising and automatic parking. In the low-speed cruising stage, the vehicle drives from the user's drop-off point to an empty space in the parking lot, or from the parking lot to the user's pick-up point. During this process, the vehicle usually drives at a lower speed. In the automatic parking stage, the vehicle needs to automatically park in or out of the parking space.

[0052] The present application provides a solution, and the valet parking method based on binocular camera includes: pre-processing the images collected by binocular camera to judge whether the car is in the low-speed cruising stage or the automatic parking stage; when judging that the car is in the automatic parking stage, calling YOLOv8 detection algorithm to identify the obstacle type and parking line of the pre-processed image; when the parking line is identified, controlling the vehicle to park according to the obstacle type and parking line. When judging that the car is in the automatic parking stage, pre-processing the images collected by binocular camera, introducing YOLOv8 to identify the obstacle type and parking line, on this basis, using binocular camera to calculate and obtain the obstacle and parking line distance in real time, and then integrating the acquired data with other sensor data, on this basis, realizing stable and efficient control of the vehicle.

[0053] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a vehicle-mounted computer, controller, mobile phone, etc., or a valet parking device based on a binocular camera that can realize the above functions, etc. The following takes the parking controller as an example to illustrate this embodiment and the following embodiments.

[0054] Based on this, the embodiment of the present application provides a valet parking method based on a binocular camera, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the valet parking method based on a binocular camera of the present application.

[0055] In this embodiment, the valet parking method based on a binocular camera includes steps S10 to S30:

[0056] Step S10: Pre-process the images collected by the binocular camera to determine whether the car is in a low-speed cruising stage or an automatic parking stage.

[0057] It is understandable that the binocular camera captures the image in the scene through two cameras, and uses the principle of triangulation to simulate the visual function of the human eye by calculating the distance and angle information between the two cameras, obtain the depth information of the scene, and thus reconstruct the surrounding three-dimensional shape and position. Specifically, the farther the object is from the camera, the smaller the pixel size it occupies in the image. In the binocular camera, the positions of the two cameras are different, so the pixel size of the same object in the two images is also different. This size difference is called parallax. The binocular camera uses this parallax information to calculate the distance of the object.

[0058] It should be noted that by detecting whether the vehicle enters the parking lot, it can be determined whether the vehicle is in the low-speed cruising stage or the automatic parking stage. At the same time, the images collected by the binocular camera are preprocessed, including scaling, cropping and color space conversion, so that they meet the input requirements of the improved YOLOv8 detection algorithm.

[0059] Step S20: When it is determined that the car is in the automatic parking stage, the YOLOv8 detection algorithm is called to identify the obstacle type and parking space line of the pre-processed image.

[0060] It should be noted that YOLOv8 (You Only Look Once version 8) is a deep learning framework for real-time object detection. The YOLOv8 algorithm only needs to browse the image once to identify the category and location of the object in the image. It uses supervised learning to achieve target detection by training a deep convolutional neural network. The supervision information of each image is the objects it contains. The information of each object includes the center position (x, y), height (h), width (w) and object category of the object.

[0061] It is understandable that this application uses a lightweight network MobileNetV3 instead of Darknet53. MobileNetV3 is a network optimized for mobile and embedded devices. They use deep separable convolution to reduce the amount of computation while maintaining high performance. The improved YOLOv8 algorithm can improve performance in mobile classification, detection, and segmentation tasks through the adoption of a novel architecture design, including an inverted residual structure and a linear bottleneck layer, as well as a new efficient segmentation decoder Lite Reduced Atrous Spatial PyramidPooling (LR-ASPP). These improvements achieve higher accuracy, lower latency, and better performance through a carefully designed lightweight architecture.

[0062] Specifically, the processed image is input into the MobileNetV3 backbone network to extract feature information; based on convolution, activation and pooling operations, the network output is calculated layer by layer; the target position and category information are obtained in the output layer, and the processed image is annotated with obstacle boxes and parking space lines.

[0063] Step S30: When the parking space markings are identified, the vehicle is controlled to park according to the type of obstacle and the parking space markings.

[0064] It is understood that obstacles are classified, such as vehicles, pedestrians, bicycles, fixed objects (such as road posts, trees), etc. The location, size and dynamic state (such as moving speed) of obstacles are determined. Parking space markings are identified, including parallel parking space markings, vertical parking space markings or diagonal parking space markings, etc. The exact location and size of the parking space are determined.

[0065] It should be noted that a safe and feasible parking path is planned based on the parking space markings and obstacle information. The type and location of obstacles are considered to avoid collisions with obstacles. If there are multiple obstacles, multiple alternative paths may need to be planned and the optimal path selected.

[0066] Step S40: During the automatic parking process of the vehicle, the distance to the obstacle and the distance to the parking space marking are detected by the binocular camera.

[0067] It is understandable that the planned parking path is converted into vehicle control instructions, such as steering, acceleration, deceleration and braking, etc. The vehicle's motion control is achieved through the vehicle's actuators (such as steering motors, brake systems, accelerator pedals, etc.). During the parking process, the position changes of obstacles and parking space markings, as well as the driving status of the vehicle, are continuously monitored to ensure the accuracy and safety of the parking process.

[0068] Step S50: adjusting the parking path according to the obstacle distance information and the parking space line distance information until the parking process is completed.

[0069] It is understandable that when the vehicle is successfully parked in the parking space, the sensor confirms whether the vehicle position is accurate and whether it meets the parking requirements. If an emergency occurs during parking (such as other vehicles breaking into the parking space, pedestrians suddenly appearing, etc.), the system should be able to respond in time to avoid collision or take other safety measures.

[0070] In this embodiment, the binocular camera collects images, pre-processes the collected images, and determines whether the car enters the parking lot through the collected images or the user controls the APP, thereby determining whether the vehicle is in the low-speed cruising stage or the automatic parking stage. When it is determined that the car is in the automatic parking stage, the pre-processed images are input into the trained YOLOv8 neural network to perform target detection, identify obstacle types and parking space markings, and automatically park when a marked parking space is identified.

[0071] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 , Figure 2 This is a flow chart of the second embodiment of the valet parking method based on binocular cameras of this application. Figure 3 This is a structural diagram provided for Embodiment 2 of the valet parking method based on a binocular camera of the present application.

[0072] In step S20, the binocular camera-based valet parking method includes:

[0073] Step S201: When no marked parking space is identified, the binocular camera is controlled to identify the front and rear vehicles on the same side, determine the spatial parking space and generate a virtual parking space line according to the spatial parking space.

[0074] It should be noted that when the image captured by the binocular camera does not identify the marked parking space, the space parking space is identified by judging the front and rear vehicles on the same side, such as Figure 3As shown, when the front and rear vehicles on the same side are parallel to the car to be parked, the parking space is a parallel parking space, and the binocular camera generates a virtual parallel-lined parking space as shown in the figure; when the front and rear vehicles on the same side are perpendicular to the car to be parked, the parking space is a vertical parking space, and the binocular camera generates a virtual vertical-lined parking space as shown in the figure.

[0075] Step S202: Controlling the vehicle to perform automatic parking according to the virtual parking space line.

[0076] It should be noted that after the virtual parking space is set, the parking controller will plan an optimal path to the virtual parking space, taking into account factors such as the vehicle's size, steering ability, driving speed, and obstacles in the surrounding environment.

[0077] In this embodiment, the binocular camera collects images. When it is determined that the car is in the automatic parking stage, the pre-processed images are input into the trained YOLOv8 neural network to perform target detection, identify the types of obstacles and parking line recognition. When no marked parking space is recognized, the spatial parking space is identified by judging the front and rear vehicles on the same side to form a virtual parking space, and then automatic parking is performed.

[0078] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those of the above-mentioned first and second embodiments can refer to the above introduction, and will not be repeated later. Figure 3 , Figure 3 A flowchart diagram of the third embodiment of the valet parking method based on a binocular camera of the present application is provided.

[0079] Before step S10, the binocular camera-based valet parking method further includes:

[0080] Step S301: When it is determined that the vehicle is in a low-speed cruising stage, the YOLOv8 detection algorithm is called to identify obstacle types and mark obstacle boxes on the preprocessed image.

[0081] It should be noted that low-speed cruising means starting the cruise system in a parking lot when the vehicle speed is low, so that the vehicle can travel steadily at the set speed and direction.

[0082] It is understandable that when the car is in the low-speed cruising stage, the binocular camera inputs the pre-processed images into the trained YOLOv8 neural network to perform target detection and identify the types of obstacles.

[0083] Step S302: Based on the obstacle type recognition result, the depth information of the frame area where the obstacle is recognized is obtained by using a binocular camera to measure the distance.

[0084] It should be noted that step S302 includes the following steps:

[0085] Step S321: pre-processing the binocular vision image based on the obstacle type recognition result and the obstacle frame area.

[0086] It should be noted that the image is preprocessed and the Scharr operator is used to replace the horizontal Sobel operator for edge detection on the left and right images. The Scharr operator is an enhancement of the difference of the Sobel operator. The size of the edge detection filter of the Scharr operator is 3×3. The difference between pixel values ​​can be increased by amplifying the weight coefficient in the filter, making up for the disadvantage that the Sobel operator has a poor effect on extracting weaker edges in the image.

[0087] It can be understood that the Scharr operator is an edge detection operator in image processing, which is mainly used for edge detection by calculating image gradients. It contains two 3x3 convolution kernels, which are used to calculate the gradients of the image in the horizontal and vertical directions respectively. Compared with the Sobel operator, the Scharr operator has a heavier central element weight in the smooth part, which is equivalent to using a Gaussian function with a smaller standard deviation for smoothing. This makes the Scharr operator perform better in terms of noise sensitivity and gradient accuracy, and is particularly suitable for processing images with high noise.

[0088] Step S322: using the SGBM algorithm to match the processed binocular vision images to generate a disparity map, and calculating depth information through the disparity map.

[0089] It should be noted that the convolution kernel of the Scharr operator has the following gradients for the horizontal and vertical directions: Horizontal gradient Scharr convolution kernel Kx:

[0090]

[0091] Vertical gradient Scharr convolution kernel Ky:

[0092]

[0093] For each pixel, the corresponding convolution kernel is used to perform weighted summation in the local area to obtain the gradient values ​​of the point in the x direction and y direction, respectively, Scharr (x) and Scharr (y), the formula is as follows:

[0094] Scharr(x)=10[P(x+1,y)-P(x-1,y)]+3[P(x+1,y-1)-P(x-1,y-1)]+3[P(x+1,y+1)-P(x-1,y+1)]

[0095] Scharr(y)=10[P(x,y-1)-P(x,y+1)]+3[P(x-1,y-1)-P(x-1,y+1)]+3[P(x+1,y-1)-P(x+1,y+1)]

[0096] Among them, P(x,y) represents the pixel value of the original image at the coordinate (x,y).

[0097] The weighting factor is calculated by the calculated gradient value to reflect the edge strength of the image:

[0098]

[0099] Through the original image, the SAD cost of each pixel under different disparities is calculated, and the formula is as follows:

[0100] SAD(p,d)=∑(x,y)∈window∣IL(x,y)-IR(xd,y)∣

[0101] Among them, IL and IR are the pixel values ​​of the left image and the right image at position p(x,y), d is the disparity, (xd,y) represents the position corresponding to position p in the right image, and window is the SAD window, which is set to 3*3 here.

[0102] Combining SAD cost and gradient information, construct the cost function:

[0103] Ccombined(p,d)=SAD(p,d)×W(p)

[0104] Then, the disparity map is obtained through dynamic programming and data post-processing, and the depth information is calculated through the disparity map.

[0105] It can be understood that the SGBM (Semi-Global Block Matching) algorithm is a semi-global matching algorithm for calculating disparity in binocular vision. By selecting the disparity of each pixel point, a disparity map is formed, and a global energy function related to the disparity map is set to minimize this energy function to achieve the purpose of solving the optimal disparity of each pixel.

[0106] Step S303: Acquire other sensor information.

[0107] It should be noted that other sensors include integrated laser radar (LiDAR), millimeter wave radar, ultrasonic sensors and other sensors to obtain all-round and multi-level environmental information. Each sensor has its own unique advantages: for example, laser radar can provide high-precision distance information, and millimeter wave radar has strong penetration ability.

[0108] Step S304: integrating the other sensor information with the depth information to calculate the position information of the obstacle.

[0109] It should be noted that the types of obstacles, such as vehicles, pedestrians, and trees, are identified through the binocular camera combined with the YOLOv8 detection algorithm. The depth information is calculated through the binocular camera combined with the SGBM algorithm. The depth information of the obstacle can be provided by the lidar and ultrasonic sensors to estimate the state of the obstacle, including position, speed, and direction.

[0110] Step S305: Acquire vehicle external environment information and internal vehicle information.

[0111] It should be noted that the vehicle's external environment information includes the vehicle's ability to communicate with surrounding vehicles and traffic infrastructure through vehicle networking technology to obtain real-time traffic information, road condition data, etc., so as to have a more comprehensive understanding of the vehicle's external environment. The vehicle's external environment information can also include high-precision maps and GPS positioning information. The vehicle can obtain its own position, driving direction, and surrounding road information in real time. Internal vehicle information includes basic information such as the vehicle's speed, acceleration, steering angle, and braking status, which is crucial for path planning and obstacle avoidance decisions.

[0112] Step S306: Control the driving direction and speed of the vehicle based on the obstacle type, obstacle location information, vehicle external environment information, and internal vehicle information.

[0113] It should be noted that the optimal driving route is generated based on obstacle information, combined with the binocular camera and the current state of the vehicle. Path planning needs to consider the dynamic characteristics of the vehicle and obstacle information. During driving, the speed and direction of the vehicle are controlled according to the real-time information until the low-speed cruising stage ends or the vehicle enters the automatic parking stage.

[0114] In this embodiment, when judging that the car enters the low-speed cruising stage, the obstacle type is identified by the improved YOLOv8 algorithm, and the obstacle frame diagram is drawn on the binocular image. Based on the obstacle frame diagram, the obstacle depth information is calculated by the optimized SGBM algorithm, and the calculation of the parts outside the frame diagram can be ignored to reduce the algorithm calculation amount. Through the obstacle type and location information, the external environment information and internal vehicle information are obtained, and the information is fused in combination with other vehicle sensors, and different driving plans are selected to reduce the error during the low-speed cruising stage.

[0115] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the valet parking method based on binocular cameras of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0116] This application also provides a valet parking device based on a binocular camera, please refer to Figure 5 , the valet parking device based on binocular camera includes:

[0117] The state determination module 10 is used to pre-process the images collected by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage;

[0118] The information processing module 20 is used to call the YOLOv8 detection algorithm to identify the obstacle type and the marked parking space on the pre-processed image when it is determined that the car is in the automatic parking stage;

[0119] The parking control module 30 controls the vehicle to park according to the obstacle type and the parking space markings when the parking space markings are identified.

[0120] The valet parking device based on binocular cameras provided in the present application adopts the valet parking method based on binocular cameras in the above-mentioned embodiments, which can solve the technical problems of being unable to identify the type of obstacles during valet parking and being unable to identify the type of obstacles when multiple ultrasonic radars emit sounds and co-frequency interference. Compared with the prior art, the beneficial effects of the valet parking device based on binocular cameras provided in the present application are the same as the beneficial effects of the valet parking method based on binocular cameras provided in the above-mentioned embodiments, and the other technical features of the valet parking device based on binocular cameras are the same as the features disclosed in the above-mentioned embodiment methods, which will not be repeated here.

[0121] The present application provides a valet parking device based on a binocular camera, and the valet parking device based on a binocular camera includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the valet parking method based on the binocular camera in the above-mentioned embodiment 1.

[0122] Reference below Figure 6, which shows a schematic diagram of the structure of a valet parking device based on a binocular camera suitable for implementing the embodiment of the present application. The valet parking device based on a binocular camera in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The valet parking device based on a binocular camera shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0123] like Figure 6 As shown, the valet parking device based on the binocular camera may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the valet parking device based on the binocular camera are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the valet parking device based on a binocular camera to communicate with other devices wirelessly or by wire to exchange data. Although the valet parking device based on a binocular camera with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0124] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0125] The valet parking device based on binocular cameras provided in the present application adopts the valet parking method based on binocular cameras in the above embodiment, which can solve the technical problem of mechanical fatigue imbalance of the dual motor spindles. Compared with the prior art, the beneficial effects of the valet parking device based on binocular cameras provided in the present application are the same as the beneficial effects of the valet parking method based on binocular cameras provided in the above embodiment, and the other technical features of the valet parking device based on binocular cameras are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0126] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0127] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0128] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the binocular camera-based valet parking method in the above-mentioned embodiment.

[0129] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0130] The computer-readable storage medium may be included in the valet parking device based on a binocular camera, or may exist independently without being assembled into the valet parking device based on a binocular camera.

[0131] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0132] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0133] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0134] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned valet parking method based on binocular cameras, and can solve the technical problems of being unable to identify the type of obstacles during valet parking and being unable to identify the type of obstacles when multiple ultrasonic radars emit sounds and co-frequency interference. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the valet parking method based on binocular cameras provided in the above-mentioned embodiments, and will not be elaborated here.

[0135] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A valet parking method based on a binocular camera, characterized in that: The valet parking method based on a binocular camera comprises: Pre-process the images collected by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage; When judging that the car is in the automatic parking stage, the YOLOv8 detection algorithm is called to identify the obstacle type and parking space line of the pre-processed image; When the parking space markings are identified, the vehicle is controlled to park according to the type of obstacle and the parking space markings.

2. The valet parking method based on binocular camera as claimed in claim 1, characterized in that: After the step of controlling the vehicle to park according to the obstacle type and the parking space markings when the parking space markings are identified, the method includes: During the automatic parking process of the vehicle, the distance to the obstacle and the distance to the parking space marking are detected by the binocular camera; The parking path is adjusted according to the obstacle distance information and the parking space line distance information until the parking process is completed.

3. The valet parking method based on binocular camera as claimed in claim 1, characterized in that: When determining that the car is in the automatic parking stage, after the step of calling the YOLOv8 detection algorithm to perform obstacle type recognition and parking space recognition on the binocular camera image, the method includes: When no marked parking space is identified, the binocular camera is controlled to identify the front and rear vehicles on the same side, determine the space parking space and generate a virtual parking space line according to the space parking space; The vehicle is controlled to perform automatic parking according to the virtual parking space line.

4. The valet parking method based on binocular camera as claimed in claim 1, characterized in that: After the step of preprocessing the image collected by the binocular camera to determine whether the vehicle is in the low-speed cruising stage or the automatic parking stage, the method includes: When it is determined that the car is in the low-speed cruising stage, the YOLOv8 detection algorithm is called to identify the obstacle types and mark the obstacle boxes on the pre-processed images; Based on the obstacle type recognition result, the depth information of the frame area where the obstacle is identified is obtained by using a binocular camera to measure the distance; The vehicle is controlled to travel at a low speed according to the depth information.

5. The valet parking method based on binocular camera as claimed in claim 4, characterized in that: The step of controlling the vehicle to travel at a low speed according to the depth information further includes: Get other sensor information; Combining the other sensor information with the depth information to calculate the location information of the obstacle; Obtain vehicle external environment information and internal vehicle information; The vehicle's driving direction and speed are controlled by combining obstacle type, obstacle location information, vehicle external environment information, and internal vehicle information.

6. The valet parking method based on binocular camera as claimed in claim 5, characterized in that: After the step of controlling the driving direction and driving speed of the vehicle in combination with the obstacle type, obstacle location information, vehicle external environment information and internal vehicle information, the method further comprises: Detect whether the vehicle enters the automatic parking stage; When the vehicle enters the automatic parking stage, the step of calling the YOLOv8 detection algorithm to perform obstacle type recognition and parking space recognition on the binocular camera image is executed.

7. The valet parking method based on binocular camera as claimed in claim 4, characterized in that: The step of obtaining depth information of the frame area where the obstacle is identified by using a binocular camera based on the obstacle type identification result includes: Preprocess the binocular vision image based on the obstacle type recognition result and the obstacle block diagram area; The SGBM algorithm is used to match the processed binocular vision images to generate a disparity map, and the depth information is calculated through the disparity map.

8. A valet parking device based on a binocular camera, characterized in that: The device comprises: The state determination module is used to pre-process the images collected by the binocular camera to determine whether the car is in the low-speed cruising stage or the automatic parking stage; The information processing module is used to call the YOLOv8 detection algorithm to identify obstacle types and marked parking spaces in the pre-processed images when judging that the car is in the automatic parking stage; The parking control module controls the vehicle to park according to the obstacle type and the parking space markings when the parking space markings are identified.

9. A valet parking device based on a binocular camera, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the binocular camera-based valet parking method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the valet parking method based on a binocular camera as described in any one of claims 1 to 7 are implemented.

Images