Method, device and equipment for visualizing drivable area of automatic parking and storage medium

Abstract

The application discloses an automatic parking drivable area visualization method, device, equipment and storage medium, relates to the parking technical field, and the automatic parking drivable area visualization method comprises: acquiring vehicle-mounted camera data, radar point cloud data, vehicle size and safety distance, utilizing a target obstacle identification model based on an encoder-decoder and a self-attention module to analyze the camera data, determining the initial position, size and type of the obstacle. Then, the positions are adjusted through the radar data to obtain accurate real-time obstacle positions, and the drivable area is constructed in combination with the vehicle size and the safety distance. Finally, the system sends the camera data, the radar data and the drivable area information to a vehicle controller, which uses a scene restoration model to reconstruct the automatic parking scene, and transmits the result to a vehicle display to present an automatic parking view. The application can display the automatic parking scene view and show the drivable area, thereby reducing the collision risk.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of parking, in particular to a drivable area visualization method, device and equipment for automatic parking and a storage medium. BACKGROUND

[0002] With the acceleration of urbanization and the increase of car ownership, the problem of parking space shortage and narrow parking space is increasingly prominent. As a solution that can reduce the burden of drivers and improve parking efficiency, the demand for automatic parking technology is increasing. In a complex parking environment, drivers need to accurately identify the drivable area around the vehicle to ensure safe and accurate parking operations.

[0003] Existing automatic parking technology mainly relies on environmental perception and data fusion technology. Through vehicle-mounted cameras, radars and other sensors, environmental information around the vehicle is collected to construct the environment information of the parking area, and at least one initial custom parking space is determined in the parking area. For example, a custom parking space automatic parking method based on parking area determination (publication number CN118387088A) and a multi-sensor fusion intelligent parking system and method (publication number CN112180373B) are both implemented through these technologies to realize the function of automatic parking.

[0004] Although the existing method can construct the parking area information inside the intelligent driving controller, these information often cannot be intuitively presented to the driver, resulting in the driver's inability to intuitively feel the perception range and drivable area of the vehicle. This may lead to the parking control system being unable to accurately identify the surrounding environment in a complex environment, thereby affecting the safety of driving. In addition, due to the lack of intuitive visual display, the driver cannot intervene in time when there is a risk of collision, increasing the risk of collision during automatic parking. Therefore, how to display the drivable area of automatic parking to avoid collision with obstacles becomes a problem to be solved.

[0005] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0006] The present application aims to provide a drivable area visualization method, device and equipment for automatic parking and a storage medium, which aims to solve the technical problem of how to display the drivable area of automatic parking to avoid collision with obstacles.

[0007] To achieve the above-mentioned purpose, the present application provides a drivable area visualization method for automatic parking, which comprises:

[0008] Obtaining vehicle-mounted camera data, radar point cloud data, vehicle size and vehicle safety distance;

[0009] The vehicle-mounted camera data is identified through a target obstacle identification model to obtain a first position, size and type of the obstacle in the image, the target obstacle identification model being constructed according to an encoder, a decoder and a self-attention module;

[0010] The first position is adjusted according to the radar point cloud data to obtain a real-time obstacle position;

[0011] A drivable area is constructed according to the vehicle size, the vehicle safety distance, the real-time obstacle position and the size and type of the obstacle in the image;

[0012] The vehicle-mounted camera data, the radar point cloud data and data corresponding to the drivable area are sent to a car controller, so that the car controller restores an automatic parking scene according to a scene restoration model and sends restored scene data to a car display, so that the car display displays an automatic parking scene restoration view according to the scene data.

[0013] In an embodiment, the step of adjusting the first position according to the radar point cloud data to obtain a real-time obstacle position comprises:

[0014] A second position of a static obstacle in the point cloud data is identified by fitting the radar point cloud data through a random sample consensus strategy;

[0015] A third position of a dynamic obstacle in the point cloud data is identified by clustering the radar point cloud data through a density clustering strategy;

[0016] The obstacle in the image and the static obstacle and the dynamic obstacle are spatially aligned according to the first position, the second position and the third position to obtain a real-time obstacle position.

[0017] In an embodiment, the step of fitting the radar point cloud data through a random sample consensus strategy to identify a second position of a static obstacle in the point cloud data comprises:

[0018] All points are extracted from the radar point cloud data to obtain an initial point set;

[0019] A random point pair is selected from the initial point set;

[0020] An initial obstacle geometric model is fitted according to the random point pair;

[0021] The distances of all points to the initial obstacle geometric model are calculated;

[0022] The initial point set is divided according to the distances to obtain an inlier point set;

[0023] maximizing the inner point set by multiple selections of the random points, to obtain a target obstacle geometric model;

[0024] determining a second position of a static obstacle according to the target obstacle geometric model.

[0025] In an embodiment, the step of clustering the radar point cloud data by a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data comprises:

[0026] defining a neighborhood radius and a minimum point number according to distribution characteristics of point clouds in the radar point cloud data;

[0027] clustering each point in the radar point cloud data according to the neighborhood radius and the minimum point number, to obtain a point cloud cluster;

[0028] determining a dynamic obstacle according to changes of consecutive frames in the radar point cloud data;

[0029] dynamically tracking the dynamic obstacle in the consecutive frames by Kalman filtering, to obtain a third position of the dynamic obstacle.

[0030] In an embodiment, the step of spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position and the third position to obtain a real-time obstacle position comprises:

[0031] obtaining camera parameters and radar parameters;

[0032] constructing a ground coordinate system with a rear axle center of a vehicle as an origin, a forward direction of the vehicle as a positive direction of a horizontal axis, and a left side of the vehicle as a positive direction of a vertical axis;

[0033] converting a two-dimensional coordinate of the first position in the image into a three-dimensional coordinate in the ground coordinate system by inverse perspective transformation according to the camera parameters, to obtain a first coordinate;

[0034] rotating and translating the second position and the third position according to the radar parameters and the ground coordinate system, to obtain a second coordinate and a third coordinate;

[0035] converting the first coordinate, the second coordinate and the third coordinate in the ground coordinate system to a bird's-eye view perspective, to obtain a real-time obstacle position.

[0036] In an embodiment, the target obstacle identification model comprises an encoder, a decoder and a self-attention module, the encoder comprises a convolutional layer and a pooling layer, the step of identifying the vehicle-mounted camera data through the target obstacle identification model to obtain the first position, size and type of the obstacle in the image comprises:

[0037] Denoising, illumination compensation and standardization are performed on the image in the vehicle-mounted camera data to obtain target image data;

[0038] The target image data is processed through the encoder and the decoder to obtain the first position of the obstacle in the image;

[0039] The target image data is convolved through the convolutional layer to obtain a feature map;

[0040] The feature map is reduced in dimension through the pooling layer to obtain a low-dimensional feature map;

[0041] The low-dimensional feature map is weighted and aggregated through the self-attention module to obtain a weighted feature map;

[0042] Boundary box regression is performed on the weighted feature map to obtain the size of the obstacle;

[0043] The weighted feature map is network encoded through the self-attention module to obtain a classification network;

[0044] The classification network is processed through a smoothed maximum function to obtain the type of the obstacle.

[0045] In an embodiment, the step of processing the target image data through the encoder and the decoder to obtain the first position of the obstacle in the image comprises:

[0046] Each pixel in the target image data is mapped to a depth value through the encoder;

[0047] The target image data is upsampled and jump connected according to the depth value through the decoder to obtain a depth map;

[0048] The first position of the obstacle in the image is obtained according to the distance information of each pixel in the depth map from the camera.

[0049] In addition, in order to achieve the above-mentioned purpose, the application further provides a drivable area visualization device for automatic parking, the device comprises:

[0050] A data acquisition module is configured to acquire vehicle-mounted camera data, radar point cloud data, vehicle size and vehicle safety distance;

[0051] An obstacle recognition module is configured to recognize the vehicle camera data by a target obstacle recognition model to obtain a first position, size and type of an obstacle in an image, wherein the target obstacle recognition model is constructed according to an encoder, a decoder and a self-attention module.

[0052] A position determination module is configured to adjust the first position according to the radar point cloud data to obtain a real-time obstacle position.

[0053] A drivable area construction module is configured to construct a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position and the size and type of the obstacle in the image.

[0054] A view restoration module is configured to send the vehicle camera data, the radar point cloud data and data corresponding to the drivable area to a car controller, so that the car controller restores an automatic parking scene according to a scene restoration model and sends restored scene data to a car display, so that the car display displays an automatic parking scene restoration view according to the scene data.

[0055] In addition, to achieve the above-mentioned purpose, the present application further provides an automatic parking drivable area visualization device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the automatic parking drivable area visualization method as described above.

[0056] In addition, to achieve the above-mentioned purpose, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the automatic parking drivable area visualization method as described above.

[0057] In addition, to achieve the above-mentioned purpose, the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the automatic parking drivable area visualization method as described above.

[0058] The one or more technical solutions provided by the present application have at least the following technical effects:

[0059] Obtain vehicle-mounted camera data, radar point cloud data, vehicle size, and vehicle safety distance; identify the vehicle-mounted camera data using a target obstacle recognition model to obtain the first position, size, and type of obstacles in the image. The target obstacle recognition model is constructed based on an encoder, a decoder, and a self-attention module; adjust the first position based on the radar point cloud data to obtain a real-time obstacle position; construct a drivable area based on the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacles in the image; send the vehicle-mounted camera data, the radar point cloud data, and data corresponding to the drivable area to the vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and send the restored scene data to the vehicle display, so that the vehicle display displays the automatic parking scene restoration view based on the scene data. The parking control system first collects vehicle-mounted camera data and radar point cloud data, and simultaneously records the vehicle size and safety distance to provide the system with comprehensive environmental information to ensure accurate perception of the vehicle's surroundings. The system then processes the camera data using a target obstacle recognition model constructed using an encoder, decoder, and self-attention module to identify the location, size, and type of obstacles, accurately identifying and classifying them. This provides critical information for collision avoidance and parking path planning. The system then compares the radar point cloud data with the obstacle positions detected by the camera to correct the real-time obstacle positions, improving the accuracy of the obstacle location data and ensuring the safety and reliability of the parking control system. Based on this, the system calculates the vehicle's drivable area, combining vehicle dimensions, safety distance, real-time obstacle positions, and obstacle size and type. This provides a safe driving area for automated parking, avoids collisions, and ensures the feasibility of the parking path. Finally, the system transmits the on-board camera data, radar point cloud data, and drivable area data to the onboard controller. The controller reconstructs the automated parking scene based on the scene reconstruction model and transmits the reconstructed scene data to the onboard display, displaying a visual representation of the automated parking scene. This improves operational transparency and safety, and enhances driver trust in the parking control system. The entire process significantly improves the accuracy, safety, and user-friendliness of the parking control system through precise environmental perception, obstacle identification, position correction, drivable area construction, and scene restoration. The parking control system collects on-board camera data and radar point cloud data, combined with vehicle size and safety distance information, to provide the system with comprehensive environmental awareness, ensuring accurate identification of obstacles around the vehicle. A target obstacle recognition model constructed using encoders, decoders, and self-attention modules further processes the camera data to accurately identify and classify obstacles, providing critical information for collision avoidance and parking path planning. The system then corrects the real-time position of obstacles based on the radar point cloud data, further improving the accuracy of the position data and enhancing the safety and reliability of the system.On this basis, the system comprehensively considers the vehicle size, safety distance, real-time obstacle position, and obstacle size and type, calculates the drivable area of the vehicle, ensures the feasibility of the parking path and avoids collision with obstacles. Finally, the system sends these data to the vehicle controller, the controller reconstructs the automatic parking scene according to the scene restoration model, and sends the restored scene data to the vehicle display, which intuitively displays the view of the automatic parking scene. This not only improves the transparency and safety of the operation, enhances the driver's trust in the parking control system, but also displays the drivable area of the automatic parking to avoid collision with obstacles, significantly improves the human-computer interaction performance and safety of the automatic parking, and reduces the collision risk when using the automatic parking. BRIEF DESCRIPTION OF DRAWINGS

[0060] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0062] Figure 1 The flowchart provided by the first embodiment of the automatic parking drivable area visualization method of the present application;

[0063] Figure 2 The flowchart provided by the second embodiment of the automatic parking drivable area visualization method of the present application;

[0064] Figure 3 The module structure diagram of the automatic parking drivable area visualization device of the embodiment of the present application;

[0065] Figure 4 The device structure diagram of the hardware running environment involved in the automatic parking drivable area visualization method in the embodiment of the present application.

[0066] The purpose of the present application, functional characteristics and advantages will be further explained with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and not to limit the present application.

[0068] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings in the specification and specific embodiments.

[0069] With the growth of urbanization and the number of cars, the problem of parking space shortage and narrow space is becoming more and more serious, and automatic parking technology has become an important solution to improve parking efficiency and reduce the burden on drivers. The existing approach mainly collects environmental information through sensors such as vehicle-mounted cameras and radars, builds a parking area and determines a custom parking space. However, although these technologies can build parking area information in the intelligent driving controller, they cannot be intuitively presented to the driver, resulting in a lack of transparency in the perception range and drivable area of the parking control system in complex environments, increasing the risk of collision and affecting driving safety and the driver's trust in the system.

[0070] The main solution of the embodiment of the present application is that the parking control system collects vehicle-mounted camera and radar point cloud data, combines vehicle size and safety distance information, and comprehensively perceives the surrounding environment. The model constructed by using the encoder-decoder and self-attention module processes the camera data, accurately identifies the position, size and type of obstacles, and corrects these positions through radar data to improve accuracy. The system calculates the drivable area accordingly to ensure the safety and feasibility of the parking path. Finally, the system sends the camera, radar and drivable area data to the car controller, reconstructs and displays the automatic parking scene.

[0071] It should be noted that the execution subject of the embodiment of the present application can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a parking controller, a parking control system, etc. that can realize the above functions. The parking control system is taken as an example to illustrate the embodiment and the following embodiments.

[0072] Based on this, the embodiment of the present application provides a drivable area visualization method for automatic parking, which is described with reference to Figure 1 , Figure 1 The flowchart of the first embodiment of the drivable area visualization method for automatic parking of the present application is shown.

[0073] In this embodiment, the drivable area visualization method for automatic parking includes steps S10-S50:

[0074] Step S10, acquiring vehicle-mounted camera data, radar point cloud data, vehicle size and vehicle safety distance;

[0075] It should be noted that the car camera data refers to the visual information captured by the cameras installed on the vehicle. In this embodiment, the cameras used include four fisheye cameras for the front, rear, left and right of the vehicle, and two wide-angle cameras for the front and rear windshields, which can provide image data of the environment around the vehicle for identifying obstacles, road signs, traffic signals, etc. Camera data is an important part of environmental perception, and through deep learning algorithms, these image data can be processed and analyzed to identify the location and type of obstacles, providing visual perception information for the parking control system. Radar point cloud data is collected by radar sensors on the vehicle, which forms a three-dimensional point cloud map of the environment around the vehicle. In this embodiment, the radar includes twelve ultrasonic radars and five millimeter wave radars around the vehicle. Ultrasonic radars detect the distance and position of obstacles by emitting ultrasonic waves and receiving their echoes, while millimeter wave radars use radio waves for more accurate distance, speed and angle measurements. The data from these radar sensors can provide accurate position information of obstacles around the vehicle, which is crucial for building a three-dimensional environmental model around the vehicle. Vehicle size refers to the physical dimensions of the vehicle, including length, width and height. In the automatic parking process, understanding the size of the vehicle is crucial for determining whether the vehicle can fit into a specific parking space and whether it will collide with other objects during parking. Vehicle size data helps the parking controller calculate the drivable area and operating space of the vehicle, ensuring the safety and feasibility of the parking operation. Vehicle safety distance refers to the minimum distance that should be maintained between the vehicle and surrounding obstacles to ensure safety. This distance takes into account the possible movement range of the vehicle during driving and parking, as well as unexpected situations that may occur due to operational errors or environmental changes. In this embodiment, the data of the vehicle safety distance helps the parking controller determine the boundary of the drivable area of the vehicle, and marks the non-drivable area as red on the display screen to remind the driver to pay attention. The setting of the safety distance can reduce the risk of collision and improve the safety of automatic parking.

[0076] It can be understood that first, the acquisition of vehicle camera data is achieved through multiple cameras installed around the vehicle, including surround-view fisheye cameras and wide-angle cameras, which can capture high-definition images of the environment around the vehicle, and then extract the position and type information of obstacles through image processing technology, which can provide visual perception data for the parking controller and help the system understand the environmental layout around the vehicle; second, the acquisition of radar point cloud data relies on ultrasonic radar and millimeter wave radar equipped on the vehicle, which measures the precise distance and speed between the vehicle and the surrounding obstacles by transmitting and receiving signals, which can provide more accurate obstacle position information and enhance the system's perception of the environment; third, the acquisition of vehicle size requires measuring the length, width and height of the vehicle, which is crucial for the parking controller to calculate the operable space of the vehicle during parking to ensure that the parking operation does not exceed the physical limitations of the vehicle; finally, the determination of the vehicle safety distance is based on safety standards and vehicle operating characteristics, setting a minimum distance to ensure that the vehicle maintains a sufficient safety interval with obstacles during parking to avoid collisions, which improves the safety and reliability of the parking control system and makes the parking process more stable and safe. Through these steps, the parking control system can accurately identify and display the drivable area of the vehicle, achieving safe and efficient automatic parking operation.

[0077] In step S20, the vehicle camera data is identified by a target obstacle identification model to obtain the first position, size and type of the obstacle in the image, and the target obstacle identification model is constructed according to an encoder, a decoder and a self-attention module.

[0078] It is important to note that the target obstacle recognition model is a deep learning model that combines Bird's Eye View (BEV) and Transformer architecture, designed to identify obstacles from vehicle-mounted camera data. This model is designed to improve the accuracy and efficiency of obstacle detection, particularly in autonomous driving and automatic parking scenarios. Here is a detailed explanation of the target obstacle recognition model, including its origin, component layers or modules: (1) Encoder: Convolutional Layers: The initial stage of the encoder typically includes multiple convolutional layers, which are responsible for extracting features from the input image data. Convolutional layers slide filters (or convolutional kernels) over the image to capture local features such as edges, textures, etc. These features are then processed by activation functions (such as ReLU) to introduce non-linearity, allowing the model to learn more complex feature representations. Pooling Layers: Following the convolutional layers are pooling layers, which are used to reduce the spatial dimensions of the feature maps, i.e., to reduce the number of parameters and computational load while increasing the invariance of the features (such as translation invariance). The most common pooling operation is max pooling, which selects the maximum value within a region. Feature Fusion: After multiple convolutional and pooling layers, there may be a feature fusion operation, such as through skip connections, to merge feature maps at different levels to preserve more spatial information. BEV Conversion: The output of the encoder is a series of feature maps, which are then converted to a Bird's Eye View (BEV) representation to provide input for the subsequent Transformer encoder. (2) Transformer Encoder: Self-Attention Module: The self-attention module is a mechanism that allows the model to focus on information at different positions in the sequence when processing sequential data. In the target obstacle recognition model, the self-attention module can help the model better understand the relationships between different regions in the image, allowing the model to consider global information when processing features and improving the accuracy of obstacle recognition. Multi-Head Attention: The self-attention module encoder usually includes multi-head attention, which means that different representations of features are processed in parallel to capture dependencies at different scales. Feedforward Network: After the self-attention layer, the self-attention module also includes a feedforward network that performs further nonlinear transformations on the output of the self-attention layer. (3) Decoder: Encoder Output: The decoder receives the BEV feature representation from the Transformer encoder and begins to reconstruct the spatial location and size information of the obstacles. Upsampling: The decoder may include up-sampling or transposed convolutional layers to increase the spatial resolution of the feature maps, allowing more accurate localization of obstacles.Feature refinement: The layers in the decoder can further refine the features to predict the precise location, size, and type of the obstacles.(4) Output layer: Finally, the output of the decoder is passed through one or more fully connected layers to generate the class, location (in the BEV coordinate system), size, and possibly other attributes (such as velocity or direction) of the obstacles.

[0079] Obstacle refers to all objects that can affect the driving and parking operations of the vehicle, including but not limited to other vehicles, pedestrians, bicycles, pillars, walls, traffic signs, etc. The first position refers to the location of the obstacle in the image, i.e., the coordinates of the obstacle in the two-dimensional image plane, usually represented by pixel values, which is the first step in determining the spatial position of the obstacle in the obstacle recognition process. Size refers to the size of the obstacle in the image, i.e., the size of the pixel area occupied by the obstacle, which helps the system to judge the actual size of the obstacle and thus evaluate its potential impact on the parking operation. Type refers to the kind or category of the obstacle, such as vehicle, pedestrian, bicycle, etc., and identifying the type of the obstacle is crucial for the parking control system to make correct driving decisions.

[0080] It can be understood that first, the system processes the image captured by the vehicle-mounted camera through the convolutional layers of the encoder part of the target obstacle recognition model, which uses a series of learnable filters to slide over the image and extract local features such as edges and textures, which can capture the basic patterns in the image and lay the foundation for subsequent recognition; then, the feature maps output by the convolutional layers are down-sampled by the pooling layer to reduce the dimension of the feature maps and extract the most important information, which helps to reduce the computational load and improve the robustness of the model to changes in image positions; then, the system converts the extracted feature maps into a bird's eye view (BEV) representation through the model, which provides a two-dimensional planar view from the top perspective of the vehicle and obstacles, enabling the model to more comprehensively understand the environment around the vehicle; subsequently, the Transformer encoder processes the BEV features using the self-attention mechanism, which calculates the attention weights of each position in the feature map to all other positions, enabling the model to capture the global relationships between obstacles, which can improve the model's ability to understand complex scenes; finally, the decoder part gradually upsamples and refines the processed feature maps, and finally outputs the precise location, size, and type of the obstacles, which can provide precise obstacle information for the parking control system, enabling the system to safely and accurately avoid obstacles and achieve efficient parking. Through these series of steps, the system can accurately identify the key information of the obstacles from the vehicle-mounted camera data, providing important environmental perception capabilities for autonomous driving and automatic parking.

[0081] As an example, the target obstacle identification model comprises an encoder, a decoder and a self-attention module, the encoder comprises a convolutional layer and a pooling layer, the step of identifying the vehicle camera data through the target obstacle identification model to obtain the first position, size and type of the obstacle in the image comprises: denoising, light compensation and standardization of the image in the vehicle camera data to obtain target image data; processing the target image data through the encoder and the decoder to obtain the first position of the obstacle in the image; performing convolutional operation on the target image data through the convolutional layer to obtain a feature map; performing dimension reduction on the feature map through the pooling layer to obtain a low-dimensional feature map; performing weighted aggregation on the low-dimensional feature map through the self-attention module to obtain a weighted feature map; performing bounding box regression on the weighted feature map to obtain the size of the obstacle; performing network coding on the weighted feature map through the self-attention module to obtain a classification network; processing the classification network through a smooth max function to obtain the type of the obstacle.

[0082] Denoising refers to the process of reducing or eliminating noise in images, which can be caused by camera sensors, environmental factors, or interference during transmission. Denoising smooths the image through algorithms such as median filtering, Gaussian filtering, etc., reducing randomly varying pixel values, improving image quality, and providing clearer image data for subsequent feature extraction. Light compensation refers to adjusting the brightness and contrast of images to reduce the impact of image quality changes under different lighting conditions. Since images captured by cameras under different lighting conditions may have uneven brightness or color distortion, light compensation improves the lighting conditions of images through algorithms such as histogram equalization, making feature extraction more accurate. Standardization refers to converting image data into a uniform format or range, such as scaling pixel values to between 0 and 1. This can speed up the convergence of model training and improve the adaptability of the model to different cameras and environmental conditions. Target image data refers to image data that has been processed by denoising, light compensation, and standardization. These data are the pre-processing results of model input, providing high-quality image input for subsequent feature extraction and obstacle recognition. The first position refers to the position of the obstacle in the image, i.e., the two-dimensional coordinates of the obstacle. Through the processing of the encoder and decoder, the model can determine the approximate position of the obstacle in the image. Convolution operation is a basic operation in convolutional neural networks, which generates a new feature map by sliding a convolution kernel (filter) over each local region of the image and calculating the dot product of the convolution kernel and the covered region. This process can extract local features of the image, such as edges, textures, etc. Feature map is the output of convolution operation, representing the extracted features in the image. Each convolution layer may produce multiple feature maps, which capture features of the image from different angles. Dimensionality reduction refers to reducing the dimensionality of data. Pooling layers reduce the spatial size of feature maps through operations such as max pooling and average pooling, while retaining the most important information. Low-dimensional feature map is the feature map after dimensionality reduction by the pooling layer, which has a smaller spatial size but still contains key information of the image. Weighted aggregation refers to the self-attention module giving different weights to different parts of the feature map according to their importance, and then performing weighted summation on these features to highlight important features and suppress unimportant features. Weighted feature map is the output of the self-attention module, which contains features that have been weighted and aggregated, making them more prominent and helpful for subsequent obstacle recognition. Boundary box regression refers to predicting the boundary box of the obstacle, i.e., the precise position and size of the obstacle in the image, through regression algorithms such as linear regression. Network encoding refers to the self-attention module further processing the weighted feature map to generate encoded features for classification. The classification network is a network composed of a series of neural network layers, used to classify obstacles, i.e., determine the type of obstacle. The smooth maximum function (such as the softmax function) is an activation function used to convert the output of the classification network into a probability distribution, representing the confidence of different classes.

[0083] First, the parking control system performs denoising on the raw image data captured by the on-board camera, reducing random noise in the image through algorithms such as Gaussian filtering, which improves the quality of the image and provides clearer image data for subsequent feature extraction. Second, the system performs illumination compensation to adjust the brightness and contrast of the image, reducing the impact of image quality changes under different lighting conditions, which helps the model to stably recognize obstacles in different environments. Third, the system standardizes the image by scaling the pixel values to a uniform range (such as 0 to 1), which can speed up the convergence of model training and improve the adaptability of the model to different cameras and environmental conditions. Then, the parking control system uses the convolutional layer in the encoder to perform convolution operations on the preprocessed image data, extracting local features such as edges and textures in the image through sliding filters, which can capture the basic patterns in the image and lay the foundation for subsequent obstacle recognition. Next, the system reduces the dimensionality of the feature map through the pooling layer by selecting the maximum or average value in the region to reduce the spatial size of the feature map, which helps to extract more abstract features while reducing computational complexity. Next, the system uses the self-attention module to aggregate the low-dimensional feature map, giving different weights to different parts of the feature map according to their importance, which can highlight important features and suppress unimportant features, improving the model's ability to recognize obstacles. Subsequently, the system performs bounding box regression on the weighted feature map, predicting the bounding box of the obstacle in the image, i.e., the precise location and size of the obstacle in the image, which can provide precise spatial information for parking operations. Finally, the parking control system encodes the weighted feature map through the self-attention module to build a classification network, and then processes the output of the classification network through the softmax function to convert the original neural network response into a probability distribution, thereby identifying the type of obstacle, which can provide specific information about the identity of the obstacle to the parking system, enabling the system to make more accurate parking decisions. Through this series of detailed processing steps, the parking control system can accurately identify the first location, size, and type of obstacles in the image from the on-board camera data, providing key environmental perception capabilities for automatic parking.

[0084] As an example, the step of processing the target image data through the encoder and the decoder to obtain the first location of the obstacle in the image includes: mapping each pixel in the target image data to a depth value through the encoder; upsampling and skip-connection the target image data according to the depth value through the decoder to obtain a depth map; and obtaining the first location of the obstacle in the image according to the distance information of each pixel in the depth map from the camera.

[0085] The depth value refers to the depth or distance of each pixel point in three-dimensional space relative to the camera. In the parking control system, the encoder analyzes the target image data and uses the features learned by the neural network to predict the depth value of each pixel point, which represents the distance from the camera to the obstacle surface. Upsampling is a process of increasing data resolution, usually used in the decoder stage. In deep learning, upsampling increases the size of the feature map through interpolation methods such as nearest neighbor interpolation, bilinear interpolation, etc. to restore the resolution close to the original input image, providing more detailed information for subsequent depth map generation and bounding box positioning. Skip connection (also known as Skip Connection) is a structure that connects different layers in the network, which directly connects the feature map of the shallow layer in the encoder to the feature map of the deep layer in the decoder. This prevents information loss when passing through deep networks, helps the network learn more rich features, and improves the training stability and performance of the model. The depth map is a two-dimensional image where each pixel value represents the depth value of that point to the camera. The depth map provides an intuitive representation of the distance of each point in the scene, which is very useful for determining the three-dimensional position and shape of the obstacle. Distance information refers to the depth value of each pixel point in the depth map, i.e. the actual physical distance of each pixel point to the camera. This information is obtained through encoder prediction and decoder upsampling and skip connection reconstruction, and it directly reflects the distribution of obstacles in space, which is the key data for precise obstacle detection and obstacle avoidance in the parking control system.

[0086] First, the encoder part of the parking control system processes the target image data through a deep convolutional neural network. The specific operation is to extract features through multiple convolutional layers and then use pooling layers to reduce the spatial dimension of the features. This allows the network to capture the key information in the image and map each pixel to a depth value, that is, predict the depth of each pixel from the camera. The effect of this is to provide the three-dimensional spatial position information of each pixel for subsequent steps. Secondly, the decoder part receives the depth value output by the encoder, increases the size of the feature map through upsampling operations, and uses jump connections to combine the deep features in the encoder with the shallow features in the decoder. Upsampling is usually achieved through transposed convolution or interpolation algorithms. Jump connections are to directly connect the outputs of certain layers in the encoder to the corresponding layers of the decoder. The effect of this is to restore and enhance the detail information of the feature map while maintaining the spatial resolution of the feature map, thereby generating an accurate depth map. Finally, based on the depth value of each pixel in the depth map, that is, the distance information between each pixel and the camera, the parking control system can determine the first position of the obstacle in the image, that is, the two-dimensional coordinate position of the obstacle on the image plane. The effect of this is to provide the parking control system with accurate obstacle position information, allowing the system to effectively identify and avoid obstacles during parking, ensuring the safety and accuracy of the parking process.

[0087] Firstly, the parking control system processes the target image data captured by the on-board camera through the encoder part of the target obstacle recognition model. Specifically, the encoder extracts features such as edges, textures, etc. in the image using convolutional layers, and then gradually reduces the spatial dimensions of the feature map through pooling layers. This way, the key information in the image can be extracted and encoded into a more compact deep feature representation. At the same time, the encoder maps each pixel to a depth value, i.e. predicts the depth of each pixel point from the camera, in order to provide the system with three-dimensional information of the obstacles in the scene. Secondly, the parking control system uses the decoder part of the target obstacle recognition model to process the depth features output by the encoder. The decoder increases the size of the feature map through upsampling operations to restore the spatial resolution of the image. This way, the depth information can match the resolution of the original image. At the same time, the decoder uses skip connections to combine deep features in the encoder with shallow features in the decoder, which can preserve more detailed information and enhance the semantic information of the feature map, and finally generate an accurate depth map. The depth map contains the distance information between each pixel point and the camera. Finally, the parking control system determines the first position of the obstacle in the image according to the depth value of each pixel point in the depth map, i.e. the distance information from the camera. Specifically, the system analyzes the depth values in the depth map and identifies the pixel points within a certain range from the camera. These pixel points form the outline of the obstacle. By analyzing these outlines, the system can determine the two-dimensional coordinate position of the obstacle in the image. This is done to provide the parking control system with accurate obstacle position information, so that the system can effectively identify and avoid obstacles during parking, ensuring the safety and accuracy of the parking process.

[0088] Step S30, adjusting the first position according to the radar point cloud data to obtain a real-time obstacle position;

[0089] It should be noted that adjustment refers to the process of correcting and optimizing the first position of the obstacle obtained through camera data and target obstacle recognition model according to radar point cloud data. Specifically, due to the influence of factors such as perspective, lighting, weather, etc., the position of the obstacle obtained by relying solely on image data may have some errors. Radar point cloud data provides more accurate distance and speed information, therefore, the parking control system will fuse radar point cloud data with camera recognition results, through comparison and verification, the position of the obstacle is fine-tuned to ensure the accuracy of the position of the obstacle. This process involves data association and state estimation algorithms such as Kalman filtering or particle filtering, which can combine the advantages of two kinds of sensor data to provide more reliable obstacle position estimation. Real-time obstacle position refers to the accurate position information of the obstacle at the current time, this information is dynamically updated, reflecting the latest state of the obstacle over time, through the above adjustment process, the parking control system can obtain an obstacle position that combines camera image data and radar point cloud data, this position information takes into account the current distance, speed and direction of the obstacle, so it can provide more accurate real-time position.

[0090] It can be understood that first, the parking control system collects radar point cloud data, which is obtained by radar sensors emitting signals and receiving reflected signals, containing distance and speed information of the obstacle; then, the system compares the radar point cloud data with the first position obtained by the camera data, using algorithms such as Kalman filtering, which is a recursive filter that estimates the position of the obstacle through prediction and update two steps, the prediction step uses the position and speed information at the last time to predict the current position, the update step combines the radar point cloud data to correct the predicted position, this can reduce the error caused by the limitation of camera perspective or lighting changes, improve the accuracy of position estimation; finally, after adjustment, the system obtains the real-time obstacle position, which is dynamically updated and can accurately reflect the exact position of the obstacle at any given time, this is done to provide accurate environmental perception information for the parking control system, so that it can safely navigate and avoid collision, ensuring the safety and accuracy of the parking process.

[0091] Step S40, constructing a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image;

[0092] It should be noted that the drivable area refers to the space area in which the vehicle can safely drive without colliding with obstacles in the parking control system, which is calculated based on the size of the vehicle, the safety distance, the real-time position of the obstacles, and the size and type of the obstacles. This area is usually determined by the minimum turning radius and passing ability of the vehicle determined by its geometric size, the buffer space between the vehicle and the obstacles determined by the safety distance, and the area that needs to be avoided on the vehicle driving path determined by the position and size of the obstacles.

[0093] It can be understood that first, the parking control system measures and records the size of the vehicle, including length, width and height, which are used to determine the minimum space range required by the vehicle when parking. Specifically, the system inputs the vehicle size data into the parking algorithm, which calculates the bounding box of the vehicle when turning and moving, which ensures that the physical limitations of the vehicle are considered when planning the path to avoid planning a path that cannot be executed. Secondly, the system will set a safety buffer area around the vehicle according to the requirements of the vehicle safety distance. Specifically, the system will add extra space around the bounding box of the vehicle, which is based on safety standards and vehicle operating characteristics, for example, adding a safety distance of 50 cm around the vehicle. This can reduce the risk of collision with other objects during parking. Then, the system uses real-time obstacle position, size and type information collected by radar and camera sensors to determine the specific position and space range occupied by the obstacles in space. Specifically, the system combines radar point cloud data and image recognition results to identify the exact position of the obstacles and mark the area occupied by the obstacles in the space around the vehicle. Finally, the parking control system integrates the vehicle size, safety buffer area and obstacle-occupied space to construct the drivable area. The specific construction method is that the system first draws the bounding box and safety buffer area of the vehicle on the two-dimensional or three-dimensional space map around the vehicle, then marks the position of the obstacles, and finally excludes the vehicle bounding box, safety buffer area and obstacle-occupied space from the entire space map, and the remaining part is the drivable area. The system can plan one or more paths for the vehicle to travel within this drivable area, ensuring that the vehicle can avoid obstacles and maintain a safe distance during parking, and ultimately achieve safe and accurate automatic parking.

[0094] In step S50, the vehicle camera data, the radar point cloud data, and the data corresponding to the drivable area are sent to the car controller, so that the car controller restores the automatic parking scene according to the scene restoration model, and sends the restored scene data to the car display, so that the car display displays the automatic parking scene restoration view according to the scene data.

[0095] Note that the head unit controller, also known as the Vehicle Control Unit (VCU), is a central processing unit within the vehicle responsible for coordinating and controlling various electronic systems and functions. In the parking control system, the head unit controller receives data from the onboard cameras and radars, processes this information, and controls the automatic parking operation of the vehicle based on the output of the scene reconstruction model. The scene reconstruction model is a software algorithm that uses data collected by onboard sensors to construct a virtual representation of the environment around the vehicle. This model typically includes multiple modules: (1) Perception module: integrates camera data and radar point cloud data to identify the location, size, and type of obstacles. (2) Localization module: determines the precise position of the vehicle in the environment. (3) Map construction module: creates or updates a map of the environment around the vehicle based on sensor data. This model is trained through machine learning and computer vision techniques to process and interpret sensor data, providing an in-depth understanding of the environment. The automatic parking scene refers to the actual environment in which the vehicle performs automatic parking operations, including the layout of the parking lot, the location of obstacles, and the relative position of the vehicle. Reconstruction refers to the process by which the head unit controller processes sensor data based on the scene reconstruction model and constructs a virtual representation of the automatic parking scene. This process involves data fusion, processing, and interpretation to generate an accurate environmental model. The reconstructed scene data refers to the detailed information about the automatic parking scene obtained after the head unit controller processes it, including the location of obstacles, the location of the vehicle, and the drivable area, which provides the necessary environmental information for automatic parking. The head unit display, also known as the display screen of the vehicle infotainment system, is a display device within the vehicle used to display various information to the driver, such as navigation, vehicle status, entertainment information, etc. In the parking control system, it is used to display the reconstructed view of the automatic parking scene. The automatic parking scene reconstruction view is the visual representation of the automatic parking scene displayed on the head unit display, which includes the location of the vehicle, the location of obstacles, and the parking path, helping the driver understand the vehicle's environment and the running status of the parking control system.

[0096] It can be understood that first, the parking control system will collect the visual data captured by the vehicle camera, the point cloud data collected by the radar sensor and the calculated drivable area data, and then transmit them to the car controller. After receiving these data, the car controller uses the built-in scene restoration model, which includes multiple modules such as perception, positioning, and map construction, to comprehensively analyze and process the automatic parking scene. The scene restoration model identifies the positions of obstacles and vehicles by fusing camera data and radar data, and constructs a detailed map of the vehicle's surroundings. The purpose of this process is to generate an accurate virtual environment model, i.e., the restored scene data, which contains key information such as vehicle position, obstacle information, and drivable area. Finally, the car controller sends the restored scene data to the car display, and the car display generates and displays the restored view of the automatic parking scene based on the received data. This view presents the parking environment and path of the vehicle to the driver in a graphical manner, allowing the driver to intuitively understand the automatic parking process and improving the transparency and safety of the parking operation. Through these detailed steps, the parking control system can provide the driver with a clear and accurate parking scene view, making the automatic parking process more reliable and easier to understand.

[0097] The embodiment provides a drivable area visualization method for automatic parking, obtains vehicle-mounted camera data, radar point cloud data, vehicle size and vehicle safety distance; a target obstacle identification model is used for identifying the vehicle-mounted camera data, to obtain a first position, size and type of an obstacle in an image, and the target obstacle identification model is constructed according to an encoder, a decoder and a self-attention module; the first position is adjusted according to the radar point cloud data, to obtain a real-time obstacle position; a drivable area is constructed according to the vehicle size, the vehicle safety distance, the real-time obstacle position and the size and type of the obstacle in the image; the vehicle-mounted camera data, the radar point cloud data and data corresponding to the drivable area are sent to a car controller, so that the car controller restores an automatic parking scene according to a scene restoration model, and sends the restored scene data to a car display, so that the car display displays an automatic parking scene restoration view according to the scene data. The parking control system first collects vehicle-mounted camera data and radar point cloud data, and records the vehicle size and safety distance, to provide comprehensive environmental information for the system and ensure accurate perception of the environment around the vehicle. Then, the system uses the target obstacle identification model constructed by the encoder, the decoder and the self-attention module to process the camera data, to identify the position, size and type of the obstacle, accurately identify and classify the obstacle, and provide key information for collision avoidance and parking path planning. Then, the system compares the radar point cloud data with the obstacle position identified by the camera, to correct the real-time position of the obstacle, improve the accuracy of the obstacle position data and ensure the safety and reliability of the parking control system. On this basis, the system combines the vehicle size, the safety distance, the real-time obstacle position and the size and type of the obstacle, to calculate the drivable area of the vehicle, provide a safe driving space for automatic parking, avoid collision with the obstacle and ensure the feasibility of the parking path. Finally, the system sends the vehicle-mounted camera data, the radar point cloud data and the drivable area data to the car controller, the controller reconstructs the automatic parking scene according to the scene restoration model, and sends the restored scene data to the car display, to display the view of the automatic parking scene, improve the transparency and safety of the operation and enhance the trust of the driver in the parking control system. The whole process greatly improves the accuracy, safety and user-friendliness of the parking control system through accurate environmental perception, obstacle identification, position correction, drivable area construction and scene restoration. The parking control system collects vehicle-mounted camera data and radar point cloud data, combines vehicle size and safety distance information, provides comprehensive environmental perception for the system and ensures accurate identification of the obstacles around the vehicle. The target obstacle identification model constructed by the encoder, the decoder and the self-attention module further processes the camera data, accurately identifies and classifies the obstacle, and thus provides key information for collision avoidance and parking path planning.The system then corrects the real-time position of the obstacle according to the radar point cloud data, further improves the accuracy of the position data, and enhances the safety and reliability of the system. On this basis, the system considers the vehicle size, safety distance, real-time obstacle position, and obstacle size and type to calculate the drivable area of the vehicle, ensuring the feasibility of the parking path and avoiding collision with obstacles. Finally, the system sends these data to the car controller, the controller reconstructs the automatic parking scene according to the scene restoration model, and sends the restored scene data to the car display to intuitively display the view of the automatic parking scene. This not only improves the transparency and safety of the operation, enhances the driver's trust in the parking control system, but also displays the drivable area of the automatic parking to avoid collision with obstacles, significantly improves the human-computer interaction performance and safety of automatic parking, and reduces the collision risk when using automatic parking.

[0098] Based on the first embodiment of the present application, the same or similar contents as the above embodiment one can be referred to the above introduction, and the subsequent will not be repeated. On this basis, please refer to Figure 2 , Figure 2 The flowchart of the second embodiment of the automatic parking drivable area visualization method of the present application is shown in the figure. The steps S30 of the automatic parking drivable area visualization method include steps S31-S33:

[0099] Step S31, fitting the radar point cloud data by using the random sample consensus strategy to identify the second position of the static obstacle in the point cloud data;

[0100] It should be noted that the random sample consensus (RANSAC) is an iterative method for estimating the parameters of a mathematical model from a set of data that contains outliers. In the parking control system, RANSAC is used to identify consistent geometric models such as lines or planes from radar point cloud data, which can be used to describe the outline of obstacles. The RANSAC algorithm randomly selects a small subset of data points to fit a model, then evaluates the fitting of this model to the overall data to identify which data points are outliers and which are consistent data points that fit the model. Fitting here refers to using a mathematical model to match or approximate a set of points in the radar point cloud data. In the parking control system, fitting usually refers to determining one or more geometric models (such as planes or lines) that best represent a set of points in the point cloud data. The fitting process can help the system identify specific structures in the point cloud, such as the ground, walls, or the surface of obstacles. Static obstacles refer to obstacles that do not change relative to the vehicle's position, such as parking posts, walls, other vehicles, etc. These obstacles do not move within the observation time of the radar sensor, so they can be stably detected and tracked through radar point cloud data. The second position refers to the precise position of the static obstacle identified by processing the radar point cloud data through the RANSAC algorithm. This position is more accurate and reliable than the initial "first position" (which may be obtained from camera data) because it is obtained by analyzing the consistent patterns in the radar point cloud data and can provide more accurate spatial position information of the obstacle.

[0101] It can be understood that first, the parking control system will use the random sample consensus (RANSAC) to process the radar point cloud data. Specifically, a small subset of point cloud data is randomly selected as a sample set, and then a pre-set geometric model (such as a plane or a line) is used to fit these sample points. This is done to identify the model parameters that best represent the static obstacle from the overall data. Second, the system evaluates the fitting of this model to all point cloud data by calculating the error of the data points to the model to distinguish which points conform to the model (i.e. consistent data points) and which points do not conform (i.e. outliers). This step is to exclude the influence of noise and error and ensure the accuracy of the model. Finally, the system repeats the above process multiple times, each time randomly selecting a new sample set to ensure the robustness of the model, and finally selects the best fitting model that best represents the actual position and shape of the static obstacle, thereby determining the second position of the static obstacle, which is a more accurate spatial position of the obstacle optimized by the algorithm. Even in the presence of noise and inaccurate data in the radar point cloud data, the system can accurately identify the position of the static obstacle, providing accurate environmental perception information for the parking control system, thereby improving the safety and accuracy of the parking process.

[0102] As an example, the step of fitting the radar point cloud data by a random sample consensus strategy to identify the second position of the static obstacle in the point cloud data comprises: extracting all points from the radar point cloud data to obtain an initial point set; selecting a random point pair from the initial point set; fitting an initial obstacle geometric model according to the random point pair; calculating the distance of all points to the initial obstacle geometric model; dividing the initial point set according to the distance to obtain an inlier point set; maximizing the inlier point set by selecting the random point pair multiple times to obtain a target obstacle geometric model; and determining the second position of the static obstacle according to the target obstacle geometric model.

[0103] All points refer to each data point in the radar point cloud data, which represents the position information of the surrounding environment detected by the radar sensor. The initial point set refers to a set composed of all points extracted from the radar point cloud data, which is the starting point for RANSAC algorithm analysis. The random point pair refers to two points randomly selected from the initial point set, which is used to start constructing a hypothetical geometric model. For example, in fitting a straight line, the random point pair can be two points on the straight line; in fitting a plane, the random point pair can be two non-collinear points on the plane. The initial obstacle geometric model refers to a hypothetical geometric model fitted according to the random point pair, which represents the possible shape of the obstacle, such as a straight line, a plane or other geometric shapes. The distance refers to the Euclidean distance from each point in the initial point set to the initial obstacle geometric model, usually measured by straight-line distance. The division refers to dividing the points in the initial point set into two groups according to the calculated distance, the points close to the geometric model (considered as inliers) and the points far away from the geometric model (considered as outliers). The inlier point set refers to the set of points close to the initial obstacle geometric model after division, which are considered to be points conforming to the current geometric model, i.e. they are within a certain error range of the model. The target obstacle geometric model refers to the geometric model finally determined by selecting random point pairs multiple times and trying to maximize the inlier point set (i.e. finding the geometric parameters that can make the most points conform to the model) each time, which is considered to be the model that best represents the static obstacle in the data.

[0104] First, the system performs a data extraction operation, collecting each scatter point in the radar point cloud data according to its coordinate position in space to form an initial point set, which is done to obtain a comprehensive, unprocessed surrounding environment data set, laying the foundation for subsequent analysis. Then, the system randomly selects two points from the initial point set to form a point pair, and uses these two points to define a hypothetical geometric model, such as a straight line or a plane, which is done to establish a preliminary model for subsequent geometric shape fitting. Next, the system calculates the distance of all points in the initial point set to the hypothetical geometric model, which is done through mathematical formulas such as the distance from a point to a line or the distance from a point to a plane, which allows for the evaluation of the proximity of each point to the model and provides a basis for subsequent point classification. Then, the system divides the initial point set into an inner point set and an outer point set based on the calculated distances, with the inner point set containing points that are closer to the model, i.e., points that are considered to be within the model, which allows for the differentiation between points that conform to the model and outliers or noise. Next, the system selects different random point pairs through multiple iterations and attempts to find a point pair that maximizes the inner point set each time, which involves comparison and selection and aims to find a best model that maximizes the number of points that conform to the model, which improves the accuracy and robustness of the model. Finally, the system determines the second position of the static obstacle based on the final target obstacle geometric model, which is obtained by analyzing the points in the inner point set, which allows for the extraction of accurate static obstacle positions from noisy radar data and provides precise environmental perception data for the parking control system, thereby improving the safety and accuracy of the parking process.

[0105] Step S32, clustering the radar point cloud data through a density clustering strategy to identify the third position of the dynamic obstacle in the point cloud data;

[0106] It should be noted that the density clustering strategy, such as the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm, is a density-based clustering method. This strategy divides data points into different clusters by identifying high-density regions separated by low-density regions. Specifically, the algorithm sets two parameters: neighborhood radius (eps) and a minimum point number (minPts). For each point, the algorithm checks how many other points are within the neighborhood radius. If the number exceeds the minimum point number, the point is considered a core point of a certain cluster. The algorithm expands the cluster based on these core points, while identifying boundary points and noise points. This method does not require the number of clusters to be specified in advance, can identify clusters of arbitrary shape, and is robust to outliers. Clustering refers to dividing samples in a dataset into several clusters, so that the similarity of samples in the same cluster is high, while the similarity of samples between different clusters is low. In radar point cloud data, clustering is to classify points that are close in space into a class. These points usually represent the same physical entity. Dynamic obstacles refer to obstacles whose positions change over time, such as walking pedestrians and moving vehicles. Compared with static obstacles, the position information of dynamic obstacles changes over time, so it needs to be updated in real time. The third position refers to the accurate position of the dynamic obstacle identified after processing the radar point cloud data by the density clustering strategy. This position information is obtained based on the clustering result.

[0107] As you can understand, the system first implements a density clustering strategy, which involves setting two key parameters for the algorithm: a neighborhood radius and a minimum number of points. This step determines which points are close enough to be considered part of the same cluster. Then, for each point in the radar point cloud data, the system calculates the number of other points within the specified neighborhood radius. This is done by comparing the distance of each point to its surrounding points, aiming to identify points that are spatially clustered together. If the number of points in a point's neighborhood exceeds the minimum number, the point is considered a core point, which helps identify dense areas in the point cloud. Next, the system expands outward from these core points to form clusters, identifying boundary points at the edge of the clusters and noise points that do not belong to any cluster. This step is achieved by iteratively adding all points in the neighborhood of the core point (including non-core points) to the same cluster, in order to distinguish between different dynamic obstacles. Finally, through this density-based clustering method, the system identifies the set of points in the radar point cloud data that are dynamic obstacles and determines the third position of these dynamic obstacles, that is, their precise coordinates in space. Doing so makes it possible to distinguish dynamic obstacles from complex point cloud data and accurately track their position changes, which is crucial for parking control systems to safely navigate and make decisions in dynamic environments because it provides accurate information about moving objects in the surrounding environment, allowing the system to react in time to avoid collisions.

[0108] As an example, the step of clustering the radar point cloud data using a density clustering strategy to identify the third position of the dynamic obstacle in the point cloud data includes: defining a neighborhood radius and a minimum number of points based on the distribution characteristics of the point cloud in the radar point cloud data; clustering each point in the radar point cloud data based on the neighborhood radius and the minimum number of points to obtain a point cloud cluster; determining a dynamic obstacle based on changes in consecutive frames in the radar point cloud data; and dynamically tracking the dynamic obstacle in the consecutive frames through Kalman filtering to obtain the third position of the dynamic obstacle.

[0109] Distribution characteristics refer to the relative positions and densities of points in the radar point cloud data, which can help the system understand the degree of clustering and distribution patterns of the point cloud, thereby providing a basis for selecting appropriate clustering parameters. Neighborhood radius is a parameter in the density clustering strategy, which defines the neighborhood range of a point, i.e. all points within a certain distance from the point in space are considered neighbors of the point. In the DBSCAN algorithm, this parameter determines the range of neighboring points considered when finding core points. Minimum point number is another parameter in the density clustering strategy, which specifies the minimum number of neighboring points required for a point to be identified as a core point. This parameter, together with the neighborhood radius, determines the density threshold for clustering. Point cloud clusters refer to sets formed by grouping spatially close points using clustering algorithms, each cluster representing an independent entity or region in the point cloud, such as an obstacle or a part of the ground. Continuous frames refer to radar point cloud data captured continuously in time series, which are usually collected by radar sensors at different time points for tracking dynamic obstacles over time. Kalman filtering is a recursive filter that uses a series of noisy measurement data to estimate the state of a dynamic system. In the parking control system, Kalman filtering is used to process radar sensor data to reduce noise and predict the position and velocity of dynamic obstacles. Dynamic tracking refers to the process of tracking dynamic obstacles in consecutive frames using algorithms such as Kalman filtering, which includes predicting the next position of the obstacle, updating the prediction results, and adjusting the prediction error to continuously estimate the motion state of the obstacle.

[0110] First, the system analyzes the distribution characteristics of points in the radar point cloud data to determine appropriate neighborhood radius and minimum point number parameters. Specifically, the system sets the neighborhood radius by counting the density and distribution range of points in the point cloud, which determines how far to consider points around each point during clustering. At the same time, the system sets a minimum point number, which is used to determine how many points are needed in a region to consider it an effective cluster. This is done to accurately divide spatially close points into the same cluster in subsequent steps. Second, the system clusters each point in the radar point cloud data to form point cloud clusters. Specifically, for each point, the system checks how many points are within the set neighborhood radius. If this number exceeds the minimum point number, the point is considered a core point. Then the system assigns all points in the core point's neighborhood to a cluster. This process is repeated until all points are assigned to a cluster or marked as noise. This is done to group spatially close points, with each cluster representing a part of an obstacle or the ground, thereby identifying static environmental structures. Then, the system determines dynamic obstacles based on changes in point cloud clusters between consecutive frames. Specifically, the system compares radar point cloud data captured at different times to observe changes in the position and shape of point cloud clusters. These changed clusters represent dynamic obstacles. This is done to distinguish obstacles that move over time, such as moving vehicles or pedestrians. Finally, the system dynamically tracks dynamic obstacles in consecutive frames using Kalman filtering to obtain the third position of dynamic obstacles. Specifically, the system uses the Kalman filtering algorithm to predict the position of each dynamic obstacle at the next time step. Then, when new radar point cloud data arrives, the system updates the prediction by comparing the predicted position with the actual observed position to adjust the prediction error, thereby obtaining a more accurate obstacle position. Through this process, the movement of obstacles can be continuously tracked, even in the presence of fast-moving obstacles or noisy sensor data, providing accurate obstacle position information, which is crucial for parking control systems that need real-time environmental information to avoid collisions and plan safe parking paths.

[0111] In step S33, the obstacle in the image is spatially aligned with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position, obtaining a real-time obstacle position.

[0112] It should be noted that spatial alignment refers to the process of integrating obstacle position information from different data sources (such as vehicle-mounted cameras, radar sensors) and different time points to ensure their consistency and accuracy in the same coordinate system. Specifically, the system compares and calibrates the first position obtained from camera data (obstacle position identified by image recognition), the second position obtained from radar data (static obstacle position fitted by radar point cloud), and the third position obtained from radar data (dynamic obstacle position clustered by radar point cloud). This process involves coordinate transformation, data fusion, and error correction techniques, aiming to obtain a unified and accurate obstacle spatial position view. This ensures consistency between different sensor data, improving the accuracy and robustness of obstacle detection. Real-time obstacle position refers to the accurate position information of obstacles at the current time obtained by the system after spatial alignment. This position information is dynamically updated and reflects the latest state of obstacles over time. Real-time obstacle position not only includes the position of static obstacles but also includes the position of dynamic obstacles (such as moving vehicles and pedestrians). This information is crucial for automatic parking as it allows the driver to respond quickly to adapt to the dynamic changes of obstacles and ensure parking safety.

[0113] It can be understood that first, the system identifies the first position of the obstacle from the image data obtained by the vehicle-mounted camera, which is achieved by the target obstacle recognition model. The model analyzes the image and marks the outline and position of the obstacle. At the same time, the system obtains the second position of the static obstacle from the radar point cloud data through the random sample consensus strategy. This strategy identifies the precise position of static obstacles by fitting geometric models. In addition, the system also obtains the third position of dynamic obstacles from the radar point cloud data through the density clustering strategy. This strategy distinguishes and tracks dynamic obstacles by identifying high-density areas in the point cloud. Second, the system spatially aligns the three different sources of position data. Specifically, all position data is converted to the same coordinate system, which involves coordinate conversion algorithms to ensure that all obstacle position data is in the same reference frame. Then, the system uses data fusion techniques such as Kalman filtering. This algorithm combines data from multiple sensors to reduce uncertainty and errors through prediction and update steps, providing more accurate and reliable obstacle position information. Finally, the system obtains real-time obstacle position through the above data fusion and spatial alignment process, which contains the latest position information of static and dynamic obstacles. In this way, the system can provide a continuously updated and accurate environmental perception picture, allowing the driver to monitor the changes in the surrounding environment in real time and respond quickly to effectively avoid collisions.

[0114] As an example, the step of spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position comprises: obtaining camera parameters and radar parameters; constructing a ground coordinate system with the center of the rear axle of the ego vehicle as the origin, the forward direction of the ego vehicle as the positive direction of the horizontal axis, and the left side of the vehicle body as the positive direction of the vertical axis; converting the two-dimensional coordinates of the first position in the image into three-dimensional coordinates in the ground coordinate system through inverse perspective transformation according to the camera parameters to obtain a first coordinate; performing rotation and displacement transformation on the second position and the third position according to the radar parameters and the ground coordinate system to obtain a second coordinate and a third coordinate; and converting the first coordinate, the second coordinate, and the third coordinate in the ground coordinate system to a bird's eye view perspective to obtain a real-time obstacle position.

[0115] The camera parameters include intrinsic parameters and extrinsic parameters. Intrinsic parameters refer to the inherent characteristics of the camera, such as focal length, principal point (image center), distortion coefficient, etc., which affect how the camera projects the three-dimensional world onto a two-dimensional image. Extrinsic parameters refer to the position and attitude of the camera in the vehicle, i.e., the position and orientation of the camera relative to the vehicle coordinate system. Radar parameters refer to the characteristics of the radar sensor, such as detection range, angular resolution, signal frequency, etc., as well as the position and orientation of the radar sensor on the vehicle. The rear axle center of the ego vehicle refers to the center point of the rear axle of the vehicle, which is usually used as the origin of the vehicle coordinate system to determine the position of the vehicle relative to the ground. The positive direction of the horizontal axis refers to the axis along the forward direction of the vehicle in the vehicle coordinate system, usually defined as the positive direction of the x-axis. The positive direction of the vertical axis refers to the axis perpendicular to the forward direction of the vehicle in the vehicle coordinate system, usually defined as the positive direction of the y-axis, pointing to the left side of the vehicle body. The ground coordinate system is a coordinate system fixed to the ground, used to describe the positions of the vehicle and obstacles on the ground. This coordinate system helps to unify the data of the vehicle sensors into a common reference frame. Two-dimensional coordinates refer to positions in the image plane, usually represented by pixel coordinates. Inverse perspective transformation conversion is a mathematical transformation that converts two-dimensional coordinates in the image to three-dimensional coordinates in the real world, taking into account the camera's perspective and the geometry of the scene. Three-dimensional coordinates refer to positions in three-dimensional space, usually represented by X, Y, Z coordinates. The first coordinate refers to the result of converting the two-dimensional coordinates of the first position in the image to three-dimensional coordinates in the ground coordinate system through inverse perspective transformation. Rotation refers to adjusting the direction of radar data according to radar parameters and ground coordinate system, so that it aligns with the axes of the ground coordinate system. Displacement transformation refers to adjusting the position of radar data according to radar parameters and ground coordinate system, so that it correctly reflects in the ground coordinate system. The second coordinate and the third coordinate refer to the positions of the second position and the third position in the point cloud data in the ground coordinate system after rotation and displacement transformation. The bird's eye view perspective refers to the perspective from directly above looking down. This perspective helps to provide a global, two-dimensional view for describing the positions of the vehicle and obstacles on the ground.

[0116] First, the system will collect specific parameters of the camera and radar, including the focal length, principal point position, distortion coefficient of the camera, and the detection range and angular resolution of the radar, etc. These parameters are crucial for subsequent data processing and conversion. This is done to ensure that the sensor data can be accurately converted into the coordinate system of the vehicle, laying the foundation for subsequent spatial alignment. Second, the system will take the center of the rear axle of the vehicle as the origin of the coordinate system, the forward direction of the vehicle as the positive direction of the horizontal axis (x-axis), and the left side of the vehicle body as the positive direction of the vertical axis (y-axis), to construct a ground coordinate system. This coordinate system is the basis for the vehicle to perceive the environment, providing a unified reference framework for the positions of the vehicle and obstacles. Then, the system will use the camera parameters to convert the two-dimensional coordinates of the obstacles identified in the image into three-dimensional coordinates in the ground coordinate system through inverse perspective transformation. Inverse perspective transformation can convert two-dimensional image coordinates into actual three-dimensional world coordinates according to the camera's perspective and known three-dimensional scene structure, which is done to eliminate the perspective distortion in the image and obtain the actual position of the obstacle in the real world, i.e., the first coordinate. Next, the system will rotate and translate the positions of static and dynamic obstacles detected by the radar according to the radar parameters and the ground coordinate system, converting these positions from the radar coordinate system to the ground coordinate system, obtaining the second coordinate and the third coordinate. This is done to ensure that all sensor data is in the same coordinate system, facilitating subsequent data fusion and processing. Finally, the system will convert the first coordinate, second coordinate, and third coordinate in the ground coordinate system to the bird's eye view perspective, i.e., viewing the positions of the vehicle and obstacles from directly above, to obtain real-time obstacle positions. This can visually demonstrate the relative positions of the vehicle and obstacles on a unified two-dimensional plane, providing accurate environmental perception information for the parking control system, thereby improving the safety and accuracy of the parking process. In this way, the system can monitor and respond to changes in the surrounding environment in real time, effectively avoiding collisions.

[0117] The embodiment fits the radar point cloud data through a random sample consensus strategy to identify a second position of a static obstacle in the point cloud data; clusters the radar point cloud data through a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data; and spatially aligns the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position. The system first fits the radar point cloud data through a random sample consensus strategy, determines a model that best fits the data by randomly selecting sample points and iterating multiple times, which can identify the second position of the static obstacle in the radar point cloud data, i.e., a more accurate three-dimensional spatial position, thereby improving the accuracy and robustness of the identification of the position of the static obstacle, and accurately identifying even in the presence of noise and outliers in the data. Next, the system clusters the radar point cloud data through a density clustering strategy, groups data points according to the density characteristics of the point cloud to form clusters, and identifies the third position of the dynamic obstacle in the radar point cloud data, which can effectively distinguish and track the dynamic obstacle, and maintain tracking even in the case of moving or changing obstacles, thereby providing real-time dynamic environmental information for the parking control system. Finally, the system spatially aligns the first position (image recognition result), the second position (accurate position of the static obstacle), and the third position (position of the dynamic obstacle) to obtain the real-time obstacle position, which provides a unified, real-time updated obstacle position information, and in this way, the parking control system can more accurately perceive the surrounding environment, thereby improving the safety and reliability of the parking operation.

[0118] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the automatic parking drivable area visualization method of the present application. More forms of simple changes based on this technical concept are within the protection scope of the present application.

[0119] The present application also provides an automatic parking drivable area visualization device, which is described in detail below. Figure 3 The automatic parking drivable area visualization device comprises:

[0120] A data acquisition module 10 is configured to acquire vehicle-mounted camera data, radar point cloud data, vehicle size, and vehicle safety distance.

[0121] An obstacle identification module 20 is configured to identify the vehicle-mounted camera data through a target obstacle identification model to obtain a first position, size, and type of an obstacle in an image, wherein the target obstacle identification model is constructed according to an encoder, a decoder, and a self-attention module.

[0122] A position determining module 30 is configured to adjust the first position according to the radar point cloud data to obtain real-time obstacle positions.

[0123] A drivable area constructing module 40 is configured to construct a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle positions, and the size and type of the obstacles in the image.

[0124] A view restoring module 50 is configured to send the vehicle camera data, the radar point cloud data, and the data corresponding to the drivable area to a car controller, so that the car controller restores an automatic parking scene according to a scene restoration model and sends the restored scene data to a car display, so that the car display displays an automatic parking scene restoration view according to the scene data.

[0125] In an embodiment, the position determining module 30 is further configured to fit the radar point cloud data by a random sample consensus strategy to identify second positions of static obstacles in the point cloud data, cluster the radar point cloud data by a density clustering strategy to identify third positions of dynamic obstacles in the point cloud data, and spatially align the obstacles in the image with the static obstacles and the dynamic obstacles according to the first positions, the second positions, and the third positions to obtain the real-time obstacle positions.

[0126] In an embodiment, the position determining module 30 is further configured to extract all points from the radar point cloud data to obtain an initial point set, select random point pairs from the initial point set, fit an initial obstacle geometric model according to the random point pairs, calculate distances of all points to the initial obstacle geometric model, divide the initial point set according to the distances to obtain an inner point set, maximize the inner point set by multiple selections of the random point pairs to obtain a target obstacle geometric model, and determine the second positions of the static obstacles according to the target obstacle geometric model.

[0127] In an embodiment, the position determining module 30 is further configured to define a neighborhood radius and a minimum point number according to distribution characteristics of point clouds in the radar point cloud data, cluster each point in the radar point cloud data according to the neighborhood radius and the minimum point number to obtain point cloud clusters, determine dynamic obstacles according to changes of consecutive frames in the radar point cloud data, and dynamically track the dynamic obstacles in the consecutive frames by Kalman filtering to obtain the third positions of the dynamic obstacles.

[0128] In an embodiment, the position determination module 30 is further configured to obtain camera parameters and radar parameters; construct a ground coordinate system with the center of the rear axle of the ego vehicle as the origin, the forward direction of the ego vehicle as the positive direction of the horizontal axis, and the left side of the vehicle body as the positive direction of the vertical axis; convert the two-dimensional coordinates of the first position in the image into three-dimensional coordinates in the ground coordinate system through inverse perspective transformation according to the camera parameters to obtain first coordinates; perform rotation and displacement transformation on the second position and the third position according to the radar parameters and the ground coordinate system to obtain second coordinates and third coordinates; and convert the first coordinates, the second coordinates, and the third coordinates in the ground coordinate system to a bird's-eye view perspective to obtain real-time obstacle positions.

[0129] In an embodiment, the obstacle identification module 20 is further configured to perform denoising, illumination compensation, and standardization on the image in the vehicle-mounted camera data to obtain target image data; process the target image data through the encoder and the decoder to obtain a first position of an obstacle in the image; perform convolution operation on the target image data through the convolution layer to obtain a feature map; perform dimension reduction on the feature map through the pooling layer to obtain a low-dimensional feature map; perform weighted aggregation on the low-dimensional feature map through the self-attention module to obtain a weighted feature map; perform bounding box regression on the weighted feature map to obtain the size of the obstacle; perform network coding on the weighted feature map through the self-attention module to obtain a classification network; and process the classification network through a smooth max function to obtain the type of the obstacle.

[0130] In an embodiment, the obstacle identification module 20 is further configured to map each pixel in the target image data to a depth value through the encoder; perform upsampling and skip connection on the target image data according to the depth value through the decoder to obtain a depth map; and obtain a first position of an obstacle in the image according to distance information of each pixel in the depth map from the camera.

[0131] The automatic parking drivable area visualization device provided in the present application adopts the automatic parking drivable area visualization method in the above embodiments, and can solve the technical problem of how to display the automatic parking drivable area to avoid collision with obstacles. Compared with the prior art, the automatic parking drivable area visualization device provided in the present application has the same beneficial effects as the automatic parking drivable area visualization method provided in the above embodiments, and other technical features in the automatic parking drivable area visualization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0132] The application provides an automatic parking drivable area visualization device, which comprises at least one processor and a memory connected with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic parking drivable area visualization method in the embodiment I.

[0133] Reference will be made to the following description Figure 4 which shows a structural diagram of an automatic parking drivable area visualization device suitable for implementing the embodiments of the application. The automatic parking drivable area visualization device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 4 The automatic parking drivable area visualization device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the application.

[0134] As Figure 4As shown, the automatic parking drivable area visualization device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the automatic parking drivable area visualization device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the automatic parking drivable area visualization device to communicate wirelessly or wired with other devices to exchange data. Although the automatic parking drivable area visualization device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0135] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0136] The automatic parking drivable area visualization device provided in the present application adopts the automatic parking drivable area visualization method in the above embodiments, and can solve the technical problem of how to display the automatic parking drivable area to avoid collision with obstacles. Compared with the prior art, the automatic parking drivable area visualization device provided in the present application has the same beneficial effects as the automatic parking drivable area visualization method provided in the above embodiments, and other technical features in the automatic parking drivable area visualization device are the same as the features disclosed in the previous embodiment method, which will not be repeated here. It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0137] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., computer programs) for performing the automatic parking drivable area visualization method in the above embodiments.

[0138] The computer readable storage medium provided in the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0139] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the automatic parking drivable area visualization device, the automatic parking drivable area visualization device is caused to: acquire vehicle-mounted camera data, radar point cloud data, vehicle size, and vehicle safety distance; identify the vehicle-mounted camera data through a target obstacle identification model to obtain a first position, size, and type of an obstacle in an image, the target obstacle identification model being constructed according to an encoder, a decoder, and a self-attention module; adjust the first position according to the radar point cloud data to obtain a real-time obstacle position; construct a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image; and send data corresponding to the vehicle-mounted camera data, the radar point cloud data, and the drivable area to a vehicle controller, so that the vehicle controller restores an automatic parking scene according to a scene restoration model, and sends restored scene data to a vehicle display, so that the vehicle display displays an automatic parking scene restoration view according to the scene data.

[0140] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0141] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0142] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the names of the modules do not limit the modules themselves.

[0143] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above-mentioned automatic parking drivable area visualization method, and can solve the technical problem of how to display the drivable area of automatic parking to avoid collision with obstacles. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the automatic parking drivable area visualization method provided by the above-mentioned embodiments, which will not be repeated here.

[0144] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned automatic parking drivable area visualization method. The computer program product provided by the present application can solve the technical problem of how to display the drivable area of automatic parking to avoid collision with obstacles. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the automatic parking drivable area visualization method provided by the above-mentioned embodiments, which will not be repeated here. The above-mentioned only some embodiments of the present application, and not limit the patent scope of the present application, any equivalent structural transformation made by using the contents of the present application specification and drawings, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for visualizing a drivable area for automatic parking, characterized in that: The method comprises: Obtain vehicle camera data, radar point cloud data, vehicle dimensions, and vehicle safety distance; Identifying the vehicle camera data using a target obstacle recognition model to obtain a first position, size, and type of the obstacle in the image, wherein the target obstacle recognition model is constructed based on an encoder, a decoder, and a self-attention module; Extract all points from the radar point cloud data to obtain an initial point set; Selecting a random point pair from the initial point set; The target obstacle geometric model is obtained by maximizing the interior point set by selecting the random point pairs multiple times; determining a second position of the static obstacle according to the target obstacle geometric model; Clustering the radar point cloud data using a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data; spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position; Constructing a drivable area based on the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image; The vehicle-mounted camera data, the radar point cloud data, and the data corresponding to the drivable area are sent to a vehicle controller, so that the vehicle controller restores the automatic parking scene according to a scene restoration model, and the restored scene data is sent to a vehicle display, so that the vehicle display displays an automatic parking scene restoration view according to the scene data.

2. The method according to claim 1, wherein Before obtaining the target obstacle geometric model by maximizing the interior point set by selecting the random point pairs multiple times, the method further includes: Fitting an initial obstacle geometry model according to the random point pairs; Calculating the distances from all points to the initial obstacle geometry model; The initial point set is divided according to the distance to obtain an inner point set.

3. The method according to claim 1, wherein The step of clustering the radar point cloud data using a density clustering strategy to identify the third position of the dynamic obstacle in the point cloud data includes: Defining a neighborhood radius and a minimum number of points according to distribution characteristics of the point cloud in the radar point cloud data; Clustering each point in the radar point cloud data according to the neighborhood radius and the minimum number of points to obtain a point cloud cluster; determining a dynamic obstacle based on changes in consecutive frames of the radar point cloud data; The dynamic obstacle in the continuous frames is dynamically tracked by using Kalman filtering to obtain a third position of the dynamic obstacle.

4. The method according to claim 1, wherein The step of spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position includes: Get camera parameters and radar parameters; Construct a ground coordinate system with the center of the vehicle's rear axle as the origin, the vehicle's forward direction as the positive horizontal axis, and the left side of the vehicle as the positive vertical axis. Converting the two-dimensional coordinates of the first position in the image into three-dimensional coordinates in the ground coordinate system through an inverse perspective transformation according to the camera parameters to obtain first coordinates; Performing rotation and displacement transformation on the second position and the third position according to the radar parameters and the ground coordinate system to obtain second coordinates and third coordinates; The first coordinate, the second coordinate, and the third coordinate in the ground coordinate system are converted to a bird's-eye view to obtain a real-time obstacle position.

5. The method according to claim 1, wherein The target obstacle recognition model includes an encoder, a decoder, and a self-attention module. The encoder includes a convolution layer and a pooling layer. The step of identifying the vehicle camera data using the target obstacle recognition model to obtain the first position, size, and type of the obstacle in the image includes: Denoising, illumination compensation, and standardization are performed on the image in the vehicle-mounted camera data to obtain target image data; Processing the target image data by the encoder and the decoder to obtain a first position of an obstacle in the image; Performing a convolution operation on the target image data through the convolution layer to obtain a feature map; Performing dimensionality reduction on the feature map through the pooling layer to obtain a low-dimensional feature map; Performing weighted aggregation on the low-dimensional feature map through the self-attention module to obtain a weighted feature map; Performing bounding box regression on the weighted feature map to obtain the size of obstacles in the image; Performing network coding on the weighted feature map through the self-attention module to obtain a classification network; The classification network is processed by a smooth maximum function to obtain the type of obstacles in the image.

6. The method according to claim 5, wherein The step of processing the target image data by the encoder and the decoder to obtain the first position of the obstacle in the image includes: Mapping each pixel in the target image data to a depth value by the encoder; Upsampling and skip-connecting the target image data according to the depth value by the decoder to obtain a depth map; The first position of the obstacle in the image is obtained according to the distance information between each pixel in the depth map and the camera.

7. A device for visualizing the drivable area of ​​an automatic parking vehicle, characterized in that: The device comprises: Data acquisition module, used to obtain vehicle camera data, radar point cloud data, vehicle size and vehicle safety distance; An obstacle recognition module, configured to identify the vehicle camera data using a target obstacle recognition model to obtain a first position, size, and type of obstacles in the image. The target obstacle recognition model is constructed based on an encoder, a decoder, and a self-attention module. a position determination module configured to extract all points from the radar point cloud data to obtain an initial point set; select random point pairs from the initial point set; obtain a target obstacle geometric model by maximizing an inner point set by selecting the random point pairs multiple times; determine a second position of a static obstacle based on the target obstacle geometric model; cluster the radar point cloud data using a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data; and spatially align the obstacle in the image with the static obstacle and the dynamic obstacle based on the first position, the second position, and the third position to obtain a real-time obstacle position; A drivable area construction module is used to construct a drivable area based on the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image; The view restoration module is used to send the vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area to the vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and sends the restored scene data to the vehicle display, so that the vehicle display displays the automatic parking scene restoration view according to the scene data.

8. A device for visualizing the drivable area of ​​an automatic parking vehicle, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for visualizing a drivable area for automatic parking according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for visualizing a drivable area of ​​automatic parking according to any one of claims 1 to 6 are implemented.

Images