Visualization method, device and equipment for drivable area of automatic parking and storage medium

Abstract

The invention discloses a driving area visualization method, device and equipment for automatic parking and a storage medium, and relates to the technical field of parking, and the driving area visualization method for automatic parking comprises the following steps: obtaining vehicle-mounted camera data, radar point cloud data, vehicle size and safety distance, a target obstacle recognition model based on an encoder-decoder and a self-attention module is used for analyzing camera data, and the initial position, size and type of an obstacle are determined. Then, the positions are adjusted through radar data to obtain accurate real-time obstacle positions, and a drivable area is constructed in combination with the vehicle size and the safe distance. And finally, the system sends camera data, radar data and drivable area information to a vehicle controller, the vehicle controller reconstructs an automatic parking scene by using a scene restoration model, and a result is transmitted to a vehicle display to present an automatic parking view. According to the invention, the automatic parking scene view can be displayed, the driving area can be displayed, and the collision risk is reduced.

Description

Technical Field

[0001] The present application relates to the field of parking technology, and in particular to a method, device, equipment and storage medium for visualizing a drivable area of ​​automatic parking. Background Art

[0002] With the acceleration of urbanization and the increase in the number of cars, the problem of tight parking spaces and small parking spaces has become increasingly prominent. As a solution that can reduce the parking burden of drivers and improve parking efficiency, the demand for automatic parking technology is growing. In complex parking environments, drivers need to accurately identify the drivable area around the vehicle to ensure safe and accurate parking operations.

[0003] Existing automatic parking technologies mainly rely on environmental perception and data fusion technologies, which collect environmental information around the vehicle through on-board cameras, radars and other sensors, construct environmental information of the parking area, and determine at least one initial custom parking space in the parking area. For example, a method for automatic parking based on a custom parking space determined by a parking area (publication number CN118387088A) and a multi-sensor fusion intelligent parking system and method (publication number CN112180373B) both use these technologies to achieve the function of automatic parking.

[0004] Although existing practices can build parking area information inside the intelligent driving controller, this information is often not intuitively presented to the driver, resulting in the driver being unable to intuitively feel the vehicle's perception range and drivable area. This may cause the parking control system to be unable to accurately identify the surrounding environment in complex environments, thereby affecting driving safety. In addition, due to the lack of intuitive visual display, the driver cannot intervene manually in time when there is a risk of collision, increasing the risk of collision during automatic parking. Therefore, how to display the drivable area of ​​automatic parking to avoid collisions with obstacles has become an urgent problem to be solved.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The purpose of the present application is to provide a method, device, equipment and storage medium for visualizing the drivable area of ​​automatic parking, aiming to solve the technical problem of how to display the drivable area of ​​automatic parking to avoid collision with obstacles.

[0007] To achieve the above objectives, the present application proposes a method for visualizing a drivable area of ​​automatic parking, the method comprising:

[0008] Obtain vehicle camera data, radar point cloud data, vehicle size, and vehicle safety distance;

[0009] Identify the vehicle-mounted camera data by using a target obstacle recognition model to obtain a first position, size and type of the obstacle in the image, wherein the target obstacle recognition model is constructed based on an encoder, a decoder and a self-attention module;

[0010] Adjusting the first position according to the radar point cloud data to obtain a real-time obstacle position;

[0011] Constructing a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image;

[0012] The vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area are sent to a vehicle controller, so that the vehicle controller restores the automatic parking scene according to a scene restoration model, and the restored scene data is sent to a vehicle display, so that the vehicle display displays an automatic parking scene restoration view according to the scene data.

[0013] In one embodiment, the step of adjusting the first position according to the radar point cloud data to obtain a real-time obstacle position includes:

[0014] Fitting the radar point cloud data by a random sampling consistency strategy to identify a second position of a static obstacle in the point cloud data;

[0015] Clustering the radar point cloud data by a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data;

[0016] The obstacle in the image is spatially aligned with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position.

[0017] In one embodiment, the step of fitting the radar point cloud data by a random sampling consistency strategy to identify the second position of the static obstacle in the point cloud data includes:

[0018] Extract all points from the radar point cloud data to obtain an initial point set;

[0019] Selecting a random point pair from the initial point set;

[0020] Fitting an initial obstacle geometry model according to the random point pairs;

[0021] Calculating the distances from all points to the initial obstacle geometry model;

[0022] Dividing the initial point set according to the distance to obtain an inner point set;

[0023] The target obstacle geometric model is obtained by maximizing the interior point set by selecting the random point pairs multiple times;

[0024] A second position of the static obstacle is determined according to the target obstacle geometric model.

[0025] In one embodiment, the step of clustering the radar point cloud data by a density clustering strategy to identify the third position of the dynamic obstacle in the point cloud data includes:

[0026] Defining a neighborhood radius and a minimum number of points according to distribution characteristics of point clouds in the radar point cloud data;

[0027] Clustering each point in the radar point cloud data according to the neighborhood radius and the minimum number of points to obtain a point cloud cluster;

[0028] Determining dynamic obstacles based on changes in consecutive frames of the radar point cloud data;

[0029] The dynamic obstacle in the continuous frames is dynamically tracked by Kalman filtering to obtain a third position of the dynamic obstacle.

[0030] In one embodiment, the step of spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain the real-time obstacle position includes:

[0031] Get camera parameters and radar parameters;

[0032] The ground coordinate system is constructed with the center of the rear axle of the vehicle as the origin, the forward direction of the vehicle as the positive direction of the horizontal axis, and the left side of the vehicle as the positive direction of the vertical axis.

[0033] According to the camera parameters, the two-dimensional coordinates of the first position in the image are converted into three-dimensional coordinates in the ground coordinate system by inverse perspective transformation to obtain first coordinates;

[0034] Rotate and displace the second position and the third position according to the radar parameters and the ground coordinate system to obtain a second coordinate and a third coordinate;

[0035] The first coordinate, the second coordinate, and the third coordinate in the ground coordinate system are converted to a bird's-eye view to obtain a real-time obstacle position.

[0036] In one embodiment, the target obstacle recognition model includes an encoder, a decoder, and a self-attention module, the encoder includes a convolution layer and a pooling layer, and the step of identifying the vehicle camera data through the target obstacle recognition model to obtain the first position, size, and type of the obstacle in the image includes:

[0037] De-noising, light compensation and standardization are performed on the image in the vehicle-mounted camera data to obtain target image data;

[0038] Processing the target image data by the encoder and the decoder to obtain a first position of an obstacle in the image;

[0039] Performing a convolution operation on the target image data through the convolution layer to obtain a feature map;

[0040] Performing dimensionality reduction on the feature map through the pooling layer to obtain a low-dimensional feature map;

[0041] Performing weighted aggregation on the low-dimensional feature map through the self-attention module to obtain a weighted feature map;

[0042] Performing bounding box regression on the weighted feature map to obtain the size of the obstacle;

[0043] Performing network encoding on the weighted feature map through the self-attention module to obtain a classification network;

[0044] The classification network is processed by a smooth maximum function to obtain the type of the obstacle.

[0045] In one embodiment, the step of processing the target image data by the encoder and the decoder to obtain the first position of the obstacle in the image includes:

[0046] Mapping each pixel in the target image data to a depth value by the encoder;

[0047] Upsampling and skip-connecting the target image data according to the depth value by the decoder to obtain a depth map;

[0048] The first position of the obstacle in the image is obtained according to the distance information between each pixel in the depth map and the camera.

[0049] In addition, to achieve the above-mentioned purpose, the present application also proposes a drivable area visualization device for automatic parking, the device comprising:

[0050] Data acquisition module, used to obtain vehicle camera data, radar point cloud data, vehicle size and vehicle safety distance;

[0051] An obstacle recognition module, used to recognize the vehicle-mounted camera data through a target obstacle recognition model to obtain a first position, size and type of an obstacle in the image, wherein the target obstacle recognition model is constructed based on an encoder, a decoder and a self-attention module;

[0052] A position determination module, used to adjust the first position according to the radar point cloud data to obtain a real-time obstacle position;

[0053] A drivable area construction module, used to construct a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image;

[0054] A view restoration module is used to send the vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area to the vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and sends the restored scene data to the vehicle display, so that the vehicle display displays the automatic parking scene restoration view according to the scene data.

[0055] In addition, to achieve the above-mentioned purpose, the present application also proposes a device for visualizing the drivable area of ​​automatic parking, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the method for visualizing the drivable area of ​​automatic parking as described above.

[0056] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for visualizing the drivable area of ​​automatic parking as described above are implemented.

[0057] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the method for visualizing the drivable area of ​​automatic parking as described above.

[0058] One or more technical solutions proposed in this application have at least the following technical effects:

[0059] Obtain vehicle camera data, radar point cloud data, vehicle size and vehicle safety distance; identify the vehicle camera data through the target obstacle recognition model to obtain the first position, size and type of the obstacle in the image, the target obstacle recognition model is constructed based on the encoder, decoder and self-attention module; adjust the first position according to the radar point cloud data to obtain the real-time obstacle position; construct a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position and the size and type of the obstacle in the image; send the vehicle camera data, the radar point cloud data and the data corresponding to the drivable area to the vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and send the restored scene data to the vehicle display, so that the vehicle display displays the automatic parking scene restoration view according to the scene data. The parking control system first collects the vehicle camera data and radar point cloud data, and records the vehicle size and safety distance at the same time, providing the system with comprehensive environmental information to ensure accurate perception of the vehicle's surrounding environment. Next, the system uses the target obstacle recognition model constructed by the encoder, decoder and self-attention module to process the camera data, identify the location, size and type of the obstacle, accurately identify and classify the obstacle, and provide key information for avoiding collisions and planning parking paths. Then, the system compares the radar point cloud data with the obstacle position identified by the camera, corrects the real-time position of the obstacle, improves the accuracy of the obstacle position data, and ensures the safety and reliability of the parking control system. On this basis, the system combines the vehicle size, safety distance, real-time obstacle position, obstacle size and type to calculate the vehicle's drivable area, provide a safe driving space for automatic parking, avoid collisions with obstacles, and ensure the feasibility of the parking path. Finally, the system sends the on-board camera data, radar point cloud data and drivable area data to the car controller. The controller reconstructs the automatic parking scene according to the scene restoration model, and sends the restored scene data to the car display to display the view of the automatic parking scene, improve the transparency and safety of the operation, and enhance the driver's trust in the parking control system. The entire process greatly improves the accuracy, safety and user-friendliness of the parking control system through precise environmental perception, obstacle identification, position correction, drivable area construction and scene restoration. The parking control system collects on-board camera data and radar point cloud data, combined with vehicle size and safety distance information, to provide the system with comprehensive surrounding environment perception, ensuring that obstacles around the vehicle can be accurately identified. The target obstacle recognition model built using encoders, decoders and self-attention modules further processes camera data to accurately identify and classify obstacles, thereby providing key information for collision avoidance and parking path planning. The system then corrects the real-time position of obstacles based on radar point cloud data, further improving the accuracy of position data and enhancing the safety and reliability of the system.On this basis, the system comprehensively considers the vehicle size, safety distance, real-time obstacle position, obstacle size and type, calculates the vehicle's drivable area, ensures the feasibility of the parking path and avoids collisions with obstacles. Finally, the system sends this data to the vehicle controller, which reconstructs the automatic parking scene based on the scene restoration model and sends the restored scene data to the vehicle display to intuitively display the view of the automatic parking scene. This not only improves the transparency and safety of the operation and enhances the driver's trust in the parking control system, but also can display the drivable area of ​​automatic parking to avoid collisions with obstacles, significantly improving the human-computer interaction performance and safety of automatic parking, and reducing the risk of collision when using automatic parking. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0062] Figure 1 A flowchart of Embodiment 1 of the method for visualizing a drivable area of ​​automatic parking of the present application;

[0063] Figure 2 A flowchart diagram of Embodiment 2 of the method for visualizing a drivable area of ​​automatic parking of the present application;

[0064] Figure 3 This is a schematic diagram of the module structure of the device for visualizing the drivable area of ​​automatic parking according to an embodiment of the present application;

[0065] Figure 4 Schematic diagram of the device structure of the hardware operating environment involved in the method for visualizing the drivable area of ​​automatic parking in an embodiment of the present application.

[0066] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0068] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0069] With the growth of urbanization and car ownership, the problem of tight parking spaces and small spaces has become more serious. Therefore, automatic parking technology has become an important solution to improve parking efficiency and reduce the burden on drivers. The existing practice mainly collects environmental information through sensors such as on-board cameras and radars to build parking areas and determine custom parking spaces. However, although these technologies can build parking area information in the intelligent driving controller, they fail to present it to the driver intuitively, resulting in the opacity of the perception range and drivable area of ​​the parking control system in complex environments, increasing the risk of collision, affecting driving safety and the driver's trust in the system.

[0070] The main solution of the embodiment of the present application is: the parking control system collects the point cloud data of the vehicle camera and radar, combined with the vehicle size and safety distance information, to fully perceive the surrounding environment. The model built using the encoder-decoder and self-attention module processes the camera data to accurately identify the location, size and type of obstacles, and corrects these positions through radar data to improve accuracy. Based on this, the system calculates the drivable area to ensure the safety and feasibility of the parking path. Finally, the system sends the data of the camera, radar and drivable area to the vehicle controller to reconstruct and display the automatic parking scene.

[0071] It should be noted that the execution subject of the embodiments of the present application may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a parking controller, a parking control system, etc. The following takes the parking control system as an example to illustrate this embodiment and the following embodiments.

[0072] Based on this, the embodiment of the present application provides a method for visualizing the drivable area of ​​automatic parking, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for visualizing the drivable area of ​​automatic parking of the present application.

[0073] In this embodiment, the method for visualizing the drivable area of ​​automatic parking includes steps S10 to S50:

[0074] Step S10, obtaining vehicle camera data, radar point cloud data, vehicle size, and vehicle safety distance;

[0075] It should be noted that the vehicle-mounted camera data refers to the visual information captured by the camera installed on the vehicle. In this embodiment, the cameras used include four surround fisheye cameras on the front, rear, left and right of the vehicle, and two wide-angle cameras on the front and rear windshields of the vehicle. These cameras can provide image data of the vehicle's surrounding environment for identifying obstacles, road signs, traffic signals, etc. Camera data is an important part of environmental perception. By processing and analyzing these image data through deep learning algorithms, the location and type of obstacles can be identified, providing visual perception information for the parking control system. Radar point cloud data is data collected by radar sensors on the vehicle. These data form a three-dimensional point cloud map of the vehicle's surrounding environment. The radar in this embodiment includes a total of twelve ultrasonic radars and five millimeter-wave radars on the front, rear, left and right sides of the vehicle body. Ultrasonic radars detect the distance and position of obstacles by emitting ultrasonic waves and receiving their echoes, while millimeter-wave radars use radio waves for more accurate distance, speed and angle measurements. The data of these radar sensors can provide accurate location information of obstacles around the vehicle, which is essential for building a three-dimensional environmental model around the vehicle. Vehicle size refers to the physical size of the vehicle, including the length, width and height of the vehicle. During the automatic parking process, understanding the size of the vehicle is crucial to determining whether the vehicle can fit into a specific parking space and whether it will collide with other objects during the parking process. The vehicle size data helps the parking controller calculate the vehicle's drivable area and operating space to ensure the safety and feasibility of the parking operation. The vehicle safety distance refers to the minimum distance that should be maintained between the vehicle and surrounding obstacles to ensure safety. This distance takes into account the possible range of movement of the vehicle during driving and parking, as well as unexpected situations that may occur due to operational errors or environmental changes. In this embodiment, the data on the vehicle safety distance helps the parking controller determine the boundaries of the vehicle's drivable area, and marks the non-drivable area in red on the display screen to alert the driver. The setting of the safety distance can reduce the risk of collision and improve the safety of automatic parking.

[0076] It can be understood that, firstly, the acquisition of vehicle camera data is achieved through multiple cameras installed around the vehicle (including surround fisheye cameras and wide-angle cameras). These cameras can capture high-definition images of the vehicle's surroundings, and then extract the location and type information of obstacles through image processing technology. This can provide visual perception data for the parking controller and help the system understand the layout of the environment around the vehicle; secondly, the acquisition of radar point cloud data depends on the ultrasonic radar and millimeter wave radar equipped on the vehicle. These radars measure the precise distance and speed between the vehicle and the surrounding obstacles by transmitting and receiving signals. This can provide more accurate obstacle location information and enhance the system's ability to perceive the environment; then, the acquisition of vehicle size requires measuring the length, width and height of the vehicle. These data are crucial for the parking controller to calculate the vehicle's operable space during parking, ensuring that the parking operation does not exceed the physical limitations of the vehicle; finally, the determination of the vehicle's safety distance is based on safety standards and vehicle operating characteristics, and a minimum distance is set to ensure that a sufficient safety interval is maintained between the vehicle and the obstacle during parking to avoid collision. The effect of this is to improve the safety and reliability of the parking control system and make the parking process smoother and safer. Through these steps, the parking control system can accurately identify and display the vehicle's drivable area, achieving safe and efficient automatic parking operations.

[0077] Step S20, identifying the vehicle camera data through a target obstacle recognition model to obtain a first position, size and type of the obstacle in the image, wherein the target obstacle recognition model is constructed according to an encoder, a decoder and a self-attention module;

[0078] It should be noted that the target obstacle recognition model is a deep learning model that combines the Bird's Eye View (BEV) and Transformer architectures to identify obstacles from vehicle camera data. This model is designed to improve the accuracy and efficiency of obstacle detection, especially in the application scenarios of autonomous driving and automatic parking. The following is a detailed explanation of the target obstacle recognition model, including its source, component layers or modules: (1) Encoder: Convolutional Layers: The initial stage of the encoder usually contains multiple convolutional layers, which are responsible for extracting features from the input image data. The convolutional layer slides over the image through filters (or convolution kernels) to capture local features such as edges, textures, etc. These features are then processed by activation functions (such as ReLU) to introduce nonlinearity, allowing the model to learn more complex feature representations. Pooling Layers: Immediately following the convolutional layers are pooling layers, which are used to reduce the spatial dimension of the feature map, that is, reduce the number of parameters and the amount of computation, while improving the invariance of the features (such as translation invariance). The most common pooling operation is max pooling, which is implemented by selecting the maximum value within a region. Feature fusion: After multiple convolution and pooling layers, there may be feature fusion operations, such as merging feature maps at different levels through skip connections to retain more spatial information. BEV conversion: The output of the encoder is a series of feature maps, which are then converted to a bird's-eye view (BEV) representation to provide input for the subsequent Transformer encoder. (2) Transformer encoder: Self-attention module: The self-attention module is a mechanism that allows the model to pay attention to information at different positions in the sequence when processing sequence data. In the object obstacle recognition model, the self-attention module can help the model better understand the relationship between different regions in the image, allowing the model to consider global information when processing features and improve the accuracy of obstacle recognition. Multi-head attention: The self-attention module encoder usually contains multi-head attention, which means that different representations of features are processed in parallel to capture dependencies at different scales. Feedforward network: After the self-attention layer, the self-attention module also includes a feedforward network, which further performs nonlinear transformations on the output of the self-attention layer. (3) Decoder: Output of the encoder: The decoder receives the BEV feature representation from the Transformer encoder and begins to reconstruct the spatial position and size information of the obstacle. Upsampling: The decoder may contain upsampling or transposed convolutional layers to increase the spatial resolution of the feature map in order to more accurately locate obstacles.Feature refinement: Layers in the decoder can further refine the features to predict the precise location, size, and type of the obstacle. (4) Output layer: Finally, the output of the decoder passes through one or more fully connected layers to generate the obstacle’s category, location (in the BEV coordinate system), size, and possibly other attributes (such as speed or direction).

[0079] Obstacles refer to all objects that may affect vehicle driving and parking operations, including but not limited to other vehicles, pedestrians, bicycles, pillars, walls, traffic signs, etc. The first position refers to the position of the obstacle in the image, that is, the coordinates of the obstacle on the two-dimensional image plane, usually expressed in pixel values, which is the first step in determining the spatial position of the obstacle in the obstacle recognition process. Size refers to the size of the obstacle in the image, that is, the size of the pixel area occupied by the obstacle, which helps the system determine the actual size of the obstacle and thus evaluate its potential impact on parking operations. Type refers to the type or category of the obstacle, such as vehicles, pedestrians, bicycles, etc. Identifying the type of obstacle is crucial for the parking control system to make correct driving decisions.

[0080] It can be understood that, first, the system processes the image captured by the on-board camera through the convolution layer of the encoder part of the target obstacle recognition model. The convolution layer uses a series of learnable filters to slide on the image to extract local features such as edges and textures. This can capture the basic patterns in the image and lay the foundation for subsequent recognition; then, the feature map output by the convolution layer is downsampled through the pooling layer to reduce the dimension of the feature map and extract the most important information. This helps to reduce the amount of calculation and improve the model's robustness to position changes in the image; then, the system converts the extracted feature map into a bird's-eye view (BEV) representation through the model, which can provide a bird's-eye view from above. The model observes the two-dimensional plane view of the vehicle and obstacles from a perspective, allowing the model to have a more comprehensive understanding of the environment around the vehicle; then, the Transformer encoder uses the self-attention mechanism to process the BEV features. By calculating the attention weight of each position in the feature map to all other positions, the model can capture the global relationship between obstacles, which can improve the model's understanding of complex scenes; finally, the decoder part gradually upsamples and refines the processed feature map, and finally outputs the precise position, size and type of the obstacle, which can provide accurate obstacle information for the parking control system, allowing the system to safely and accurately avoid obstacles and achieve efficient parking. Through this series of steps, the system can accurately identify the key information of obstacles from the on-board camera data, providing important environmental perception capabilities for autonomous driving and automatic parking.

[0081] As an example, the target obstacle recognition model includes an encoder, a decoder and a self-attention module, the encoder includes a convolution layer and a pooling layer, and the step of identifying the vehicle camera data through the target obstacle recognition model to obtain the first position, size and type of the obstacle in the image includes: denoising, illumination compensation and standardization of the image in the vehicle camera data to obtain target image data; processing the target image data through the encoder and the decoder to obtain the first position of the obstacle in the image; performing a convolution operation on the target image data through the convolution layer to obtain a feature map; reducing the dimension of the feature map through the pooling layer to obtain a low-dimensional feature map; performing weighted aggregation on the low-dimensional feature map through the self-attention module to obtain a weighted feature map; performing bounding box regression on the weighted feature map to obtain the size of the obstacle; performing network encoding on the weighted feature map through the self-attention module to obtain a classification network; processing the classification network through a smooth maximum function to obtain the type of the obstacle.

[0082] Denoising refers to the process of reducing or eliminating noise in an image. Noise may be caused by camera sensors, environmental factors, or interference during transmission. Denoising smoothes the image through algorithms (such as median filtering, Gaussian filtering, etc.), reduces randomly changing pixel values, improves image quality, and provides clearer image data for subsequent feature extraction. Lighting compensation refers to adjusting the brightness and contrast of the image to reduce the impact of image quality changes under different lighting conditions. Since the images captured by the camera under different lighting conditions may have uneven brightness or color distortion, lighting compensation improves the lighting conditions of the image through algorithms (such as histogram equalization) to make feature extraction more accurate. Standardization refers to converting image data into a unified format or range, such as scaling pixel values ​​to between 0 and 1. This can speed up the convergence of model training and improve the adaptability of the model to different cameras and environmental conditions. Target image data refers to image data after denoising, lighting compensation, and standardization. These data are the preprocessing results of the model input and provide high-quality image input for subsequent feature extraction and obstacle recognition. The first position refers to the position of the obstacle in the image, that is, the two-dimensional coordinates of the obstacle. Through the processing of the encoder and decoder, the model can determine the approximate position of the obstacle in the image. The convolution operation is the basic operation in the convolutional neural network. By sliding the convolution kernel (filter) to cover each local area of ​​the image, calculating the dot product between the convolution kernel and the covered area, and generating a new feature map, this process can extract local features of the image, such as edges, textures, etc. The feature map is the output of the convolution operation, which represents the features extracted from the image. Each convolution layer may produce multiple feature maps, which capture the features of the image from different angles. Dimensionality reduction refers to reducing the dimension of the data. The pooling layer reduces the spatial size of the feature map through operations (such as maximum pooling and average pooling) while retaining the most important information. Low-dimensional feature maps are feature maps after dimensionality reduction by the pooling layer. They have fewer spatial dimensions but still contain key information of the image. Weighted aggregation means that the self-attention module gives different weights according to the importance of different parts of the feature map, and then weighted sums these features to highlight important features and suppress unimportant features. The weighted feature map is the output of the self-attention module. It contains weighted aggregated features that are more prominent and helpful for subsequent obstacle recognition. Bounding box regression refers to predicting the bounding box of an obstacle, that is, the exact position and size of the obstacle in the image, through a regression algorithm (such as linear regression). Network coding refers to the further processing of the weighted feature map by the self-attention module to generate coded features for classification. The classification network is a network composed of a series of neural network layers, which is used to classify obstacles, that is, determine the type of obstacles. The smooth maximum function (such as the softmax function) is an activation function used to convert the output of the classification network into a probability distribution, representing the confidence of different categories.

[0083] First, the parking control system performs denoising on the raw image data captured by the on-board camera, and reduces random noise in the image through algorithms (such as Gaussian filtering). This can improve the image quality and provide clearer image data for subsequent feature extraction. Secondly, the system performs illumination compensation and adjusts the brightness and contrast of the image to reduce the impact of image quality changes under different lighting conditions. This helps the model to stably identify obstacles in different environments. Next, the system normalizes the image and scales the pixel values ​​to a uniform range (such as 0 to 1). This can speed up the convergence of model training and improve the adaptability of the model to different cameras and environmental conditions. Then, the parking control system uses the convolution layer in the encoder to perform convolution operations on the preprocessed image data, and extracts local features in the image, such as edges and textures, through sliding filters. This can capture the basic patterns in the image and lay the foundation for subsequent obstacle recognition. Next, the system reduces the dimension of the feature map through the pooling layer, and reduces the spatial size of the feature map by selecting the maximum value or average value in the region. This helps to extract more abstract features while reducing the amount of calculation. Next, the system uses the self-attention module to perform weighted aggregation on the low-dimensional feature map, giving different weights according to the importance of different parts of the feature map. This can highlight important features and suppress unimportant features, improving the model's ability to identify obstacles. Subsequently, the system performs bounding box regression on the weighted feature map, and predicts the bounding box of the obstacle, that is, the exact position and size of the obstacle in the image through the algorithm. This can provide accurate spatial information for parking operations. Finally, the parking control system uses the self-attention module to perform network encoding on the weighted feature map, build a classification network, and then process the output of the classification network through a smooth maximum function (such as softmax) to convert the original neural network response into a probability distribution, thereby identifying the type of obstacle. This can provide the parking system with specific information about the identity of the obstacle, allowing the system to make more accurate parking decisions. Through this series of detailed processing steps, the parking control system can accurately identify the first position, size and type of obstacles in the image from the on-board camera data, providing key environmental perception capabilities for automatic parking.

[0084] As an example, the step of processing the target image data through the encoder and the decoder to obtain the first position of the obstacle in the image includes: mapping each pixel in the target image data to a depth value through the encoder; upsampling and jump-connecting the target image data according to the depth value through the decoder to obtain a depth map; and obtaining the first position of the obstacle in the image based on the distance information between each pixel in the depth map and the camera.

[0085] The depth value refers to the depth or distance of each pixel relative to the camera in three-dimensional space. In the parking control system, the encoder predicts the depth value of each pixel by analyzing the target image data and using the features learned by the neural network. This value represents the distance from the camera to the obstacle surface. Upsampling is a process of increasing the resolution of data, usually used in the decoder stage. In deep learning, upsampling increases the size of the feature map through interpolation methods (such as nearest neighbor interpolation, bilinear interpolation, etc.) to restore the resolution close to the original input image, providing more detailed information for subsequent depth map generation and bounding box positioning. Skip connection (also known as Skip Connection) is a structure that connects different layers in the network. It directly connects the feature map of the shallower layer in the encoder to the feature map of the deeper layer in the decoder. This prevents information from being lost when it is transmitted in the deep network, helps the network learn richer features, and improves the training stability and performance of the model. The depth map is a two-dimensional image in which each pixel value represents the depth value of the point to the camera. The depth map provides an intuitive representation of the distance of each point in the scene, which is very useful for determining the three-dimensional position and shape of obstacles. Distance information refers to the depth value corresponding to each pixel in the depth map, that is, the actual physical distance from each pixel to the camera. This information is obtained through encoder prediction and decoder upsampling and jump connection reconstruction. It directly reflects the distribution of obstacles in space and is the key data for the parking control system to perform accurate obstacle detection and avoidance.

[0086] First, the encoder part of the parking control system processes the target image data through a deep convolutional neural network. The specific operation is to extract features through multiple convolutional layers, and then use the pooling layer to reduce the spatial dimension of the features. This allows the network to capture the key information in the image and map each pixel to a depth value, that is, predict the depth of each pixel from the camera. The effect of this is to provide the three-dimensional spatial position information of each pixel for subsequent steps. Secondly, the decoder part receives the depth value output by the encoder, increases the size of the feature map through upsampling operations, and uses jump connections to combine the deep features in the encoder with the shallow features in the decoder. Upsampling is usually achieved through transposed convolution or interpolation algorithms, and jump connections directly connect the outputs of certain layers in the encoder to the corresponding layers of the decoder. The effect of this is to restore and enhance the detail information of the feature map while maintaining the spatial resolution of the feature map, thereby generating an accurate depth map. Finally, based on the depth value of each pixel in the depth map, that is, the distance information between each pixel and the camera, the parking control system can determine the first position of the obstacle in the image, that is, the two-dimensional coordinate position of the obstacle on the image plane. The effect of this is to provide the parking control system with accurate obstacle position information, so that the system can effectively identify and avoid obstacles during the parking process, ensuring the safety and accuracy of the parking process.

[0087] First, the parking control system processes the target image data captured by the on-board camera through the encoder part of the target obstacle recognition model. Specifically, the encoder uses convolutional layers to extract features in the image, such as edges, textures, etc., and then gradually reduces the spatial dimensions of the feature map through pooling layers. This can extract key information from the image and encode it into a more compact deep feature representation. At the same time, the encoder maps each pixel to a depth value, that is, predicts the depth of each pixel from the camera. This is done to provide the system with three-dimensional information about obstacles in the scene. Secondly, the parking control system uses the decoder part of the target obstacle recognition model to process the deep features output by the encoder. The decoder increases the size of the feature map through upsampling operations and restores the spatial resolution of the image. This allows the depth information to match the resolution of the original image. At the same time, the decoder uses jump connections to combine the deep features in the encoder with the shallow features in the decoder. This can retain more detail information and enhance the semantic information of the feature map, and finally generate an accurate depth map. The depth map contains the distance information between each pixel and the camera. Finally, the parking control system determines the first position of the obstacle in the image based on the depth value of each pixel in the depth map, that is, the distance information from the camera. Specifically, the system analyzes the depth values ​​in the depth map and identifies the pixels within a certain range from the camera, which form the outline of the obstacle. By analyzing these outlines, the system can determine the two-dimensional coordinate position of the obstacle in the image. This is done to provide the parking control system with accurate obstacle location information, allowing the system to effectively identify and avoid obstacles during parking, ensuring the safety and accuracy of the parking process.

[0088] Step S30, adjusting the first position according to the radar point cloud data to obtain a real-time obstacle position;

[0089] It should be noted that adjustment refers to the process of correcting and optimizing the first position of the obstacle previously obtained through the camera data and the target obstacle recognition model based on the radar point cloud data. Specifically, since the camera data may be affected by factors such as viewing angle, lighting, and weather, the obstacle position obtained by relying solely on image data may have certain errors. The radar point cloud data provides more accurate distance and speed information. Therefore, the parking control system will fuse the radar point cloud data with the camera recognition results, and fine-tune the position of the obstacle through comparison and verification to ensure the accuracy of the obstacle position. This process involves data association and state estimation algorithms, such as Kalman filtering or particle filtering, which can combine the advantages of the two sensor data to provide more reliable obstacle position estimation. The real-time obstacle position refers to the precise position information of the obstacle at the current moment. This information is dynamically updated and reflects the latest state of the obstacle changing over time. Through the above adjustment process, the parking control system can obtain an obstacle position that combines the camera image data and radar point cloud data. This position information takes into account the current distance, speed and direction of the obstacle, so it can provide a more accurate real-time position.

[0090] It can be understood that, first, the parking control system collects radar point cloud data, which is obtained by the radar sensor transmitting signals and receiving reflected signals, and contains the distance and speed information of the obstacle; then, the system compares the radar point cloud data with the first position obtained by the camera data, and uses algorithms such as Kalman filtering, which is a recursive filter. It estimates the position of the obstacle through two steps of prediction and update. The prediction step uses the position and speed information of the previous moment to predict the current position, and the update step combines the radar point cloud data to correct the predicted position. This can reduce errors caused by camera viewing angle limitations or lighting changes and improve the accuracy of position estimation; finally, after adjustment, the system obtains the real-time obstacle position, which is dynamically updated and can accurately reflect the exact position of the obstacle at any given moment. This is done to provide the parking control system with accurate environmental perception information, so that it can navigate safely and avoid collisions, ensuring the safety and accuracy of the parking process.

[0091] Step S40, constructing a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image;

[0092] It should be noted that the drivable area refers to the space area where the vehicle can safely drive without colliding with obstacles, calculated in the parking control system based on the vehicle's size, safety distance, real-time obstacle location, and obstacle size and type. This area is usually determined by the vehicle's geometric size to determine its minimum turning radius and passing capacity, the safety distance to determine the buffer space that should be maintained between the vehicle and the obstacle, and the location and size of the obstacle to determine the area that needs to be avoided on the vehicle's driving path.

[0093] It is understood that first, the parking control system measures and records the dimensions of the vehicle, including length, width and height, which are used to determine the minimum space required for the vehicle to park. Specifically, the system inputs the vehicle dimension data into the parking algorithm, and the algorithm calculates the bounding box of the vehicle when turning and moving. This ensures that the physical limitations of the vehicle are taken into account when planning the path, avoiding planning an unfeasible path. Secondly, the system will draw a safety buffer area around the vehicle according to the requirements of the vehicle's safety distance. Specifically, the system will add additional space around the vehicle's bounding box. This space is set based on safety standards and vehicle operating characteristics, such as adding a safety distance of 50 cm on all sides of the vehicle. This can reduce the risk of collision with other objects during parking. Then, the system will use the real-time obstacle location, size and type information collected by radar and camera sensors to determine the specific location of the obstacle in space and the space occupied by the obstacle. Specifically, the system combines radar point cloud data and image recognition results to identify the exact location of the obstacle and mark the area occupied by the obstacle in the space around the vehicle. Finally, the parking control system constructs a drivable area based on the vehicle size, safety buffer area, and the space occupied by obstacles. The specific construction method is that the system first draws the vehicle's bounding box and safety buffer area on the two-dimensional or three-dimensional space map around the vehicle, then marks the location of the obstacle, and finally deducts the vehicle's bounding box, safety buffer area, and the space occupied by obstacles from the entire space map. The remaining part is the drivable area. The system can plan one or more paths for the vehicle to travel within this drivable area, ensuring that the vehicle can avoid obstacles and maintain a safe distance during parking, ultimately achieving safe and accurate automatic parking.

[0094] Step S50, sending the vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area to the vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and sending the restored scene data to the vehicle display, so that the vehicle display displays the automatic parking scene restoration view according to the scene data.

[0095] It should be noted that the vehicle control unit, also known as the vehicle control unit (VCU), is a central processing unit inside the vehicle that is responsible for coordinating and controlling the various electronic systems and functions of the vehicle. In the parking control system, the vehicle control unit receives data from the vehicle's cameras and radars, processes this information, and controls the vehicle's automatic parking operation based on the output of the scene restoration model. The scene restoration model is a software algorithm that uses data collected by the vehicle's sensors to build a virtual representation of the vehicle's surroundings. This model usually includes multiple modules: (1) Perception module: Integrates the vehicle's camera data and radar point cloud data to identify the location, size, and type of obstacles. (2) Positioning module: Determines the precise location of the vehicle in the environment. (3) Map building module: Creates or updates a map of the vehicle's surroundings based on sensor data. This model is trained using machine learning and computer vision technology to process and interpret sensor data and provide a deep understanding of the environment. The automatic parking scene refers to the actual environment in which the vehicle is performing the automatic parking operation, including the layout of the parking lot, the location of obstacles, the relative position of the vehicle, etc. Restoration refers to the process in which the vehicle controller processes sensor data according to the scene restoration model and constructs a virtual representation of the automatic parking scene. This process involves data fusion, processing and interpretation to generate an accurate environmental model. The restored scene data refers to the detailed information about the automatic parking scene obtained after processing by the vehicle controller, including the location of obstacles, the location of the vehicle, the drivable area, etc. These data provide the necessary environmental information for automatic parking. The vehicle display, also known as the display screen of the vehicle infotainment system, is a display device inside the vehicle that is used to display various information to the driver, such as navigation, vehicle status, entertainment information, etc. In the parking control system, it is used to display the restored view of the automatic parking scene. The automatic parking scene restoration view is a visual representation of the automatic parking scene displayed on the vehicle display. This view includes the location of the vehicle, the location of obstacles, the parking path, etc., to help the driver understand the vehicle environment and the operating status of the parking control system.

[0096] It is understandable that, first, the parking control system will aggregate the visual data captured by the on-board camera, the point cloud data collected by the radar sensor, and the calculated drivable area data, and then transmit them to the vehicle controller. After receiving these data, the vehicle controller uses the built-in scene restoration model, which includes multiple modules such as perception, positioning, and map construction, to comprehensively analyze and process the automatic parking scene. The scene restoration model identifies the location of obstacles and vehicles by fusing camera data and radar data, and constructs a detailed map of the vehicle's surrounding environment. The purpose of this process is to generate an accurate virtual environment model, that is, the restored scene data, which contains key information such as vehicle location, obstacle information, and drivable area. Finally, the vehicle controller sends these restored scene data to the vehicle display, which generates and displays the restored view of the automatic parking scene based on the received data. This view graphically shows the driver the parking environment and path of the vehicle, allowing the driver to intuitively understand the automatic parking process, improving the transparency and safety of the parking operation. Through this series of detailed steps, the parking control system can provide the driver with a clear and accurate view of the parking scene, making the automatic parking process more reliable and easy to understand.

[0097] This embodiment provides a method for visualizing a drivable area of ​​automatic parking, which obtains vehicle-mounted camera data, radar point cloud data, vehicle size, and vehicle safety distance; identifies the vehicle-mounted camera data through a target obstacle recognition model to obtain a first position, size, and type of an obstacle in the image, wherein the target obstacle recognition model is constructed based on an encoder, a decoder, and a self-attention module; adjusts the first position based on the radar point cloud data to obtain a real-time obstacle position; constructs a drivable area based on the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image; sends the vehicle-mounted camera data, the radar point cloud data, and the data corresponding to the drivable area to a vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and sends the restored scene data to a vehicle display, so that the vehicle display displays an automatic parking scene restoration view according to the scene data. The parking control system first collects the vehicle-mounted camera data and radar point cloud data, and records the vehicle size and safety distance at the same time, providing the system with comprehensive environmental information to ensure accurate perception of the vehicle's surrounding environment. Next, the system uses the target obstacle recognition model constructed by the encoder, decoder and self-attention module to process the camera data, identify the location, size and type of the obstacle, accurately identify and classify the obstacle, and provide key information for avoiding collisions and planning parking paths. Then, the system compares the radar point cloud data with the obstacle position identified by the camera, corrects the real-time position of the obstacle, improves the accuracy of the obstacle position data, and ensures the safety and reliability of the parking control system. On this basis, the system combines the vehicle size, safety distance, real-time obstacle position, obstacle size and type to calculate the vehicle's drivable area, provide a safe driving space for automatic parking, avoid collisions with obstacles, and ensure the feasibility of the parking path. Finally, the system sends the on-board camera data, radar point cloud data and drivable area data to the car controller. The controller reconstructs the automatic parking scene according to the scene restoration model, and sends the restored scene data to the car display to display the view of the automatic parking scene, improve the transparency and safety of the operation, and enhance the driver's trust in the parking control system. The entire process greatly improves the accuracy, safety and user-friendliness of the parking control system through precise environmental perception, obstacle identification, position correction, drivable area construction and scene restoration. The parking control system collects on-board camera data and radar point cloud data, combined with vehicle size and safety distance information, to provide the system with comprehensive surrounding environment perception, ensuring that obstacles around the vehicle can be accurately identified. The target obstacle recognition model built using encoders, decoders and self-attention modules further processes camera data to accurately identify and classify obstacles, thereby providing key information for collision avoidance and parking path planning.The system then corrects the real-time position of the obstacle based on the radar point cloud data, further improving the accuracy of the position data and enhancing the safety and reliability of the system. On this basis, the system comprehensively considers the vehicle size, safety distance, real-time obstacle position, obstacle size and type, calculates the vehicle's drivable area, ensures the feasibility of the parking path and avoids collisions with obstacles. Finally, the system sends this data to the vehicle controller, which reconstructs the automatic parking scene based on the scene restoration model and sends the restored scene data to the vehicle display to intuitively display the view of the automatic parking scene. This not only improves the transparency and safety of the operation and enhances the driver's trust in the parking control system, but also can display the drivable area of ​​​​automatic parking to avoid collisions with obstacles, significantly improving the human-computer interaction performance and safety of automatic parking, and reducing the risk of collision when using automatic parking.

[0098] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2 , Figure 2 This is a flow chart of a second embodiment of the method for visualizing a drivable area of ​​automatic parking of the present application. Step S30 of the method for visualizing a drivable area of ​​automatic parking includes steps S31 to S33:

[0099] Step S31, fitting the radar point cloud data by a random sampling consistency strategy to identify a second position of a static obstacle in the point cloud data;

[0100] It should be noted that the Random Sampling Consensus Strategy (RANSAC) is an iterative method for estimating the parameters of a mathematical model from a set of data containing outliers. In parking control systems, RANSAC is used to identify consistent geometric models, such as lines or planes, from radar point cloud data. These models can be used to describe the outline of obstacles. The RANSAC algorithm fits the model by randomly selecting a small number of data points, and then evaluates the fit of the model to the overall data, so as to identify which data points are outliers and which are consistent data points that conform to the model. Fitting here refers to using a mathematical model to match or approximate a set of points in the radar point cloud data. In parking control systems, fitting usually refers to determining one or more geometric models (such as planes or lines) so that these models can best represent a set of points in the point cloud data. The fitting process can help the system identify specific structures in the point cloud, such as the surface of the ground, wall or obstacle. Static obstacles refer to obstacles that do not change relative to the vehicle's position, such as parking piles, walls, other vehicles, etc. These obstacles will not move during the observation time of the radar sensor, so they can be stably detected and tracked through radar point cloud data. The second position refers to the precise position of the static obstacle identified after processing the radar point cloud data through the RANSAC algorithm. This position is more accurate and reliable than the initial "first position" (which may be obtained from camera data) because it is obtained by analyzing the consistent patterns in the radar point cloud data and can provide more accurate obstacle spatial position information.

[0101] It is understandable that, first of all, the parking control system will use a random sampling consensus strategy (RANSAC) to process the radar point cloud data. The specific approach is to randomly select a small part of the point cloud data as a sample set, and then try to fit these sample points with a preset geometric model (such as a plane or a straight line). This is done to identify the model parameters that best represent the static obstacles from the overall data; secondly, the system will evaluate the degree of fit of the model to all point cloud data, and distinguish which points conform to the model (i.e., consistent data points) and which points do not conform (i.e., outliers) by calculating the error from the data points to the model. This step is to eliminate the influence of noise and errors and ensure the accuracy of the model; finally, the system will repeat the above process many times, randomly selecting a new sample set each time to ensure the robustness of the model, and finally select the best fitting model, which can best represent the actual position and shape of the static obstacle, thereby determining the second position of the static obstacle, that is, a more accurate spatial position of the obstacle after algorithm optimization. In this way, even in the presence of noise and inaccurate data in the radar point cloud data, the system can accurately identify the location of static obstacles and provide precise environmental perception information to the parking control system, thereby improving the safety and accuracy of the parking process.

[0102] As an example, the step of fitting the radar point cloud data through a random sampling consistency strategy to identify the second position of the static obstacle in the point cloud data includes: extracting all points from the radar point cloud data to obtain an initial point set; selecting random point pairs from the initial point set; fitting an initial obstacle geometric model based on the random point pairs; calculating the distances from all points to the initial obstacle geometric model; dividing the initial point set according to the distances to obtain an inner point set; maximizing the inner point set by selecting the random point pairs multiple times to obtain a target obstacle geometric model; and determining the second position of the static obstacle based on the target obstacle geometric model.

[0103] All points refer to every data point in the radar point cloud data, which represent the various position information in the surrounding environment detected by the radar sensor. The initial point set refers to the set of all points extracted from the radar point cloud data, which is the starting point for the RANSAC algorithm analysis. A random point pair refers to two points randomly selected from the initial point set, which are used to start building a hypothetical geometric model. For example, when fitting a straight line, a random point pair can be two points on the straight line; when fitting a plane, a random point pair can be two non-collinear points on the plane. The initial obstacle geometry model refers to a hypothetical geometric model fitted based on the random point pair, which represents the possible shape of the obstacle, such as a straight line, a plane or other geometric shapes. The distance refers to the Euclidean distance from each point in the initial point set to the initial obstacle geometry model, usually measured by the straight line distance. Partitioning refers to dividing the points in the initial point set into two groups according to the calculated distance, points closer to the geometry model (considered to be points within the model) and points farther away (considered to be points outside the model). The internal point set refers to the set of points that are closer to the initial obstacle geometry model after division. These points are considered to be points that conform to the current geometry model, that is, they are within a certain error range of the model. The target obstacle geometry model refers to the geometric model that is finally determined by selecting random point pairs through multiple iterations and trying to maximize the internal point set each time (that is, finding the geometric parameters that can make the most points conform to the model). This model is considered to be the model that best represents the static obstacles in the data.

[0104] First, the system performs a data extraction operation to collect each scattered point in the radar point cloud data according to its coordinate position in space to form an initial point set. This is done to obtain a comprehensive, unprocessed data set of the surrounding environment, laying the foundation for subsequent analysis. Then, the system randomly selects two points from this initial point set to form a point pair, and uses these two points to define a hypothetical geometric model, such as a line or a plane. This step is to establish a preliminary model for subsequent geometric shape fitting; then, the system calculates the distance from all points in the initial point set to this hypothetical geometric model. This calculation is completed through mathematical formulas, such as the distance formula from point to line or point to plane. This can evaluate the proximity of each point to the model and provide a basis for subsequent point classification. Then, the system divides the initial point set into an inner point set and an outer point set based on these calculated distances. The inner point set contains those points that are closer to the model, that is, they are considered to be points within the model. This can distinguish which points are in line with the model and which are outliers or noise. Next, the system selects different random point pairs through multiple iterations, and each time tries to find the point pair that can maximize the inner point set. This process involves comparison and selection, and the purpose is to find an optimal model so that the most points conform to this model, which can improve the accuracy and robustness of the model. Finally, the system determines the second position of the static obstacle based on the final target obstacle geometry model. This position is obtained by analyzing the points in the inner point set. This can extract the accurate static obstacle position from the noisy radar data, provide accurate environmental perception data for the parking control system, and thus improve the safety and accuracy of the parking process.

[0105] Step S32, clustering the radar point cloud data by a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data;

[0106] It should be noted that density clustering strategies, such as the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm, are density-based clustering methods. This strategy divides data points into different clusters by identifying high-density areas separated by low-density areas. Specifically, the algorithm sets two parameters: neighborhood radius (eps) and a minimum number of points (minPts). For each point, the algorithm checks how many other points it has within the neighborhood radius. If this number exceeds the minimum number of points, the point is considered to be the core point of a cluster. The algorithm will expand the cluster based on these core points and identify boundary points and noise points. This method does not require the number of clusters to be specified in advance, can identify clusters of any shape, and has a certain robustness to outliers. Clustering refers to dividing the samples in the data set into several clusters, so that the samples in the same cluster have high similarity, while the samples between different clusters have low similarity. In radar point cloud data, clustering is to classify points that are close in space into one category, and these points usually represent the same physical entity. Dynamic obstacles refer to obstacles whose positions change within a certain period of time, such as pedestrians and moving vehicles. Compared with static obstacles, the position information of dynamic obstacles changes over time, so they need to be updated in real time. The third position refers to the precise position of dynamic obstacles identified after processing radar point cloud data through density clustering strategy. This position information is obtained based on the clustering results.

[0107] It can be understood that first, the system implements a density clustering strategy, which is to set two key parameters for the algorithm: the neighborhood radius and a minimum number of points. This step is to determine which points are close enough to each other to be considered part of the same cluster. Then, for each point in the radar point cloud data, the system calculates how many other points are within the set neighborhood radius. This step is done by comparing the distance of each point to its surrounding points, in order to find those points that are clustered together in space. If the number of points in the neighborhood of a point exceeds the minimum number of points, then the point is considered a core point, which can help identify dense areas in the point cloud. Next, the system expands outward to form clusters based on these core points, and at the same time identifies those boundary points at the edge of the cluster and noise points that do not belong to any cluster. This step is achieved by iteratively adding all points in the neighborhood of the core point (including non-core points) to the same cluster, in order to distinguish different dynamic obstacles. Finally, through this density-based clustering method, the system identifies the point set that belongs to dynamic obstacles in the radar point cloud data and determines the third position of these dynamic obstacles, that is, their precise coordinates in space. Doing so makes it possible to distinguish dynamic obstacles from complex point cloud data and accurately track their position changes, which is critical for parking control systems to safely navigate and make decisions in dynamic environments, as it provides accurate information about moving objects in the surrounding environment, allowing the system to react in time to avoid collisions.

[0108] As an example, the step of clustering the radar point cloud data by a density clustering strategy to identify the third position of the dynamic obstacle in the point cloud data includes: defining a neighborhood radius and a minimum number of points according to the distribution characteristics of the point cloud in the radar point cloud data; clustering each point in the radar point cloud data according to the neighborhood radius and the minimum number of points to obtain a point cloud cluster; determining a dynamic obstacle according to changes in continuous frames in the radar point cloud data; and dynamically tracking the dynamic obstacle in the continuous frames by Kalman filtering to obtain the third position of the dynamic obstacle.

[0109] Distribution characteristics refer to the relative position and density of each point in the radar point cloud data in space. These characteristics can help the system understand the degree of aggregation and distribution pattern of the point cloud, thereby providing a basis for selecting appropriate clustering parameters. Neighborhood radius is a parameter in the density clustering strategy, which defines the neighborhood range of a point, that is, all points within a certain distance from the point in space are considered to be neighbors of the point. In the DBSCAN algorithm, this parameter determines the range of neighboring points considered when finding core points. Minimum number of points is another parameter in the density clustering strategy, which specifies the minimum number of neighboring points required for a point to be identified as a core point. This parameter and neighborhood radius together determine the density threshold of clustering. Point cloud cluster refers to a set formed by grouping points close in space through a clustering algorithm. Each cluster represents an independent entity or area in the point cloud, such as an obstacle or part of the ground. Continuous frames refer to radar point cloud data captured continuously in a time series. These data are usually collected by radar sensors at different time points and are used to track the changes of dynamic obstacles over time. Kalman filtering is a recursive filter that uses a series of noisy measurement data to estimate the state of a dynamic system. In parking control systems, Kalman filtering is used to process radar sensor data to reduce noise and predict the position and velocity of dynamic obstacles. Dynamic tracking refers to the process of tracking dynamic obstacles in consecutive frames using algorithms such as Kalman filtering. This process includes predicting the next position of the obstacle, updating the prediction results, and adjusting the prediction error to achieve continuous estimation of the obstacle's motion state.

[0110] First, the system analyzes the distribution characteristics of the points in the radar point cloud data and determines the appropriate neighborhood radius and minimum number of points. Specifically, the system sets the neighborhood radius by counting the density and distribution range of the points in the point cloud. This value determines how close the points around each point are considered when clustering; at the same time, the system sets a minimum number of points. This value is used to determine how many points are needed in an area to consider it as a valid clustering area. This is done to accurately divide the points close in space into the same cluster in the subsequent steps. Secondly, the system clusters each point in the radar point cloud data to form a point cloud cluster. The specific operation is that for each point, the system checks how many points it has within the set neighborhood radius. If this number exceeds the minimum number of points, then this point is considered to be a core point, and then the system classifies this core point and all the points in its neighborhood into a cluster. This process is repeated until all points are assigned to a cluster or marked as noise. This is done to group points close in space, each cluster represents an obstacle or part of the ground, thereby identifying static environmental structures. Then, the system determines the dynamic obstacles based on the changes in the point cloud clusters in the continuous frames. Specifically, the system compares the radar point cloud data captured at different time points to observe which point cloud clusters have changed in position and shape. These changing clusters represent dynamic obstacles. This is done to distinguish obstacles that move over time, such as moving vehicles or walking pedestrians. Finally, the system dynamically tracks the dynamic obstacles in the continuous frames through Kalman filtering to obtain the third position of the dynamic obstacles. Specifically, the system uses the Kalman filter algorithm to predict the position of each dynamic obstacle at the next moment, and then when new radar point cloud data arrives, the system updates this prediction and adjusts the prediction error by comparing the predicted position with the actual observed position, thereby obtaining a more accurate obstacle position. This process can continuously track the movement of obstacles and provide accurate obstacle position information even when the obstacles move quickly or the sensor data is noisy, which is critical for parking control systems because they require real-time updated environmental information to avoid collisions and plan safe parking paths.

[0111] Step S33: spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position.

[0112] It should be noted that spatial alignment refers to the process of comprehensively processing obstacle position information from different data sources (such as on-board cameras, radar sensors) and different time points to ensure that they are consistent and accurate in the same coordinate system. Specifically, the system will compare and calibrate the first position obtained by camera data (obstacle position obtained by image recognition), the second position obtained by radar data (static obstacle position obtained by radar point cloud fitting) and the third position (dynamic obstacle position obtained by radar point cloud clustering). This process involves technologies such as coordinate transformation, data fusion and error correction, with the aim of obtaining a unified and accurate view of the obstacle's spatial position, which can ensure consistency between different sensor data and improve the accuracy and robustness of obstacle detection. Real-time obstacle position refers to the precise position information of the obstacle obtained by the system at the current moment after spatial alignment. This position information is dynamically updated and reflects the latest state of the obstacle changing over time. The real-time obstacle position includes not only the position of static obstacles, but also the position of dynamic obstacles (such as moving vehicles and walking pedestrians). This information is crucial for automatic parking because it enables the driver to respond quickly to adapt to the dynamic changes of obstacles and ensure parking safety.

[0113] It can be understood that, first, the system will identify the first position of the obstacle from the image data obtained by the on-board camera, which is done by the target obstacle recognition model, which analyzes the image and marks the outline and position of the obstacle; at the same time, the system obtains the second position of the static obstacle from the radar point cloud data through the random sampling consistency strategy, which identifies the precise position of the static obstacle by fitting the geometric model; in addition, the system will also obtain the third position of the dynamic obstacle from the radar point cloud data through the density clustering strategy, which distinguishes and tracks dynamic obstacles by identifying high-density areas in the point cloud. Secondly, the system spatially aligns the position data from these three different sources. The specific operation is to convert all the position data into the same coordinate system. This process involves a coordinate transformation algorithm to ensure that all obstacle position data are in the same reference frame; then, the system will use data fusion technology, such as Kalman filtering, which can combine data from multiple sensors, reduce uncertainty and error through prediction and update steps, and provide more accurate and reliable obstacle position information. Finally, the system obtains the real-time obstacle position, which is obtained through the above-mentioned data fusion and spatial alignment process, and contains the latest position information of static and dynamic obstacles. In this way, the system can provide a continuously updated and accurate environmental perception picture, allowing the driver to monitor the changes in the surrounding environment in real time, respond quickly, and effectively avoid collisions.

[0114] As an example, the step of spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position and the third position to obtain the real-time obstacle position includes: acquiring camera parameters and radar parameters; constructing a ground coordinate system with the center of the rear axle of the vehicle as the origin, the forward direction of the vehicle as the positive direction of the horizontal axis, and the left side of the vehicle body as the positive direction of the vertical axis; converting the two-dimensional coordinates of the first position in the image into three-dimensional coordinates in the ground coordinate system through an inverse perspective transformation according to the camera parameters to obtain the first coordinate; rotating and displacing the second position and the third position according to the radar parameters and the ground coordinate system to obtain the second coordinate and the third coordinate; converting the first coordinate, the second coordinate and the third coordinate in the ground coordinate system to a bird's-eye view to obtain the real-time obstacle position.

[0115] Camera parameters include internal parameters and external parameters. Internal parameters refer to the intrinsic characteristics of the camera, such as focal length, principal point (image center), distortion coefficient, etc. These parameters affect how the camera projects the three-dimensional world onto a two-dimensional image. External parameters refer to the position and posture of the camera in the vehicle, that is, the position and orientation of the camera relative to the vehicle coordinate system. Radar parameters refer to the characteristics of the radar sensor, such as detection range, angular resolution, signal frequency, etc., as well as the position and orientation of the radar sensor on the vehicle. The rear axle center of the vehicle refers to the center point of the rear axle of the vehicle. This point is usually used as the origin of the vehicle coordinate system to determine the position of the vehicle relative to the ground. The positive direction of the horizontal axis refers to the axis along the vehicle's forward direction in the vehicle coordinate system, usually defined as the positive direction of the x-axis. The positive direction of the vertical axis refers to the axis perpendicular to the vehicle's forward direction in the vehicle coordinate system, usually defined as the positive direction of the y-axis, pointing to the left side of the vehicle body. The ground coordinate system is a coordinate system fixed to the ground and used to describe the position of the vehicle and obstacles on the ground. This coordinate system helps to unify the data of the vehicle sensors into a common reference frame. Two-dimensional coordinates refer to the position on the image plane, usually expressed in pixel coordinates. The inverse perspective transformation is a mathematical transformation used to convert 2D coordinates in an image to 3D coordinates in the real world. This transformation needs to take into account the camera's viewpoint and the scene's geometry. 3D coordinates refer to positions in 3D space, usually expressed as X, Y, and Z coordinates. The first coordinate refers to the result of converting the 2D coordinates of the first position in the image to 3D coordinates in the ground coordinate system through the inverse perspective transformation. Rotation refers to adjusting the orientation of the radar data to align with the axes of the ground coordinate system based on the radar parameters and the ground coordinate system. Displacement transformation refers to adjusting the position of the radar data to correctly reflect it in the ground coordinate system based on the radar parameters and the ground coordinate system. The second and third coordinates refer to the positions of the second and third positions in the point cloud data in the ground coordinate system after rotation and displacement transformation. The bird's-eye view refers to the view from directly above, which helps provide a global, 2D view of the position of vehicles and obstacles on the ground.

[0116] First, the system collects specific parameters of the camera and radar, including the focal length, principal point position, distortion coefficient of the camera, and the detection range and angular resolution of the radar, which are crucial for subsequent data processing and conversion. This is done to ensure that the sensor data can be accurately converted into the vehicle's coordinate system, laying the foundation for subsequent spatial alignment. Secondly, the system uses the center of the vehicle's rear axle as the coordinate origin, the vehicle's forward direction as the positive direction of the horizontal axis (x-axis), and the left side of the vehicle body as the positive direction of the vertical axis (y-axis) to construct a ground coordinate system. This coordinate system is the basis for the vehicle to perceive the environment, and it provides a unified reference frame for the position of the vehicle and obstacles. Then, the system uses the camera parameters to convert the two-dimensional coordinates of the obstacles identified in the image into three-dimensional coordinates in the ground coordinate system through inverse perspective transformation. Inverse perspective transformation can convert two-dimensional image coordinates into actual three-dimensional world coordinates based on the camera's viewing angle and the known three-dimensional scene structure. This is done to eliminate perspective distortion in the image and obtain the actual position of the obstacle in the real world, that is, the first coordinate. Next, the system will rotate and translate the positions of static and dynamic obstacles detected by the radar according to the radar parameters and the ground coordinate system, and convert these positions from the radar coordinate system to the ground coordinate system to obtain the second and third coordinates. This is done to ensure that all sensor data are in the same coordinate system, which is convenient for subsequent data fusion and processing. Finally, the system will convert the first, second, and third coordinates in the ground coordinate system to a bird's-eye view, that is, view the position of the vehicle and obstacles from directly above to obtain the real-time obstacle position. This can intuitively display the relative position of the vehicle and obstacles on a unified two-dimensional plane, provide accurate environmental perception information for the parking control system, and thus improve the safety and accuracy of the parking process. In this way, the system can monitor and respond to changes in the surrounding environment in real time and effectively avoid collisions.

[0117] This embodiment fits the radar point cloud data through a random sampling consistency strategy to identify the second position of the static obstacle in the point cloud data; clusters the radar point cloud data through a density clustering strategy to identify the third position of the dynamic obstacle in the point cloud data; and spatially aligns the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain the real-time obstacle position. The system first fits the radar point cloud data using a random sampling consistency strategy, and determines the model that best fits the data by randomly selecting sample points and iterating multiple times. This process can identify the second position of the static obstacle in the radar point cloud data, that is, a more accurate three-dimensional spatial position, which improves the accuracy and robustness of the recognition of the static obstacle position, and can accurately identify it even when there is noise and outliers in the data. Next, the system uses a density clustering strategy to cluster the radar point cloud data, grouping the data points according to the density characteristics of the point cloud to form clusters, and identifying the third position of dynamic obstacles in the radar point cloud data. This step can effectively distinguish and track dynamic obstacles, and can keep tracking even when the obstacles move or change, providing real-time dynamic environment information for the parking control system. Finally, the system spatially aligns the first position (image recognition result), the second position (the precise position of the static obstacle) and the third position (the position of the dynamic obstacle) to obtain the real-time obstacle position. This step provides a unified, real-time updated obstacle position information. In this way, the parking control system can perceive the surrounding environment more accurately and improve the safety and reliability of parking operations.

[0118] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for visualizing the drivable area of ​​automatic parking in the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0119] This application also provides a device for visualizing the drivable area of ​​automatic parking, please refer to Figure 3 The automatic parking drivable area visualization device comprises:

[0120] The data acquisition module 10 is used to acquire vehicle camera data, radar point cloud data, vehicle size and vehicle safety distance;

[0121] An obstacle recognition module 20 is used to recognize the vehicle-mounted camera data through a target obstacle recognition model to obtain a first position, size and type of an obstacle in the image, wherein the target obstacle recognition model is constructed according to an encoder, a decoder and a self-attention module;

[0122] A position determination module 30, configured to adjust the first position according to the radar point cloud data to obtain a real-time obstacle position;

[0123] A drivable area construction module 40, configured to construct a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image;

[0124] The view restoration module 50 is used to send the vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area to the vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and sends the restored scene data to the vehicle display, so that the vehicle display displays the automatic parking scene restoration view according to the scene data.

[0125] In one embodiment, the position determination module 30 is further used to fit the radar point cloud data through a random sampling consistency strategy to identify a second position of a static obstacle in the point cloud data; cluster the radar point cloud data through a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data; and spatially align the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position.

[0126] In one embodiment, the position determination module 30 is further used to extract all points from the radar point cloud data to obtain an initial point set; select random point pairs from the initial point set; fit an initial obstacle geometry model based on the random point pairs; calculate the distances from all points to the initial obstacle geometry model; divide the initial point set according to the distances to obtain an inner point set; maximize the inner point set by selecting the random point pairs multiple times to obtain a target obstacle geometry model; and determine a second position of the static obstacle based on the target obstacle geometry model.

[0127] In one embodiment, the position determination module 30 is further used to define a neighborhood radius and a minimum number of points according to the distribution characteristics of the point cloud in the radar point cloud data; cluster each point in the radar point cloud data according to the neighborhood radius and the minimum number of points to obtain a point cloud cluster; determine a dynamic obstacle according to changes in consecutive frames in the radar point cloud data; and dynamically track the dynamic obstacle in the consecutive frames through Kalman filtering to obtain a third position of the dynamic obstacle.

[0128] In one embodiment, the position determination module 30 is also used to obtain camera parameters and radar parameters; construct a ground coordinate system with the center of the rear axle of the vehicle as the origin, the forward direction of the vehicle as the positive direction of the horizontal axis, and the left side of the vehicle body as the positive direction of the vertical axis; according to the camera parameters, the two-dimensional coordinates of the first position in the image are converted into three-dimensional coordinates in the ground coordinate system through an inverse perspective transformation to obtain a first coordinate; according to the radar parameters and the ground coordinate system, the second position and the third position are rotated and displaced to obtain a second coordinate and a third coordinate; the first coordinate, the second coordinate and the third coordinate in the ground coordinate system are converted to a bird's-eye view to obtain a real-time obstacle position.

[0129] In one embodiment, the obstacle recognition module 20 is further used to denoise, compensate for illumination, and standardize the image in the vehicle-mounted camera data to obtain target image data; process the target image data through the encoder and the decoder to obtain the first position of the obstacle in the image; perform a convolution operation on the target image data through the convolution layer to obtain a feature map; perform dimensionality reduction on the feature map through the pooling layer to obtain a low-dimensional feature map; perform weighted aggregation on the low-dimensional feature map through the self-attention module to obtain a weighted feature map; perform bounding box regression on the weighted feature map to obtain the size of the obstacle; perform network encoding on the weighted feature map through the self-attention module to obtain a classification network; and process the classification network through a smoothed maximum function to obtain the type of the obstacle.

[0130] In one embodiment, the obstacle recognition module 20 is further used to map each pixel in the target image data to a depth value through the encoder; upsample and jump-connect the target image data according to the depth value through the decoder to obtain a depth map; and obtain the first position of the obstacle in the image according to the distance information between each pixel in the depth map and the camera.

[0131] The automatic parking drivable area visualization device provided by the present application adopts the automatic parking drivable area visualization method in the above embodiment, and can solve the technical problem of how to display the automatic parking drivable area to avoid collision with obstacles. Compared with the prior art, the beneficial effects of the automatic parking drivable area visualization device provided by the present application are the same as the beneficial effects of the automatic parking drivable area visualization method provided by the above embodiment, and the other technical features of the automatic parking drivable area visualization device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0132] The present application provides a device for visualizing a drivable area for automatic parking, and the device for visualizing a drivable area for automatic parking includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for visualizing a drivable area for automatic parking in the above-mentioned embodiment one.

[0133] Reference below Figure 4 , which shows a schematic diagram of the structure of a drivable area visualization device suitable for implementing the automatic parking embodiment of the present application. The drivable area visualization device of the automatic parking embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The drivable area visualization device for automatic parking shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0134] like Figure 4As shown, the drivable area visualization device for automatic parking may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the drivable area visualization device for automatic parking are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the drivable area visualization device for automatic parking to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a drivable area visualization device for automatic parking with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0135] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0136] The automatic parking drivable area visualization device provided in the present application adopts the automatic parking drivable area visualization method in the above-mentioned embodiment, which can solve the technical problem of how to display the automatic parking drivable area to avoid collision with obstacles. Compared with the prior art, the beneficial effects of the automatic parking drivable area visualization device provided in the present application are the same as the beneficial effects of the automatic parking drivable area visualization method provided in the above-mentioned embodiment, and the other technical features in the automatic parking drivable area visualization device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here. It should be understood that the various parts disclosed in the present application can be implemented in hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0137] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, the computer-readable program instructions being used to execute the method for visualizing a drivable area for automatic parking in the above-mentioned embodiment.

[0138] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0139] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the automatic parking drivable area visualization device, the automatic parking drivable area visualization device: obtains vehicle-mounted camera data, radar point cloud data, vehicle size and vehicle safety distance; identifies the vehicle-mounted camera data through a target obstacle recognition model to obtain a first position, size and type of an obstacle in the image, wherein the target obstacle recognition model is constructed according to an encoder, a decoder and a self-attention module; adjusts the first position according to the radar point cloud data to obtain a real-time obstacle position; constructs a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position and the size and type of the obstacle in the image; sends the vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area to a vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and sends the restored scene data to a vehicle display, so that the vehicle display displays an automatic parking scene restoration view according to the scene data.

[0140] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0143] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned method for visualizing the drivable area of ​​automatic parking, and can solve the technical problem of how to display the drivable area of ​​automatic parking to avoid collision with obstacles. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the method for visualizing the drivable area of ​​automatic parking provided by the above-mentioned embodiment, and will not be described in detail here.

[0144] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method for visualizing the drivable area of ​​automatic parking as described above. The computer program product provided by the present application can solve the technical problem of how to display the drivable area of ​​automatic parking to avoid collision with obstacles. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the method for visualizing the drivable area of ​​automatic parking provided by the above-mentioned embodiment, and will not be repeated here. The above description is only a partial embodiment of the present application, and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the specification and drawings of this application under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for visualizing a drivable area of ​​an automatic parking vehicle, characterized in that: The method comprises: Obtain vehicle camera data, radar point cloud data, vehicle size, and vehicle safety distance; Identify the vehicle-mounted camera data by using a target obstacle recognition model to obtain a first position, size and type of the obstacle in the image, wherein the target obstacle recognition model is constructed based on an encoder, a decoder and a self-attention module; Adjusting the first position according to the radar point cloud data to obtain a real-time obstacle position; Constructing a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image; The vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area are sent to a vehicle controller, so that the vehicle controller restores the automatic parking scene according to a scene restoration model, and the restored scene data is sent to a vehicle display, so that the vehicle display displays an automatic parking scene restoration view according to the scene data.

2. The method according to claim 1, characterized in that The step of adjusting the first position according to the radar point cloud data to obtain a real-time obstacle position includes: Fitting the radar point cloud data by a random sampling consistency strategy to identify a second position of a static obstacle in the point cloud data; Clustering the radar point cloud data by a density clustering strategy to identify a third position of a dynamic obstacle in the point cloud data; The obstacle in the image is spatially aligned with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position.

3. The method according to claim 2, characterized in that The step of fitting the radar point cloud data by a random sampling consistency strategy to identify the second position of the static obstacle in the point cloud data comprises: Extract all points from the radar point cloud data to obtain an initial point set; Selecting a random point pair from the initial point set; Fitting an initial obstacle geometry model according to the random point pairs; Calculating the distances from all points to the initial obstacle geometry model; Dividing the initial point set according to the distance to obtain an inner point set; The target obstacle geometric model is obtained by maximizing the interior point set by selecting the random point pairs multiple times; A second position of the static obstacle is determined according to the target obstacle geometric model.

4. The method according to claim 2, characterized in that The step of clustering the radar point cloud data by a density clustering strategy to identify the third position of the dynamic obstacle in the point cloud data comprises: Defining a neighborhood radius and a minimum number of points according to distribution characteristics of point clouds in the radar point cloud data; Clustering each point in the radar point cloud data according to the neighborhood radius and the minimum number of points to obtain a point cloud cluster; Determining dynamic obstacles based on changes in consecutive frames of the radar point cloud data; The dynamic obstacle in the continuous frames is dynamically tracked by Kalman filtering to obtain a third position of the dynamic obstacle.

5. The method according to claim 2, characterized in that The step of spatially aligning the obstacle in the image with the static obstacle and the dynamic obstacle according to the first position, the second position, and the third position to obtain a real-time obstacle position comprises: Get camera parameters and radar parameters; The ground coordinate system is constructed with the center of the rear axle of the vehicle as the origin, the forward direction of the vehicle as the positive direction of the horizontal axis, and the left side of the vehicle as the positive direction of the vertical axis. According to the camera parameters, converting the two-dimensional coordinates of the first position in the image into three-dimensional coordinates in the ground coordinate system through an inverse perspective transformation to obtain first coordinates; Rotate and displace the second position and the third position according to the radar parameters and the ground coordinate system to obtain a second coordinate and a third coordinate; The first coordinate, the second coordinate, and the third coordinate in the ground coordinate system are converted to a bird's-eye view to obtain a real-time obstacle position.

6. The method according to claim 1, characterized in that The target obstacle recognition model includes an encoder, a decoder and a self-attention module, the encoder includes a convolution layer and a pooling layer, and the step of identifying the vehicle camera data through the target obstacle recognition model to obtain the first position, size and type of the obstacle in the image includes: De-noising, light compensation and standardization are performed on the image in the vehicle-mounted camera data to obtain target image data; Processing the target image data by the encoder and the decoder to obtain a first position of an obstacle in the image; Performing a convolution operation on the target image data through the convolution layer to obtain a feature map; Performing dimensionality reduction on the feature map through the pooling layer to obtain a low-dimensional feature map; Performing weighted aggregation on the low-dimensional feature map through the self-attention module to obtain a weighted feature map; Performing bounding box regression on the weighted feature map to obtain the size of the obstacle; Performing network encoding on the weighted feature map through the self-attention module to obtain a classification network; The classification network is processed by a smooth maximum function to obtain the type of the obstacle.

7. The method according to claim 6, characterized in that The step of processing the target image data by the encoder and the decoder to obtain the first position of the obstacle in the image comprises: Mapping each pixel in the target image data to a depth value by the encoder; Upsampling and skip-connecting the target image data according to the depth value by the decoder to obtain a depth map; The first position of the obstacle in the image is obtained according to the distance information between each pixel in the depth map and the camera.

8. A device for visualizing a drivable area for automatic parking, characterized in that: The device comprises: Data acquisition module, used to obtain vehicle camera data, radar point cloud data, vehicle size and vehicle safety distance; An obstacle recognition module, used to recognize the vehicle-mounted camera data through a target obstacle recognition model to obtain a first position, size and type of an obstacle in the image, wherein the target obstacle recognition model is constructed based on an encoder, a decoder and a self-attention module; A position determination module, used to adjust the first position according to the radar point cloud data to obtain a real-time obstacle position; A drivable area construction module, used to construct a drivable area according to the vehicle size, the vehicle safety distance, the real-time obstacle position, and the size and type of the obstacle in the image; A view restoration module is used to send the vehicle-mounted camera data, the radar point cloud data and the data corresponding to the drivable area to the vehicle controller, so that the vehicle controller restores the automatic parking scene according to the scene restoration model, and sends the restored scene data to the vehicle display, so that the vehicle display displays the automatic parking scene restoration view according to the scene data.

9. A device for visualizing the drivable area of ​​an automatic parking vehicle, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for visualizing a drivable area of ​​automatic parking according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for visualizing a drivable area of ​​automatic parking according to any one of claims 1 to 7 are implemented.

Images