Parking space determination method and device, vehicle and storage medium

Abstract

The application provides a parking space determination method and device, a vehicle and a storage medium. The method obtains point cloud data and image data collected by a target vehicle; at least one parking space candidate frame is determined from the image data, and ground point cloud is selected from the point cloud data; image data corresponding to each parking space candidate frame and corresponding ground point cloud are fused in features to obtain fusion parking space features corresponding to each parking space candidate frame, and the target parking space candidate frame is determined based on the fusion parking space features. By using the parking space determination method provided by the application, the point cloud data and the image data are fused in features, the ground point cloud and the image data are mutually supplemented, the robustness of the fusion parking space features is improved, the influence of the environment, data noise, deformation and the like on the fusion parking space features is reduced, and the accuracy of parking space detection is improved. Meanwhile, by selecting the ground point cloud, the data processing amount is greatly reduced, and the efficiency of parking space detection is improved.

Description

Technical Field

[0001] This application relates to the automotive field, specifically to parking space determination methods, devices, vehicles, and storage media. Background Technology

[0002] With the development of intelligent vehicles, driving methods are becoming increasingly diverse. As an important function of intelligent driving, automatic parking is crucial in finding suitable vacant parking spaces.

[0003] Currently, visual detection is commonly used to identify available parking spaces. However, visual detection is easily affected by the external environment, resulting in low accuracy. Therefore, a new method for identifying parking spaces is needed to improve the accuracy of parking space detection. Summary of the Invention

[0004] In view of the above problems, this application provides a parking space determination method, device, vehicle and storage medium, which can, based on detecting the candidate boxes behind the vehicle through image data, fuse the image data and point cloud data corresponding to each parking space candidate box, and determine the candidate boxes of vacant parking spaces from the determined candidate boxes, thereby improving the accuracy of parking space detection.

[0005] The embodiments of this application are implemented using the following technical solutions:

[0006] In a first aspect, this application provides a parking space determination method, the method comprising: acquiring point cloud data and image data collected from a target vehicle; selecting ground point cloud data from the point cloud data, wherein the ground point cloud data is point cloud data corresponding to the ground plane; determining at least one parking space candidate box from the image data, wherein each parking space candidate box is used to represent a parking space; performing feature fusion on the image data corresponding to each parking space candidate box and the ground point cloud corresponding to the parking space candidate box to obtain a fused parking space feature corresponding to each parking space candidate box, wherein the fused parking space feature is used to represent the occupancy status of the parking space candidate box; and determining a target parking space candidate box from the at least one parking space candidate box based on the fused parking space feature corresponding to each parking space candidate box, wherein the parking space corresponding to the target parking space candidate box is an vacant parking space.

[0007] Secondly, this application provides a parking space determination device, comprising: an acquisition module for acquiring point cloud data and image data collected from a target vehicle; a selection module for selecting ground point cloud data from the point cloud data, wherein the ground point cloud data is point cloud data corresponding to the ground plane; a confirmation module for determining at least one parking space candidate box from the image data, wherein each parking space candidate box represents a parking space; a fusion module for fusing the image data corresponding to each parking space candidate box and the ground point cloud corresponding to the parking space candidate box to obtain a fused parking space feature corresponding to each parking space candidate box, wherein the fused parking space feature represents the occupancy status of the parking space candidate box; and an output module for determining a target parking space candidate box from the at least one parking space candidate box based on the fused parking space feature corresponding to each parking space candidate box, wherein the parking space corresponding to the target parking space candidate box is an vacant parking space.

[0008] In some embodiments, the confirmation module further includes a feature extraction unit and a processing unit. The feature extraction unit is used to extract features from the image data to obtain image features, the image features including the features of each feature point in the image data, each feature point corresponding to at least one pixel in the image data; the processing unit is used to perform Gaussian filtering on the image features to obtain the Gaussian distribution value of each feature point in the image features; determine a target feature point from the image features based on the Gaussian distribution value of each feature point in the image features, the Gaussian distribution value of the target feature point being greater than the Gaussian distribution value of the feature points adjacent to the target feature point; and determine at least one parking space candidate box in the image data based on the target feature point.

[0009] In some embodiments, the parking space confirmation device further includes an association module, which is used to convert the image data and the ground point cloud to a target coordinate system; for each parking space candidate box, a radial threshold is determined based on the coordinates of at least two target feature points in the target coordinate system along its length, and an azimuth threshold is determined based on the coordinates of at least two target feature points in the target coordinate system along its width; based on the azimuth threshold and the radial threshold, the ground point cloud corresponding to the parking space candidate box is obtained from the ground point cloud in the target coordinate system.

[0010] In some embodiments, the fusion module further includes an image-radar processing unit and a radar-image processing unit. The image-radar processing unit is used to project the ground point cloud corresponding to each parking space candidate frame onto the image data to obtain the projection position of each ground point cloud on the image data; for each ground point cloud, the ground point cloud is fused with image data within a preset range from the projection position of the ground point cloud to obtain a fused point cloud corresponding to the ground point cloud, wherein the preset range is determined by the acquisition distance of the ground point cloud; the radar-image processing unit is used to fuse the image data corresponding to each parking space candidate frame and the fused point cloud of the ground point cloud corresponding to the parking space candidate frame to obtain the fused parking space feature of the parking space candidate frame.

[0011] Specifically, in some embodiments, the image-radar processing unit is further configured to, for each ground point cloud, downsample image data within a preset range of the projection position of the ground point cloud based on preset sampling parameters to obtain sampled image data corresponding to the ground point cloud; and perform feature fusion between the ground point cloud and the sampled image data corresponding to the ground point cloud to obtain a fused point cloud corresponding to the ground point cloud.

[0012] In some implementations, the selection module includes a filtering unit and a calculation unit. The filtering unit is used to select candidate point cloud data whose acquisition distance is within a preset distance range from the point cloud data based on the acquisition distance of each point cloud in the point cloud data. The calculation unit is used to select ground point clouds from the candidate point cloud data using a random sampling consensus algorithm.

[0013] In some implementations, the output module is specifically used to determine the confidence level of a parking space candidate box as an vacant parking space based on the fused parking space features corresponding to the parking space candidate box; and to determine a target parking space candidate box from at least one parking space candidate box based on the confidence level corresponding to each of the parking space candidate boxes, wherein the confidence level corresponding to the target parking space candidate box is greater than the confidence level of other parking space candidate boxes in the at least one parking space candidate box excluding the target parking space candidate box.

[0014] Thirdly, this application provides a vehicle comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the methods described above.

[0015] Fourthly, this application provides a computer-readable storage medium storing program code that can be invoked by a processor to execute the above-described method.

[0016] The parking space determination method, apparatus, vehicle, and storage medium provided in this application are as follows: The method acquires point cloud data and image data collected from a target vehicle; determines at least one parking space candidate box from the image data; selects ground point clouds from the point cloud data; associates the parking space candidate boxes with the corresponding ground point clouds; fuses the image data corresponding to each parking space candidate box and the ground point cloud corresponding to that parking space candidate box to obtain fused parking space features corresponding to each parking space candidate box; and determines the target parking space candidate box based on the fused parking space features corresponding to each parking space candidate box. By adopting the parking space determination method provided in this application, based on determining the parking space candidate box according to the image data, the ground point cloud corresponding to the parking space candidate box and the image data are fused, so that the spatial features contained in the ground point cloud and the semantic features contained in the image data complement each other, reducing the impact of environment, data noise, deformation, etc. on the fused parking space features, improving the robustness of the fused parking space features, and thus improving the accuracy of parking space determination; at the same time, by selecting ground point clouds, the amount of data processing is reduced, and the efficiency of parking space detection is improved.

[0017] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the parking space determination method provided in the embodiments of this application.

[0020] Figure 2 This is provided by the embodiments of this application. Figure 1 A flowchart of step S120.

[0021] Figure 3 This is provided by the embodiments of this application. Figure 1 A flowchart of step S130.

[0022] Figure 4 This is provided by the embodiments of this application. Figure 1 A flowchart of step S140.

[0023] Figure 5 This is a flowchart illustrating the ground point cloud corresponding to the confirmed parking space candidate frame provided in an embodiment of this application.

[0024] Figure 6This is a flowchart illustrating the application of the parking space determination method provided in this embodiment to automatic parking.

[0025] Figure 7 This is a schematic block diagram of the parking space confirmation device provided in the embodiments of this application.

[0026] Figure 8 This is a structural block diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0027] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0029] Automated parking, as an important function of intelligent driving technology, works by first identifying available parking spaces and then planning a parking path. Therefore, detecting available and suitable parking spaces is a crucial prerequisite for automated parking. Currently, commonly used parking space detection methods mainly rely on visual detection, which uses surround-view cameras to perceive the parking area. By identifying parking lines, the overlapping part of two parking lines is used to determine the corner point of the parking space, thereby identifying available parking spaces. However, visual detection is greatly affected by the environment. When the environment is dark or the parking lines are unclear, it may cause misidentification or missed identification of parking space corner points, resulting in low accuracy in determining parking spaces.

[0030] To address the aforementioned issues, this application provides a parking space determination method, apparatus, vehicle, and storage medium. The method involves acquiring point cloud data and image data collected from a target vehicle; selecting ground point clouds from the point cloud data (point cloud data corresponding to the ground plane); determining at least one parking space candidate box from the image data (each candidate box represents a parking space); fusing features between the image data corresponding to each candidate box and the ground point cloud corresponding to that candidate box to obtain a fused parking space feature for each candidate box, which represents the occupancy status of that candidate box; and determining a target parking space candidate box from the at least one candidate box based on the fused parking space feature, where the parking space corresponding to the target candidate box is an vacant parking space. By employing the parking space determination method provided in this application, feature fusion between point cloud data and image data allows for mutual complementarity between the ground point cloud and image data, improving the robustness of the image data, reducing the impact of data noise and deformation on the image data, and improving the accuracy of parking space detection. Simultaneously, by selecting ground point clouds, the amount of data processing is greatly reduced, improving the efficiency of parking space detection.

[0031] The embodiments provided in this application will now be described with reference to the accompanying drawings.

[0032] like Figure 1 , Figure 1 A flowchart illustrating the parking space determination method provided in this application is given. The parking space determination method includes:

[0033] S110. Acquire point cloud data and image data collected from the target vehicle.

[0034] The target vehicle can be a gasoline-powered vehicle, a new energy vehicle, or a vehicle powered by other energy sources.

[0035] It should be noted that the target vehicle is equipped with devices for collecting point cloud data and image data, such as sensors and cameras.

[0036] For example, in some implementations, point cloud data is acquired by the vehicle's onboard millimeter-wave radar; image data is acquired by the vehicle's onboard cameras (such as surround-view cameras or fisheye cameras).

[0037] It is worth mentioning that in practical applications, in order to use the collected point cloud data or image data to guide the operation of the target vehicle, it is necessary to calibrate the extrinsic parameters of the millimeter-wave radar or camera before collecting data on the external environment using millimeter-wave radar or camera. The extrinsic parameters refer to the parameters that transform the millimeter-wave radar coordinate system or camera coordinate system with the vehicle coordinate system.

[0038] For example, in the embodiments of this application, before data acquisition, it is necessary to calibrate the external parameters of the millimeter-wave radar and the camera, such as coordinate axes (x, y, z), roll angle, pitch angle, yaw angle, etc.

[0039] S120. Select the ground point cloud from the point cloud data; where the ground point cloud is the point cloud data corresponding to the ground plane.

[0040] It should be noted that since the detection range of millimeter-wave radar is generally between 150m and 200m, the collected point cloud data will contain point cloud data of various objects, such as walls, ground, obstacles, etc. Considering that parking spaces must be located on the ground plane, other point cloud data outside the ground plane are meaningless for parking space determination. Therefore, selecting ground point cloud data from the point cloud data can greatly reduce the amount of point cloud data to be processed in the subsequent process, thereby improving the efficiency of parking space detection.

[0041] It is worth mentioning that, since millimeter-wave radar detects points by sending a cone-shaped beam and obtaining point cloud data from the received return beam, the closer a location is to the millimeter-wave radar, the more ground point cloud data it contains, and the farther away a location is from the millimeter-wave radar, the fewer ground point cloud data it contains. Therefore, in some implementations, selecting ground point clouds from the point cloud data can be done by directly determining a preset distance range based on the acquisition distance of the millimeter-wave radar, and selecting the point cloud data within the preset distance range as the ground point cloud. In other implementations, a neural network can be used to extract features from the ground point cloud, and the ground point cloud can be selected from the point cloud data based on the extracted features. No specific restrictions are imposed here, and the selection can be made according to actual needs.

[0042] For example, in the embodiments of this application, such as Figure 2 , Figure 2 The embodiments provided in this application are given Figure 1 A flowchart of step S120 is shown. Step S120 includes:

[0043] S121. Based on the acquisition distance of each point cloud in the point cloud data, select candidate point cloud data whose acquisition distance is within a preset distance range from the point cloud data.

[0044] The acquisition distance of each point cloud can be calculated based on the spatial distance of the point cloud's three-dimensional coordinates.

[0045] It is understandable that locations closer to the millimeter-wave radar contain more ground point clouds, while locations farther away from the millimeter-wave radar contain fewer ground point clouds. By selecting candidate point cloud data within a preset distance range, the candidate point cloud data contains more ground point clouds. For example, in this embodiment of the application, the preset distance range is x<20m, y<20m and z<1m.

[0046] It should be noted that the preset distance range may vary for different vehicle-mounted millimeter-wave radars due to their different data acquisition capabilities.

[0047] S122. Select ground point clouds from candidate point cloud data using the random sampling consensus algorithm.

[0048] Random Sample Consensus (RANSAC) is an iterative method for estimating mathematical model parameters using observed data points.

[0049] Specifically, the distribution of point cloud data can be described by a mathematical model. Points that fit the mathematical model are defined as inliers, and points that do not fit the mathematical model are defined as outliers. In this embodiment, ground point clouds are used as inliers, and non-ground point clouds are used as outliers. The selection of ground point clouds from candidate point cloud data using a random sampling consensus algorithm specifically includes:

[0050] From the candidate point cloud data, n point cloud data are randomly selected, where n is the minimum number of input samples required to determine the parameters of the mathematical model. Assuming that these n point cloud data are all inliers, the mathematical model is fitted using these n point cloud data. The fitted mathematical model is then used to determine whether other candidate point cloud data belong to inliers. If a sufficient number of point clouds can be classified as assumed inliers through the mathematical model, that is, the number of ground point clouds exceeds a preset threshold, then the fitted model is considered to be usable for selecting ground point clouds. The fitted model is then used to classify the point cloud data into ground point clouds and non-ground point clouds.

[0051] If the number of point cloud data classified as inliers is less than the preset threshold, the mathematical model is refitted using the point cloud data classified as inliers. The fitted data model is then used to determine whether other candidate point cloud data belong to inliers again. This process continues until the number of point cloud data classified as inliers exceeds the preset threshold, at which point iterative training is performed until a suitable mathematical model is trained to classify point cloud data into ground point cloud and non-ground point cloud.

[0052] It should be noted that the parameters in the random sampling consensus algorithm (such as the number of samples n, the number of iterations, the preset threshold, etc.) directly affect the output results of the algorithm, that is, the accuracy of the ground point cloud. Therefore, when using the random sampling consensus algorithm, the parameters in the algorithm can be set according to actual needs so that the ground point cloud determined by the algorithm meets the detection requirements.

[0053] The parking space determination method provided in this application selects candidate point cloud data from the point cloud data based on the acquisition distance, so that the candidate point cloud data contains more ground point clouds, ensuring the effectiveness of the random sampling consensus algorithm. At the same time, using the random sampling consensus algorithm to select ground point clouds from the candidate point cloud data can greatly reduce the amount of point cloud data in the subsequent processing, and avoid interference from non-ground point clouds in the subsequent recognition process, thereby improving the efficiency of parking space detection.

[0054] S130. Determine at least one parking space candidate box from the image data, each parking space candidate box representing a parking space.

[0055] Each parking space candidate box represents a parking space; furthermore, the parking space corresponding to the parking space candidate box may be an vacant parking space or a parking space with obstacles.

[0056] In some implementations, determining at least one parking space candidate box from image data can be achieved by extracting parking space features from the image data. Furthermore, parking space feature extraction can be achieved through a neural network or a feature extraction algorithm. For example, in the embodiments of this application, parking space features can be extracted from the image data based on the backbone network of a convolutional neural network (CNN) (the network that extracts features from the image is called the backbone network) - a residual neural network (ResNet), thereby determining at least one parking space candidate box from the image data.

[0057] Furthermore, since a parking space is a rectangular frame, and a rectangle can be determined by its edges or vertices, when extracting parking space features from image data and determining candidate parking space frames based on these features, the extracted features can be overall features of the parking space, such as the overall parking space frame, which can be used to directly determine candidate parking space frames in the image data. Alternatively, they can be local features of the parking space, such as parking space corners or parking space lines, which can be used to construct candidate parking space frames in the image data. The specific method is not limited here and can be selected according to actual needs.

[0058] It should be noted that extracting different parking space features directly affects the detection rate and accuracy of parking spaces. For example, when the extracted parking space feature is a complete parking space bounding box, partially occluded parking spaces, parking spaces with blurred lines, or parking spaces with poor lighting cannot be identified, resulting in a low detection rate. However, since a complete parking space bounding box is identified, the accuracy of the identified parking space bounding box is high. When the extracted parking space feature is the parking space edge line, as long as at least two edge lines can be identified, a parking space candidate bounding box can be confirmed. However, when the edge line is blurred, the parking space cannot be identified. Compared to the overall identification of the parking space bounding box, this improves the detection rate but reduces the accuracy. When the extracted feature is the corner point of the parking space, as long as multiple corner points can be extracted to construct a rectangle that meets certain length and width requirements, a parking space candidate bounding box can be constructed, greatly improving the detection rate. It can also effectively detect some parking spaces with blurred or incomplete edge lines.

[0059] In this embodiment of the application, in order to enhance the detection capability of parking spaces and avoid missed detections due to blurred parking space lines, the corner points of parking spaces are extracted to confirm the candidate boxes of parking spaces.

[0060] For example, in some implementations, such as Figure 3 , Figure 3 The embodiments provided in this application are given Figure 1 A flowchart of step S130 is shown. Step S130 includes:

[0061] S131. Extract features from the image data to obtain image features.

[0062] The image features include the features of each feature point in the image data, and each feature point corresponds to at least one pixel in the image data. Furthermore, in this embodiment, feature extraction of the image data is actually the extraction of parking space corner features from the image data. The features of each feature point in the image data are actually the parking space corner features of each feature point.

[0063] Feature extraction can be achieved through a neural network model or a feature extraction algorithm; no specific restrictions are imposed here.

[0064] S132. Perform Gaussian filtering on the image features to obtain the Gaussian distribution value of each feature point in the image features.

[0065] Gaussian filtering is an image processing technique that effectively reduces noise in images, making them clearer. The basic principle of Gaussian filtering is to use the probability density function of a normal distribution (i.e., Gaussian distribution) as weights for noise points, then take a weighted average of the values ​​of the pixels surrounding the pixel to be filtered, and finally assign this average value to the pixel being filtered, thus achieving the filtering effect.

[0066] It is understandable that after Gaussian filtering, the Gaussian distribution value of each feature point represents the weighted sum of the parking space corner features contained in all feature points within a certain distance from that feature point. In other words, the larger the Gaussian distribution value of a feature point, the more parking space corner features it contains.

[0067] S133. Determine target feature points from image features based on the Gaussian distribution values ​​of each feature point in the image features.

[0068] Wherein, the Gaussian distribution value of the target feature point is greater than the Gaussian distribution value of the feature points adjacent to the target feature point; furthermore, the feature points adjacent to the target feature point can be the feature points closest to the target feature point, or feature points within a certain distance range from the target feature point, without specific restrictions.

[0069] It is understandable that, considering that the corner of a real parking space is actually the intersection of two parking lines, the corner has a certain area. The closer to the center of the parking space corner, the more parking space corner features it contains within a certain distance, and the larger the Gaussian distribution value. Since the Gaussian distribution value of the target feature point is greater than that of the feature points adjacent to it, it means that the target feature point contains more parking space corner features than the adjacent feature points. Therefore, the target feature point is also the possible center point of the parking space corner.

[0070] S134. Determine at least one parking space candidate box in the image data based on the target feature points.

[0071] In some implementations, determining at least one parking space candidate box in the image data based on target feature points can be achieved by connecting all target feature points to obtain multiple connected shapes; then filtering the multiple connected shapes based on preset requirements (such as length, width, and angle) to confirm the parking space candidate box.

[0072] In other implementations, at least one parking space candidate box is determined in the image data based on target feature points. Alternatively, the length and width of the parking space candidate box can be preset. If the distance between any two target feature points meets the length or width requirement of the parking space candidate box, the two target feature points are connected. Then, the connecting lines are filtered according to the angle between different connecting lines on the same target feature point to confirm the parking space candidate box. Furthermore, considering that there is a certain deviation between the target feature points and the corner points of the actual parking spaces, when determining the length or width requirements, a certain length threshold range or width threshold range can be set. If the distance between any two target feature points is within the length threshold range or width threshold range of the parking space candidate box, the two target feature points are connected.

[0073] It should be noted that, considering that the constructed parking space candidate boxes need to meet certain size requirements, not all target feature points meet the requirements for constructing parking space candidate boxes during the construction process.

[0074] It is worth mentioning that, in some implementations, before using the ResNet backbone network of a Convolutional Neural Network (CNN) to extract features from image data, it is necessary to train the neural network model. The training process specifically includes:

[0075] Step 1: Use the model to extract parking space features from the sample image data to obtain the image features of the sample image. Then, use Gaussian filtering to confirm the preliminary predicted center point. The preliminary predicted center point is determined by selecting a certain number of points on the image features of the sample image that are greater than or equal to the values ​​of the eight surrounding adjacent Gaussian distributions. The specific Gaussian function is as follows:

[0076] Gaussian function:

[0077] Step 2: Using a pre-defined label image, mark the center point of the parking space corner on the image features of the sample image. The center point p of the parking space corner in the pre-defined label image can be determined by the following formula:

[0078] Where (x1, y1) and (x2, y2) are the coordinates of the two ends of the diagonal of the corner point of the parking space;

[0079] Step 3: Calculate the confidence level of each preliminary predicted center point and the deviation value between each preliminary predicted center point and the labeled center point. The confidence level is used to characterize the model's acceptance of the preliminary predicted center point. Construct prediction boxes for center points that meet the confidence threshold and the preset deviation value. Then, through the preset length and width of the parking space candidate boxes, make the model output parking space candidate boxes that meet the length and width requirements, thereby realizing the training of the model.

[0080] During training, considering that there may be a large number of predicted center points, a downsampling factor can be set to downsample the predicted center points, thereby optimizing the computational load and improving the model training efficiency.

[0081] It is understandable that the parking space candidate boxes determined by the trained model have the length and width that meet the preset length and width requirements, and the target feature points corresponding to the parking space candidate boxes also meet the preset confidence requirements and preset deviation value requirements.

[0082] The parking space determination method provided in this application detects the center point of the corner point by center point detection, thereby confirming the corner point of the parking space and constructing a parking space candidate box that meets the requirements. This reduces the influence of parking space lines and parking space shape on corner point recognition, avoids the inability to recognize corner points due to blurred or incomplete parking space lines, and improves the detection rate of parking spaces.

[0083] S140. The image data corresponding to each parking space candidate box and the ground point cloud corresponding to the parking space candidate box are fused to obtain the fused parking space feature corresponding to each parking space candidate box. The fused parking space feature is used to characterize the occupancy status of the parking space candidate box.

[0084] Feature fusion refers to the process of combining the advantages of features given features with different attributes, thereby improving the performance of the model. For example, spatial features of point cloud data can be fused into image data to obtain image data with spatial features; semantic features of image data can also be fused into point cloud data to obtain point cloud data with semantic features.

[0085] Furthermore, feature fusion can be achieved through a neural network model. For example, in the embodiments of this application, feature fusion is achieved using the cross-attention layer of the transformer model. Of course, in other embodiments, feature fusion can also be achieved in other ways.

[0086] It should be noted that, due to limitations in data acquisition methods, point cloud data has a large angular resolution error, but its depth and distance recognition is relatively accurate; image data has a strong ability to recognize angular resolution, but it cannot effectively predict depth and distance; through feature fusion, the distance recognition advantage of point cloud data can be maximized, while combining it with the angular resolution advantage of images to provide more accurate information for the identification of vacant parking spaces.

[0087] In some implementations, such as Figure 4 , Figure 4 A flowchart illustrating step S140 provided in an embodiment of this application is given. Step S140 specifically includes:

[0088] S141. For each parking space candidate box, project the ground point cloud corresponding to the parking space candidate box onto the image data to obtain the projection position of each ground point cloud on the image data.

[0089] Among them, projection, or projection transformation, is used to transform one coordinate system into another. This method often requires corresponding transformation parameters. Furthermore, the projection position refers to the image coordinate position of the ground point cloud after it has been transformed into the image coordinate system.

[0090] S142. For each ground point cloud, feature fusion is performed between the ground point cloud and image data within a preset range of the projection position of the ground point cloud to obtain the fused point cloud corresponding to the ground point cloud.

[0091] For ease of description, in this application, the image data within a preset range of the projection position of the ground point cloud is referred to as the image patch of the ground point cloud.

[0092] The preset range is determined by the acquisition distance of the ground point cloud, which means that the size of the image patch is determined by the acquisition distance of the ground point cloud. For example, the closer the acquisition distance of the ground point cloud, the smaller the preset range and the smaller the image patch; the farther the acquisition distance, the larger the preset range and the larger the image patch.

[0093] It is worth mentioning that the preset range is determined by the acquisition distance of the ground point cloud. This is because the perspective principle exists in the image data, which means that for the same image area, the closer to the target vehicle, the smaller the actual area of ​​the acquired image and the fewer image features it contains; the farther away from the target vehicle, the larger the actual area of ​​the acquired image and the more image features it contains. Therefore, by determining a larger preset range for ground point clouds acquired at close range and a smaller preset range for ground point clouds acquired at distant range, the actual area of ​​the image data corresponding to each ground point cloud can be made similar. This avoids the ground point cloud having too few or too many image features for feature fusion, thus improving the feature fusion effect between ground point cloud and image data.

[0094] Understandably, the fused point cloud corresponding to the ground point cloud retains the depth information of the ground point cloud and adds semantic features of the image patches corresponding to the ground point cloud, thereby enabling the fused point cloud to more accurately describe spatial location information.

[0095] Considering that ground point cloud image patches may contain many image features, in order to further optimize the computational load, in some implementations, step S142 includes:

[0096] S142a. For each ground point cloud, downsample the image data within a preset range of the projection position of the ground point cloud based on preset sampling parameters to obtain the sampled image data corresponding to that ground point cloud.

[0097] It should be noted that, in order to ensure the consistency of the fusion features corresponding to the ground point clouds in different locations, the same sampling parameters are required to downsample the image patches (also called image data within a preset range of the projection position of the ground point cloud) corresponding to different ground point clouds.

[0098] S142b: Perform feature fusion between the ground point cloud and the sampled image data corresponding to the ground point cloud to obtain the fused point cloud corresponding to the ground point cloud.

[0099] By downsampling the image patches, the amount of image data that needs to be fused for features is reduced, which improves the efficiency of feature fusion between ground point clouds and image patches, thereby improving the efficiency of parking space detection.

[0100] S143. For each parking space candidate box, fuse the image data corresponding to the parking space candidate box and the fused point cloud of the ground point cloud corresponding to the parking space candidate box to obtain the fused parking space feature of the parking space candidate box.

[0101] The parking space determination method provided in this application embodiment ensures that the fused parking space features of the obtained parking space candidate boxes are image features after fused point cloud encoding. The fused parking space features not only include the image features of the parking space candidate boxes but also the fused point cloud features corresponding to the parking space candidate boxes. Since the fused point cloud itself contains features of the ground point cloud and the image features of the corresponding image patches, fusing the features of the ground point cloud with the image features of the parking space candidate boxes enhances the depth features of the image data. Furthermore, fusing the image features of the corresponding image patches with the image features of the parking space candidate boxes supplements and verifies the image features of the parking space candidate boxes, thereby enhancing the robustness of the fused parking space features.

[0102] In some implementations, considering that the ground point cloud may contain other functional ground areas besides the ground area corresponding to the parking space, such as lanes and pedestrian walkways, in order to avoid interference from other ground point clouds on feature fusion and to ensure that the image data corresponding to the parking space candidate box can be fused with the correct ground point cloud, it is necessary to confirm the ground point cloud corresponding to the parking space candidate box before step S140. Since each parking space candidate box in this embodiment includes multiple target feature points, confirming the ground point cloud corresponding to the parking space candidate box in this embodiment may include the following steps, such as... Figure 5 , Figure 5 This application provides a flowchart illustrating the ground point cloud corresponding to the confirmed parking space candidate frame, including:

[0103] S144. Convert the image data and ground point cloud to the target coordinate system.

[0104] Since image data is two-dimensional data and point cloud data is three-dimensional data, the target coordinate system can be a two-dimensional coordinate system or a three-dimensional coordinate system. In some embodiments, the target coordinate system is a two-dimensional polar coordinate system. Furthermore, the two-dimensional polar coordinates can be established based on the target vehicle or based on the image coordinate system. In the embodiments of this application, the two-dimensional polar coordinates are established based on the image coordinate system.

[0105] S145. For each parking space candidate box, determine the radial threshold based on the coordinates of at least two target feature points in the target coordinate system along its length direction, and determine the azimuth threshold based on the coordinates of at least two target feature points in the target coordinate system along its width direction.

[0106] It should be noted that since the parking space candidate box is in polar coordinate system, the orientation threshold is also the angle threshold in polar coordinate system, and the radial threshold is also the radius threshold in polar coordinate system.

[0107] It is understandable that the radial threshold is determined based on at least two target feature points along the length direction, where the at least two target feature points can be on the same sideline along the length direction of the parking space candidate box or on different sidelines.

[0108] Similarly, the orientation threshold is determined based on at least two target feature points in the width direction, wherein the at least two target feature points can be on the same sideline in the width direction of the parking space candidate box or on different sidelines.

[0109] S146. Based on the orientation threshold and radial threshold, obtain the ground point cloud corresponding to the candidate parking space from the ground point cloud in the target coordinate system.

[0110] It is understandable that after converting the ground point cloud to the target coordinate system, each ground point cloud has corresponding coordinates.

[0111] For example, in some embodiments, for a parking space candidate box m, where Φ m,l Φ represents the angle between the coordinates of the leftmost target feature point along the width direction of the parking space candidate box m in polar coordinates. m,r The angle r represents the coordinates of the rightmost target feature point along the width direction. m,f r represents the radial distance of the front target feature point along the length direction of the parking space candidate box m in polar coordinates. m,b The radial distance of the target feature point behind the parking space candidate box m along its length direction, where the front, back, left, and right sides of the parking space candidate are described based on the polar coordinate system.

[0112] The polar coordinates Φ of the target point cloud associated with the parking space candidate box m m,i It needs to satisfy:

[0113] Φ m,l ≤Φ m,i ≤Φ m,r ;

[0114] The polar radius coordinates r of the target point cloud associated with the parking space candidate bounding box m m,f It needs to satisfy:

[0115] r m,f ≤r m,i ≤r m,b ;

[0116] In some embodiments, considering that the depth information of point cloud data is corrected when converted to a two-dimensional polar coordinate system, the polar radius coordinates r of the target point cloud are determined... m,f r m,f The following conditions must be met:

[0117] Where λ is the preset minimum length of the parking space, δ is the preset acquisition radius adjustment parameter of the millimeter-wave radar, σ is the depth variance of the parking space candidate box m, and r c The distance from the center point of the parking space candidate box m.

[0118] The parking space determination method provided in this application converts image data and ground point cloud to the same polar coordinate system, and uses the target feature points contained in the parking space candidate box to determine the selection range of the ground point cloud corresponding to the parking space candidate box. This establishes a correlation between the image data and the ground point cloud corresponding to the parking space candidate box, which facilitates the subsequent feature fusion of the image data and ground point cloud data of the parking space candidate box to obtain the fused parking space feature corresponding to each parking space candidate box. This avoids interference from other ground point clouds during feature fusion and improves the effectiveness of fused parking space features.

[0119] S150. Based on the fused parking space features corresponding to each parking space candidate box, determine the target parking space candidate box from at least one parking space candidate box. The parking space corresponding to the target parking space candidate box is an vacant parking space.

[0120] In some implementations, the fused parking space features can be binary classified into "occupied" and "vacant". Based on the fused parking space features corresponding to each parking space candidate, the target parking space candidate box is determined from at least one parking space candidate box, that is, the parking space candidate box with the fused parking space feature as "vacant" is selected as the target parking space candidate box.

[0121] In other implementations, confidence scores, also known as fusion scores, can be calculated for each fused parking space feature. Confidence score is the degree of certainty that the corresponding parking space candidate box is an available parking space based on the fused parking space features. It can also be understood as the correctness of the fusion of image data and ground point cloud. The higher the confidence score of the fused parking space features, the higher the degree of certainty that the corresponding parking space candidate box is an available parking space based on the fused parking space features. Based on the fused parking space features corresponding to each parking space candidate, the target parking space candidate box is determined from at least one parking space candidate box. This can be done by selecting the parking space candidate box with a confidence score of fused parking space features higher than a preset confidence threshold as the target parking space candidate box, or by selecting the parking space candidate box with the highest confidence score of fused parking space features as the target parking space candidate box.

[0122] In some implementations, step S150 specifically includes: determining the confidence level of a parking space candidate box as an vacant parking space based on the fused parking space features corresponding to the parking space candidate box; and determining a target parking space candidate box from at least one parking space candidate box based on the confidence level corresponding to each parking space candidate box.

[0123] Among them, the confidence level corresponding to the target parking space candidate box is greater than the confidence level of at least one other parking space candidate box besides the target parking space candidate box.

[0124] By selecting the parking space candidate box corresponding to the fused parking space feature with the highest confidence as the target parking space candidate box, the parking space corresponding to the selected target parking space candidate box is the best vacant parking space, which facilitates subsequent parking.

[0125] In some implementations, considering that the determined target parking space candidate box is established in an image coordinate system, in order to facilitate the target vehicle to plan its parking path based on the target parking space candidate box, after step S150, the parking space determination method may further include: transforming the target parking space candidate box to a vehicle body coordinate system, and planning the parking path of the target vehicle based on the coordinates of the target parking space candidate box in the vehicle body coordinate system. Here, the vehicle body coordinate system is a spatial coordinate system established with the target vehicle as the reference.

[0126] The parking space determination method provided in this application acquires point cloud data and image data of the target vehicle, identifies target feature points in the image data to obtain parking space candidate boxes, that is, identifies the center point of the corner of the parking space to obtain the parking space candidate box; then, selects ground point cloud from the point cloud data, and uses the determined parking space candidate box to select the ground point cloud corresponding to the parking space candidate box; performs feature fusion between the image data and the corresponding ground point cloud of each parking space candidate box to obtain fused parking space features; confirms the target parking space candidate box based on the confidence level of the fused parking space, and thus determines the target vacant parking space; through the parking space determination method provided in this application embodiment, the vehicle... Parking space candidate boxes are constructed based on the center points of the identified parking space corners, reducing the influence of parking lines and parking space shapes on corner recognition. This avoids the inability to identify corners due to blurred or incomplete parking lines, thus improving the detection rate of parking spaces. Simultaneously, the fused parking space features of the candidate boxes are image features encoded with fused point clouds. These features include not only the image features of the candidate boxes but also the corresponding fused point cloud features. This allows the fused point cloud to supplement and verify the image features of the candidate boxes, enhancing the spatial characteristics of the image data and thus improving the robustness of the fused parking space features and the accuracy of vacant parking space detection.

[0127] For a better understanding of this solution, please refer to [link / reference]. Figure 6 , Figure 6 A flowchart illustrating the application of the parking space determination method provided in this application to automatic parking is provided, including:

[0128] Step 1: After starting the parking task, acquire the point cloud data and image data of the target vehicle.

[0129] Step 2: For point cloud data, first filter the point cloud data according to the preset distance range, then use the random smoke consistency algorithm to extract the ground point cloud, and then extract the features of the ground point cloud to obtain the ground point cloud features.

[0130] For image data, a backbone network is used to extract parking space corner features from the image data, thereby identifying the target feature points, which are the center points of the parking space corners, and constructing parking space candidate boxes based on the target feature points.

[0131] Step 3: Convert the determined ground point cloud and image data to polar coordinates, and use the determined parking space candidate boxes to perform soft correlation between the ground point cloud and image data. That is, in this application, the determined parking space candidate boxes are used to determine the image data corresponding to the parking space candidate boxes and the ground point cloud corresponding to the parking space candidate boxes.

[0132] Step 4: Spatial-semantic fusion is performed on the softly correlated ground point cloud and image data, which is also the feature fusion of this application. Spatial-semantic fusion includes sequential image-radar feature encoding and radar-image encoding. Image-radar feature encoding is the process in this application of fusing the ground point cloud corresponding to the parking space candidate box with the image data corresponding to the ground point cloud to obtain the fused point cloud. Radar-image encoding is the process in this application of fusing the image data of the parking space candidate box with the fused point cloud corresponding to the ground point cloud of the parking space candidate box to obtain the fused parking space features.

[0133] Step 5: Use the fusion detection head to decode the encoded image features to obtain the decoded fused parking space features. Then, determine the target parking space candidates based on the confidence level of the fused parking space features. Finally, perform coordinate transformation on the target parking space candidate boxes to obtain the target empty parking spaces.

[0134] It should be noted that in the above application process, steps two through five are all implemented through the object detection model (Detection Transformer, also known as DETR). The object detection model includes multiple sub-modules, which are used to perform various functions, such as feature extraction and feature fusion.

[0135] In some embodiments, this application also provides a parking space confirmation device, such as... Figure 7 , Figure 7 A schematic block diagram of the parking space confirmation device provided in this application is given. The parking space confirmation device 200 includes:

[0136] The acquisition module 210 is used to acquire point cloud data and image data collected from the target vehicle;

[0137] The selection module 220 is used to select ground point cloud from point cloud data, where ground point cloud is the point cloud data corresponding to the ground plane.

[0138] The confirmation module 230 is used to determine at least one parking space candidate box from the image data, each parking space candidate box being used to represent a parking space;

[0139] The fusion module 240 is used to fuse the image data corresponding to each parking space candidate box and the ground point cloud corresponding to the parking space candidate box to obtain the fused parking space feature corresponding to each parking space candidate box. The fused parking space feature is used to characterize the occupancy status of the parking space candidate box.

[0140] The output module 250 is used to determine the target parking space candidate box from at least one parking space candidate box based on the fused parking space features corresponding to each parking space candidate box, and the parking space corresponding to the target parking space candidate box is an empty parking space.

[0141] In some embodiments, the confirmation module 230 further includes a feature extraction unit and a processing unit. The feature extraction unit is used to extract features from the image data to obtain image features, which include the features of each feature point in the image data, and each feature point corresponds to at least one pixel in the image data. The processing unit is used to perform Gaussian filtering on the image features to obtain the Gaussian distribution value of each feature point in the image features. Based on the Gaussian distribution value of each feature point in the image features, a target feature point is determined from the image features, where the Gaussian distribution value of the target feature point is greater than the Gaussian distribution value of the feature points adjacent to the target feature point. Based on the target feature point, at least one parking space candidate box is determined in the image data.

[0142] In some embodiments, the parking space confirmation device 200 further includes an association module, which is used to convert image data and ground point cloud to a target coordinate system; for each parking space candidate box, a radial threshold is determined based on the coordinates of at least two target feature points in its length direction in the target coordinate system, and an azimuth threshold is determined based on the coordinates of at least two target feature points in its width direction in the target coordinate system; based on the azimuth threshold and the radial threshold, the ground point cloud corresponding to the parking space candidate box is obtained from the ground point cloud in the target coordinate system.

[0143] In some embodiments, the fusion module 240 further includes an image-radar processing unit and a radar-image processing unit. The image-radar processing unit is used to project the ground point cloud corresponding to each parking space candidate box onto the image data to obtain the projection position of each ground point cloud on the image data. For each ground point cloud, the ground point cloud is fused with the image data within a preset range of the projection position of the ground point cloud to obtain the fused point cloud corresponding to the ground point cloud. The preset range is determined by the acquisition distance of the ground point cloud. The radar-image processing unit is used to fuse the image data corresponding to each parking space candidate box and the fused point cloud of the ground point cloud corresponding to the parking space candidate box to obtain the fused parking space feature of the parking space candidate box.

[0144] Specifically, in some implementations, the image-radar processing unit is further configured to, for each ground point cloud, downsample image data within a preset range of the projection position of the ground point cloud based on preset sampling parameters, to obtain sampled image data corresponding to the ground point cloud; and perform feature fusion between the ground point cloud and the sampled image data corresponding to the ground point cloud to obtain a fused point cloud corresponding to the ground point cloud.

[0145] In some implementations, the selection module 220 includes a filtering unit and a calculation unit. The filtering unit is used to select candidate point cloud data whose acquisition distance is within a preset distance range from the point cloud data based on the acquisition distance of each point cloud in the point cloud data. The calculation unit is used to select ground point clouds from the candidate point cloud data using a random sampling consensus algorithm.

[0146] In some implementations, the output module 250 is specifically used to determine the confidence level of a parking space candidate box as an vacant parking space based on the fused parking space features corresponding to the parking space candidate box; and to determine a target parking space candidate box from at least one parking space candidate box based on the confidence level corresponding to each parking space candidate box, wherein the confidence level corresponding to the target parking space candidate box is greater than the confidence level of other parking space candidate boxes in at least one parking space candidate box excluding the target parking space candidate box.

[0147] In some implementations, based on the parking space determination method provided in the above embodiments, this application also provides a vehicle, such as... Figure 8 , Figure 8 A structural block diagram of a vehicle 300 provided in an embodiment of this application is given. The vehicle includes one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs are configured to perform the above-described method.

[0148] Among them, vehicle 300 can be a gasoline-powered vehicle, an electric vehicle, or a vehicle powered by other energy sources.

[0149] The processor 310 may include one or more processing cores. The processor 310 connects to various parts of the wearable device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 320, and by calling data stored in the memory 320. Optionally, the processor 310 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 310 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and Modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 310 and may be implemented separately through a communication chip.

[0150] The memory 320 may include random access memory (RAM) or read-only memory (ROM). The memory 320 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 320 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created during the use of the electronic device.

[0151] In some embodiments, this application also provides a computer-readable storage medium storing program code that can be invoked by a processor to execute the above-described methods.

[0152] Computer-readable storage media can be electronic storage devices such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, computer-readable storage media includes non-transitory computer-readable storage media. The computer-readable storage medium has storage space for program code that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code can be compressed in an appropriate form.

[0153] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although this application has disclosed preferred embodiments as above, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A method for determining parking spaces, characterized in that, include: Acquire point cloud data and image data collected from the target vehicle; Select ground point cloud from the point cloud data, wherein the ground point cloud is the point cloud data corresponding to the ground plane; At least one parking space candidate box is determined from the image data, each parking space candidate box representing a parking space; each parking space candidate box includes multiple target feature points, the target feature points representing parking space corner points; The image data and the ground point cloud are transformed to a target coordinate system; the target coordinate system is a two-dimensional polar coordinate system. For each parking space candidate box, a radial threshold is determined based on the coordinates of at least two target feature points in the target coordinate system along its length, and an azimuth threshold is determined based on the coordinates of at least two target feature points in the target coordinate system along its width. The radial threshold is a radius threshold in polar coordinates, and the azimuth threshold is an angle threshold in polar coordinates; the radius threshold includes an upper radius threshold and a lower radius threshold; the angle threshold includes an upper angle threshold and a lower angle threshold. Based on the azimuth threshold and the radial threshold, a ground point cloud with polar coordinates between the upper limit angle threshold and the lower limit angle threshold and polar radius coordinates between the upper limit radius threshold and the lower limit radius threshold is obtained from the ground point cloud in the target coordinate system, and is used as the ground point cloud corresponding to the parking space candidate box; The image data corresponding to each parking space candidate box and the ground point cloud corresponding to the parking space candidate box are fused to obtain the fused parking space feature corresponding to each parking space candidate box. The fused parking space feature is used to characterize the occupancy status of the parking space candidate box. Based on the fused parking space features corresponding to each of the parking space candidate boxes, a target parking space candidate box is determined from the at least one parking space candidate box, and the parking space corresponding to the target parking space candidate box is an vacant parking space.

2. The method according to claim 1, characterized in that, Determining at least one parking space candidate box from the image data includes: The image data is subjected to feature extraction to obtain image features, the image features include the features of each feature point in the image data, and each feature point corresponds to at least one pixel in the image data; The image features are subjected to Gaussian filtering to obtain the Gaussian distribution value of each feature point in the image features; Target feature points are determined from the image features based on the Gaussian distribution values ​​of each feature point in the image features, wherein the Gaussian distribution value of the target feature point is greater than the Gaussian distribution value of the feature points adjacent to the target feature point; At least one parking space candidate box is determined in the image data based on the target feature points.

3. The method according to claim 1, characterized in that, The step of fusing the image data corresponding to each parking space candidate box and the ground point cloud corresponding to that parking space candidate box to obtain the fused parking space feature corresponding to each parking space candidate box includes: For each parking space candidate box, the ground point cloud corresponding to the parking space candidate box is projected onto the image data to obtain the projection position of each ground point cloud on the image data. For each ground point cloud, feature fusion is performed between the ground point cloud and image data within a preset range of the projection position of the ground point cloud to obtain the fused point cloud corresponding to the ground point cloud. The preset range is determined by the acquisition distance of the ground point cloud. For each parking space candidate box, the image data corresponding to the parking space candidate box and the fused point cloud of the ground point cloud corresponding to the parking space candidate box are fused to obtain the fused parking space feature of the parking space candidate box.

4. The method according to claim 3, characterized in that, For each of the ground point clouds, feature fusion is performed between the ground point cloud and image data within a preset range from the projection position of the ground point cloud to obtain a fused point cloud corresponding to the ground point cloud, including: For each ground point cloud, image data within a preset range from the projection position of the ground point cloud is downsampled based on preset sampling parameters to obtain the sampled image data corresponding to that ground point cloud; The ground point cloud is fused with the sampled image data corresponding to the ground point cloud to obtain the fused point cloud corresponding to the ground point cloud.

5. The method according to claim 1, characterized in that, The step of selecting ground point clouds from the point cloud data based on point cloud features includes: Based on the acquisition distance of each point cloud in the point cloud data, candidate point cloud data whose acquisition distance is within a preset distance range are selected from the point cloud data; Ground point clouds are selected from the candidate point cloud data using a random sampling consensus algorithm.

6. The method according to claim 1, characterized in that, The step of determining the parking space candidate box corresponding to the vacant parking space from the at least one parking space candidate box based on the fused parking space features corresponding to each of the parking space candidate boxes includes: The confidence level of the parking space candidate box as an vacant parking space is determined based on the fused parking space features corresponding to the parking space candidate box; A target parking space candidate box is determined from at least one parking space candidate box based on the confidence level corresponding to each of the aforementioned parking space candidate boxes, wherein the confidence level corresponding to the target parking space candidate box is greater than the confidence levels of other parking space candidate boxes in the at least one parking space candidate box excluding the target parking space candidate box.

7. A parking space determination device, characterized in that, include: The acquisition module is used to acquire point cloud data and image data collected from the target vehicle; The selection module is used to select ground point clouds from the point cloud data, wherein the ground point clouds are point cloud data corresponding to the ground plane; A confirmation module is used to determine at least one parking space candidate box from the image data, each parking space candidate box being used to represent a parking space; each parking space candidate box includes multiple target feature points, the target feature points representing parking space corner points; The association module is used to convert the image data and the ground point cloud to the target coordinate system; the target coordinate system is a two-dimensional polar coordinate system; for each parking space candidate box, a radial threshold is determined based on the coordinates of at least two target feature points in the target coordinate system in its length direction, and an azimuth threshold is determined based on the coordinates of at least two target feature points in the target coordinate system in its width direction. The radial threshold is a radius threshold in polar coordinates, and the azimuth threshold is an angle threshold in polar coordinates; the radius threshold includes an upper radius threshold and a lower radius threshold; the angle threshold includes an upper angle threshold and a lower angle threshold; based on the azimuth threshold and the radial threshold, a ground point cloud with polar angular coordinates between the upper angle threshold and the lower angle threshold and polar radial coordinates between the upper radius threshold and the lower radius threshold is obtained from the ground point cloud in the target coordinate system, and is used as the ground point cloud corresponding to the parking space candidate box; The fusion module is used to fuse the image data corresponding to each parking space candidate box and the ground point cloud corresponding to the parking space candidate box to obtain the fused parking space feature corresponding to each parking space candidate box. The fused parking space feature is used to characterize the occupancy status of the parking space candidate box. The output module is used to determine a target parking space candidate box from the at least one parking space candidate box based on the fused parking space features corresponding to each of the parking space candidate boxes, wherein the parking space corresponding to the target parking space candidate box is an vacant parking space.

8. A vehicle, characterized in that, include: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1-6.

Images