Training method and device of parking space detection model, parking space detection method, device and equipment

Abstract

The embodiments of the present disclosure disclose a training method of a parking space detection model, a parking space detection method, device and equipment, wherein the method comprises: determining a target point cloud map of a target region and image data corresponding to at least one vehicle pose in a map coordinate system of the target point cloud map; determining a region range corresponding to each vehicle pose based on a preset range determination rule; obtaining first parking space label data in a map coordinate system corresponding to each region range; determining second parking space label data in a vehicle coordinate system corresponding to each vehicle pose based on each first parking space label data; and training a first parking space detection model obtained in advance based on the image data and the second parking space label data corresponding to each vehicle pose, to obtain a target parking space detection model. The embodiments of the present disclosure realize the training of a model for directly detecting a parking space in a vehicle coordinate system based on multi-view images, and can effectively improve the accuracy of the parking space detection result.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to computer vision technology, and in particular to a parking space detection model training method, a parking space detection method, an apparatus and a device. BACKGROUND

[0002] With the rapid development of automobile intelligence, automatic parking, memory parking, autonomous guest parking and other parking functions have been applied to vehicles, and accurate detection of parking spaces is the basis for the implementation of various parking functions. In related technologies, parking space detection is usually based on IPM (Inverse Perspective Mapping) to convert multiple view images to a bird's eye view, and then stitch a panoramic bird's eye view at the bird's eye view for parking space detection. However, IPM is based on the assumption of a road plane, and when the vehicle is in a vibration state during driving, the obtained panoramic bird's eye view will be distorted, resulting in poor accuracy of the parking space detection result. SUMMARY

[0003] To solve the above technical problems of poor accuracy of parking space detection results, embodiments of the present disclosure provide a parking space detection model training method, a parking space detection method, an apparatus and a device to avoid the adverse effects of converting multi-view images to bird's eye view images through IPM transformation on parking space detection, effectively improving the accuracy of the parking space detection result.

[0004] According to an aspect of an embodiment of the present disclosure, a parking space detection model training method is provided, comprising: determining a target point cloud map of a target region and image data corresponding to at least one vehicle pose in a map coordinate system of the target point cloud map, the image data corresponding to any vehicle pose comprising at least one view image collected at the vehicle pose; determining a region range corresponding to each vehicle pose based on a preset range determination rule; obtaining first parking space label data in the map coordinate system corresponding to each region range; determining second parking space label data in a vehicle coordinate system corresponding to each vehicle pose based on the first parking space label data corresponding to each region range; and training a pre-obtained first parking space detection model based on the image data corresponding to each vehicle pose and the second parking space label data, to obtain a target parking space detection model.

[0005] According to another aspect of the embodiments of the present disclosure, a parking space detection method is provided, including: acquiring images to be processed corresponding to each of at least one view angle; processing the images to be processed corresponding to each of the view angles by using a target parking space detection model to obtain image processing results, the target parking space detection model being obtained based on the training method of the parking space detection model provided in any of the above embodiments; and determining a parking space detection result in a vehicle coordinate system based on the image processing results.

[0006] According to still another aspect of the embodiments of the present disclosure, a training device of a parking space detection model is provided, including: a first acquisition module configured to determine a target point cloud map of a target region and image data corresponding to at least one vehicle pose in a map coordinate system of the target point cloud map, the image data corresponding to any of the vehicle poses including images of at least one view angle collected at the vehicle pose; a first processing module configured to determine a region range corresponding to each of the vehicle poses based on a preset range determination rule; a second processing module configured to acquire first parking space label data in the map coordinate system corresponding to each of the region ranges; a third processing module configured to determine second parking space label data in a vehicle coordinate system corresponding to each of the vehicle poses based on the first parking space label data corresponding to each of the region ranges; and a fourth processing module configured to train a first parking space detection model obtained in advance based on the image data corresponding to each of the vehicle poses and the second parking space label data to obtain a target parking space detection model.

[0007] According to still another aspect of the embodiments of the present disclosure, a parking space detection device is provided, including: a second acquisition module configured to acquire images to be processed corresponding to each of at least one view angle; a fifth processing module configured to process the images to be processed corresponding to each of the view angles by using a target parking space detection model to obtain image processing results, the target parking space detection model being obtained based on the training method of the parking space detection model provided in any of the above embodiments; and a sixth processing module configured to determine a parking space detection result in a vehicle coordinate system based on the image processing results.

[0008] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, the storage medium storing a computer program, the computer program being configured to execute the training method of the parking space detection model provided in any of the above embodiments of the present disclosure, or execute the parking space detection method provided in any of the above embodiments of the present disclosure.

[0009] According to a further aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the training method of the parking space detection model according to any one of the embodiments of the present disclosure, or implement the parking space detection method according to any one of the embodiments of the present disclosure.

[0010] The training method of the parking space detection model, the parking space detection method, the device and the electronic device provided by the embodiments of the present disclosure can determine the parking space label data in the region range of each vehicle pose based on the point cloud map of the target region and the image data corresponding to at least one vehicle pose stored during mapping, thereby obtaining training samples for training the parking space detection model, obtaining a target parking space detection model, and realizing the training of the model for directly detecting the parking space in the vehicle coordinate system based on multi-view images. The method can avoid the adverse effects of converting multi-view images to bird's eye view images through IPM transformation on parking space detection, effectively improve the accuracy of the parking space detection result, and can also avoid the splicing of multi-view images, reduce the adverse effects of the conflict of the spliced multi-view images in the overlapping area after IPM transformation on parking space detection, and further improve the accuracy of the parking space detection result. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 is an exemplary application scenario of the training method of the parking space detection model provided by the present disclosure;

[0012] Figure 2 is a flowchart of the training method of the parking space detection model provided by an exemplary embodiment of the present disclosure;

[0013] Figure 3 is a schematic diagram of a target point cloud map and a parking space annotation result provided by an exemplary embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of the training method of the parking space detection model provided by another exemplary embodiment of the present disclosure;

[0015] Figure 5 is a schematic diagram of a corner point identification provided by an exemplary embodiment of the present disclosure;

[0016] Figure 6 is a schematic diagram of a truncated target parking space provided by an exemplary embodiment of the present disclosure;

[0017] Figure 7 is a flowchart of step 2051 provided by an exemplary embodiment of the present disclosure;

[0018] Figure 8is a schematic diagram of a network structure of a first parking space detection model provided by an example embodiment of the present disclosure.

[0019] Figure 9 is a schematic diagram of a network structure of a first parking space detection model provided by another example embodiment of the present disclosure.

[0020] Figure 10 is a schematic diagram of a flow of a parking space detection method provided by an example embodiment of the present disclosure.

[0021] Figure 11 is a schematic diagram of a structure of a training device of a parking space detection model provided by an example embodiment of the present disclosure.

[0022] Figure 12 is a schematic diagram of a structure of a training device of a parking space detection model provided by another example embodiment of the present disclosure.

[0023] Figure 13 is a schematic diagram of a structure of a parking space detection device provided by an example embodiment of the present disclosure.

[0024] Figure 14 is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to explain the present disclosure, the example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present disclosure, but not all the embodiments. It should be understood that the present disclosure is not limited by the example embodiments.

[0026] It should be noted that: unless otherwise specified, the relative arrangement, numerical expression and numerical value of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0027] Summary of the disclosure

[0028] In the process of implementing the present disclosure, the inventors found that with the rapid development of automobile intelligence, automatic parking, memory parking, autonomous guest parking and other parking functions have been applied to vehicles. Accurate detection of parking spaces is the basis for the realization of various parking functions. In related technologies, parking space detection is usually based on IPM (Inverse Perspective Mapping) to convert multiple perspective images to a bird's eye view perspective, and then splice into a panoramic bird's eye view for parking space detection. However, IPM is based on the assumption of road plane, and when the vehicle is in a vibration state during driving, the obtained panoramic bird's eye view will be distorted, resulting in poor accuracy of the parking space detection result.

[0029] Example overview

[0030] Figure 1 is one exemplary application scenario of the training method of the parking space detection model provided by the present disclosure.

[0031] In the parking scenarios such as automatic parking, memory parking, autonomous valet parking, etc., for any target area containing parking spaces, such as a parking lot, a parking area, etc., a target point cloud map of the target area can be established in advance based on at least one of a camera, a laser radar, a millimeter wave radar, an ultrasonic radar, a combined inertial navigation, etc. on the vehicle, using any implementable mapping algorithm, and the driving trajectory of the vehicle in the mapping process and the sensor data collected at each trajectory point of the driving trajectory, such as laser radar point cloud data, image data, combined inertial navigation data, etc., are recorded. The present disclosure requires image data, so at least the image data collected at each trajectory point is recorded. The driving trajectory can include the vehicle pose when each sensor collects data during the mapping process, and can also include the corresponding collection time, which can be set according to actual needs. The mapping algorithm can be any implementable Simultaneous Localization and Mapping (SLAM) algorithm, which can be set according to actual needs. Using the training method of the parking space detection model of the present disclosure, the target point cloud map of the target area can be determined based on the above method, and the image data corresponding to at least one vehicle pose in the map coordinate system of the target point cloud map can be determined. The image data corresponding to each vehicle pose can include at least one perspective image collected at the vehicle pose, and the area range corresponding to each vehicle pose can be determined based on a predetermined range determination rule. The area range is the range of the effective sensing area of the camera at each perspective in the corresponding vehicle pose. First parking label data in the map coordinate system corresponding to each area range is obtained, and second parking label data in the vehicle coordinate system corresponding to each vehicle pose is determined based on the first parking label data corresponding to each area range. Then, the first parking space detection model obtained in advance can be trained based on the image data and the second parking label data corresponding to each vehicle pose, to obtain a target parking space detection model. The first parking label data in the map coordinate system corresponding to each area range can be obtained from the label data set of the overall range of the target point cloud map obtained by labeling the target point cloud map in advance. The specific labeling method for labeling the target point cloud map can be set according to actual needs, such as at least one of automatic labeling, semi-automatic labeling, manual labeling, etc. The labeling timing can be at any timing after determining the target point cloud map of the target area and before obtaining the first parking label data of each area range, which can be set according to actual needs. For example, the labeling can be performed before determining the area range corresponding to each vehicle pose, or after determining the area range, as long as the first parking label data in the map coordinate system corresponding to each area range can be obtained. The first parking space detection model can be a pre-established initial parking space detection model, or a basic parking space detection model trained based on certain training samples, which is not limited.The network structure of the first parking space detection model can be set according to actual needs, for example, a network structure based on a Transformer can be used. After obtaining the target parking space detection model, the target parking space detection model can be configured to a parking space detection device of a vehicle, which can include the mapping vehicle and any other vehicle, and the specific configuration is not limited. When the vehicle needs to park in the target area, the corresponding parking space detection method can be performed by the parking space detection device, the target parking space detection model is used to realize parking space detection of the to-be-processed image based on at least one view angle, and a parking space detection result is obtained, which is used to assist parking. The training of the model for directly detecting the parking space in the vehicle coordinate system based on the multi-view image is realized, which can avoid the adverse effects of converting the multi-view image to the bird's eye view image through the IPM transformation on the parking space detection, effectively improve the accuracy of the parking space detection result, and also avoid the splicing of the multi-view image, reduce the adverse effects of the conflict of the spliced multi-view image in the overlapping area after the IPM transformation on the parking space detection, and further improve the accuracy of the parking space detection result.

[0032] Example method

[0033] Figure 2 is a flowchart of a training method of a parking space detection model provided by an example embodiment of the present disclosure. The present embodiment can be applied on an electronic device, such as a server, a terminal device, and the like, as shown in Figure 2 includes the following steps:

[0034] In step 201, image data corresponding to at least one vehicle pose in a map coordinate system of a target point cloud map of a target area is determined.

[0035] The image data corresponding to any vehicle pose includes at least one view angle image collected at the vehicle pose. The target area can be any area containing a parking space, and the target point cloud map can be a point cloud map of the target area established in advance. The target point cloud map can be established in advance based on any implementable mapping algorithm. For example, the target point cloud map of the target area can be established based on a Simultaneous Localization and Mapping (SLAM) algorithm. The image data corresponding to at least one vehicle pose in the map coordinate system of the target point cloud map can be stored during the construction of the target point cloud map. For example, when the target point cloud map is constructed, the vehicle travels along a certain path in the target area, and the sensor on the vehicle collects data at a certain frame rate during the travel. The vehicle-mounted computing platform or related device therein can obtain the sensor data and record the vehicle pose, timestamp, and other related information when the data is collected.

[0036] In an optional embodiment, the number of vehicle poses included in the at least one vehicle pose can be set according to actual needs, such as sampling from a large number of vehicle poses corresponding to the stored target point cloud map according to a certain sampling interval to obtain each sampling pose as the at least one vehicle pose of the present disclosure. The sampling interval can be a time interval or a vehicle travel distance interval, which can be set according to actual needs. For example, the sampling interval is the timestamp of the vehicle pose of the stored vehicle during the construction of the target point cloud map, and at least one vehicle pose is obtained by sampling according to a preset time interval. For another example, at least one vehicle pose is obtained by sampling according to a preset distance interval according to the driving trajectory of the vehicle in the target area.

[0037] In some optional embodiments, the map coordinate system of the target point cloud map can be a coordinate system with any point in the target area as the origin, such as a map coordinate system of the target area can be established according to the pose of the vehicle at any time in the target area, which can be set according to actual needs.

[0038] Step 202, determining the region range corresponding to each vehicle pose based on a preset range determination rule.

[0039] The preset range determination rule can be set according to actual needs to determine the region range around the vehicle pose, and the preset range determination rule can include a shape rule and a size rule of the region range.

[0040] In an optional embodiment, for any vehicle pose, the shape of the region range corresponding to the vehicle pose can be at least one of a rectangular region, a circular region, an elliptical region, and other arbitrary possible shape regions, which can be set according to actual needs. The size of the region range corresponding to the vehicle pose can be set according to actual needs, such as for a rectangular region, the size can be set as the range of a rectangular region formed by a first distance in front of the vehicle, a second distance behind the vehicle, a third distance to the left of the vehicle, and a fourth distance to the right of the vehicle. The first distance and the second distance can be the same or different, and the third distance and the fourth distance can be the same or different, which is not limited. For another example, for a circular region, the size can be set as a circular region with the vehicle position corresponding to the vehicle pose as the center and a first preset radius as the radius. The first preset radius can be set according to actual needs.

[0041] Step 203, obtaining first parking space label data in the map coordinate system corresponding to each region range.

[0042] The first parking space label data corresponding to any region range can be obtained from a target parking space label data set corresponding to the target point cloud map, or can be obtained by performing parking space labeling on a local point cloud map corresponding to the region range, and the specific implementation is not limited. The target parking space label data set can be obtained by performing parking space labeling on the target point cloud map. The labeling of the local point cloud map and the target point cloud map can adopt any implementable labeling manner, such as at least one of automatic labeling, semi-automatic labeling, manual labeling, and the like, and the specific implementation is not limited. For automatic labeling, the parking space in the target point cloud map can be identified based on a pre-trained point cloud map parking space detection model to obtain the target parking space label data set corresponding to the target point cloud map. For semi-automatic labeling, the results of automatic labeling can be audited by a human being to correct the inaccurate or incorrect part, thereby improving the accuracy of the labeling results. For manual labeling, the parking space in the target point cloud map can be labeled by a relevant labeling personnel to obtain the target parking space label data set.

[0043] In an optional embodiment, when the first parking space label data corresponding to a region range needs to be obtained from the target parking space label data set, the labels of the parking spaces in the region range can be extracted from the target parking space label data set as the first parking space label data corresponding to the region range based on the positional relationship between the region range and the target point cloud map.

[0044] In some optional embodiments, the first parking space label data can include labels corresponding to the parking spaces in the corresponding region range respectively. For any parking space, the representation of the label can be set according to actual needs, such as being represented as a parking space corner point coordinate, a parking space center point coordinate, an entrance corner point coordinate, or the like, and the specific representation is not limited as long as the shape and size of the parking space can be determined.

[0045] An exemplary Figure 3 is a schematic diagram of a target point cloud map and a parking space labeling result provided by an exemplary embodiment of the present disclosure. In this example, the label of the parking space is the corner point coordinate of the labeled parking space.

[0046] In step 204, the second parking space label data in the vehicle coordinate system corresponding to each vehicle pose is determined based on the first parking space label data corresponding to each region range respectively.

[0047] Since the vehicle poses corresponding to different region ranges are different, that is, the vehicle coordinate systems corresponding to different region ranges are different, in order to use the first parking space label data for training of the parking space detection model of the present disclosure, it is necessary to convert the first parking space label data to the corresponding vehicle coordinate system.

[0048] In an optional embodiment, for each vehicle pose, the vehicle pose represents a rotation and translation relationship of the corresponding vehicle coordinate system relative to the unified map coordinate system, based on which a conversion relationship between the vehicle coordinate system of the vehicle pose and the map coordinate system can be determined, and the first parking space label data of the region range corresponding to the vehicle pose can be converted into the vehicle coordinate system of the vehicle pose based on the conversion relationship to obtain second parking space label data.

[0049] In step 205, the first parking space detection model obtained in advance is trained based on the image data and the second parking space label data corresponding to each vehicle pose respectively, to obtain a target parking space detection model.

[0050] The first parking space detection model can be an initialized parking space detection model or a parking space detection model after a certain training. The network structure of the first parking space detection model can be set according to actual needs, such as a network structure based on Transformer, a network structure based on YOLO, etc., and the specific implementation is not limited. The first parking space detection model describes the mapping relationship from the image data of at least one view to the parking space prediction result in the vehicle coordinate system, realizing the detection of the parking space in the vehicle coordinate system based on multi-view images directly.

[0051] In some optional embodiments, for the training of the first parking space detection model, the first parking space detection model can be used to process the image data corresponding to each vehicle pose respectively to obtain the parking space prediction result corresponding to each vehicle pose respectively, the network loss is determined based on the parking space prediction result and the second parking space label data, and the network parameter is updated. The updated model is continuously iteratively updated according to the above process until the training end condition is met, and the trained target parking space detection model is obtained. The training end condition can include at least one of network loss convergence and iteration number reaching a preset number threshold.

[0052] In some optional embodiments, the type of the parking space in the target region can include at least one of a vertical parking space, a horizontal parking space, an inclined parking space, a stereo parking space, etc. The first parking space label data can also include the type of each parking space corresponding to each parking space, and the output of the first parking space detection model can also include the predicted type of each parking space corresponding to each parking space. The specific implementation can be set according to actual needs.

[0053] In some optional embodiments, the first parking space label data can also include at least one of the wheel stopper, the ground lock, and whether the parking space is occupied in each parking space, so that the model can learn the related attributes of each parking space, and can predict the related attributes of each parking space when applied, to provide more effective parking space information for vehicle parking.

[0054] The training method of the parking space detection model provided in the embodiment can determine parking space label data in the region range of each vehicle pose based on the point cloud map of the target region and the image data corresponding to at least one vehicle pose stored during mapping, thereby obtaining training samples for training of the parking space detection model, obtaining a target parking space detection model, and realizing training of a model for directly detecting parking spaces in a vehicle coordinate system based on multi-view images. The method can avoid the adverse effects of converting multi-view images to bird's eye view image through IPM transformation on parking space detection, effectively improve the accuracy of the parking space detection result, and can also avoid splicing of multi-view images, reduce the adverse effects of conflicts of spliced multi-view images in the overlapping area after IPM transformation on parking space detection, and further improve the accuracy of the parking space detection result.

[0055] Figure 4 FIG. 1 is a flowchart of a training method of a parking space detection model provided by another example embodiment of the disclosure.

[0056] In some optional embodiments, the step 201 of determining the target point cloud map of the target region and the image data corresponding to at least one vehicle pose in the map coordinate system of the target point cloud map comprises:

[0057] The step 2011 establishes a target point cloud map of the target region based on at least one of original image data, radar point cloud data, and integrated inertial navigation data collected by the vehicle at each vehicle pose.

[0058] The original image data can be image data collected by a camera on the vehicle, and the radar point cloud data can include at least one of laser radar point cloud data, millimeter wave radar point cloud data, and ultrasonic wave radar point cloud data. The integrated inertial navigation data can include inertial measurement unit (IMU) data, wheel speed meter data, etc. Vehicle pose estimation can be performed based on at least one of corresponding visual odometry, radar odometry, inertial odometry, and wheel speed odometry, and then simultaneous localization and mapping can be performed based on a graph optimization algorithm to establish the target point cloud map of the target region.

[0059] In some optional embodiments, for a multi-layer parking lot, each layer can be regarded as a target region, and a target point cloud map corresponding to the target region can be established, and corresponding identifiers can be set for each layer to identify the layer to which the target point cloud map belongs, facilitating subsequent application.

[0060] The step 2012 obtains image data corresponding to at least one vehicle pose from the original image data collected at each vehicle pose.

[0061] The image data corresponding to at least one vehicle pose can be obtained at a certain sampling interval. The specific acquisition manner can be referred to the foregoing content, and will not be described here again.

[0062] In the embodiment, the vehicle pose and the corresponding image data are recorded in the point cloud map construction process of the target region, so that the image data corresponding to at least one vehicle pose can be sampled from the recorded image data of the vehicle pose as the sample data of the parking space detection model of the disclosure, and the effectiveness and richness of the sample data are improved.

[0063] In some optional embodiments, the step 202 of determining the region range corresponding to each vehicle pose based on the preset range determination rule comprises:

[0064] In step 2021, for any vehicle pose, the rectangular region in which the vehicle pose is located is determined based on the first distance in the first direction, the second distance in the second direction, the third distance in the third direction and the fourth distance in the fourth direction of the vehicle pose.

[0065] The first direction, the second direction, the third direction and the fourth direction can respectively refer to the front, the back, the left and the right of the vehicle pose. The front of the vehicle pose refers to the direction indicated by the vehicle attitude in the vehicle pose, and the back of the vehicle pose refers to the direction opposite to the front. The first distance, the second distance, the third distance and the fourth distance can be set according to actual needs, and are not limited in particular. The first distance in the first direction representing the front and the second distance representing the back can be the same or different. The third distance representing the left and the fourth distance representing the right can be the same or different.

[0066] In step 2022, the rectangular region is taken as the region range corresponding to the vehicle pose.

[0067] In the embodiment, the region range corresponding to the vehicle pose is set as the rectangular region around the vehicle, so that the first parking space label data in the region range is facilitated to be obtained, and the calculation amount is reduced.

[0068] In some optional embodiments, the step 203 of obtaining the first parking space label data in the map coordinate system corresponding to each region range can specifically comprise the following steps:

[0069] In step 2031, the target parking space label data set labeled in the map coordinate system corresponding to the target point cloud map is obtained.

[0070] The target parking space label data set comprises labels corresponding to each parking space included in the target point cloud map.

[0071] In some optional embodiments, the target parking space label data set can be obtained in any implementable standard manner, such as at least one of automatic labeling, semi-automatic labeling, manual labeling, etc. The specific manner is not limited.

[0072] In some optional embodiments, the label corresponding to each parking space can be represented in any implementable manner, such as including the coordinates of at least one corner point of the parking space, also including the center point coordinates of the parking space, also including the identification of each corner point, etc. The identification of the corner point is used to distinguish the relative position of the corner point in the parking space, so as to facilitate the determination of the entry line and the side line of the parking space.

[0073] In step 2032, for any region range, based on the region range, the first parking space label data corresponding to the region range is determined from the target parking space label data set.

[0074] The first parking space label data corresponding to the region range can be extracted from the target parking space label data set through the position range of the region range in the target point cloud map.

[0075] The embodiment obtains the parking space label data set in the map coordinate system by labeling the target point cloud map of the target region, provides effective data support for obtaining the parking space label data corresponding to each region range, and further provides the ground truth of the parking space in the vehicle coordinate system for the model training of the present disclosure.

[0076] In some optional embodiments, the obtaining of the target parking space label data set labeled in the map coordinate system corresponding to the target point cloud map in step 2031 comprises:

[0077] The target point cloud map is subjected to parking space detection based on the point cloud map parking space detection model obtained through pre-training, to obtain the target parking space label data set corresponding to the target point cloud map.

[0078] The point cloud map parking space detection model can be any implementable target detection model, such as a target detection model based on a convolutional neural network and its series, which can be set according to actual needs.

[0079] In some optional embodiments, the output result of the point cloud map parking space detection model can include at least one corner point coordinate of the four corner point coordinates of each parking space respectively corresponding to the detected parking space, and can also include the corner point identification corresponding to each corner point coordinate. The representation of the corner point identification can be set according to actual needs, such as four different numbers respectively representing the four corner points in the parking space.

[0080] In some optional embodiments, when one or two of the four corner points of the parking space are blocked during parking space labeling, the coordinates of the blocked corner points can be estimated according to the parking lines of the adjacent parking spaces.

[0081] The embodiment realizes automatic labeling of parking spaces of a target point cloud map by using a pre-trained point cloud map parking space detection model, and quickly obtains a target parking space label data set corresponding to the target point cloud map, thereby greatly improving labeling efficiency.

[0082] In some optional embodiments, the obtaining, in step 2031, of the target parking space label data set labeled in the map coordinate system corresponding to the target point cloud map comprises:

[0083] Based on the pre-trained point cloud map parking space detection model, a parking space detection is performed on the target point cloud map to obtain a target parking space label data set corresponding to the target point cloud map. The target parking space label data set and the target point cloud map are audited to obtain an audited parking space label data. The audited parking space label data is taken as the target parking space label data set.

[0084] The auditing of the target parking space label data set and the target point cloud map can be displaying the target point cloud map and the target parking space label data set on a screen together, determining the accuracy of the labeling result based on the matching of the labels of each parking space in the target parking space label data set and the actual parking spaces in the target point cloud map, correcting the labeling result that is inaccurate or incorrect, taking the corrected parking space label data as the audited parking space label data, and taking the audited parking space label data as the target parking space label data set when the auditing of the label data of all parking spaces is completed.

[0085] The embodiment can correct the parking space labels that are inaccurate or incorrect by further auditing the target parking space label data set labeled based on the target point cloud map, so as to improve the accuracy of the target parking space label data set, thereby improving the performance of the target parking space detection model of the present disclosure.

[0086] In some optional embodiments, the determining, in step 204, of the second parking space label data in the vehicle coordinate system corresponding to each vehicle pose based on the first parking space label data respectively corresponding to each region range comprises:

[0087] In step 2041, for any region range, based on the vehicle pose corresponding to the region range, the first parking space label data corresponding to the region range is converted from the map coordinate system to the vehicle coordinate system of the vehicle pose to obtain the second parking space label data in the vehicle coordinate system corresponding to the vehicle pose.

[0088] The vehicle pose represents the rotation and translation of the vehicle coordinate system relative to the map coordinate system, so the conversion relationship between the vehicle coordinate system and the map coordinate system can be determined based on the vehicle pose, and the conversion of the first parking space label data to the second parking space label data is realized based on the conversion relationship.

[0089] The embodiment effectively obtains the label data of the parking space label in the vehicle coordinate system which can be used as the parking space detection model of the disclosure by converting the first parking space label data in the map coordinate system of each region range to the corresponding vehicle coordinate system, thereby providing an effective reference true value for model training.

[0090] In some optional embodiments, the first parking space label data includes at least one parking space corner point coordinate corresponding to each parking space and a corner point identifier corresponding to each parking space corner point coordinate.

[0091] The corner point identifier is used to identify the relative position relationship of each corner point of the parking space and is used to determine the entry line, side line, etc. in the parking space. The entry line refers to the boundary line of the parking space through which the vehicle needs to pass when parking.

[0092] Exemplarily, Figure 5 is a schematic diagram of the corner point identifier provided by an exemplary embodiment of the disclosure. In this example, the corner point identifier is represented as a sequential number of 1, 2, 3, and 4, so that the entry line, the first side line, the distal end line, and the second side line can be determined according to the corner point identifier and the corner point coordinate. For example, the line segment between the corner point coordinates of the corner point identifiers 1 and 4 is the entry line, the corner point coordinates of the corner point identifiers 1 and 2 can determine the first side line, and so on. In actual application, the corner point identifier is not limited to the number in the figure, for example, 0, 1, 2, and 3 can be used, and P0-P3 can also be used, etc. As long as each corner point can be distinguished, it can be identified. Each parking space can be identified according to a unified identification method to facilitate subsequent application.

[0093] Step 2041 converts the first parking space label data corresponding to the region range from the map coordinate system to the vehicle coordinate system of the vehicle pose corresponding to the region range based on the vehicle pose corresponding to the region range, and obtains the second parking space label data in the vehicle coordinate system corresponding to the vehicle pose, including:

[0094] 1. For any parking space in the first parking space label data corresponding to the region range, determine the truncation state of the boundary of the region range to the parking space.

[0095] For the parking space located on the boundary of the region range, the boundary will be truncated, resulting in that part of the corner points of the parking space are in the region range and the other part of the corner points are outside the region range. Based on this, the truncation state of the boundary of the region range to the parking space can be determined. The truncation state can include two states of being truncated and not being truncated.

[0096] 2. For any parking space corner point coordinate of the parking space, determine the occlusion state of the parking space corner point coordinate in the vehicle pose corresponding to the region range based on a preset occlusion rule.

[0097] The preset occlusion rule can be set according to actual needs. The occlusion state can include two states of occlusion and non-occlusion. Occlusion means that the camera of each view angle cannot observe the parking angle point under the vehicle pose. Non-occlusion means that the parking angle point can be observed by at least one camera of the view angle under the vehicle pose.

[0098] 3. Determine the second parking label data based on the truncation state of each parking space corresponding to the first parking label data, the angle point identifier corresponding to each parking angle point coordinate, and the occlusion state corresponding to the angle point identifier.

[0099] The determination condition of the parking label can be set according to the different truncation states of the parking space and the different occlusion states of the parking angle point coordinate. The parking angle point coordinate of the parking space that meets the condition in the first parking label data is taken as the effective parking label. The parking space that does not meet the condition is filtered out, so as to improve the accuracy and effectiveness of the parking label data.

[0100] In some optional embodiments, the second parking label data is determined based on the truncation state of each parking space corresponding to the first parking label data, the angle point identifier corresponding to each parking angle point coordinate, and the occlusion state corresponding to the angle point identifier. Specifically, the determination can include:

[0101] a. For any parking space, in response to the truncation state corresponding to the parking space being truncated, the entry line state of the parking space is determined based on the parking angle point coordinate and the angle point identifier corresponding to the parking space. The entry line state includes a complete state and an incomplete state.

[0102] When the parking space is truncated, only part of the area of the parking space is within the range of the area, resulting in the parking space being an incomplete parking space within the range of the area. In order to further improve the accuracy of the parking label data, the accurate parking label of the incomplete parking space within the range of the area needs to be obtained by corresponding processing. The entry line can be represented by two entry angle points at both ends. When both of the two entry angle points are within the range of the area, it indicates that the entry line state is a complete state. If at least one of the two entry angle points is not within the range of the area, it indicates that the entry line state is an incomplete state.

[0103] b. In response to the entry line state of the parking space being a complete state, and the occlusion state of at least one of the two entry angle point coordinates of the parking space being non-occlusion, the two entry angle point coordinates and the two truncation point coordinates of the parking space are taken as the label of the parking space. The two truncation point coordinates are the intersection coordinates of the two side lines of the parking space and the boundary of the area range.

[0104] The entry line state is a complete state, indicating that both entry corner point coordinates are within the range of the region, and for both entry corner point coordinates, it is necessary to further determine whether the parking space can be an effective parking space in combination with the occlusion state. If at least one of the two entry corner point coordinates is not occluded, the parking space is an effective parking space.

[0105] In some optional embodiments, the two truncation point coordinates can be obtained by finding the intersection points of the two side lines of the parking space and the boundary of the region range. For example, the boundary of the region range is represented by a curve or a straight line, and the two side lines of the parking space are represented by two straight lines, and the intersection points of the boundary curve or straight line of the region range and the two straight lines are found. When the entry line state is a complete state, it indicates that the entry line of the parking space is within the effective perception range of the vehicle during parking, and can be an optional parking space for the vehicle. Therefore, during the model training stage, the two entry corner point coordinates and the two truncation point coordinates of the parking space are used as the label of the parking space, so that the model can learn more accurate parking space information around the vehicle in the vehicle pose.

[0106] An exemplary parking space is shown in FIG. 1. The parking space is truncated by the boundary of the region range. The boundary of the region range intersects the first side line of the parking space at a first truncation point and intersects the second side line of the parking space at a second truncation point. The label of the parking space includes the coordinates of the entry corner point 1, the coordinates of the first truncation point, the coordinates of the second truncation point, and the coordinates of the entry corner point 4. Figure 6 An exemplary truncated parking space is shown in FIG. 2. The parking space is truncated by the boundary of the region range. The boundary of the region range intersects the first side line of the parking space at a first truncation point and intersects the second side line of the parking space at a second truncation point. The label of the parking space includes the coordinates of the entry corner point 1, the coordinates of the first truncation point, the coordinates of the second truncation point, and the coordinates of the entry corner point 4.

[0107] c. In response to the truncation state of the parking space corresponding to the parking space being not truncated, and at least one of the occlusion states of the parking space corner point coordinates being not occluded, the parking space corner point coordinates of the parking space are used as the label of the parking space.

[0108] For a parking space that is not truncated, it indicates that the complete region of the parking space is within the region range. When at least one of the occlusion states of the parking space corner point coordinates is not occluded, the other corner point coordinates of the parking space in the vehicle coordinate system can be predicted according to the unoccluded parking space corner point coordinates and the related corner point coordinates of the adjacent parking space. Therefore, the parking space can be used as an effective parking space, and the parking space corner point coordinates of the parking space are used as the label of the parking space for model training. The model can predict the complete parking space in the vehicle coordinate system when at least one of the parking space corner point coordinates is observed, and provide effective parking space information for parking. When the occlusion states of all the corner point coordinates of the parking space are occluded, the parking space is an invalid parking space and is not used as the label of the parking space for model training.

[0109] d. Based on the labels of the parking spaces, the third parking space label data corresponding to the region range is determined.

[0110] According to the foregoing steps, the labels of the parking spaces in the first parking space label data that can be effective parking spaces can be obtained. The third parking space label data corresponding to the region range can be determined based on the labels of the parking spaces. For the parking space truncated in the region range, if it is ignored in the vehicle pose, it can be identified in other vehicle poses.

[0111] In some optional embodiments, for a parking space with an incomplete entry line state, the truncation points of the partial region range and the boundary of the region range can also be used as the label of the parking space. For example, only one entry corner point of the four corner points of the truncated parking space is in the region range. The entry corner point coordinate of the region range, the truncation point of the entry line and the boundary of the region range, and the truncation point of the side line where the entry corner point is located and the boundary of the region range are used as the label of the parking space. For another example, when both entry corner points of the parking space are not in the region range, and both distal end points are in the region range, the coordinates of the two distal end points and the truncation points of the two side lines and the boundary of the region range are used as the label of the parking space, and so on. The specific settings can be made according to actual needs.

[0112] e. converting the third parking space label data from the map coordinate system to the vehicle coordinate system of the vehicle pose to obtain the second parking space label data corresponding to the vehicle pose.

[0113] The conversion from the map coordinate system to the vehicle coordinate system can be referred to the foregoing content, which will not be repeated here.

[0114] The embodiment distinguishes the corner points of the parking space through the corner point identification, so that the label data of the parking space truncated by the boundary of the region range can be determined according to the truncated state, and the accuracy of the label data of the parking space is effectively improved.

[0115] In some optional embodiments, the occlusion state of the parking space corner point coordinate in the vehicle pose corresponding to the region range is determined based on a preset occlusion rule, which includes:

[0116] Converting the parking space corner point coordinate to the vehicle coordinate system corresponding to the vehicle pose to obtain a first corner point coordinate; projecting the first corner point coordinate to the image coordinate system corresponding to each view angle of the vehicle pose to obtain a projection point corresponding to each view angle; determining the segmentation type corresponding to each projection point based on the semantic segmentation result corresponding to the image of each view angle of the vehicle pose; in response to the segmentation type corresponding to the projection point of at least one view angle being the first type, determining that the occlusion state of the parking space corner point coordinate is unoccluded; or, in response to the segmentation types corresponding to the projection points of each view angle being the second type, or the projection point not being on the image of the view angle, determining that the occlusion state of the parking space corner point coordinate is occluded.

[0117] The specific operation principle of converting the parking space corner point coordinates to the vehicle coordinate system can be referred to the foregoing content, and will not be described here. The first corner point coordinates can be projected to the image coordinate system of each view based on the camera intrinsic and extrinsic parameters of each view. The semantic segmentation result corresponding to each view image can be obtained based on the pre-trained semantic segmentation model. The semantic segmentation model can be any implementable model, such as a semantic segmentation model based on FCN (Fully Convolutional Networks), a semantic segmentation model based on U-NET, a semantic segmentation model based on Deeplabv3 and its series, etc. The specific implementation can be set according to actual needs, and the present disclosure is not limited. The first type can include ground type or non-obstacle type, etc. representing the type that can be observed. The second type can include any obstacle type, representing that the obstacle is observed at the projection point, and the corner point cannot be observed. When the projection point of the first corner point coordinates in any view is of the first type in the segmentation type of the image of the view, it means that the corner point corresponding to the first corner point coordinates is not blocked in the view and can be observed, and thus it is determined that the blocking state of the first corner point coordinates in the view is unblocked. As long as the blocking state of the first corner point coordinates in at least one view is unblocked, it is determined that the blocking state of the corner point corresponding to the first corner point coordinates in the vehicle pose is unblocked. If the blocking state of the first corner point coordinates in all views is blocked, or the projection point of the first corner point coordinates is not on the image of each view, it is determined that the blocking state of the corner point corresponding to the first corner point coordinates in the vehicle pose is blocked.

[0118] In the embodiment, by projecting each parking space corner point coordinate to the image coordinate system of each view in the vehicle pose corresponding to the region range, the blocking state of the corner point can be effectively determined according to the semantic segmentation type corresponding to the projection point in the image of each view, so as to provide accurate and effective observable corner points for determining the parking space label data corresponding to the region range, and further improve the accuracy of the parking space label data.

[0119] In some optional embodiments, the blocking state of the parking space corner point coordinate in the vehicle pose corresponding to the region range is determined based on a preset blocking rule, including:

[0120] The parking space corner point coordinate is converted to the vehicle coordinate system corresponding to the vehicle pose to obtain a first corner point coordinate; the first point cloud data in the vehicle coordinate system corresponding to the vehicle pose is obtained; the distance between the first corner point coordinate and each point in the first point cloud data is determined; and in response to the distance between the first corner point coordinate and each point in the first point cloud data satisfying a preset condition, it is determined that the blocking state of the parking space corner point coordinate is unblocked.

[0121] The first point cloud data represents the situation of the obstacle points around the vehicle at the vehicle pose. The first point cloud data can be stored when the target point cloud map of the target area is established, or can be obtained from the target point cloud map, and the specific implementation is not limited. The preset condition can be set according to actual needs, for example, the preset condition can be that there is at least one distance less than the distance threshold. If the corner point corresponding to the first corner point coordinate is not blocked at the vehicle pose, the corner point should be an obstacle point at the vehicle pose, and then the first corner point coordinate should be near the first point cloud data. Based on this, the distance between the first corner point coordinate and each point in the first point cloud data can be calculated. If there is at least one point in the first point cloud data, the distance between the point and the first corner point coordinate is less than the distance threshold, which indicates that the point is the corner point corresponding to the first corner point coordinate, and the corner point is not blocked. Based on this, the blocking state corresponding to each parking space corner point coordinate can be determined.

[0122] In the embodiment, the blocking state corresponding to each parking space corner point coordinate is determined through the positional relationship between the point cloud data in the region range and the parking space corner point, so that accurate and effective observable corner points for determining the parking space label data corresponding to the region range are provided, and the accuracy of the parking space label data is further improved.

[0123] In some optional embodiments, the step 205 of training the first parking space detection model obtained in advance based on the image data and the second parking space label data corresponding to each vehicle pose to obtain a target parking space detection model includes:

[0124] The step 2051 takes any one of the vehicle poses as a target vehicle pose, and processes the images corresponding to each view angle of the target vehicle pose by using the first parking space detection model to obtain an image processing result.

[0125] The image processing result includes a parking space prediction result in the vehicle coordinate system corresponding to the target vehicle pose. The parking space prediction result can include prediction box information of the parking spaces included in the predicted images in the vehicle coordinate system. The prediction box information can include at least one corner point coordinate of each parking space in the vehicle coordinate system.

[0126] The step 2052 determines a network loss based on the parking space prediction result corresponding to each vehicle pose and the second parking space label data.

[0127] The network loss can be determined based on a preset loss function. The preset loss function can be any implementable loss function, such as an intersection over union (IOU) and its series of loss functions, an L1 loss function, an L2 loss function, and the like. The IOU loss function is determined based on the intersection over union of the parking space in the parking space prediction result and the corresponding parking space in the second parking space label data. The L1 loss function is a mean absolute error (MAE) loss function based on the parking space corner point position. The L2 loss function is a mean square error (MSE) loss function based on the parking space corner point position.

[0128] At step 2053, the network parameters of the first parking space detection model are updated based on the network loss to obtain a second parking space detection model.

[0129] The update of the network parameters can be determined by the gradient of the network loss and implemented by any implementable gradient descent algorithm, such as a stochastic gradient descent algorithm, a learning rate adaptive descent algorithm, and the like. The specific implementation can be set according to actual needs, and the present disclosure is not limited.

[0130] At step 2054, in response to the second parking space detection model satisfying a training end condition, the second parking space detection model is taken as a target parking space detection model.

[0131] The second parking space detection model can be determined to satisfy the training end condition based on the current iteration number and the network loss of the second parking space detection model. For example, the second parking space detection model can be determined to satisfy the training end condition when the current iteration number reaches a preset number threshold or the network loss of the second parking space detection model converges. The specific training end condition can be set according to actual needs, and the present disclosure is not limited.

[0132] At step 2055, in response to the second parking space detection model not satisfying the training end condition, the second parking space detection model is taken as the first parking space detection model, and the step of taking any vehicle pose in the vehicle poses as a target vehicle pose and processing the images of each view corresponding to the target vehicle pose by using the first parking space detection model to obtain an image processing result is repeatedly performed.

[0133] If the second parking space detection model does not satisfy the training end condition, the second parking space detection model needs to be iteratively trained. Specifically, the second parking space detection model can be taken as the first parking space detection model, and the steps 2051 to 2055 are repeatedly performed until the second parking space detection model obtained by the update satisfies the training end condition, and the second parking space detection model is taken as the target parking space detection model.

[0134] Figure 7is a flowchart of step 2051 provided by an example embodiment of the present disclosure.

[0135] In some optional embodiments, the processing of the images of each perspective corresponding to the target vehicle pose in step 2051 by using the first parking space detection model to obtain the image processing result includes:

[0136] In step 20511, the first feature extraction network in the first parking space detection model is used to process the images of each perspective respectively to obtain the image features corresponding to each perspective respectively.

[0137] The first feature extraction network can be any implementable feature extraction network, such as a feature extraction network based on a convolutional neural network, specifically a feature extraction network based on VGG (Visual Geometry Group), a feature extraction network based on Resnet, etc., without limitation.

[0138] In some optional embodiments, the first feature extraction network can include branch networks corresponding to each perspective respectively, or can include an overall feature extraction network for extracting features from the images of each perspective simultaneously, without limitation to the specific network structure.

[0139] In step 20512, the feature mapping network in the first parking space detection model is used to process the image features corresponding to each perspective respectively to obtain the first bird's eye view features corresponding to each perspective respectively.

[0140] The feature mapping network is used to map the image features corresponding to each perspective respectively to the bird's eye view perspective of the vehicle coordinate system. Similar to the first feature extraction network, the feature mapping network can include branch mapping networks corresponding to each perspective respectively, or can be an overall feature mapping network for mapping each perspective simultaneously, which can be set according to actual needs.

[0141] In some optional embodiments, the feature mapping network can adopt a network structure of an attention mechanism, a network structure of a Multilayer Perceptron (MLP), etc. The specific setting can be made according to actual needs.

[0142] In some optional embodiments, the feature mapping network can also be based on a homography matrix, depth estimation, etc. to realize feature mapping, which can be set according to actual needs. The homography matrix refers to determining a mapping conversion matrix based on the camera intrinsic and extrinsic parameters of each perspective respectively to realize feature mapping. The depth estimation refers to estimating the depth information of each perspective image to establish a conversion matrix between the image coordinate system and the vehicle coordinate system to realize feature mapping.

[0143] In step 20513, the first bird's eye view feature corresponding to each view angle is fused by using a feature fusion network in the first parking space detection model to obtain a bird's eye view fusion feature.

[0144] In the above process, since different view angles correspond to different regions in the bird's eye view, after obtaining the first bird's eye view feature corresponding to each view angle, the first bird's eye view features of each view angle need to be fused into the overall feature in the bird's eye view. The network structure of the specific feature fusion network can be set according to actual needs. For example, the feature fusion can be performed by using a concatenation (Concat) layer, the feature fusion can also be performed by using addition (add), the time sequence feature fusion can also be performed by using a feature fusion network based on a recurrent neural network, and the like, and the specific implementation is not limited.

[0145] In step 20514, the bird's eye view fusion feature is processed by using a second feature extraction network in the first parking space detection model to obtain a second bird's eye view feature.

[0146] In the above process, the second feature extraction network can be any implementable feature extraction network, such as a feature extraction network based on a convolutional neural network, and the specific implementation can be set according to actual needs. The second feature extraction network is used to further extract the bird's eye view feature, so as to extract the feature more related to the parking space, which is convenient for subsequent prediction of the parking space.

[0147] In step 20515, the second bird's eye view feature is processed by using a prediction head network in the first parking space detection model to obtain an image processing result.

[0148] In the above process, the prediction head network can be any implementable network result, such as a network structure based on a convolutional neural network or a network structure based on a multilayer perceptron, and the specific implementation is not limited.

[0149] In the above process, the first feature extraction network is used to extract the features of the images in each view angle, the feature mapping network is used to map the image features in each view angle to the bird's eye view, the feature fusion network is used to fuse the bird's eye view features in each view angle, and then the second feature extraction network is used to further extract the bird's eye view fusion feature, so as to provide more effective bird's eye view features for the parking space prediction of the prediction head network, thereby obtaining the parking space prediction result in the vehicle coordinate system, realizing the mapping relationship from the multi-view image to the parking space in the vehicle coordinate system, and effectively improving the accuracy of the parking space detection result.

[0150] Figure 8is a schematic diagram of a network structure of a first parking space detection model provided by an example embodiment of the present disclosure. The network structure includes a first feature extraction network, a feature mapping network, a feature fusion network, a second feature extraction network, and a prediction head network. The first feature extraction network is configured to perform feature extraction on images of multiple perspectives to obtain image features corresponding to each perspective, the feature mapping network is configured to convert the image features corresponding to each perspective to a bird's eye view perspective to obtain first bird's eye view features corresponding to each perspective, the feature fusion network is configured to fuse the first bird's eye view features corresponding to each perspective to obtain bird's eye view fusion features, the second feature extraction network is configured to perform feature extraction on the bird's eye view fusion features to obtain second bird's eye view features, and the prediction head network is configured to process the second bird's eye view features to obtain an image processing result.

[0151] In some optional embodiments, the feature fusion network is a time sequence-based feature fusion network; and the step of fusing, by the feature fusion network in the first parking space detection model, the first bird's eye view features corresponding to each perspective to obtain the bird's eye view fusion features includes:

[0152] obtaining previous bird's eye view features, the previous bird's eye view features including bird's eye view features corresponding to a first number of vehicle poses before a target vehicle pose; converting the previous bird's eye view features to a bird's eye view perspective of the target vehicle pose to obtain target bird's eye view features corresponding to the previous bird's eye view features; and fusing, by the feature fusion network in the first parking space detection model, the first bird's eye view features corresponding to each perspective and the target bird's eye view features to obtain the bird's eye view fusion features.

[0153] The time sequence-based feature fusion network can be implemented based on a recurrent neural network. The time sequence-based feature fusion network is configured to fuse the first bird's eye view features corresponding to each perspective and the target bird's eye view features corresponding to the previous bird's eye view features to obtain time sequence-based bird's eye view fusion features. For the first number of vehicle poses before the target vehicle pose, a collection time corresponding to the target vehicle pose can be represented as t n , the first number of vehicle poses can be represented as t n-1 , t n-2 , …, t n-M , and M represents the first number.

[0154] In some optional embodiments, the first quantity of vehicle poses respectively correspond to aerial view features can include aerial view features corresponding to each view angle at each vehicle pose, and correspondingly, the target aerial view feature includes features converted to the target vehicle pose from aerial view features at each vehicle pose. For the fusion of the first aerial view features at each view angle and the target aerial view feature, the multi-view aerial view feature fusion can be performed first for each vehicle pose and the target vehicle pose to obtain fusion features respectively corresponding to each vehicle pose and a fusion feature corresponding to the target vehicle pose, and then the temporal fusion is performed on the fusion features respectively corresponding to each vehicle pose and the fusion feature corresponding to the target vehicle pose to obtain the temporal aerial view fusion feature. Alternatively, the temporal fusion can be performed first, and then the multi-view fusion is performed. The specific fusion manner is not limited.

[0155] In some optional embodiments, the first quantity of vehicle poses respectively correspond to aerial view features can include multi-view fused aerial view fusion features at each vehicle pose, and correspondingly, the target aerial view feature includes target fusion features converted to the target vehicle pose from the aerial view fusion features at each vehicle pose. For the fusion of the first aerial view features at each view angle and the target aerial view feature, the multi-view fusion can be performed first on the first aerial view features at each view angle to obtain the aerial view fusion feature corresponding to the target vehicle pose, and then the temporal fusion is performed on the aerial view fusion feature corresponding to the target vehicle pose and the target fusion features corresponding to each vehicle pose to obtain the temporal aerial view fusion feature.

[0156] In an optional embodiment, Figure 9 is a network structure schematic diagram of the first parking space detection model provided by another exemplary embodiment of the present disclosure. In the present example, the first feature extraction network includes feature extraction branch networks respectively corresponding to each view angle, the feature mapping network includes branch mapping networks respectively corresponding to each view angle, the feature fusion network is a temporal network structure, the target vehicle pose is t n , the preceding aerial view feature is a preceding aerial view fusion feature, and the preceding aerial view fusion feature includes aerial view fusion features respectively corresponding to M time points of t n-1 , t n-2 , …, t n-M , M is the first quantity, and correspondingly, the vehicle pose corresponding to the preceding aerial view fusion feature includes vehicle poses respectively corresponding to M time points of t n-1 , t n-2 , …, t n-M , the first aerial view features at each view angle at t n are fused to obtain the aerial view fusion feature at t n , and the aerial view fusion features respectively corresponding to each time point are converted to the target vehicle pose t nThe time sequence bird's eye view feature is fused to obtain a time sequence bird's eye view fusion feature, and the time sequence bird's eye view fusion feature is further subjected to feature extraction and prediction to obtain an image processing result. In this example, the bird's eye view fusion features of different time sequences can also carry the vehicle pose of the corresponding time sequence to further improve the model performance. n The time sequence bird's eye view feature is fused to obtain a time sequence bird's eye view fusion feature, and the time sequence bird's eye view fusion feature is further subjected to feature extraction and prediction to obtain an image processing result. In this example, the bird's eye view fusion features of different time sequences can also carry the vehicle pose of the corresponding time sequence to further improve the model performance.

[0157] The embodiments of the present disclosure can densely select vehicle poses in the established target point cloud map, obtain images and annotated observation continuous true values corresponding to each vehicle pose, realize time sequence fusion of bird's eye view features, and further improve the model performance.

[0158] The above-mentioned embodiments of the present disclosure can be implemented alone or in combination in a non-conflicting manner, and can be set according to actual needs. The present disclosure is not limited.

[0159] The training method of any one of the parking space detection models provided by the embodiments of the present disclosure can be executed by any appropriate device with data processing capability, including but not limited to terminal devices and servers. Alternatively, the training method of any one of the parking space detection models provided by the embodiments of the present disclosure can be executed by a processor, such as a processor executing the training method of any one of the parking space detection models mentioned by the embodiments of the present disclosure by calling corresponding instructions stored in a memory. The following will not be repeated.

[0160] Figure 10 is a flowchart of a parking space detection method provided by an exemplary embodiment of the present disclosure. The present embodiment can be applied on an electronic device, such as a vehicle-mounted computing platform, as shown in Figure 10 The present embodiment includes the following steps:

[0161] Step 301: Obtain images to be processed corresponding to each view angle in at least one view angle.

[0162] The images to be processed corresponding to each view angle can be images of the surroundings of the vehicle collected by the camera when the vehicle is driving in the target area.

[0163] Step 302: Process the images to be processed corresponding to each view angle by using a target parking space detection model to obtain an image processing result.

[0164] The target parking space detection model is obtained based on the training method of the parking space detection model provided by any one of the above-mentioned embodiments. For details, please refer to the above-mentioned embodiments, which will not be repeated here.

[0165] Step 303: Determine the observed parking space detection result in the vehicle coordinate system based on the image processing result.

[0166] The image processing result includes a probability that each point in a vehicle coordinate system range output by the target parking space detection model belongs to a parking space corner point, and a corner point coordinate of at least one corner point of each observed parking space is determined based on the image processing result and a preset probability threshold as a parking space detection result, which can be used to assist a vehicle in planning and controlling various parking.

[0167] Any of the parking space detection methods provided in the embodiments of the present disclosure can be executed by any appropriate device with data processing capability, including but not limited to terminal devices and servers, etc. Alternatively, any of the parking space detection methods provided in the embodiments of the present disclosure can be executed by a processor, such as a processor executing any of the parking space detection methods mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. Details are not described herein.

[0168] Example apparatus

[0169] Figure 11 FIG. 1 is a structural schematic diagram of a training device of a parking space detection model provided in an example embodiment of the present disclosure. The training device of the parking space detection model of this embodiment can be used to implement the training method embodiments of the corresponding parking space detection model of the present disclosure, such as the training device shown in FIG. 1. Figure 11 The device shown in FIG. 1 includes a first acquisition module 501, a first processing module 502, a second processing module 503, a third processing module 504, and a fourth processing module 505.

[0170] The first acquisition module 501 is configured to determine a target point cloud map of a target region and image data corresponding to at least one vehicle pose in a map coordinate system of the target point cloud map. The image data corresponding to any vehicle pose includes at least one image collected at a vehicle pose from a perspective.

[0171] The first processing module 502 is configured to determine a region range corresponding to each vehicle pose based on a preset range determination rule.

[0172] The second processing module 503 is configured to obtain first parking space label data in a map coordinate system corresponding to each region range.

[0173] The third processing module 504 is configured to determine second parking space label data in a vehicle coordinate system corresponding to each vehicle pose based on the first parking space label data corresponding to each region range.

[0174] The fourth processing module 505 is configured to train a first parking space detection model obtained in advance based on the image data and the second parking space label data corresponding to each vehicle pose, to obtain a target parking space detection model.

[0175] Figure 12FIG. 1 is a schematic structural diagram of a parking space detection model training device provided by another example embodiment of the present disclosure.

[0176] In some optional embodiments, the first acquisition module 501 includes:

[0177] The map establishing unit 5011 is configured to establish a target point cloud map of the target region based on at least one of original image data, radar point cloud data, and combined inertial navigation data of the target region collected by the vehicle at each vehicle pose.

[0178] The first acquisition unit 5012 is configured to acquire image data corresponding to at least one vehicle pose from original image data collected at each vehicle pose.

[0179] In some optional embodiments, the first processing module 502 includes:

[0180] The first determination unit 5021 is configured to determine, for any vehicle pose, a rectangular region in which the vehicle pose is located based on a first distance in a first direction, a second distance in a second direction, a third distance in a third direction, and a fourth distance in a fourth direction of the vehicle pose.

[0181] The second determination unit 5022 is configured to take the rectangular region as a region range corresponding to the vehicle pose.

[0182] In some optional embodiments, the second processing module 503 includes:

[0183] The second acquisition unit 5031 is configured to acquire a target parking space label data set labeled in a map coordinate system corresponding to the target point cloud map.

[0184] The target parking space label data set includes labels corresponding to each parking space included in the target point cloud map.

[0185] The third determination unit 5032 is configured to determine, for any region range, first parking space label data corresponding to the region range from the target parking space label data set based on the region range.

[0186] In some optional embodiments, the second acquisition unit 5031 is specifically configured to:

[0187] Perform parking space detection on the target point cloud map based on a point cloud map parking space detection model obtained through pre-training, to obtain a target parking space label data set corresponding to the target point cloud map.

[0188] In some optional embodiments, the second acquisition unit 5031 is specifically configured to:

[0189] The target point cloud map is subjected to parking space detection based on a pre-trained point cloud map parking space detection model to obtain target parking space label data corresponding to the target point cloud map; the target parking space label data and the target point cloud map are subjected to auditing to obtain audited parking space label data; and the audited parking space label data is taken as the target parking space label data.

[0190] In some optional embodiments, the third processing module 504 comprises:

[0191] The first processing unit 5041 is configured to, for any area range, convert first parking space label data corresponding to the area range from a map coordinate system to a vehicle coordinate system of a vehicle pose corresponding to the area range based on the vehicle pose, to obtain second parking space label data in the vehicle coordinate system corresponding to the vehicle pose.

[0192] In some optional embodiments, the first parking space label data comprises at least one parking space corner point coordinate corresponding to each parking space and a corner point identifier corresponding to each parking space corner point coordinate; and the first processing unit 5041 is specifically configured to:

[0193] For any parking space in the first parking space label data corresponding to the area range, determine a truncation state of a boundary of the area range to the parking space; for any parking space corner point coordinate of the parking space, determine an occlusion state of the parking space corner point coordinate in the vehicle pose corresponding to the area range based on a preset occlusion rule; and determine the second parking space label data based on the truncation state corresponding to each parking space in the first parking space label data, the corner point identifier corresponding to each parking space corner point coordinate, and the corresponding occlusion state.

[0194] In some optional embodiments, the first processing unit 5041 is specifically configured to:

[0195] For any parking space, in response to the truncation state corresponding to the parking space being truncated, determine an entry line state of the parking space based on the parking space corner point coordinate and the corner point identifier corresponding to the parking space, the entry line state comprising a complete state and an incomplete state; in response to the entry line state of the parking space being the complete state and the occlusion state of at least one of the two entry corner point coordinates of the parking space being unoccluded, take the two entry corner point coordinates and two truncation point coordinates of the parking space as labels of the parking space, the two truncation point coordinates being intersection coordinates of two side lines of the parking space and the boundary of the area range; in response to the truncation state corresponding to the parking space being untruncated and the occlusion state of at least one of the parking space corner point coordinates of the parking space being unoccluded, take each parking space corner point coordinate of the parking space as a label of the parking space; determine third parking space label data corresponding to the area range based on the labels of each parking space; and convert the third parking space label data from the map coordinate system to the vehicle coordinate system of the vehicle pose to obtain second parking space label data corresponding to the vehicle pose.

[0196] In some optional embodiments, the first processing unit 5041 is specifically configured to:

[0197] convert the parking space corner point coordinate into a vehicle coordinate system corresponding to the vehicle pose to obtain a first corner point coordinate; obtain first point cloud data in the vehicle coordinate system corresponding to the vehicle pose; determine distances between the first corner point coordinate and each point in the first point cloud data; and in response to the distances between the first corner point coordinate and each point in the first point cloud data satisfying a preset condition, determine that the occlusion state of the parking space corner point coordinate is unoccluded.

[0198] In some optional embodiments, the fourth processing module 505 includes:

[0199] The second processing unit 5051 is configured to take any one of the vehicle poses as a target vehicle pose, and process images of each view corresponding to the target vehicle pose by using the first parking space detection model to obtain an image processing result.

[0200] The image processing result includes a parking space prediction result in a vehicle coordinate system corresponding to the target vehicle pose.

[0201] The third processing unit 5052 is configured to determine a network loss based on the parking space prediction result corresponding to each vehicle pose and the second parking space label data.

[0202] The fourth processing unit 5053 is configured to update network parameters of the first parking space detection model based on the network loss to obtain a second parking space detection model.

[0203] The fifth processing unit 5054 is configured to take the second parking space detection model as a target parking space detection model in response to the first parking space detection model satisfying a training end condition.

[0204] The sixth processing unit 5055 is configured to take the second parking space detection model as the first parking space detection model in response to the second parking space detection model not satisfying the training end condition, and repeat the step of taking any one of the vehicle poses as a target vehicle pose and processing images of each view corresponding to the target vehicle pose by using the first parking space detection model to obtain an image processing result.

[0205] In some optional embodiments, the second processing unit 5051 is specifically configured to:

[0206] The first feature extraction network in the first parking space detection model is used to process images of each view angle respectively to obtain image features corresponding to each view angle respectively; the feature mapping network in the first parking space detection model is used to process the image features corresponding to each view angle respectively to obtain first bird's eye view features corresponding to each view angle respectively; the feature fusion network in the first parking space detection model is used to fuse the first bird's eye view features corresponding to each view angle respectively to obtain bird's eye view fusion features; the second feature extraction network in the first parking space detection model is used to process the bird's eye view fusion features to obtain second bird's eye view features; and the prediction head network in the first parking space detection model is used to process the second bird's eye view features to obtain the image processing result.

[0207] In some optional embodiments, the feature fusion network is a time sequence-based feature fusion network; and the second processing unit 5051 is specifically configured to:

[0208] The feature fusion network in the first parking space detection model is used to fuse the first bird's eye view features corresponding to each view angle respectively and the previous bird's eye view features corresponding to each view angle respectively to obtain bird's eye view fusion features, and the previous bird's eye view features corresponding to any view angle include bird's eye view features of the first number of vehicle poses before the target vehicle pose.

[0209] The beneficial technical effects corresponding to the exemplary embodiments of the device can be referred to the corresponding beneficial technical effects of the exemplary method embodiments described above, which will not be described herein.

[0210] Figure 13 is a structural schematic diagram of a parking space detection device provided by an exemplary embodiment of the present disclosure. The parking space detection device of this embodiment can be used to implement the corresponding parking space detection method embodiments of the present disclosure, such as Figure 13 The device shown in the figure includes a second acquisition module 601, a fifth processing module 602, and a sixth processing module 603.

[0211] The second acquisition module 601 is configured to acquire images to be processed corresponding to each view angle in at least one view angle.

[0212] The fifth processing module 602 is configured to process the images to be processed corresponding to each view angle respectively by using a target parking space detection model to obtain an image processing result, and the target parking space detection model is obtained based on the training method of the parking space detection model provided in any of the above embodiments.

[0213] The sixth processing module 603 is configured to determine a parking space detection result in a vehicle coordinate system based on the image processing result.

[0214] The beneficial technical effects corresponding to the exemplary embodiments of the device can be referred to the corresponding beneficial technical effects of the exemplary method embodiments described above, which will not be described herein.

[0215] Example electronic device

[0216] Figure 14 is a structural diagram of an electronic device provided by an embodiment of the present disclosure, comprising at least one processor 11 and a memory 12.

[0217] The processor 11 can be a central processing unit (CPU) or other forms of processing unit having data processing capability and / or instruction execution capability, and can control other components in the electronic device 10 to perform desired functions.

[0218] The memory 12 can comprise one or more computer program products, which can comprise various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache memory, and / or the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer readable storage medium, and the processor 11 can run the one or more computer program instructions to implement the methods and / or other desired functions of various embodiments of the present disclosure described above.

[0219] In one example, the electronic device 10 can further comprise an input device 13 and an output device 14, which are interconnected through a bus system and / or other forms of connection mechanism (not shown).

[0220] The input device 13 can further comprise, for example, a keyboard, a mouse, and / or the like.

[0221] The output device 14 can output various information to the outside, which can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0222] Of course, in order to simplify, Figure 14 Only some of the components in the electronic device 10 related to the present disclosure are shown in the figure, and components such as buses, input / output interfaces, and / or the like are omitted. In addition, the electronic device 10 can further comprise any other appropriate components according to specific application cases.

[0223] Example computer program product and computer readable storage medium

[0224] In addition to the above methods and devices, embodiments of the present disclosure can also provide a computer program product comprising computer program instructions, which, when run by a processor, cause the processor to perform the steps in the methods of various embodiments of the present disclosure described in the “Exemplary Methods” section above.

[0225] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments of the present disclosure can be a computer program product, which can include a computer program tangibly embodied in a machine readable storage medium.

[0226] Moreover, an embodiment of the present disclosure can also be a computer readable storage medium having stored thereon a computer program instructing, which can make a processor execute the steps of the methods of various embodiments of the present disclosure described in the above "Exemplary Methods" section when the computer program is run by the processor.

[0227] The computer readable storage medium can be any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0228] The above describes the basic principles of the present disclosure in combination with specific embodiments, but the advantages, benefits, effects and the like mentioned in the present disclosure are only examples and are not limiting, and it cannot be considered that each embodiment of the present disclosure must have them. In addition, the above specific details are only for the purpose of example and for the purpose of understanding, and are not limiting, and the above details do not limit the present disclosure to be necessarily implemented with the above specific details.

[0229] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A training method for a parking space detection model, comprising: The target area is determined by a target point cloud map and image data corresponding to at least one vehicle pose in the map coordinate system of the target point cloud map. The image data corresponding to any vehicle pose includes images from at least one viewpoint acquired under the vehicle pose. The target area is an area that includes parking spaces. Based on the preset range determination rules, the region range corresponding to each vehicle pose is determined. Obtain the first parking space label data in the map coordinate system corresponding to each of the aforementioned regions; the first parking space label data includes the labels corresponding to each parking space within the respective region. Based on the first parking space label data corresponding to each of the said regions, determine the second parking space label data in the vehicle coordinate system corresponding to each of the said vehicle poses; Based on the image data corresponding to the poses of each vehicle and the second parking space label data, the pre-obtained first parking space detection model is trained to obtain the target parking space detection model.

2. The method according to claim 1, wherein, The step of obtaining the first parking space label data in the map coordinate system corresponding to each of the aforementioned regions includes: Obtain the target parking space label dataset corresponding to the target point cloud map and annotated in the map coordinate system. The target parking space label dataset includes the labels corresponding to each parking space included in the target point cloud map. For any of the aforementioned regions, the first parking space label data corresponding to that region is determined from the target parking space label dataset.

3. The method according to claim 2, wherein, The step of obtaining the target parking space label dataset corresponding to the target point cloud map and annotated in the map coordinate system includes: Based on the pre-trained point cloud map parking space detection model, parking spaces are detected on the target point cloud map to obtain the target parking space label dataset corresponding to the target point cloud map.

4. The method according to claim 3, wherein, After the parking space detection model based on the pre-trained point cloud map performs parking space detection on the target point cloud map and obtains the target parking space label dataset corresponding to the target point cloud map, the method further includes: The target parking space label dataset and the target point cloud map are reviewed to obtain the reviewed parking space label data. The reviewed parking space label data is used as the target parking space label dataset.

5. The method according to claim 1, wherein, The step of training a pre-obtained first parking space detection model based on the image data corresponding to each of the vehicle poses and the second parking space label data to obtain a target parking space detection model includes: Take any one of the vehicle poses as the target vehicle pose, and use the first parking space detection model to process the images of each viewpoint corresponding to the target vehicle pose to obtain image processing results. The image processing results include parking space prediction results in the vehicle coordinate system corresponding to the target vehicle pose. Based on the parking space prediction results corresponding to each of the vehicle poses and the second parking space label data, the network loss is determined. The network parameters of the first parking space detection model are updated based on the network loss to obtain the second parking space detection model. In response to the second parking space detection model meeting the training termination condition, the second parking space detection model is used as the target parking space detection model; or... In response to the second parking space detection model not meeting the training termination condition, the second parking space detection model is used as the first parking space detection model, and the steps of taking any one of the vehicle poses as the target vehicle pose and using the first parking space detection model to process the images of each viewpoint corresponding to the target vehicle pose to obtain the image processing results are repeated.

6. The method according to claim 5, wherein, The step of processing the images of the target vehicle's pose from various viewpoints using the first parking space detection model to obtain image processing results includes: Using the first feature extraction network in the first parking space detection model, the images from each of the aforementioned viewpoints are processed to obtain the image features corresponding to each of the aforementioned viewpoints. Using the feature mapping network in the first parking space detection model, the image features corresponding to each of the aforementioned viewpoints are processed to obtain the first bird's-eye view features corresponding to each of the aforementioned viewpoints. By using the feature fusion network in the first parking space detection model, the first bird's-eye view features corresponding to each of the aforementioned viewpoints are fused to obtain the bird's-eye view fusion features; The second feature extraction network in the first parking space detection model is used to process the bird's-eye view fusion features to obtain the second bird's-eye view features; The image processing result is obtained by using the prediction head network in the first parking space detection model to process the features of the second bird's-eye view.

7. The method according to claim 6, wherein, The feature fusion network is a time-series based feature fusion network; The step of using the feature fusion network in the first parking space detection model to fuse the first bird's-eye view features corresponding to each of the aforementioned viewpoints to obtain fused bird's-eye view features includes: Obtain the preceding bird's-eye view features, which include the bird's-eye view features corresponding to a first number of vehicle poses preceding the target vehicle pose. The foreground bird's-eye view features are converted to the bird's-eye view perspective of the target vehicle's pose to obtain the target bird's-eye view features corresponding to the foreground bird's-eye view features; Using the feature fusion network in the first parking space detection model, the first bird's-eye view features corresponding to each of the aforementioned viewpoints are fused with the target bird's-eye view features to obtain the fused bird's-eye view features.

8. The method according to claim 1, wherein, The step of determining the second parking space label data in the vehicle coordinate system corresponding to each of the vehicle poses based on the first parking space label data corresponding to each of the aforementioned regions includes: For any of the aforementioned regions, based on the vehicle pose corresponding to that region, the first parking space label data corresponding to that region is transformed from the map coordinate system to the vehicle coordinate system of the vehicle pose to obtain the second parking space label data in the vehicle coordinate system corresponding to the vehicle pose.

9. The method according to claim 8, wherein, The first parking space label data includes the coordinates of at least one corner point corresponding to each parking space and the corner point identifier corresponding to each corner point coordinate; Based on the vehicle pose corresponding to the area, the first parking space label data corresponding to the area is transformed from the map coordinate system to the vehicle coordinate system of the vehicle pose to obtain the second parking space label data in the vehicle coordinate system corresponding to the vehicle pose, including: For any parking space in the first parking space tag data corresponding to the area range, determine the truncation state of the parking space by the boundary of the area range; For any corner point coordinate of the parking space, the occlusion status of the corner point coordinate under the vehicle pose in the area is determined based on the preset occlusion rules. The second parking space label data is determined based on the truncation state corresponding to each parking space in the first parking space label data, the corner point identifier corresponding to the corner point coordinates of each parking space, and the corresponding occlusion state.

10. The method according to claim 9, wherein, The step of determining the second parking space label data based on the truncation state corresponding to each parking space in the first parking space label data, the corner marker corresponding to the corner coordinates of each parking space, and the corresponding occlusion state includes: For any of the parking spaces, in response to the truncation state corresponding to the parking space being truncated, the entrance line state of the parking space is determined based on the corner coordinates and the corner identifier of the parking space. The entrance line state includes a complete state and an incomplete state. In response to the parking space's entrance line being in a complete state, and at least one of the two entrance corner coordinates of the parking space being in an unobstructed state, the two entrance corner coordinates and two cutoff point coordinates of the parking space are used as the parking space's label. The two cutoff point coordinates are the intersection points of the two side lines of the parking space with the boundaries of the area; or, In response to the truncation state corresponding to the parking space being untruncation, and the occlusion state of the parking space having at least one corner coordinate being unoccluded, the corner coordinates of the parking space are used as the label of the parking space. Based on the labels of each parking space, determine the label data of the third parking space corresponding to the area. The third parking space label data is transformed from the map coordinate system to the vehicle coordinate system of the vehicle pose to obtain the second parking space label data corresponding to the vehicle pose.

11. The method according to claim 9, wherein, The process of determining the occlusion state of the parking space corner coordinates within the area corresponding to the vehicle's pose based on preset occlusion rules includes: Transform the coordinates of the parking space corner point to the vehicle coordinate system corresponding to the vehicle pose to obtain the coordinates of the first corner point; Project the coordinates of the first corner point onto the image coordinate system corresponding to each of the viewpoints of the vehicle pose to obtain the projection points of the first corner point coordinates corresponding to each of the viewpoints. Based on the semantic segmentation results corresponding to the images of each viewpoint of the vehicle pose, the segmentation type corresponding to each projection point is determined. In response to the existence of at least one projection point corresponding to the first type of the viewpoint, the occlusion state of the corner point coordinates of the parking space is determined to be unoccluded; or, In response to the fact that the segmentation type of the projection point corresponding to each of the aforementioned viewpoints is the second type, or that the projection point is not on the image of the viewpoint, the occlusion state of the parking space corner coordinates is determined to be occlusion.

12. The method according to claim 9, wherein, The process of determining the occlusion state of the parking space corner coordinates within the area corresponding to the vehicle's pose based on preset occlusion rules includes: Transform the coordinates of the parking space corner point to the vehicle coordinate system corresponding to the vehicle pose to obtain the coordinates of the first corner point; Obtain the first point cloud data in the vehicle coordinate system corresponding to the vehicle pose; Determine the distances between the coordinates of the first corner point and each point in the first point cloud data; In response to the fact that the distance between the first corner point coordinates and each point in the first point cloud data meets a preset condition, the occlusion state of the parking space corner point coordinates is determined to be unoccluded.

13. The method according to claim 1, wherein, The determination of the region range corresponding to each vehicle pose based on the preset range determination rule includes: For any of the vehicle poses, the rectangular region where the vehicle pose is located is determined based on the first distance in the first direction, the second distance in the second direction, the third distance in the third direction, and the fourth distance in the fourth direction. The rectangular area is taken as the region corresponding to the vehicle's pose.

14. The method according to claim 1, wherein, The target point cloud map of the determined target area and the image data corresponding to at least one vehicle pose in the map coordinate system of the target point cloud map include: Based on at least one of the original image data, radar point cloud data, and combined inertial navigation data of the target area collected by the vehicle in various vehicle poses, a target point cloud map of the target area is established. The image data corresponding to the at least one vehicle pose is obtained from the original image data collected under each of the vehicle poses.

15. A parking space detection method, comprising: Obtain the images to be processed corresponding to each of the at least one viewpoint; The target parking space detection model is used to process the images to be processed corresponding to each of the aforementioned viewpoints to obtain image processing results. The target parking space detection model is obtained based on the training method of the parking space detection model according to any one of claims 1-14. Based on the image processing results, the parking space detection results in the observed vehicle coordinate system are determined.

16. A training device for a parking space detection model, comprising: The first acquisition module is used to determine the target point cloud map of the target area and the image data corresponding to at least one vehicle pose in the map coordinate system of the target point cloud map. The image data corresponding to any vehicle pose includes images from at least one viewpoint acquired under the vehicle pose. The target area is an area including parking spaces. The first processing module is used to determine the region range corresponding to each of the vehicle poses based on a preset range determination rule. The second processing module is used to obtain first parking space label data in the map coordinate system corresponding to each of the said area ranges; the first parking space label data includes labels corresponding to each parking space within the said area range; The third processing module is used to determine the second parking space label data in the vehicle coordinate system corresponding to each of the vehicle poses based on the first parking space label data corresponding to each of the area ranges. The fourth processing module is used to train the pre-obtained first parking space detection model based on the image data corresponding to each of the vehicle poses and the second parking space label data to obtain the target parking space detection model.

17. A parking space detection device, comprising: The second acquisition module is used to acquire the image to be processed corresponding to each of the at least one viewpoint; The fifth processing module is used to process the images to be processed corresponding to each of the aforementioned viewpoints using the target parking space detection model to obtain image processing results. The target parking space detection model is obtained based on the training method of the parking space detection model according to any one of claims 1-14. The sixth processing module is used to determine the parking space detection result in the observed vehicle coordinate system based on the image processing result.

18. A computer-readable storage medium storing a computer program for executing the training method of the parking space detection model according to any one of claims 1-14, or executing the parking space detection method according to claim 15.

19. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the training method of the parking space detection model according to any one of claims 1-14, or to implement the parking space detection method according to claim 15.

Images