Parking space detection model training method, parking space detection method and related equipment

Abstract

The application discloses a parking space detection model training method, a parking space detection method and related equipment. The parking space detection model training method comprises the following steps: a parking space detection model training device acquires a spliced image and multiple fisheye images, wherein the spliced image is obtained according to the multiple fisheye images. Then, the parking space detection model training device jointly labels the spliced image and the multiple fisheye images to obtain the coordinates of feature points on the spliced image. Next, the parking space detection model training device inputs an unlabeled spliced image into a parking space detection model to obtain a prediction result, and updates the parameters of the parking space detection model according to the prediction result and the coordinates of the feature points on the spliced image to obtain a trained parking space detection model. On the one hand, the method can guarantee the accuracy of parking space detection by jointly labeling the fisheye image and the spliced image. On the other hand, the method can use relatively accurate labeled data, so that the requirements for image resolution and platform computing power are relatively low, and the training cost is reduced.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and in particular to a parking space detection model training method, a parking space detection method, a parking space detection model training device, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Intelligent driving technology is a technology that uses sensors and electronic systems to completely or partially replace the driver in controlling a vehicle. Because intelligent driving technology can improve driving safety and convenience, it has been widely applied in various scenarios such as emergency braking, cruise control, automatic parking, and sign recognition.

[0003] In parking scenarios, mechanical parking spaces present a significant challenge for drivers due to factors such as dim lighting, unclear space features, and the risk of collisions. Intelligent driving technology can detect parking spaces through object detection and other techniques, enabling vehicles to park more accurately in the desired location, thus improving parking safety and user experience.

[0004] However, the relevant technologies often employ high-performance computing platforms and high-resolution image input models, making it difficult to balance the accuracy and universality of parking space detection. As a result, the technology is difficult to promote due to its cost and the difficulty of image acquisition. Summary of the Invention

[0005] In view of this, this application provides a parking space detection model training method, a parking space detection method, and related equipment to solve the problem that related technologies are difficult to balance the accuracy and universality of parking space detection, ensuring the quality of parking space detection while reducing image acquisition costs and computing power costs.

[0006] Firstly, this application provides a method for training a parking space detection model, the method comprising:

[0007] Obtain a stitched image and multiple fisheye images, wherein the stitched image is obtained based on the multiple fisheye images;

[0008] The stitched image and the multiple fisheye images are jointly annotated to obtain the coordinates of feature points on the stitched image;

[0009] Input the unlabeled stitched image into the parking space detection model to obtain the prediction results;

[0010] The parameters of the parking space detection model are updated based on the prediction results and the coordinates of the feature points on the stitched image to obtain the trained parking space detection model.

[0011] In some possible implementations, the stitched image is obtained by fusing multiple fisheye images based on vehicle calibration parameters. The step of jointly annotating the stitched image and the multiple fisheye images to obtain the coordinates of feature points on the stitched image includes:

[0012] Obtain the initial labeled coordinates of the feature points on the stitched image;

[0013] The initial labeled coordinates of the feature points on the target fisheye image are obtained based on the vehicle calibration parameters;

[0014] The feature points are re-annotated to obtain the target annotation coordinates of the feature points on the target fisheye image;

[0015] The target coordinates of the feature points on the stitched image are obtained based on the vehicle calibration parameters.

[0016] In some possible implementations, the target coordinates of the feature points on the mosaic map include the vertices of at least one bounding box in the mosaic map.

[0017] In some possible implementations, the method further includes:

[0018] Based on the target annotation coordinates of the feature points on the stitched image, obtain the coordinates of the center point of the marker box on the stitched image and the vector from the center point of the marker box to the vertex of the marker box;

[0019] The prediction results include the predicted coordinates of the center point of the bounding box and the predicted vector from the center point of the bounding box to the vertex of the bounding box. Updating the parameters of the parking space detection model based on the prediction results and the coordinates of the feature points on the stitched image includes:

[0020] The loss value of the parking space detection model is determined based on the predicted values ​​of the center point coordinates of the sign box, the predicted values ​​of the vector from the center point of the sign box to the vertex of the sign box, the center point coordinates of the sign box on the stitched image, and the vector from the center point of the sign box to the vertex of the sign box.

[0021] The parameters of the parking space detection model are updated based on the loss value.

[0022] In some possible implementations, the method further includes:

[0023] The unlabeled mosaic image and the mosaic image including the labeled results are downsampled at the same ratio to obtain a smaller version of the unlabeled mosaic image and the mosaic image including the labeled results, so as to update the parameters of the parking space detection model under low computing power conditions.

[0024] Secondly, this application provides a parking space detection method, which is applied to a parking space detection model trained by the method in the first aspect or any possible implementation of the first aspect, the method comprising:

[0025] Obtain the image to be detected, which is a stitched image obtained from multiple fisheye images;

[0026] The image to be detected is input into the trained parking space detection model to obtain the parking space prediction result. The trained parking space detection model updates its parameters based on the prediction result of the parking space detection model and the coordinates of the feature points on the stitched image. The coordinates of the feature points on the stitched image are obtained by jointly annotating the stitched image and the multiple fisheye images.

[0027] Thirdly, this application provides a parking space detection model training apparatus, which includes modules for executing the parking space detection model training method in the first aspect or any possible implementation of the first aspect, specifically including:

[0028] The acquisition module is used to acquire a stitched image and multiple fisheye images, wherein the stitched image is obtained based on the multiple fisheye images;

[0029] The annotation module is used to jointly annotate the stitched image and the multiple fisheye images to obtain the coordinates of feature points on the stitched image;

[0030] The prediction module is used to obtain prediction results based on the unlabeled stitched image of the input parking space detection model;

[0031] The parameter update module is used to update the parameters of the parking space detection model based on the prediction results and the coordinates of the feature points on the stitched image, so as to obtain the trained parking space detection model.

[0032] Fourthly, this application provides an electronic device. The electronic device includes a processor and a memory. The memory stores computer instructions; the processor executes the methods described in the first aspect of this application or any possible implementation thereof, according to the computer instructions.

[0033] Fifthly, this application provides a computer-readable medium storing instructions that, when executed on a computer device, cause the computer device to perform the method described in the first aspect of this application or any possible implementation thereof.

[0034] Sixthly, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in the first aspect of this application or any possible implementation thereof.

[0035] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods.

[0036] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0037] This application provides a method for training a parking space detection model. On the one hand, the method trains the parking space detection model by using a joint annotation of fisheye images and stitched images to ensure the accuracy of parking space detection. On the other hand, the method can utilize relatively accurate annotation data, thereby reducing the requirements for image resolution and platform computing power, and lowering the cost of image acquisition and computing power. Attached Figure Description

[0038] Figure 1 This is a flowchart of a parking space detection model training method disclosed in an embodiment of this application;

[0039] Figure 2 This is a schematic diagram illustrating the correspondence between a stitched image and a fisheye image disclosed in an embodiment of this application;

[0040] Figure 3 This is a flowchart of a parking space detection method disclosed in an embodiment of this application;

[0041] Figure 4 This is a schematic diagram of the structure of a parking space detection model training device disclosed in an embodiment of this application. Detailed Implementation

[0042] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0043] The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. The terms "first" and "second" in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, chronological order of operations, or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0044] First, some technical terms involved in the embodiments of this application will be introduced.

[0045] Mechanical storage units are a type of storage design that uses automated mechanical equipment to achieve three-dimensional storage of vehicles or goods. Utilizing lifting, lateral, and other mechanical devices, a single-level planar space is expanded into multi-level storage units. This allows for applications such as multi-level parking platforms in mechanical garages and automated racking systems in high-density warehousing centers. Compared to traditional storage units, mechanical storage units are more economical to operate and require less floor space, significantly improving space utilization and alleviating land resource constraints.

[0046] However, because mechanical storage expands a single-level planar space into multiple storage units to make the most of space resources, its structure is relatively complex. For example, it may have multiple layers of shelves or narrow aisles, which may create blind spots for vehicles / equipment and pose a risk of collision. At the same time, the environment of mechanical storage is usually dimly lit, making parking space features less obvious, thus posing a challenge to the driver's parking accuracy.

[0047] In related technologies, object detection technology is often used to detect parking spaces to assist drivers in accurately parking in mechanical parking scenarios. To improve parking accuracy and user experience, these technologies typically employ methods such as acquiring high-resolution images and utilizing high-computing platforms to enhance the accuracy of object detection models in detecting parking spaces.

[0048] However, this object detection method has two main drawbacks. First, it requires high image resolution, making it difficult to acquire and limiting its application scenarios. Second, it requires a high-performance computing platform to run the model, resulting in high computational and deployment costs. This makes it difficult to balance the accuracy and universality of parking space detection, hindering the widespread adoption of the technology.

[0049] In view of this, this application provides a parking space detection model training method to solve the problem that related technologies are difficult to balance between the accuracy and universality of parking space detection, ensuring the quality of parking space detection while reducing image acquisition cost and computing power cost.

[0050] Specifically, the method includes: a parking space detection model training device acquiring a stitched image and multiple fisheye images, wherein the stitched image is obtained from the multiple fisheye images. Then, the parking space detection model training device performs joint annotation on the stitched image and the multiple fisheye images to obtain the coordinates of feature points on the stitched image. Next, the parking space detection model training device inputs the unannotated stitched image into the parking space detection model to obtain a prediction result, and updates the parameters of the parking space detection model based on the prediction result and the coordinates of the feature points on the stitched image, thus obtaining the trained parking space detection model.

[0051] On the one hand, this method trains the parking space detection model and can ensure the accuracy of parking space detection by using jointly labeled fisheye images and stitched images; on the other hand, this method can use relatively accurate labeled data, thus requiring lower image resolution and platform computing power, reducing image acquisition costs and computing power costs.

[0052] The method provided in this application will be described below with reference to specific embodiments.

[0053] Figure 1 This is a flowchart illustrating a parking space detection model training method disclosed in an embodiment of this application. The method involves a parking space detection model training device to ultimately obtain a parking space detection model that meets performance requirements. The method includes:

[0054] S102: The parking space detection model training device acquires stitched images and multiple fisheye images.

[0055] The parking space detection model training device can be any device with model training capabilities. For example, the parking space detection model training device can include a desktop computer, a laptop computer, a mobile phone, or a vehicle controller. Of course, the parking space detection model training device can also be a server.

[0056] The stitched image is obtained by fusing multiple fisheye images, which are captured by multiple onboard fisheye cameras deployed at different locations on the vehicle. This allows for a more comprehensive acquisition of information about the surrounding environment, providing a more accurate information source for parking space detection. It should be noted that the model training device can periodically send image acquisition requests to the onboard fisheye cameras to obtain multiple fisheye images and the stitched image; alternatively, the onboard fisheye cameras can autonomously acquire multiple fisheye images and the stitched image and send them to the model training device.

[0057] Multiple fisheye images can be fused to create a stitched image. See also Figure 2 This diagram illustrates the correspondence between a stitched image and a fisheye image. Figure 2 The vehicle-mounted fisheye camera captured four fisheye images: a front-view fisheye image, a left-view fisheye image, a rear-view fisheye image, and a right-view fisheye image.

[0058] When performing image fusion to obtain a stitched image, since the edges or corners of each fisheye image are easily joined with other fisheye images, the pixels located at the edges, especially the corners, in the stitched image should reflect the features of the two fisheye images being stitched / fused. That is, pixels located at the edges, especially the corners, in the stitched image should reflect the features of the two fisheye images being stitched / fused. Figure 2 The pixels in the area indicated by the bold black box should be obtained by fusing pixels from two fisheye images, for example, through weighted fusion. This fusion method can be represented by a relational table, such as... Figure 2As shown, solid arrows in the relationship table represent pixels at that location in the stitched image obtained by fusing the front-view and left-view fisheye images, while dashed arrows represent pixels at that location obtained solely from either the front-view or left-view fisheye image. Furthermore, the relationship table may be affected by the parameters of the fisheye camera, such as internal camera parameters, external camera parameters, and distortion parameters. It should be noted that... Figure 2 Only one example of the relationship table is given. This application embodiment does not make any limitation on the area ratio of the fused region (the area shown in the black bold box) to the stitched image or the specific weighting coefficient of the pixels of the two fisheye images. Relevant technicians should select appropriate parameters for fusion according to actual needs.

[0059] By summarizing the correspondence between the stitched fisheye images and the stitched image in each fusion region, a complete relationship diagram can be obtained, thereby determining the correspondence between the pixels in the stitched image and multiple fisheye images. It should be noted that... Figure 2 This document only demonstrates a scenario and relationship diagram of stitching together four fisheye images. Relevant technicians can obtain stitched images and corresponding relationship tables by stitching together other numbers of fisheye images using similar methods based on actual circumstances. This application does not impose any limitations on this.

[0060] S104: The parking space detection model training device performs joint annotation on the stitched image and multiple fisheye images to obtain the coordinates of feature points on the stitched image.

[0061] After obtaining the stitched image and multiple fisheye images, the parking space detection model training device can jointly annotate them to obtain more accurate annotation information. When using the stitched image for annotation, the use of an image with more detailed surrounding environment allows for greater uniformity in features such as parking space size and shape; for example, each parking space has a uniformly sized rectangular marker, which facilitates model training. When using fisheye images for annotation, the fisheye image uses more pixels to represent features close to the camera's optical center, thus more clearly and accurately annotating parking space features on the bottom surface, resulting in more accurate model detection. Therefore, jointly annotating both provides highly accurate parking space feature annotation information that is beneficial for model training.

[0062] In some possible implementations, the parking space detection model training device can first obtain the initial labeled coordinates of the feature points on the stitched image, and then obtain the initial labeled coordinates of the feature points on the target fisheye image according to the vehicle calibration parameters. Next, the feature points are re-labeled to obtain the target labeled coordinates of the feature points on the target fisheye image, and then the target labeled coordinates of the feature points on the stitched image are obtained according to the vehicle calibration parameters. It should be noted that the vehicle calibration parameters specifically include the relationship table described in S102, as well as other parameters that may affect the coordinates of feature points in the image (such as the parameters of the fisheye camera described in S102).

[0063] Where a feature point on the stitched image corresponds to a feature point on a fisheye image (e.g.) Figure 2 The dashed arrows in the relationship table indicate points / pixels. The parking space detection model training device can directly perform image and coordinate system transformations based on vehicle calibration parameters to obtain the target labeled coordinates. However, when feature points on the stitched image correspond to feature points on two fisheye images (e.g., ...), ... Figure 2 The points / pixels indicated by the solid arrows in the relationship table are used to determine the specific fisheye images and corresponding feature point coordinates for re-annotation of the parking space detection model training device.

[0064] Since parking spaces are typically regular in shape, such as rectangles, the corresponding fisheye image and feature point coordinates that the parking space detection model can identify after coordinate transformation should be used for re-annotation. Furthermore, since the size and shape of parking spaces are consistent, directly regressing the coordinates of the four vertices of the rectangle is more beneficial for model training than using heatmaps to predict the probability of each location. In one possible example, the four vertices used to identify the parking space could be the starting points of the side beams.

[0065] In other words, the target coordinates of the feature points after coordinate transformation on the stitched image should include at least one vertex of a bounding box in the stitched image. This allows the parking space detection model to be trained using feature point coordinates of uniform size and shape. The bounding box describes the location and orientation of the target (e.g., a parking space). For example, the bounding box can be a two-dimensional (2D) bounding box or a three-dimensional (3D) bounding box. A 2D bounding box identifies the size and position of the target in the image to be detected, while a 3D bounding box identifies the size, position, and orientation of the target in space.

[0066] In this way, we can obtain parking space feature annotation information with high accuracy that is beneficial to model training. That is, we can obtain the ground truth of the stitched image after joint annotation for model training.

[0067] S106: The parking space detection model training device inputs the unlabeled stitched image into the parking space detection model to obtain the prediction result.

[0068] In the embodiments of this application, the parking space detection model training device can input an unlabeled stitched image into the parking space detection model, and the output of the parking space detection model is the prediction result.

[0069] The parking space detection model can specifically employ object detection technology, which can be broadly categorized into single-stage object detection methods and two-stage object detection methods. Single-stage object detection methods directly output object detection results through a single neural network model, such as the YOLO (You Only Look Once) algorithm and the SingleShot MultiBox Detector (SSD) algorithm. Two-stage object detection methods typically consist of two main stages. The first stage generates candidate object regions, usually implemented using a Region Proposal Network (RPN) or other methods. The second stage classifies and refines the localization of these candidate regions. Common two-stage object detection methods include Fast Region-based Convolutional Neural Network (Fast R-CNN), Mask Region-based Convolutional Neural Network (Mask R-CNN), and Cascade Region-based Convolutional Neural Network (Cascade R-CNN). It should be noted that the above content only provides some examples of parking space detection models, and the embodiments of this application do not specifically limit which model to use.

[0070] S108: The parking space detection model training device updates the parameters of the parking space detection model based on the prediction results and the coordinates of the feature points on the stitched image, and obtains the trained parking space detection model.

[0071] The parking space detection model training device can update the parameters of the parking space detection model based on the prediction results and the coordinates of feature points in the stitched image after joint annotation. In other words, it can obtain the true data based on the coordinates of feature points on the stitched image, which can be compared with the model prediction results and used to update the model parameters.

[0072] In some possible implementations, the parking space detection model training device can obtain the coordinates of the center point of the marker box on the stitched image and the vector from the center point of the marker box to its vertex based on the target annotation coordinates of the feature points on the stitched image. In this case, the prediction result can include the predicted values ​​of the center point coordinates of the marker box and the predicted value of the vector from the center point of the marker box to its vertex. Specifically, the parking space detection model training device updates the model parameters in the following way:

[0073] The parking space detection model training device first determines the loss value of the parking space detection model based on the predicted coordinates of the center point of the marker frame, the predicted vector from the center point to the vertex of the marker frame, and the coordinates and vector of the center point of the marker frame on the stitched image. The loss value can be determined using one or more loss functions (such as absolute error regression loss function, squared error regression loss function, etc.). Then, the parking space detection model training device can update the parameters of the parking space detection model based on the loss value, for example, by optimizing the model weights based on the gradient of the loss value. In this way, the model parameters can be updated more accurately based on feedback information, resulting in a parking space detection model with better performance.

[0074] In some possible implementations, the parking space detection model training device can also preprocess the stitched image, thereby reducing the amount of data and complexity of model computation.

[0075] Specifically, the parking space detection model training device can downsample both the unlabeled stitched image and the stitched image including the labeled results at the same ratio to obtain a scaled-down unlabeled stitched image and a stitched image including the labeled results, thereby updating the parameters of the parking space detection model under low computing power conditions. At this point, to ensure the accuracy of the prediction results and reduce the impact of detail loss caused by image scaling, the parking space detection model can employ a model that is better at extracting spatial features (e.g., a 3-layer dilated spatial convolutional pooling pyramid) to capture features at different scales. By adjusting the stride of the dilated convolution, the receptive field size of each convolutional branch can be controlled to adapt to targets of different sizes, ultimately integrating the results to obtain the output prediction.

[0076] In some possible implementations, the preprocessing of the stitched image may also include using data augmentation techniques such as horizontal flipping, vertical flipping, and affine transformations to process the image and input it into the model for training, thereby improving the model's generalization ability.

[0077] In some possible implementations, the parking space detection model can also be deployed using lightweight networks, such as efficient convolutional neural networks using deepwise separable convolutions, or by reducing the number of model parameters, for example by extracting the output of the backbone network from downsampled feature results and performing feature fusion, to reduce the computational burden.

[0078] In some possible implementations, the parking space detection model can also undergo model compression to reduce its storage space and computational cost. For example, Quantization-Aware Training (QAT) can be performed to convert floating-point parameters and activation values ​​in the model into low-bit-width integers (e.g., 8 bits) to reduce model storage requirements and accelerate the inference process. In this way, the parking space detection model can save computational costs and maintain low-computing-power operation while maintaining a certain level of accuracy.

[0079] Based on the above description, this application provides a method for training a parking space detection model. On the one hand, this method trains the parking space detection model by using jointly labeled fisheye images and stitched images to ensure the accuracy of parking space detection; on the other hand, this method can utilize relatively accurate labeled data, thus requiring lower image resolution and platform computing power, reducing image acquisition costs and computing power costs.

[0080] The following section introduces a parking space detection method provided in this application, which is based on a trained parking space detection model. See also... Figure 3 The flowchart shown is a parking space detection method, which specifically includes:

[0081] S302: The trained parking space detection model acquires the image to be detected.

[0082] The image to be detected is a stitched image obtained from multiple fisheye images, which are captured by multiple vehicle-mounted fisheye cameras deployed at different locations on the vehicle. There is a correspondence between the stitched image and the fisheye images. Specifically, when multiple fisheye images are fused to obtain the stitched image, since the edges or corners of each fisheye image are easily connected to other fisheye images, the pixels located at the edges, especially the corners, in the stitched image should reflect the features of the two fisheye images being stitched / fused.

[0083] In some possible implementations, the input image to be detected can be cropped or compressed in size, which allows the model to process less data, improves inference speed, and reduces computing power requirements and costs.

[0084] In some possible implementations, the method can also preprocess the image to be detected, such as performing image denoising, main feature extraction, or image enhancement, thereby reducing interference and improving the parking space detection effect.

[0085] S304: The trained parking space detection model obtains parking space prediction results based on the input image to be detected.

[0086] The parking space detection model can specifically be a target detection model that predicts parking spaces by forecasting the coordinates of feature points (mainly the vertices of the bounding boxes used to indicate parking spaces). In mechanical parking spaces, the bounding boxes are typically rectangles of uniform size and shape. In some possible implementations, the inference result of the parking space detection model can include the predicted coordinates of the center point of the bounding box and the predicted vector from the center point to the vertices of the bounding box, thus obtaining the specific coordinates of the vertices.

[0087] In some possible implementations, the parking space detection model can also display the identification boxes as a visual result to the user (such as the driver), so that the driver can more intuitively understand the location, direction and other information of the parking space and take more reasonable actions.

[0088] It should be noted that the parking space detection model in this embodiment is achieved through methods such as... Figure 1 The training is performed using the method shown above, and the relevant steps have been described in detail above, so they will not be repeated here.

[0089] Based on the above description, this application provides a parking space detection method. This method is based on the aforementioned parking space detection model training method and the trained parking space detection model. It can detect parking spaces more accurately with low computing power requirements, reduce detection costs, and improve detection efficiency.

[0090] This application also provides a parking space detection model training device. The device of this application will be described in detail below with reference to the accompanying drawings.

[0091] See Figure 4 The diagram shows a structural schematic of a parking space detection model training device. The parking space detection model training device 400 may include:

[0092] The acquisition module 402 is used to acquire a stitched image and multiple fisheye images, wherein the stitched image is obtained from the multiple fisheye images;

[0093] The annotation module 404 is used to jointly annotate the stitched image and multiple fisheye images to obtain the coordinates of feature points on the stitched image;

[0094] The prediction module 406 is used to obtain prediction results based on the unlabeled stitched image of the input parking space detection model;

[0095] The parameter update module 408 is used to update the parameters of the parking space detection model based on the prediction results and the coordinates of the feature points on the stitched image, so as to obtain the trained parking space detection model.

[0096] In some possible implementations, the stitched image is obtained by fusing multiple fisheye images based on vehicle calibration parameters. When jointly annotating the stitched image and the multiple fisheye images to obtain the coordinates of feature points on the stitched image, the annotation module 404 is specifically used for:

[0097] The initial labeled coordinates of the feature points on the stitched image are obtained, and then the initial labeled coordinates of the feature points on the target fisheye image are obtained according to the vehicle calibration parameters. Next, the feature points are re-labeled to obtain the target labeled coordinates of the feature points on the target fisheye image, and then the target labeled coordinates of the feature points on the stitched image are obtained according to the vehicle calibration parameters.

[0098] In some possible implementations, the target annotation coordinates of feature points on the mosaic map include the vertices of at least one bounding box in the mosaic map.

[0099] In some possible implementations, the prediction module 406 is also used for:

[0100] Based on the target annotation coordinates of the feature points on the stitched image, obtain the coordinates of the center point of the bounding box and the vector from the center point to the vertex of the bounding box. The prediction results include the predicted values ​​of the center point coordinates and the predicted value of the vector from the center point to the vertex of the bounding box.

[0101] At this point, when the parameter update module 408 updates the parameters of the parking space detection model based on the prediction results and the coordinates of the feature points on the stitched image, it is specifically used for:

[0102] Based on the predicted values ​​of the center point coordinates of the sign box, the predicted value of the vector from the center point of the sign box to the vertex of the sign box, the center point coordinates of the sign box on the stitched image, and the vector from the center point of the sign box to the vertex of the sign box, the loss value of the parking space detection model is determined, and the parameters of the parking space detection model are updated according to the loss value.

[0103] In some possible implementations, the device also includes a preprocessing module 410, specifically used for:

[0104] The unlabeled mosaic and the mosaic including the labeled results are downsampled at the same ratio to obtain a scaled-down unlabeled mosaic and a mosaic including the labeled results, so as to update the parameters of the parking space detection model under low computing power conditions.

[0105] Based on the aforementioned parking space detection model training method, parking space detection method, and parking space detection model training device, this application also provides an electronic device. This electronic device may be, for example, a vehicle control unit (VCU) or an electronic control unit (ECU), including a processor and a memory. The memory stores computer-readable instructions, and the processor executes these computer-readable instructions to perform the aforementioned method. In some examples, this electronic device is used to implement the functions of the aforementioned device.

[0106] This application provides a computer-readable storage medium storing instructions that, when executed on a computer device, cause the computer device to perform the above-described method.

[0107] This application provides a computer program product containing instructions that, when run on a computer device, cause the computer device to perform the above-described method.

[0108] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0109] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0110] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0111] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0112] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a parking space detection model, characterized in that, The method includes: Obtain a stitched image and multiple fisheye images, wherein the stitched image is obtained based on the multiple fisheye images; The stitched image and the multiple fisheye images are jointly annotated to obtain the coordinates of feature points on the stitched image; Input the unlabeled stitched image into the parking space detection model to obtain the prediction results; The parameters of the parking space detection model are updated based on the prediction results and the coordinates of the feature points on the stitched image to obtain the trained parking space detection model.

2. The method according to claim 1, characterized in that, The stitched image is obtained by fusing multiple fisheye images based on vehicle calibration parameters. The joint annotation of the stitched image and the multiple fisheye images to obtain the coordinates of feature points on the stitched image includes: Obtain the initial labeled coordinates of the feature points on the stitched image; The initial labeled coordinates of the feature points on the target fisheye image are obtained based on the vehicle calibration parameters; The feature points are re-annotated to obtain the target annotation coordinates of the feature points on the target fisheye image; The target coordinates of the feature points on the stitched image are obtained based on the vehicle calibration parameters.

3. The method according to claim 2, characterized in that, The target coordinates of the feature points on the stitched image include the vertices of at least one bounding box in the stitched image.

4. The method according to claim 3, characterized in that, The method further includes: Based on the target annotation coordinates of the feature points on the stitched image, obtain the coordinates of the center point of the marker box on the stitched image and the vector from the center point of the marker box to the vertex of the marker box; The prediction results include the predicted coordinates of the center point of the bounding box and the predicted vector from the center point of the bounding box to the vertex of the bounding box. Updating the parameters of the parking space detection model based on the prediction results and the coordinates of the feature points on the stitched image includes: The loss value of the parking space detection model is determined based on the predicted values ​​of the center point coordinates of the sign box, the predicted values ​​of the vector from the center point of the sign box to the vertex of the sign box, the center point coordinates of the sign box on the stitched image, and the vector from the center point of the sign box to the vertex of the sign box. The parameters of the parking space detection model are updated based on the loss value.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The unlabeled mosaic image and the mosaic image including the labeled results are downsampled at the same ratio to obtain a smaller version of the unlabeled mosaic image and the mosaic image including the labeled results, so as to update the parameters of the parking space detection model under low computing power conditions.

6. A parking space detection method, characterized in that, The method includes: Obtain the image to be detected, which is a stitched image obtained from multiple fisheye images; The image to be detected is input into the trained parking space detection model to obtain the parking space prediction result. The trained parking space detection model updates its parameters based on the prediction result of the parking space detection model and the coordinates of the feature points on the stitched image. The coordinates of the feature points on the stitched image are obtained by jointly annotating the stitched image and the multiple fisheye images.

7. A parking space detection model training device, characterized in that, The device includes: The acquisition module is used to acquire a stitched image and multiple fisheye images, wherein the stitched image is obtained based on the multiple fisheye images; The annotation module is used to jointly annotate the stitched image and the multiple fisheye images to obtain the coordinates of feature points on the stitched image; The prediction module is used to obtain prediction results based on the unlabeled stitched image of the input parking space detection model; The parameter update module is used to update the parameters of the parking space detection model based on the prediction results and the coordinates of the feature points on the stitched image, so as to obtain the trained parking space detection model.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store computer programs or computer instructions; A processor for executing a computer program or computer instructions stored in the memory, causing the electronic device to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed, is used to implement the method of any one of claims 1 to 6.

10. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1 to 6.

Images