Method, device and equipment for determining parking space, vehicle and medium

Abstract

The embodiment of the invention relates to a method, device and equipment for determining a parking space, a vehicle and a medium. In the method, a first heat map and a second heat map are generated based on images captured by a camera, wherein the size of the first heat map is smaller than that of the second heat map. The method further includes determining first information associated with the parking space based on the first heat map, determining second information associated with the parking space based on the second heat map, and determining location information of the parking space based on the first information and the second information. Through the method provided by the embodiment of the invention, the determined parking space information can be modified and updated more quickly in real time.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of data processing, and more particularly, to methods, devices, equipment, vehicles, and media for determining parking spaces. Background Art

[0002] With the rapid development of artificial intelligence, the research on autonomous driving and driver assistance systems has attracted extensive attention in the industry. Technologies such as automatic parking assistance and valet parking have become increasingly important for drivers. Automatic parking assistance systems are dedicated to helping drivers complete the parking operation of vehicles to address the challenges of parking in narrow spaces or complex environments. Such systems utilize sensors, cameras, and computer vision technologies to detect the surrounding environment and achieve precise and safe parking by automatically controlling the steering, acceleration, and braking of the vehicle.

[0003] Valet parking technology enables vehicles to complete the entire parking process independently without the driver being inside the vehicle. This technology relies on advanced sensors, real-time positioning systems, and intelligent control algorithms, enabling the vehicle to autonomously search for suitable parking spaces in a parking lot or designated area and complete the parking operation. This convenience not only enhances the user experience but also provides a potential solution to traffic congestion problems. However, there are still many technical problems that need to be solved in the application of technologies such as automatic parking assistance and valet parking. Summary of the Invention

[0004] Embodiments of the present disclosure provide a method, device, equipment, vehicle, and medium for determining a parking space.

[0005] According to a first aspect of the present disclosure, there is provided a method for determining a parking space, the method including generating a first heatmap and a second heatmap based on an image captured by a camera, wherein the size of the first heatmap is smaller than the size of the second heatmap. The method further includes determining first information associated with the parking space based on the first heatmap, determining second information associated with the parking space based on the second heatmap, and determining the location information of the parking space according to the first information and the second information.

[0006] According to a second aspect of the present disclosure, there is provided a device for determining a parking space, the device including a generating unit configured to generate a first heatmap and a second heatmap based on an image captured by a camera, wherein the size of the first heatmap is smaller than the size of the second heatmap. The device further includes a first determining unit configured to determine first information associated with the parking space based on the first heatmap, a second determining unit configured to determine second information associated with the parking space based on the second heatmap, and a third determining unit configured to determine the location information of the parking space according to the first information and the second information.

[0007] According to a third aspect of the present disclosure, a controller is provided. The controller includes at least one processor; and a memory coupled to the at least one processor and having instructions stored thereon that, when executed by the at least one processor, cause the controller to perform the steps of the method in the first aspect of the present disclosure.

[0008] According to a fourth aspect of the present disclosure, a vehicle is provided, the vehicle including the controller in the third aspect of the present disclosure.

[0009] According to a fifth aspect of the present disclosure, a machine-readable storage medium is provided. Computer-executable instructions are stored on the machine-readable storage medium, wherein the computer-executable instructions are executed by a processor to implement the steps of the method in the first aspect of the present disclosure. Description of the Drawings

[0010] By describing the exemplary embodiments of the present disclosure in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present disclosure will become more apparent. Among them, in the exemplary embodiments of the present disclosure, the same reference numerals generally represent the same components.

[0011] Figure 1A A schematic diagram illustrating an example environment in which the device and / or method according to the embodiments of the present disclosure may be implemented;

[0012] Figure 1B A schematic diagram showing the functions of an example parking space determination system according to the embodiments of the present disclosure;

[0013] Figure 2 A flowchart illustrating a method for determining a parking space according to the embodiments of the present disclosure;

[0014] Figure 3 A schematic diagram illustrating a process for determining a parking space according to the embodiments of the present disclosure;

[0015] Figure 4 A schematic diagram illustrating a process for training a machine learning model according to the embodiments of the present disclosure;

[0016] Figure 5 A schematic diagram illustrating a device for determining a parking space according to the embodiments of the present disclosure;

[0017] Figure 6 A schematic block diagram of an example device that can be used to implement the embodiments of the present disclosure. Detailed Description of the Embodiments

[0018] The embodiments of the present disclosure described below with reference to the accompanying drawings are only for exemplary purposes and are not intended to limit the protection scope of the present disclosure. Additionally, before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to users and user authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0019] Parking spaces have different attributes, such as parking space function, parking space shape, parking space material, parking space availability, etc. Among the parking space function attributes, there are different types such as graffiti, disabled, female, micro, and multi-storey parking spaces. Generally, there are three types of parking space shapes: parallel, inclined, and perpendicular. For the parking space material, it can include materials such as painted concrete, multi-storey, grass, and bricks. In order to ensure that a car can park in the parking space, signals such as parking space function, parking space shape, and parking space availability attributes need to be obtained.

[0020] In addition, whether the parking space is available is also extremely important. Only a parking space with an empty attribute can be parked in. Otherwise, when the parking space is occupied, an indication that the parking space is unavailable should be given. Due to changes in lighting conditions, the existence of shadows, and the occlusion of surrounding objects, some parking spaces may only partially show the situation of the parking space. There are also some corner case scenarios. If duplicate data is always added in one scenario, it is not helpful for solving parking space scenarios that include multiple corner cases. Therefore, it is necessary to combine active learning with the system to select information data. It is important to select information-rich data and balance the dataset to improve performance. Therefore, a method for quickly determining parking spaces and quickly and real-time modifying and updating the determined parking space information is needed.

[0021] To at least solve the above and other potential problems, embodiments of the present disclosure provide a method for determining parking spaces. In this method, a controller in a vehicle or a parking space determination system generates a first heatmap and a second heatmap based on an image collected by an on-vehicle camera, where the size of the first heatmap is smaller than the size of the second heatmap. A basic module in the controller of the vehicle determines or predicts first information or global information associated with the parking space based on the first heatmap. An update module in the controller of the vehicle determines or predicts second information or local information associated with the parking space based on the second heatmap. The controller of the vehicle then determines or modifies the current position information of the parking space according to the first information and the second information.

[0022] The method implemented through the present disclosure can predict attributes such as parking space functions, parking space shapes, and parking space availability. The method implemented through the present disclosure can also combine heatmap prediction with regression to address situations of light changes, shadows, and occlusions to obtain the corner points of the parking space. The method implemented through the present disclosure can also predict the angle of the entrance line points to refine the parking space. The method implemented through the present disclosure can also combine active learning to select information data to address the corner point use case, ultimately enabling the rapid and real-time modification and update of the determined parking space information.

[0023] Embodiments of the present disclosure will be described in further detail below in conjunction with the accompanying drawings, where Figure 1A FIG. 100A is a schematic diagram showing an example environment in which an apparatus and / or method according to an embodiment of the present disclosure may be implemented.

[0024] As Figure 1A shown, a vehicle 101 equipped with a parking space determination system implemented according to an embodiment of the present disclosure can utilize its own camera to collect images related to a parking space 103 to generate a plurality of heatmaps, such as a first heatmap 105 for a base module 109 and a second heatmap 107 for an update module 111. In some embodiments, the first heatmap 105 may have a smaller size than the second heatmap 107. Additionally or alternatively, the first heatmap 105 may have parameters such as a lower resolution than the second heatmap 107.

[0025] In some embodiments, the base module 109 can use the first heatmap 105 to determine global information associated with the parking space 103, such as the availability of the parking space (occupied or vacant), the length and width of the parking space, and other information. The update module 111 can use the second heatmap 107 to determine local information associated with the parking space 103, such as corner point information, angle information, the shape of the parking space, and other information. In some embodiments, the update module 111 can use the determined local information to update or modify one or more of the global information determined by the base module 109.

[0026] Figure 1B FIG. 100B is a schematic diagram showing the functions of an example parking space determination system 100B according to an embodiment of the present disclosure. The example parking space determination system 100B can include a trained machine learning model 102 according to the method of an embodiment of the present disclosure. For example, in some embodiments, the machine learning model 102 can include a decision tree model for classification and regression that makes decisions by hierarchically partitioning features. In some embodiments, the machine learning model 102 can include a linear regression model that establishes a linear relationship between features and a target variable and predicts a continuous target variable based on a linear combination of input features.

[0027] In some embodiments, the machine learning model 102 may include a support vector machine model that finds an optimal hyperplane to separate data points of different classes or fits data and predicts continuous values in a regression task. In some embodiments, the machine learning model 102 may also include a k-nearest neighbor model that, for an unknown sample, finds the k known-class samples that are its nearest neighbors in the feature space to determine the class to which the unknown sample belongs.

[0028] In some embodiments, the machine learning model 102 may include a neural network model, which may include multiple layers such as an input layer, hidden layers, and an output layer, and elements such as a large number of interconnected nodes, and learns and makes predictions through weight adjustment between these nodes. The neural network model learns the patterns and features of the data through training so as to make predictions or classifications when given new data. It should be understood that the types of the machine learning model 102 shown above are only examples and not limitations. According to the embodiments of the present disclosure, the machine learning model 102 may include any type of model or element, and the present disclosure makes no limitation thereto.

[0029] Reference Figure 1B , the machine learning model 102 may process the image 104 collected by the camera. As an example, the image 104 according to the present disclosure may be an Around View Image (AVM). In some embodiments, the image 104 may be captured by multiple cameras installed at different positions of the vehicle to collect the scenes around the vehicle, and these images are fused into a panoramic or top-down image through software processing. The AVM image can provide an all-round perspective for the example parking space determination system 100B, helping it to observe the surrounding environment more clearly, especially when performing complex operations such as parking, reversing, or turning around a corner, improving driving safety and convenience.

[0030] In response to receiving the image 104 such as AVM image data, the machine learning model 102 may identify the image 104 to confirm one or more elements in the image 104, such as through the parking space identification function 106, parking space shape determination function 108, and parking space availability determination function 110 of the machine learning model 102, etc. In some embodiments, the parking space identification function 106 in the machine learning model 102 may be used to identify parking spaces in different scenarios. For example, in some embodiments, the parking space identification function 106 may determine that the parking space is a normal ordinary parking space 112 based on the AVM image. In some embodiments, the parking space function 106 may determine that the position is a non-parking space 114 based on the elements in the AVM image.

[0031] For example, in some embodiments, the parking space identification function 106 may determine that the parking space is a reserved parking space 116, i.e., a parking space where a vehicle can only park by reservation. In some embodiments, the parking space identification function 106 may also determine that the parking space is a graffiti parking space 118 based on features such as specific patterns, graffiti, signs, or colors on the parking space. In some embodiments, the parking space identification function 106 may also determine that the parking space is a disabled parking space 120 based on the sign indicating a disabled parking space in the AVM image.

[0032] Additionally or alternatively, the parking space identification function 106 may also determine that the parking space is a women-only parking space 122, i.e., a parking space where only women can park. In some embodiments, the parking space identification function 106 may determine that the parking space is a compact parking space 124, which is smaller in size than a standard parking space and is specifically designed for small vehicles and thus is generally not suitable for larger cars. In some embodiments, the parking space identification function 106 may determine that the parking space is a multi-story parking space 126, which is designed to effectively utilize space to park vehicles by vertically moving or stacking vehicles and provides a parking solution for multiple cars. In some embodiments, the parking space identification function 106 may also identify parking spaces for charging, etc.

[0033] In some embodiments, the parking space shape determination function 108 may determine the shape of the parking space based on the AVM image data. For example, the parking space shape determination function 108 may analyze the spatial information, markings, edges, and / or other ground markings around the parking space to determine its shape, such as a perpendicular parking space 128, a parallel parking space 130, an angled parking space 132, etc.

[0034] Additionally or alternatively, in some embodiments, the parking space availability determination function 110 may determine the availability of the current parking space. For example, the parking space availability determination function 110 determines whether the parking space is occupied by identifying parking space markings, vehicle outlines, or other features, such as the parking space can be marked as an available parking space 134 and / or an occupied parking space 136. It should be understood that the above description of the functions of the exemplary parking space determination system 100B is exemplary, and the exemplary parking space determination system 100B may also include any other known or unknown functions, such as parking space size and adaptability detection, parking space reservation, parking space electronic payment, remote control, security monitoring, etc., without departing from the scope of the embodiments of the present disclosure.

[0035] Additionally or alternatively, in some embodiments, the example parking space determination system 100B and / or one or more of the machine learning model 102, the parking space identification function 106, the parking space shape determination function 108, and the parking space availability determination function 110 may be implemented by any suitable computing device, including but not limited to a personal computer, a handheld or laptop device, a mobile device, a multiprocessor system, a consumer electronic product, a minicomputer, and a distributed computing environment including any one of the above systems or devices, etc.

[0036] The above has been described in conjunction with Figure 1A and Figure 1B a block diagram of the example parking space determination system 100B in which embodiments of the present disclosure can be implemented. The following will be described in conjunction with Figure 2 a flowchart of a method 200 for determining a parking space according to an embodiment of the present disclosure. The method 200 can be executed at the machine learning model 102 in FIG. 1 and any suitable computing device.

[0037] At block 202, a first heatmap and a second heatmap are generated based on an image acquired by a camera, where the size of the first heatmap is smaller than the size of the second heatmap. As an example, the machine learning model 102 in the example parking space determination system 100B generates one or more heatmaps through the processing of elements such as a backbone network and a neck component therein.

[0038] At block 204, first information associated with the parking space is determined based on the first heatmap. As an example, a basic module in the machine learning model 102 of the example parking space determination system 100B may first determine global information associated with the parking space based on the first heatmap with a lower resolution. An example implementation of the basic module will be described in detail below in conjunction with Figure 3 to describe.

[0039] For example, the basic module may determine the center point of an outer rectangle associated with the parking space in the first heatmap. The outer rectangle may be a bounding box for identifying the approximate area of the parking space. In some embodiments, the basic module may also predict the length and width of the outer rectangle. In some embodiments, the basic module may also directly determine the center point of the parking space in the first heatmap, as well as the length and width of the parking space. Additionally or alternatively, the basic module may also determine the offset between the center point of the outer rectangle and the center point of the parking space, for example, by subtracting the coordinate values of the two center points to determine the horizontal and vertical offsets therebetween for further adjustment and refinement.

[0040] In some embodiments, the base module can also use the first heatmap to determine whether the parking space to be parked in is occupied or available. For example, the base module can identify and analyze specific patterns, shapes, colors, etc. of the parking spaces. The base module can then use the extracted features to determine the occupancy status of the parking spaces.

[0041] Additionally or alternatively, in some embodiments, the base module can determine the offset from the center point to the corner points in the parking space based on offset regression. For example, the base module can use an offset regression method to obtain the four corner points of the parking space, directly regressing the offset from the center of the parking space to the four corners, which can obtain the approximate positions of the corner points. For example, during the training process of training the base module with training data, the base module learns the offset from the center point to the four corner points.

[0042] During the training process, the base module can optimize the predicted offset to make it as close as possible to the actual annotation value. In actual applications, the base module can predict the offset of the four corner points based on the center point of the parking space detected in the image. The base module can use the predicted offset to calculate and adjust the positions from the center point of the parking space to the four corner points, thereby determining the complete parking space bounding box or outer rectangle. Therefore, when the computing resources are insufficient, the base module can also meet the basic parking needs.

[0043] At block 206, the second information associated with the parking space is determined based on the second heatmap. As an example, the update module in the machine learning model 102 of the exemplary parking space determination system 100B can determine data of the corner points in the parking space and the angle information of the parking space based on information such as that in the second heatmap, such as pixel feature information. For example, in some embodiments, the update module can identify or predict local information such as the corner point position information, the offset information of the corner points, the angle information of the parking space, and the shape of the parking space associated with the parking space in the second heatmap with a higher resolution. An example implementation of the update module will be described in detail below in conjunction with Figure 3 to be described in detail.

[0044] The update module can add an angle factor during the process of refining the two corner points of the parking space far from the entrance line. For example, techniques such as geometry or line fitting can be used to calculate the parking space angle. These two points should be on a straight line in the angled direction. The shape module of the update module can reshape the parking space into a parallelogram, for example, through techniques such as geometric transformation, perspective transformation, or parallelogram fitting. Thus, the update module can more accurately describe the position and shape of the parking space, providing more information about the geometric features of the parking space. In some embodiments, the base module and the update module can jointly implement one or more of the parking space identification function 106, the parking space shape determination function 108, and the parking space availability determination function 110.

[0045] At block 208, the location information of the parking space is determined based on the first information and the second information. In some embodiments, the exemplary parking space determination system 100B may first use the global information provided by the basic module to navigate the vehicle to the parking space or park the vehicle. For example, the exemplary parking space determination system 100B may first present the global information associated with the parking space to the user. In some embodiments, the exemplary parking space determination system 100B may then use the local information provided by the update module to first refine the location information of the parking space of the vehicle and the parking process.

[0046] For example, the exemplary parking space determination system 100B may update or modify one or more pieces of information in the global information based on local information such as corner position information, offset information of the corners, angle information of the parking space, and shape of the parking space provided by the update module. In some embodiments, the update module and the basic module may operate independently and simultaneously to determine the corresponding information respectively.

[0047] Thus, through operations such as identifying and predicting different heatmaps by the basic module and the update module, the present disclosure improves technologies such as automatic parking assistance and automatic valet parking, enabling the vehicle to more quickly and accurately determine the location data of the parking space and automatically drive into the parking space without manual intervention. The following combines Figure 3 FIG. 300 is a schematic diagram of a process for determining a parking space according to an embodiment of the present disclosure.

[0048] As Figure 3 shown, the AVM image data 302 collected by the camera may be fed into the machine learning model 102 implemented according to the embodiments of the present disclosure. As an example, in the embodiments of the present disclosure, the machine learning model 102 may be a model implemented based on a neural network, which may include multiple elements such as a backbone network 304 and a neck component 306. For example, the backbone network may generally be composed of a convolutional neural network such as convolutional layers, pooling layers, and activation functions. The backbone network performs a series of operations such as convolution and pooling to extract feature information from the original image, identify one or more objects in the image, image segmentation, etc.

[0049] The neck component can then further process and refine the features, for example, through operations such as dimensionality reduction, attention mechanisms, upsampling, or other means, to generate more useful or more suitable feature representations for subsequent processing, such as multiple different heatmaps with different sizes or resolutions. For example, a Gaussian kernel can be used for weighting during the generation of multiple heatmaps to extract features and produce smooth heatmaps. Parameters of the Gaussian kernel, such as the size (dimension) and standard deviation of the kernel, will affect the generation result of the final heatmap. For example, in some embodiments, a smaller Gaussian kernel and standard deviation can produce a more detailed heatmap, while a larger Gaussian kernel and standard deviation will make the heatmap smoother but may lose some details.

[0050] The machine learning model can perform operations such as feature extraction processing like sampling on the AVM image data 302 to generate one or more different heatmaps. In some embodiments, the Gaussian kernel of the machine learning model 102 can be determined according to the visible range of the target. The visible size of the parking space target is larger than that of the parking space corner points. Relative to the parking space, the parking space corner points are small targets. For example, the base module 308 can make predictions at a high scale or dimension. For example, the original image can be scaled down to a scale of 1 / 8, at which scale one pixel can represent a parking space.

[0051] In other embodiments, the update module can make predictions at a low scale. For example, the original image can be scaled down to a scale of 1 / 4, at which scale one pixel can represent a corner point of the parking space. One pixel at a high scale contains more information than at a low scale. Thus, heatmaps with different scales, sizes, or resolutions can be generated, such as a first heatmap 326 with a first size and a second heatmap 328 with a second size, where the first size can be smaller than the second size. In some embodiments, a first heatmap 326 with a first resolution and a second heatmap 328 with a second resolution can also be generated, where the first resolution can be smaller than the second resolution.

[0052] The generated heatmaps 326 and 328 can be fed into the base module 308 and the update module 310 respectively via a multi-task head in the neural network. The multi-task head is a structure in the neural network that allows the neural network to perform multiple tasks simultaneously. These tasks may be related, and the neural network learns the correlations between different tasks by sharing the underlying representations to improve the overall performance. Sharing can accelerate training and improve the generalization ability of the model to unseen data because different tasks can jointly learn useful feature representations. The multi-task head can balance the relationships between different tasks so that some tasks can promote each other.

[0053] The base module 308 can generate an outer rectangle 312 for identifying a general area of a parking space based on the first heat map 326, and determine whether the parking space is occupied or available based on the outer rectangle. In some embodiments, the base module 308 can also predict or determine the length and width 314 of the outer rectangle. Additionally or alternatively, the base module 308 can determine the offset 316 between the center of the outer rectangle and the center point of the parking space. In some embodiments, the base module can determine the offset 4*2 from the center coordinates to the corner points in the parking space based on offset regression, and determine the difference or shape loss 316 between the predicted parking space shape and the actual parking space shape, such as one or more of the parking space angle or length, etc.

[0054] Meanwhile, the update module 310 can generate local information associated with the parking space based on another heat map 328. For example, in some embodiments, the update module 310 can determine the position data and heat map information 318 of the four corner points of the parking space. In some embodiments, the update module 310 can also determine the offset 320 between the corner points of the outer rectangle 312 and the four corner points of the parking space. Additionally or alternatively, in some embodiments, the update module 310 can also determine the angle information 322 and shape information 324 of the parking space, etc. The exemplary parking space determination system 100B can then use the local information determined by the update module 310 to update one or more of the global information determined by the base module.

[0055] As an example, in some embodiments, the exemplary parking space determination system 100B can compare the difference between the corner point position data provided by the base module 308 and the corner point position data provided by the update module 310. If the difference between the corner point position data provided by the base module 308 and the corner point position data provided by the update module 310 is within a predetermined threshold range, the corner point position data provided by the base module 308 can be replaced with the corner point position data provided by the update module 310. The predetermined threshold range can be determined by the manufacturer or can be determined or updated by the machine learning model 102 during the training and debugging process. Thus, rapid determination of various attribute information of the parking space can be achieved.

[0056] The following Figure 4 describes a schematic diagram of a process 400 for training the base module 308 and the update module 310 in a machine learning model according to an embodiment of the present disclosure. As Figure 4 shown, the machine learning model 402 can be a model that is the same as or similar to the machine learning model 102 shown in FIG. 1.

[0057] The overall data 404 may include data on various parking spaces, such as data on various parking space functions, availability, shape, size, restrictions, the parking lot number to which they belong, geographical location coordinates, or area identifiers, etc. The overall data 404 may be further divided into labeled data 406 and unlabeled data 408.

[0058] For example, an annotator may label the function of a parking space (e.g., a disabled parking space), shape (parallel, angled, perpendicular), size, etc. based on on-site observations or relevant information to generate the labeled data 406. In some embodiments, the unlabeled data 408 may be parking space data automatically collected by devices such as sensors and camera cameras, which has not been manually confirmed or corrected.

[0059] The labeled data 406 can then be used to train the machine learning model 402. For example, the labeled data 406 can be subjected to feature extraction and then divided into a training set, a validation set, and a test set. A conventional ratio is usually used for division (e.g., 70%-20%-10%). Subsequently, the training set can be used to train the machine learning model 402. The machine learning model 402 is adjusted by learning the patterns and features in the training set. In some embodiments, the validation set can be used to evaluate the machine learning model 402, examine its performance, and perform model parameter tuning. Finally, the test set can be used to evaluate the performance of the machine learning model 402 to ensure that the model performs well on new data and avoid overfitting or underfitting.

[0060] In some embodiments, the trained machine learning model 402 can also perform model inference 410 using the unlabeled data 408. For example, the trained machine learning model 402 can make predictions on the unlabeled image 408, such as whether there is a parking space in the image, the availability of the parking space, or the location of the parking space. In some embodiments, the unlabeled data 408 can be further classified into images with dimensions higher than a certain predetermined dimension and images with dimensions lower than a certain predetermined dimension. Images with dimensions higher than a certain predetermined dimension can be used to further train the update module 310, while images with dimensions lower than a certain predetermined dimension can be used to further train the basic module 308.

[0061] The machine learning model 402 calculates the entropy 412 of the feature map of the prediction for each unlabeled image. The sum of the uncertainties of each responsive target represents the uncertainty of that image. A high entropy value indicates a high prediction uncertainty of the machine learning model 402 for this image. At the same time, the machine learning model 402 can use the k-center set covering problem to select the next sample to be labeled from the unlabeled set, minimizing the maximum distance from other samples in the unlabeled set to the labeled sample set. This ensures data diversity and avoids selecting duplicate samples by only choosing samples with high uncertainty.

[0062] In some embodiments, images with entropy values higher than a threshold can be regarded as information-rich corner cases and marked as images requiring manual intervention or further processing. These images may contain special situations that are currently difficult for the model to accurately predict, such as corners or other difficult-to-process scenarios. For example, the top K images with entropy values higher than the threshold can be screened 414 for labeling. Conversely, if the entropy value is low, it means that the uncertainty of the image is low, and the current machine learning model 402 is capable of identifying the parking space in the image.

[0063] In some embodiments, the screened high-entropy images can be labeled and added to the training set for further training of the machine learning model 402. Through the active learning method, the machine learning model 402 can be optimized for these difficult-to-process situations and improve its ability to identify special situations. By this method, active learning is combined with the parking space detection system to discover information-rich corner cases.

[0064] Figure 5 A schematic diagram of a device for determining a parking space according to an embodiment of the present disclosure is further shown. The device 500 can be applied to the exemplary parking space determination system 100, which can include multiple modules for performing corresponding steps in the process 200 as Figure 2 discussed. As Figure 5 shown, the device 500 can include: a generation unit 502 configured to generate a first heat map and a second heat map based on an image acquired by a camera, where the size of the first heat map is smaller than the size of the second heat map; a first determination unit 504 configured to determine first information associated with a parking space based on the first heat map; a second determination unit 506 configured to determine second information associated with a parking space based on the second heat map; and a third determination unit 508 configured to determine the position information of the parking space according to the first information and the second information.

[0065] In some embodiments, the apparatus 500 further includes: a determination unit configured to determine the center point of a parking space in a first heat map; a determination unit configured to determine an offset from the center point to a corner point in the parking space based on offset regression; a determination unit configured to determine first position data of the corner point, and the length and width of the parking space based on the offset and the center point; and a determination unit configured to determine the availability of the parking space based on the first heat map, where the availability includes the occupied state and the vacant state of the parking space.

[0066] In some embodiments, the apparatus 500 further includes: a determination unit configured to determine second position data of a corner point in the parking space and angle information of the parking space based on information in a second heat map, where the second position data is more accurate than the first position data.

[0067] In some embodiments, the apparatus 500 further includes: a presentation unit configured to present information associated with the parking space to a user using first information; and an update unit configured to update information associated with the parking space based on second information.

[0068] In some embodiments, the apparatus 500 further includes: a comparison unit configured to compare the first position data of the corner point with the second position data; and a replacement unit configured to replace the first position data of the corner point with the second position data in response to a difference between the first position data and the second position data being within a predetermined threshold range.

[0069] In some embodiments, the apparatus 500 further includes: a unit for a neural network configured to process an image acquired by a camera to generate a first heat map for a base module and a second heat map for an update module. The neural network is trained using labeled data; and the trained neural network makes predictions on unlabeled images.

[0070] In some embodiments, the apparatus 500 further includes: a unit for a neural network configured to cluster and identify unlabeled images to determine one or more data among the parking space position, shape, corner point information, and availability in the unlabeled images; and a generation unit configured to generate a training base module and an update module based on the unlabeled images, where the base module is trained using a first-size image in the unlabeled images; and the update module is trained using a second-size image in the unlabeled images, where the first-size image is smaller than the second-size image.

[0071] In some embodiments, the apparatus 500 further includes: a determination unit configured to determine a prediction entropy value associated with an unlabeled image; a comparison unit configured to compare the prediction entropy value with a predetermined threshold; and a labeling unit configured to label the unlabeled image in response to the prediction entropy value being higher than the predetermined threshold.

[0072] In some embodiments, the apparatus 500 further includes a unit for first information, configured to include one or more of the location information, availability, length, width, and profile shape of a parking space; and a unit for second information, configured to include one or more of the function, corner point data, and angle data of the parking space.

[0073] Figure 6 A schematic block diagram of an example device 600 that can be used to implement the embodiments of the present disclosure is shown. As shown, the device 600 includes a processor 601, which can execute various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 602 and loaded into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0074] The various processes and treatments described above, such as method 200 and process 300, can be executed by the processor 601. For example, in some embodiments, method 200 and process 300 can be implemented as a computer software program, which is tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602. When the computer program is loaded into the RAM 603 and executed by the processor 601, one or more actions of the methods 500 and processes 600 described above can be performed.

[0075] The present disclosure can be a method, apparatus, system, and / or computer program product. The computer program product can include a computer-readable storage medium having computer-readable program instructions thereon for performing various aspects of the present disclosure.

[0076] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being an instantaneous signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0077] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or downloaded to an external computer or an external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0078] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine - related instructions, microcode, firmware instructions, state - setting data, or source code or object code written in any combination of one or more programming languages, including object - oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer - readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand - alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer - readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field - programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer - readable program instructions to implement various aspects of the present disclosure.

[0079] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer - readable program instructions.

[0080] These computer - readable program instructions can be provided to a processing unit of a general - purpose computer, a special - purpose computer, or other programmable data - processing apparatus to produce a machine such that, when the instructions are executed by the processing unit of the computer or other programmable data - processing apparatus, a device is produced that implements the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer - readable program instructions can also be stored in a computer - readable storage medium, which causes a computer, a programmable data - processing apparatus, and / or other devices to operate in a particular manner. Thus, the computer - readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, causing a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or by combinations of special purpose hardware and computer instructions.

[0083] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of technologies in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for determining a parking space, the method comprises: generating a first heatmap and a second heatmap based on an image acquired by a camera, wherein the size of the first heatmap is smaller than the size of the second heatmap; determining first information associated with the parking space based on the first heatmap; determining second information associated with the parking space based on the second heatmap; and determining location information of the parking space according to the first information and the second information.

2. The method according to claim 1, wherein determining the first information associated with the parking space based on the first heatmap comprises: determining a center point of the parking space in the first heatmap; determining an offset from the center point to a corner point in the parking space based on offset regression; determining first position data of the corner point and the length and width of the parking space based on the offset and the center point; and determining the availability of the parking space based on the first heatmap, wherein the availability includes the occupied state and the free state of the parking space.

3. The method according to claim 2, wherein generating the second information associated with the parking space based on the second heatmap comprises: determining second position data of the corner point in the parking space and angle information of the parking space based on information in the second heatmap, wherein the second position data is more accurate than the first position data.

4. The method according to claim 3, further comprises: presenting information associated with the parking space to a user by using the first information; and updating the information associated with the parking space based on the second information.

5. The method according to claim 4, wherein updating the information associated with the parking space based on the second information comprises: comparing the first position data of the corner point with the second position data; and in response to a difference between the first position data and the second position data being within a predetermined threshold range, replacing the first position data of the corner point with the second position data.

6. The method according to claim 1, wherein an image acquired by the camera is processed by a neural network to generate the first heatmap and the second heatmap, wherein the first heatmap is for a basic module and the second heatmap is for an update module; the neural network is trained by using labeled data; and the trained neural network makes predictions on unlabeled images.

7. The method according to claim 6, wherein making predictions on unlabeled images by the trained neural network comprises: the neural network clustering and identifying unlabeled images to determine one or more data of parking space locations, shapes, corner point information, availability in the unlabeled images; and training the basic module and the update module based on the unlabeled images, wherein: using a first-sized image in the unlabeled images to train the basic module; and The updating module is trained using the second-sized images in the unlabeled images, where the first-sized images are smaller than the second-sized images.

8. The method according to claim 6, wherein predicting the unlabeled images by the trained neural network further comprises: determining a prediction entropy value associated with the unlabeled image; comparing the prediction entropy value with a predetermined threshold; and in response to the prediction entropy value being higher than the predetermined threshold, labeling the unlabeled image.

9. The method according to claim 1, wherein the first information includes one or more of location information, availability, length, width, and contour shape of a parking space; and the second information includes one or more of the function of the parking space, corner point data, and angle data.

10. A device for determining a parking space, the device comprises: a generating unit configured to generate a first heatmap and a second heatmap based on an image acquired by a camera, wherein the size of the first heatmap is smaller than the size of the second heatmap; a first determining unit configured to determine first information associated with the parking space based on the first heatmap; a second determining unit configured to determine second information associated with the parking space based on the second heatmap; and a third determining unit configured to determine the location information of the parking space according to the first information and the second information.

11. A controller, comprises: at least one processor; and a memory coupled to the at least one processor and having instructions stored thereon, the instructions when executed by the at least one processor cause the controller to perform the method according to any one of claims 1-9.

12. A vehicle comprising the controller according to claim 11.

13. A machine-readable storage medium having machine-executable instructions stored thereon, wherein the machine-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 9.

Images