Parking space detection method and device, computer device and storage medium

Abstract

The application relates to a parking space detection method and device, computer equipment and a storage medium. The method comprises the following steps: performing feature extraction on a to-be-detected parking space image through a backbone network of a parking space detection model to obtain image features; extracting global features in the image features through a global branch network of the parking space detection model; wherein the global features comprise a parking space type, a parking space occupancy state and a first parking space corner point position; extracting local features in the image features through a local branch network of the parking space detection model; wherein the local features comprise a second parking space corner point position and a parking space direction angle; the network structures of the global branch network and the local branch network are different; and the global features and the local features of the parking space image are integrated and processed to obtain parking space detection data of the parking space image. The method can improve the parking space detection accuracy.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a parking space detection method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the increasing maturity of autonomous driving technology, many technologies have reached the requirements for road use and mass production. In autonomous driving systems, parking space detection provides perception target information for autonomous parking systems, which is crucial for subsequent path planning. Its reliability and accuracy are vital to the safety and stability of autonomous parking. Furthermore, with the ever-increasing number of motor vehicles, parking difficulties have become a significant problem in urban streets, making the construction of intelligent parking management systems extremely urgent. Parking space detection technology is one of the key technologies in intelligent parking management systems.

[0003] Traditional techniques extract global and local features from parking space images using an end-to-end single-stage network model, and then fuse these two pieces of information to obtain parking space data. However, because traditional techniques do not differentiate between global and local features, the accuracy of parking space detection is low. Summary of the Invention

[0004] Therefore, it is necessary to provide a parking space detection method, device, computer equipment, and storage medium that can improve the accuracy of parking space detection, addressing the aforementioned technical problems.

[0005] Firstly, this application provides a parking space detection method. The method includes:

[0006] The backbone network of the parking space detection model is used to extract features from the images of the parking spaces to be detected, thus obtaining image features.

[0007] Global features are extracted from the image features through the global branch network of the parking space detection model. The global features include: the type of parking space, the occupancy status of the parking space, and the location of the corner point of the first parking space.

[0008] The local branch network of the parking space detection model extracts local features from the image features; these local features include: the position of the second parking space corner and the parking space orientation angle; the network structure of the global branch network is different from that of the local branch network.

[0009] The global and local features of the parking space image are integrated to obtain the parking space detection data.

[0010] In one embodiment, the convolution kernel used in the global branch network is larger than the convolution kernel used in the local branch network; the size of the local features is equal to the size of the image features, and the size of the global features is smaller than the size of the image features.

[0011] In one embodiment, the global branch network includes: a parking space type regression branch, a parking space status regression branch, and a corner position regression branch;

[0012] The global branch network of the parking space detection model is used to extract global features from the image features, including:

[0013] By using the parking space type regression branch, the target point belonging to the parking space among the feature points corresponding to the image features is determined, as well as the parking space type to which the target point belongs;

[0014] The parking space occupancy status is determined by the parking space status regression branch.

[0015] By using the corner point position regression branch, the corner point position of the parking space corresponding to the target point is determined.

[0016] In one embodiment, the parking space type regression branch includes at least one channel, and each channel corresponds to a candidate parking space type;

[0017] The parking space type regression branch determines the target point belonging to a parking space among the feature points corresponding to the image features, as well as the parking space type to which the target point belongs, including:

[0018] By using each channel of the parking space type regression branch, the probability of each feature point corresponding to the image feature belonging to each candidate parking space type is determined;

[0019] Based on the relationship between the probability of each feature point belonging to each candidate parking space type and the preset probability threshold, target points belonging to parking spaces are determined from each feature point;

[0020] The parking space type of the target point is determined based on the probability that the target point belongs to each candidate parking space type.

[0021] In one embodiment, the local branch network of the parking space detection model extracts local features from the image features, including:

[0022] The image features are classified by angle using a local branch network to obtain the parking direction angle in the local features of the image features; the local branch network is trained by a circular Gaussian window.

[0023] In one embodiment, the backbone network is determined as follows:

[0024] Obtain the scaling factor of each network channel in the lightweight neural network MobilenetV3;

[0025] Network channels with a scaling factor less than a preset threshold are designated as channels to be deleted.

[0026] The channels to be removed in the lightweight neural network MobilenetV3 are pruned to obtain the backbone network.

[0027] In one embodiment, the global and local features of the parking space image are integrated to obtain parking space detection data, including:

[0028] Based on the position of the second parking space corner point, the position of the first parking space corner point is corrected to obtain the corrected parking space corner point position;

[0029] The parking space detection data is determined based on the corrected parking space corner location, parking space type, parking space occupancy status, and parking space orientation angle.

[0030] Secondly, this application also provides a parking space detection device. The device includes:

[0031] The feature extraction module is used to extract features from the image of the parking space to be detected through the backbone network of the parking space detection model, so as to obtain image features;

[0032] The global extraction module is used to extract global features from the image features through the global branch network of the parking space detection model. The global features include: the type of parking space, the occupancy status of the parking space, and the location of the corner point of the first parking space.

[0033] The local extraction module is used to extract local features from the image features through the local branch network of the parking space detection model. The local features include: the position of the second parking space corner and the parking space orientation angle. The network structure of the global branch network is different from that of the local branch network.

[0034] The feature integration module is used to integrate global and local features to obtain parking space detection data from the parking space image.

[0035] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0036] The backbone network of the parking space detection model is used to extract features from the images of the parking spaces to be detected, thus obtaining image features.

[0037] Global features are extracted from the image features through the global branch network of the parking space detection model. The global features include: the type of parking space, the occupancy status of the parking space, and the location of the corner point of the first parking space.

[0038] The local branch network of the parking space detection model extracts local features from the image features; these local features include: the position of the second parking space corner and the parking space orientation angle; the network structure of the global branch network is different from that of the local branch network.

[0039] The global and local features of the parking space image are integrated to obtain the parking space detection data.

[0040] Fourthly, this application also provides a computer-readable storage medium. This computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0041] The backbone network of the parking space detection model is used to extract features from the images of the parking spaces to be detected, thus obtaining image features.

[0042] Global features are extracted from the image features through the global branch network of the parking space detection model. The global features include: the type of parking space, the occupancy status of the parking space, and the location of the corner point of the first parking space.

[0043] The local branch network of the parking space detection model extracts local features from the image features; these local features include: the position of the second parking space corner and the parking space orientation angle; the network structure of the global branch network is different from that of the local branch network.

[0044] The global and local features of the parking space image are integrated to obtain the parking space detection data.

[0045] Fifthly, this application also provides a computer program product. This computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0046] The backbone network of the parking space detection model is used to extract features from the images of the parking spaces to be detected, thus obtaining image features.

[0047] Global features are extracted from the image features through the global branch network of the parking space detection model. The global features include: the type of parking space, the occupancy status of the parking space, and the location of the corner point of the first parking space.

[0048] The local branch network of the parking space detection model extracts local features from the image features; these local features include: the position of the second parking space corner and the parking space orientation angle; the network structure of the global branch network is different from that of the local branch network.

[0049] The global and local features of the parking space image are integrated to obtain the parking space detection data.

[0050] The aforementioned parking space detection method, apparatus, computer equipment, and storage medium extract features by inputting parking space images into a pre-built parking space detection model. Because the global and local branch networks in the pre-built model have different structures, the extracted global and local features also have different resolutions. This allows for the design of features that can be differentiated between the two branches, enabling the distinction between different types of features. Compared to traditional methods that extract global and local features using the same network structure and only perform branch-based classification without feature differentiation, this improves the accuracy of parking space detection. Furthermore, the parking space detection model only needs to extract three global features: the type of parking space, the occupancy status, and the location of the first corner point, reducing the computational load and thus improving inference efficiency. Attached Figure Description

[0051] Figure 1 This is a diagram illustrating the application environment of a parking space detection method in one embodiment.

[0052] Figure 2 This is a flowchart illustrating a parking space detection method in one embodiment;

[0053] Figure 3 This is a schematic diagram of the network structure of a parking space detection model in one embodiment;

[0054] Figure 4 This is a flowchart illustrating the steps of extracting global features from image features using a global branch network of a parking space detection model in one embodiment.

[0055] Figure 5 This is a structural block diagram of a parking space detection device in one embodiment;

[0056] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] The parking space detection method provided in this application can be applied to, for example, Figure 1The example shown is an autonomous driving application environment. The vehicle is pre-installed with onboard sensors 102 and an onboard computer device 104. The onboard computer device can be simply referred to as the computer device. The onboard sensors 102 communicate with the computer device 104 via a network. The onboard sensors 102 acquire images of the parking spaces to be detected and send these images to the computer device 104. The computer device 104 extracts features from the parking space images using the backbone network of the parking space detection model, obtaining image features. It then extracts global features from the image features using the global branch network of the parking space detection model; these global features include: the type of parking space, the occupancy status of the parking space, and the position of the first corner point of the parking space. Subsequently, it extracts local features from the image features using the local branch network of the parking space detection model; these local features include: the position of the second corner point of the parking space and the parking space orientation angle. The network structures of the global branch network and the local branch network are different. Finally, the computer device 104 integrates and processes the global and local features of the parking space image to obtain parking space detection data. The vehicle-mounted sensor 102 may be, but is not limited to, various image acquisition devices and video acquisition devices.

[0059] In one embodiment, such as Figure 2 As shown, a parking space detection method is provided, which is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0060] Step 202: Extract features from the image of the parking space to be detected using the backbone network of the parking space detection model to obtain image features.

[0061] The parking space detection model refers to a network model used to detect parking space information to achieve parking space localization. The parking space image to be detected refers to an image generated by capturing the parking space environment around the vehicle. Image features refer to the high-order semantic features in the parking space image to be detected.

[0062] Specifically, the vehicle is pre-installed with onboard sensors and computer equipment. During autonomous driving, the vehicle uses the onboard sensors to capture images of the parking spaces around it, obtaining images of the parking spaces to be detected, and then sends these images to the computer equipment. For example, the onboard sensors can be onboard cameras. The computer equipment pre-stores a parking space detection model, which can be trained using a large number of samples. The parking space detection model includes a backbone network and a detection head network. The backbone network is used to extract features from the images of the parking spaces to be detected, obtaining image features. The detection head network can include global branch networks and local branch networks. The detection head network is used to regress specific data about the parking spaces based on the image features. After acquiring the images of the parking spaces to be detected, the computer equipment calls the parking space detection model, inputs the images into the model, and uses the backbone network of the model to downsample the images to obtain high-order semantic features, which are then used as image features.

[0063] Optionally, the parking space image to be detected can be a 360-degree view of the parking space around the vehicle captured by multiple vehicle-mounted cameras.

[0064] For example, the parking space image to be detected is 512×512 in size. After the backbone network downsamples the parking space image to be detected five times, a feature map of size 13×13 is obtained, which is the image feature.

[0065] Step 204: Extract global features from the image features through the global branch network of the parking space detection model; wherein, the global features include: the type of parking space, the occupancy status of the parking space, and the position of the corner point of the first parking space.

[0066] Among them, global features refer to the overall data of parking spaces.

[0067] After extracting image features, these features can be input into the detection head network. The global branch network within the detection head network then performs overall parking space data detection on the image features. Specifically, the global branch network downsamples the image features to obtain a low-resolution feature map, i.e., global features. These global features can include the parking space type, parking space occupancy status, and the location of the first parking space corner. The parking space type can be a perpendicular, horizontal, or angled parking space. The parking space occupancy status includes occupied and unoccupied spaces. The location of the first parking space corner can be the coordinates of the corner determined by the global branch network based on the image features. Global features are more precise global features than image features. By detecting the overall parking space data, the parking space area, parking space type, and parking space occupancy status of each parking space can be accurately located.

[0068] Step 206: Extract local features from the image features through the local branch network of the parking space detection model; the local features include: the position of the second parking space corner and the parking space orientation angle; the network structure of the global branch network is different from that of the local branch network.

[0069] The global branch network is used for overall parking space data detection, while the local branch network is used for parking space corner data detection and regression. While extracting global features through the global branch network, the local branch network can be used to detect and regress parking space corner data from the image features, obtaining local features within the image. These local features may include the location of the second parking space corner and the parking space orientation angle. The location of the second parking space corner is the coordinate of the corner extracted by the local branch network based on image features. The parking space orientation angle refers to the direction the parking space faces.

[0070] Since the global branch network and the local branch network have different network structures, the extracted global features and local features also have different sizes. This allows for the design of features that can be differentiated between the two branches, which is beneficial for further improving the accuracy of parking space detection.

[0071] Step 208: Integrate the global and local features of the parking space image to obtain the parking space detection data of the parking space image.

[0072] After outputting global and local features, the global and local branch networks match the first parking space corner position from the global features with the second parking space corner position from the local features. Based on the matching result, the final parking space corner position in the parking space image is determined. This integrates the final parking space corner position, the parking space type, the parking space occupancy status, and the parking space orientation angle to obtain the parking space detection data for the parking space image.

[0073] In the aforementioned parking space detection method, the image of the parking space to be detected is input into a pre-built parking space detection model. Because the global and local branch networks in the pre-built model have different network structures, the resolutions of the global features extracted by the global branch network and the local features extracted by the local branch network also differ. This allows for the design of features from both branches to differentiate between different types of features. Compared to traditional methods that extract global and local features using the same network structure and only perform classification at the branch level without feature differentiation, this method improves the accuracy of parking space detection. Furthermore, the parking space detection model only needs to extract three global features: the type of parking space, the occupancy status, and the location of the first corner point, reducing the computational load and thus improving the model's inference efficiency.

[0074] In one embodiment, the convolution kernel used in the global branch network is larger than the convolution kernel used in the local branch network; the size of the local features is equal to the size of the image features, and the size of the global features is smaller than the size of the image features.

[0075] Image features still contain a large number of feature points. A global branch network is used to downsample these image features, resulting in global features smaller than the original image features. Since the global branch network needs to capture global information about the entire parking space, while the local branch network extracts local detail features, the extracted features differ. Therefore, the convolutional kernel used in the global branch network needs to be larger than that used in the local branch network, and the size of the local features needs to be the same as the size of the image features.

[0076] For example, the global branch network uses a 5×5 convolutional kernel. When the image features are 13×13 feature maps, a parking space bounding box will still contain many feature points. Therefore, the global branch network downsamples the image features to 7×7 global features using a 5×5 convolutional kernel on top of the 13×13 feature map. The local branch network uses a 3×3 convolution. Considering that the corner points of the parking spaces are small and local details are more important, no downsampling is performed. It still uses a feature map with the same size as the image features, that is, the local features are also 13×13 feature maps.

[0077] In this embodiment, by setting the convolution kernel used in the global branch network to be larger than that used in the local branch network, and the size of the global features to be smaller than the size of the image features, image features are downsampled. This reduces the model's memory and computational load, expands the receptive field of the features, and facilitates the rapid and accurate acquisition of the global features of the entire parking space. The local branch network uses a larger feature map with the same size as the image features, enabling it to better perceive local details.

[0078] In one embodiment, such as Figure 3 As shown, the global branch network of the parking space detection model extracts global features from the image features, including:

[0079] Step 302: Through the parking space type regression branch, determine the target point belonging to the parking space among the feature points corresponding to the image features, and the parking space type to which the target point belongs.

[0080] Step 304: Determine the parking space occupancy status corresponding to the target point through the parking space status regression branch.

[0081] Step 306: Determine the parking space corner position corresponding to the target point through the corner position regression branch.

[0082] A global branch network can be obtained by pruning a traditional global branch network. A traditional global branch network includes a parking space feature point recognition branch, a parking space type regression branch, a parking space status regression branch, and a corner position regression branch. Each branch is used to regress the following information: whether a feature point belongs to a parking space, the parking space type, the parking space occupancy status, and corner position data. Having too many branches increases both the computational load of the model and the difficulty of feature integration. Therefore, by pruning the parking space feature point recognition branch, a global branch network is obtained. Thus, the global branch network in this embodiment includes: a parking space type regression branch, a parking space status regression branch, and a corner position regression branch.

[0083] Since determining whether a feature point belongs to a parking space is essential information for parking space detection, this information can be regressed using a parking space type regression branch. Specifically, image features are input into a global branch network. The parking space type regression branch within this network identifies target points belonging to parking spaces among the feature points corresponding to the image features, and simultaneously determines the parking space type of each target point. The target points belonging to parking spaces, output from the parking space type regression branch, and the image features are then input into a parking space status regression branch to obtain the parking space occupancy status corresponding to the target point. Finally, the target points belonging to parking spaces, output from the parking space type regression branch, and the image features are input into a corner position regression branch to obtain the corner position of the parking space corresponding to the target point. Therefore, the resulting global features include target points belonging to parking spaces, the parking space type of each target point, the parking space occupancy status of each target point, and the corner position of each target point.

[0084] In this embodiment, by pruning the parking space feature point recognition branch, the number of branches in the global branch network is reduced, the branch structure of the global branch network is simplified, and the computational load of the model is reduced.

[0085] Furthermore, the parking space type regression branch includes at least one channel, and each channel corresponds to a candidate parking space type. Through the parking space type regression branch, the target point belonging to the parking space among the feature points corresponding to the image features, and the parking space type to which the target point belongs, are determined, including: determining the probability that each feature point corresponding to the image features belongs to each candidate parking space type through each channel of the parking space type regression branch; determining the target point belonging to the parking space from each feature point according to the relationship between the probability that each feature point belongs to each candidate parking space type and a preset probability threshold; and determining the parking space type to which the target point belongs according to the probability that the target point belongs to each candidate parking space type.

[0086] Among them, candidate parking space type refers to the parking space type that can be detected by the parking space type regression branch, and the parking space type to which the target point belongs is one of the candidate parking space types.

[0087] The parking space type regression branch includes at least one channel. The number of channels in the parking space type regression branch is determined by the number of parking space types that the branch can detect. For example, if three parking space types can be detected, the number of channels is set to 3. The probability of each feature point corresponding to an image feature belonging to each candidate parking space type is determined by each channel of the parking space type regression branch. The probability of each feature point belonging to each candidate parking space type determined by each channel is compared with a preset probability threshold. When the probability of each feature point belonging to each candidate parking space type determined by any channel is greater than the preset probability threshold, it indicates that the feature point belongs to a parking space, and the feature point is identified as the target point belonging to a parking space. Specifically, each channel can use a sigmoid activation function to determine the probability of each feature point corresponding to an image feature belonging to each candidate parking space type. Then, based on the probability of the target point belonging to each candidate parking space type, the candidate parking space type with the highest probability is determined as the parking space type to which the target point belongs.

[0088] In this embodiment, by using each channel of the parking space type regression branch, the probability of each feature point corresponding to the image feature belonging to each candidate parking space type is determined. Based on this probability, the target point belonging to the parking space can be determined from each feature point, which can quickly and accurately determine the feature point belonging to the parking space.

[0089] In one embodiment, extracting local features from image features through the local branch network of the parking space detection model includes: performing angle regression on the image features through the local branch network to obtain the parking space orientation angle in the local features of the image features; the local branch network is trained using a circular Gaussian window.

[0090] Since the local features include the parking space orientation angle, the local branch network can perform corner point position regression on the image features to determine the location of the second parking space corner point in the image features. This allows for angle classification of the image features, obtaining the probability that the second parking space corner point belongs to each angle. The probability of the second parking space corner point belonging to each angle exhibits periodicity, and the probability values ​​between adjacent angles are relatively close. Therefore, the angle with the highest probability is selected as the parking space orientation angle corresponding to the second parking space corner point.

[0091] The circular Gaussian window (CGWindow) is a technique used for rotating target detection. The labels processed by the CGWindow function are periodic circular labels, each representing a corresponding angle. Training the local branch network using the CGWindow ensures that the multiple angle probabilities output by the local branch network meet the periodicity requirement, thus addressing the issue of angle periodicity in parking direction angle regression. The labels processed by the CGWindow function are smooth labels, characterized by small differences in label values ​​between adjacent labels. Training the local branch network using the CGWindow ensures that the angle probabilities output by the local branch network have small differences between adjacent angles, satisfying the requirement of angle similarity between adjacent angles.

[0092] In this embodiment, since the labels processed by the circular Gaussian window are periodic circular labels and are smooth labels with small differences in label values ​​between adjacent labels, the probability of the second parking space corner point extracted by the local branch network trained by the circular Gaussian window belonging to each angle has periodicity and the probability values ​​between adjacent angles are small. This solves the problems of angle periodicity and adjacent similarity, and can more accurately determine the parking space orientation angle, thereby effectively improving the accuracy of parking space detection.

[0093] In one embodiment, the backbone network is the lightweight neural network MobileNet v3 (the third version of the mobile network). Compared to the original VGGNet16 (Visual Geometry Group Network), MobileNetv3 has fewer parameters, lower computational cost, and shorter inference time, making it more suitable for scenarios with limited storage space and power consumption, such as deployment and porting to edge computing devices like mobile embedded devices. This embodiment uses MobileNetv3 as the backbone network.

[0094] For example, the network structure of the parking space detection model can be as follows: Figure 4As shown. The parking space image is input into the parking space detection model, which includes a backbone network and a detection head network. The detection head network outputs parking space detection data. The dashed area in the parking space image input to the backbone network represents the parking space area. The backbone network is MobileNet v3, which includes: (1) the start part: convolutional layer (Conv3_s2), BN layer (BatchNormalization), and hswitch layer (activation layer). (2) the middle part: multiple blocks (bneck) containing convolutional layers. The middle part includes different Large and Small versions. The Large version includes 15 bnecks, and the Small version includes 11 bnecks. (3) the last part: average pooling layer (Avg_Pool), 1x1 convolution (implementing FC (fully connected) operation), BN layer, and hswitch layer. In the parking space detection data, the solid line area represents the specific location of the parking space frame. Parking space frame 1 represents an inclined parking space, parking space frame 2 represents a perpendicular parking space, and parking space frame 3 represents a parallel parking space.

[0095] Furthermore, the backbone network is determined as follows: the scaling factor of each network channel in the lightweight neural network MobilenetV3 is obtained; network channels with scaling factors less than a preset threshold are designated as channels to be deleted; and the channels to be deleted in the lightweight neural network MobilenetV3 are pruned to obtain the backbone network.

[0096] The lightweight neural network MobilenetV3 is a network trained with the addition of scaling factors for network channels. MobilenetV3 consists of a start section, a middle section, and a final section, each containing multiple network channels. Each network channel is used for feature extraction. The scaling factor for each network channel is a dynamically changing weight used to represent the importance of the corresponding network channel to the image features output by the network.

[0097] The backbone network used in this embodiment is a pruned version of the lightweight MobilenetV3 neural network, with less important channels removed. Specifically, the original neural network is obtained; this is the traditional MobilenetV3 without a scaling factor. A scaling factor is introduced for each channel of the original neural network; for example, the initial value of the scaling factor can be 0.5. A loss function containing the scaling factor for each channel is obtained, and this loss function is trained through multiple backpropagation iterations. When the number of training iterations reaches a preset number or the loss value of the loss function no longer decreases, network training is stopped, resulting in the lightweight MobilenetV3 neural network. Since the scaling factor contained in the loss function also changes dynamically during network training, the scaling factor of each channel in the lightweight MobilenetV3 neural network can be obtained when network training stops.

[0098] The scaling factor is compared with a preset threshold to identify network channels whose scaling factor is less than the preset threshold. The preset threshold can be a value close to zero. A scaling factor less than the preset threshold indicates that the channel is less important to the image features output by the network; the feature map output by this channel can be ignored, and removing this channel will not affect network performance. Network channels with scaling factors less than the preset threshold are selected for removal to obtain the backbone network. By removing unimportant network channels, the number of channels in the backbone network is reduced, thus reducing the computational load of parking space detection. It is understood that the training method of MobileNetV3 is a model training method known to those skilled in the art.

[0099] In this embodiment, since MobilenetV3 is a lightweight model with fast inference speed, further lightweighting of MobilenetV3 can further reduce model complexity, thereby greatly improving model inference speed.

[0100] In one embodiment, the global and local features of the parking space image are integrated to obtain parking space detection data, including: correcting the position of the first parking space corner point based on the position of the second parking space corner point to obtain the corrected parking space corner point position; and determining the parking space detection data based on the corrected parking space corner point position, the type of parking space, the occupancy status of the parking space, and the parking space orientation angle.

[0101] Specifically, the global branch network detects the corner points of parking spaces to obtain the position of the first parking space corner point. The local branch network also detects the corner points of parking spaces to obtain the position of the second parking space corner point. Since the regression of the local branch network is more accurate, the positions of the first and second parking space corner points can be matched to determine the distance between them. The second parking space corner point position with a distance less than a distance threshold is selected. If the distance is less than the distance threshold, it means that the first and second parking space corner points regress to the same corner point. Optionally, the second parking space corner point position can be selected by calculating the Euclidean distance between the first and second parking space corner points.

[0102] Since the first parking space corner point position only provides a rough regression of the corner point position, while the second parking space corner point position provides a more accurate regression, the corresponding first parking space corner point position is corrected by replacing the corresponding first parking space corner point position with the selected second parking space corner point position, resulting in a corrected parking space corner point position. Then, based on the corrected parking space corner point position, the type of parking space, the occupancy status, and the parking space orientation angle, the parking space detection data is determined.

[0103] In this embodiment, since the parking space corner positions regressed by the local branch network are more accurate, the first parking space corner position extracted by the global branch network can be corrected by using the second parking space corner position extracted by the local branch network, which can further improve the accuracy of the parking space detection data.

[0104] In another embodiment, a parking space detection method is provided, the method comprising:

[0105] The backbone network of the parking space detection model is used to extract features from the parking space image to be detected, thus obtaining image features.

[0106] By using the channels of the parking space type regression branch in the global branch network of the parking space detection model, the probability of each feature point corresponding to the image feature belonging to each candidate parking space type is determined; based on the relationship between the probability of each feature point belonging to each candidate parking space type and the preset probability threshold, the target point belonging to the parking space is determined from each feature point; based on the probability of the target point belonging to each candidate parking space type, the parking space type to which the target point belongs is determined.

[0107] The parking space occupancy status of the target point is determined by the parking space status regression branch in the global branch network.

[0108] The corner position of the target point is determined by the corner position regression branch in the global branch network; the global features are obtained based on the parking space type, parking space occupancy status and the corner position of the first parking space.

[0109] The local branch network of the parking space detection model extracts local features from the image features. The local features include the position of the second parking space corner and the parking space orientation angle. The convolution kernel used in the global branch network is larger than the convolution kernel used in the local branch network. The size of the local features is equal to the size of the image features, and the size of the global features is smaller than the size of the image features.

[0110] Based on the position of the second parking space corner, the position of the first parking space corner is corrected to obtain the corrected parking space corner position.

[0111] The parking space detection data is determined based on the corrected parking space corner location, parking space type, parking space occupancy status, and parking space orientation angle.

[0112] In this embodiment, by setting the convolution kernel used in the global branch network to be larger than that used in the local branch network, and the size of the global features to be smaller than that of the image features, image features are downsampled. This reduces the model's memory and computational load, expands the receptive field of the features, and facilitates the rapid and accurate acquisition of the global features of the entire parking space. The local branch network uses a feature map of a larger size, the same as the image feature size, which can better perceive local details. Determining whether a feature point belongs to a parking space through a parking space type regression branch simplifies the branch structure of the global branch network, reduces the model's computational load, and also reduces the difficulty of feature integration. Since the parking space corner positions regressed by the local branch network are more accurate, the first parking space corner position extracted by the global branch network is corrected by the second parking space corner position extracted by the local branch network, which can further improve the accuracy of the parking space detection data.

[0113] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0114] Based on the same inventive concept, this application also provides a parking space detection device for implementing the parking space detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more parking space detection device embodiments provided below can be found in the limitations of the parking space detection method described above, and will not be repeated here.

[0115] In one embodiment, such as Figure 5 As shown, a parking space detection device is provided, including: a feature extraction module 502, a global extraction module 504, a local extraction module 506, and a feature integration module 508, wherein:

[0116] The feature extraction module 502 is used to extract features from the parking space image to be detected through the backbone network of the parking space detection model to obtain image features.

[0117] The global extraction module 504 is used to extract global features from the image features through the global branch network of the parking space detection model; wherein, the global features include: the type of parking space, the occupancy status of the parking space, and the position of the corner point of the first parking space.

[0118] The local extraction module 506 is used to extract local features from the image features through the local branch network of the parking space detection model; wherein, the local features include: the position of the second parking space corner point and the parking space orientation angle; the network structure of the global branch network is different from that of the local branch network.

[0119] The feature integration module 508 is used to integrate global and local features through the parking space detection model to obtain parking space detection data of the parking space image.

[0120] In one embodiment, the convolution kernel used in the global branch network is larger than the convolution kernel used in the local branch network; the size of the local features is equal to the size of the image features, and the size of the global features is smaller than the size of the image features.

[0121] In one embodiment, the global branch network includes: a parking space type regression branch, a parking space status regression branch, and a corner position regression branch; the global extraction module 504 is further used to determine the target point belonging to the parking space among each feature point corresponding to the image feature through the parking space type regression branch, and the parking space type to which the target point belongs; to determine the parking space occupancy status corresponding to the target point through the parking space status regression branch; and to determine the parking space corner position corresponding to the target point through the corner position regression branch.

[0122] In one embodiment, the parking space type regression branch includes at least one channel, and each channel corresponds to a candidate parking space type; the global extraction module 504 is further used to determine the probability that each feature point corresponding to the image feature belongs to each candidate parking space type through each channel of the parking space type regression branch; determine the target point belonging to the parking space from each feature point according to the relationship between the probability that each feature point belongs to each candidate parking space type and a preset probability threshold; and determine the parking space type to which the target point belongs according to the probability that the target point belongs to each candidate parking space type.

[0123] In one embodiment, the local extraction module 506 is further configured to perform angle classification on image features through a local branch network to obtain the parking direction angle in the local features of the image features; the local branch network is trained using a circular Gaussian window.

[0124] In one embodiment, the feature extraction module 502 is further configured to extract features from the parking space image to be detected by using the backbone network of the lightweight neural network MobilenetV3 after cropping, so as to obtain image features.

[0125] In one embodiment, the feature integration module 508 is further configured to correct the position of the first parking space corner point based on the position of the second parking space corner point to obtain the corrected parking space corner point position; and to determine the parking space detection data based on the corrected parking space corner point position, the type of parking space, the occupancy status of the parking space, and the parking space orientation angle.

[0126] Each module in the aforementioned parking space detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0127] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores images of parking spaces to be detected and pre-built parking space detection models. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a parking space detection method.

[0128] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0130] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0131] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0133] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0134] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A parking space detection method characterized by comprising: The method comprises: extracting features of a to-be-detected parking space image through a backbone network of a parking space detection model to obtain image features; extracting global features in the image features through a global branch network of the parking space detection model; wherein the global features comprise a parking space type, a parking space occupancy state, and a first parking space corner point position; extracting local features in the image features through a local branch network of the parking space detection model; wherein the local features comprise a second parking space corner point position and a parking space direction angle; the network structure of the global branch network is different from that of the local branch network; integrating the global features and the local features of the parking space image to obtain parking space detection data of the parking space image; wherein a convolution kernel used by the global branch network is larger than a convolution kernel used by the local branch network; the size of the local features is equal to the size of the image features, and the size of the global features is smaller than the size of the image features.

2. The method of claim 1, wherein, The global branch network comprises a parking space type regression branch, a parking space state regression branch, and a corner point position regression branch. extracting global features in the image features through a global branch network of the parking space detection model comprises: determining, through the parking space type regression branch, a target point belonging to a parking space among feature points corresponding to the image features, and a parking space type to which the target point belongs; determining, through the parking space state regression branch, a parking space occupancy state corresponding to the target point; determining, through the corner point position regression branch, a parking space corner point position corresponding to the target point.

3. The method of claim 2, wherein, The parking space type regression branch comprises at least one channel, and each channel corresponds to a candidate parking space type. The determination, through the parking space type regression branch, of a target point belonging to a parking space among feature points corresponding to the image features, and a parking space type to which the target point belongs, comprises: determining, through each channel of the parking space type regression branch, a probability that each feature point corresponding to the image features belongs to each candidate parking space type; determining, from each feature point, a target point belonging to a parking space according to a relationship between the probability that each feature point belongs to each candidate parking space type and a preset probability threshold; determining, according to the probability that the target point belongs to each candidate parking space type, a parking space type to which the target point belongs.

4. The method of claim 1, wherein, The extraction, through the local branch network of the parking space detection model, of local features in the image features comprises: performing angle classification on the image features through the local branch network to obtain a parking space direction angle in the local features of the image features; the local branch network is trained through a ring-shaped Gaussian window.

5. The method of claim 1, wherein, The backbone network is determined in the following manner: obtaining proportion factors of each network channel in a lightweight neural network MobilenetV3; regarding a network channel with a proportion factor smaller than a preset threshold as a to-be-deleted channel; performing pruning processing on the to-be-deleted channel in the lightweight neural network MobilenetV3 to obtain the backbone network.

6. The method of claim 1, wherein, The integration of the global features and the local features of the parking space image to obtain parking space detection data of the parking space image comprises: According to the second parking space angle point position, the first parking space angle point position is corrected to obtain a corrected parking space angle point position; According to the corrected parking space angle point position, the parking space type to which the parking space belongs, the parking space occupancy state, and the parking space direction angle, the parking space detection data is determined.

7. A parking space detection device, characterized by The device comprises: The feature extraction module is configured to perform feature extraction on the to-be-detected parking space image through a backbone network of a parking space detection model to obtain image features. The global feature extraction module is configured to extract global features in the image features through a global branch network of the parking space detection model; the global features comprise a parking space type to which the parking space belongs, a parking space occupancy state, and a first parking space angle point position. The local feature extraction module is configured to extract local features in the image features through a local branch network of the parking space detection model; the local features comprise a second parking space angle point position and a parking space direction angle; the network structure of the global branch network is different from that of the local branch network. The feature integration module is configured to perform integration processing on the global features and the local features to obtain parking space detection data of the parking space image. The global branch network adopts a convolution kernel larger than that of the local branch network; the size of the local features is equal to that of the image features, and the size of the global features is smaller than that of the image features.

8. The apparatus of claim 7, wherein, The global branch network comprises a parking space type regression branch, a parking space state regression branch, and an angle point position regression branch. The global feature extraction module is further configured to determine, through the parking space type regression branch, target points belonging to a parking space among feature points corresponding to the image features and the parking space type to which the target points belong; determine, through the parking space state regression branch, a parking space occupancy state corresponding to the target points; and determine, through the angle point position regression branch, a parking space angle point position corresponding to the target points. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images