Method and device for detecting parking space, storage medium and vehicle
By introducing an interactive attention enhancement network in parking space detection, combined with the interaction enhancement of local and global features, the problem of insufficient parking space detection accuracy in the existing technology is solved, and higher detection accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202310629487.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
The existing parking space detection algorithm cannot accurately predict the specific location and related information of the parking space, resulting in insufficient detection accuracy.
An interactive attention enhancement network is introduced, and the local feature map and global feature map are integrated interactively. Through spatial attention and channel attention modeling, more robust and accurate feature maps are generated, and model parameters are optimized to improve parking space detection accuracy.
Through the application of interactive attention enhancement network, the accuracy of parking space detection is significantly improved, and the parking space location and related information can be predicted more accurately, enhancing the robustness of the detection results.
Smart Images

Figure CN119068448A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and more specifically, to a method and device for detecting a parking space, a storage medium, and a vehicle. Background Art
[0002] Automatic parking systems play an important role in the field of autonomous driving. The detection of parking spaces is the first problem that needs to be solved in automatic parking systems. Traditional parking space detection methods mainly rely on the results of vehicle-mounted images, manually extract relevant parking space feature information such as parking space lines, parking space corners, and special colors, and then use the pre-designed geometric relationships to detect the marked parking spaces in the image. With the advent of the deep learning era, the image features of parking spaces can be automatically extracted by deep neural networks, and the parking space detection model has been unprecedentedly improved. Parking space detection technology based on deep learning has also become a hot topic of research in recent years. However, existing detection algorithms often cannot accurately predict the specific location and related information of parking spaces. Summary of the invention
[0003] One object of the present application is to provide a method for detecting a parking space, a device storage medium and a vehicle, which can improve the accuracy of parking space detection.
[0004] According to a first aspect of the present disclosure, a method for detecting a parking space is provided, comprising: obtaining a local feature map and a global feature map of a picture of a parking spot; inputting the local feature map and the global feature map into an interactive attention enhancement network to perform interactive enhancement to obtain an enhanced local feature map and an enhanced global feature map; and integrating the enhanced local feature map and the enhanced global feature map to output a parking space detection result.
[0005] According to a second aspect of the present disclosure, a computer-readable storage medium is further provided, wherein the computer-readable storage medium is used to store instructions, and the instructions are loaded by a processor to execute any one of the methods described in the first aspect.
[0006] According to a third aspect of the present disclosure, a device for detecting a parking space is also provided, the device for detecting a parking space comprising a memory, a processor, and a control program stored in the memory and executable on the processor, and when the processor executes the control program, the method for detecting a parking space according to any embodiment of the first aspect is implemented.
[0007] According to a fourth aspect of the present disclosure, a vehicle is also provided, comprising the device for detecting a parking space as described in the third aspect.
[0008] The embodiment of the present application introduces an interactive attention enhancement network into the method flow of parking space detection, interactively enhances and fuses local features and global features to achieve parking space detection, optimizes model parameters, and improves the accuracy of parking space detection.
[0009] Other features and advantages of the embodiments of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 This is a flow chart of a method for detecting a parking space provided by an embodiment of the present application.
[0011] Figure 2 This is a flow chart of a method for detecting a parking space provided in another embodiment of the present application.
[0012] Figure 3 This is a block diagram of a device for detecting parking spaces provided by an embodiment of the present application.
[0013] Figure 4 It is a block diagram of a vehicle provided by an embodiment of the present application. DETAILED DESCRIPTION
[0014] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.
[0015] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.
[0016] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.
[0017] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0018] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0019] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.
[0020] Figure 1 This is a flow chart of a method for detecting a parking space provided by an embodiment of the present application.
[0021] 101, obtaining a local feature map and a global feature map of the image of the parking spot.
[0022] 102, inputting the local feature map and the global feature map into the interactive attention enhancement network, performing interactive enhancement to obtain an enhanced local feature map and an enhanced global feature map.
[0023] 103, integrating the enhanced local feature map and the enhanced global feature map, and outputting the parking space detection result.
[0024] In one embodiment of the present application, an interactive attention enhancement network is introduced into the method flow of parking space detection, and local features and global features are interactively enhanced and fused to achieve parking space detection, thereby optimizing model parameters and improving the accuracy of parking space detection.
[0025] The interactive attention enhancement network in the embodiment of the present application can be a plug-and-play network designed based on the Non-local mechanism, which is internally defined as IA-block (interact attention block). IA-block can effectively integrate the information learned from the local feature map and the global feature map in the dual-branch network. Furthermore, IA-block can help the basic model generate more robust and accurate feature maps through spatial attention and channel attention modeling.
[0026] As an embodiment, the enhanced local feature map and the enhanced global feature map are integrated to output the parking space detection result, including: extracting information from the enhanced local feature map and the enhanced global feature map respectively to obtain local information and global information; and integrating the local information and the global information based on non-maximum suppression (NMS) to output the parking space detection result.
[0027] As an embodiment, the non-maximum suppression (NMS)-based integration of the local information and the global information to output the parking space detection result includes: based on the NMS, obtaining corner point information according to the local information and the global information; based on the NMS, obtaining parking space information according to the global information; and fusing the parking space information with the corner point information to output the parking space detection result.
[0028] As an embodiment, the inputting the local feature map and the global feature map into the interactive attention enhancement network and performing interactive enhancement to obtain enhanced local feature map and enhanced global feature map includes: performing element-by-element multiplication of the local feature map and the global feature map to obtain an interactive feature map; and obtaining the enhanced local feature map and the enhanced global feature map according to the interactive feature map.
[0029] As an embodiment, the interaction feature map is obtained according to the following method: M = F g ⊙F l , where F g represents the global feature map, F l represents the local feature map, ⊙ represents the element-by-element multiplication of the matrix, and M is the interaction feature map.
[0030] As an embodiment, obtaining the enhanced local feature map and the enhanced global feature map according to the interactive feature map includes: g and F l The enhanced local feature enhancement map and the enhanced global feature enhancement map are obtained by assigning values in the following manner:
[0031]
[0032]
[0033] Among them, softmax represents the activation function, conv is a 1×1 convolution operation, Represents matrix multiplication.
[0034] As an embodiment, the enhanced local feature map and the enhanced global feature map are integrated to output the parking space detection result, including: inputting the enhanced local feature enhancement map and the enhanced global feature enhancement map into the corresponding channel attention network to obtain the final enhanced feature map, and the channel attention network is configured as: convolution layer-activation function layer-convolution layer-activation function layer; passing the final enhanced feature map through an information extractor to obtain local information and global information, wherein the local information includes information for characterizing corner points, and the global information includes information for characterizing parking spaces; and outputting the parking space detection result according to the local information and the global information.
[0035] As an embodiment, the global information includes the confidence of the parking space, the position of the paired corner point and the occupancy status of the parking space; and / or, the local information includes the confidence of the corner point, the position of the corner point, and the direction of the corner point.
[0036] As an embodiment, the global information and the local information obtained by detection are compared with the actual parking space information and the actual corner point information, and the network parameters are corrected.
[0037] As an embodiment, obtaining a local feature map and a global feature map of a parking spot image includes: inputting the parking spot image into a front-end network to obtain a feature map; inputting the feature map into a back-end network to obtain the local feature map and the global feature map.
[0038] As an embodiment, inputting the picture of the parking spot into the front-end network to obtain the feature map includes: performing image resolution processing on the picture of the parking spot; and inputting the picture after image resolution processing into the front-end network for encoding to obtain the feature map.
[0039] As an embodiment, inputting the feature map into a back-end network to obtain a local feature map and a global feature map includes: inputting the feature map into a dual-branch back-end network for decoding to obtain the local feature map and the global feature map.
[0040] As an embodiment, the network used in the method is Darknet53.
[0041] The following will be combined Figure 2 The method and process of detecting parking spaces are introduced in detail. Figure 2 The method flow is only one embodiment of the present application.
[0042] Figure 2 is a flow chart of a method for detecting parking spaces provided by another embodiment of the present application. Figure 2 The method comprises the following steps.
[0043] 201, obtain a picture of the parking spot.
[0044] As an embodiment, a fisheye camera may be used to collect image data to obtain a panoramically fused parking spot image.
[0045] 202, processing the image of the parking spot to obtain a processed image.
[0046] As an embodiment, the image data collected by the fisheye camera can be synthesized and stitched into a top view using conventional methods, and the image data can be preprocessed to obtain parking space data for training or reasoning, such as obtaining pictures for training, or obtaining pictures for reasoning. For example, the parking space data for training can include 70,640 pictures, and the parking space data for reasoning can include 2,289 pictures, and the image resolution can all be 512×512. The parking space data in the corner point input network can be converted from a picture with an image resolution of 512×512 to a picture with an image resolution of 384×384 through the resize function in opencv and sent to the network for training and reasoning. Changing the image resolution can reduce the amount of image calculation. The image resolution of the parking space detection model can generally be set to 384×384, which satisfies the requirements of clear pictures and low calculation.
[0047] 203, the processed image is input into the front-end network Darknet53 to obtain a feature map.
[0048] The front-end network is used to encode the image to generate a feature map. The encoded image can be an image used for inference or an image used for training.
[0049] During the training phase, the processed images are input into the front-end network Darknet53, and the processed images for training are encoded to obtain feature maps for training.
[0050] In the inference stage, the processed image is input into the front-end network Darknet53, and the processed image for inference is encoded to obtain the feature map for inference.
[0051] Image features are feature images, which mainly include color features, texture features, shape features and spatial relationship features of the image.
[0052] 204, input the feature map into the backend network to obtain a local feature map and a global feature map.
[0053] As an embodiment, decoding the feature map obtained in the front-end network in step 203 can obtain a local feature map and a global feature map, and the decoding can be achieved through a convolution operation.
[0054] Specifically, the image can be sent to the deep neural network DarkNet53 to generate a feature map with 2024 channels and a scale of 12×12, that is, 12×12×2024. Local feature maps and global feature maps are obtained based on the feature map, which can be specifically achieved through decoding. For example, the feature map in the front-end network is convolved to obtain local feature maps and global feature maps. The feature maps in the two branches can be decoded through 1×1 convolution to obtain a 24×24×512 feature map. Among them, the global feature map is input into one branch, which is used to learn global information based on parking spaces. The local feature map is input into the other branch, which is used to learn local information based on corner points.
[0055] 205, the interactive attention enhancement network is used to interactively enhance the local feature map and the global feature map to obtain the final enhanced feature map.
[0056] The interactive attention enhancement network in the embodiment of the present application can be a plug-and-play network designed based on the Non-local mechanism, which is internally defined as IA-block (interact attention block). IA-block can effectively integrate the information learned from the local feature map and the global feature map in the dual-branch network. Furthermore, IA-block can help the basic model generate more robust and accurate feature maps through spatial attention and channel attention modeling.
[0057] In one embodiment of the present invention, in the model training stage, the local feature map used for training and the global feature map used for training can be sent to the two branches of Darknet53 respectively. Similarly, in the model inference stage, the local feature map used for reasoning and the global feature map used for reasoning can also be sent to the two branches of Darknet53 respectively. Among them, one branch is used to learn the global feature information of the parking space, and the other branch is used to learn the local feature information of the parking space. For example, the obtained local feature map and the global feature map can be sent to the interactive attention enhancement network to realize the interaction and enhancement of the local feature information and the global feature information in the two branches. Specifically, the feature maps of the two branches are used to realize feature interaction and enhancement using a method based on the attention mechanism. The calculation principle of the interactive attention enhancement network is shown as follows:
[0058] M=F g ⊙F l (Formula 1)
[0059] where F g represents the global feature map, F l represents the local feature map, ⊙ represents the element-by-element multiplication of the matrix, and M is the result of the interaction between the global feature map and the local feature map, that is, the obtained interactive feature map. In one embodiment of the present application, feature interaction can be achieved through Formula 1.
[0060]
[0061]
[0062] Among them, softmax is the activation function layer, conv is a 1×1 convolution operation, Represents matrix multiplication. The above formula 2 and formula 3 are both commonly used assignments in computer programming. According to the interactive feature map combined with the above formula 2 and formula 3, an enhanced local feature map and an enhanced global feature map can be obtained. In the training stage, the network parameters can be optimized. In one embodiment of the present application, feature enhancement can be achieved by formula 2 and formula 3.
[0063] The interactive attention enhancement network can be a module designed based on channel attention and spatial attention. The spatial attention in the feature map can be modeled by the above formulas 2 and 3. These feature maps are then introduced into the backend network to implement channel attention.
[0064] Spatial attention is to model the spatial pixels in the feature map to find important information in the pixel space. For example, spatial attention can associate a pixel in an image with other pixels and calculate a weight to facilitate the network to focus on identifying and learning relatively important parts.
[0065] Channel attention refers to the change in the number of channels of a feature map when it passes through a convolutional layer. The introduction of a channel attention network here actually optimizes the channel change of the feature map by calculating a weight. Channel attention is the modeling between channels in the feature map to optimize the transformation of the feature map channels. The channel attention network can be a simple and effective network, and the network can be configured as: 1×1 convolution layer-ReLU activation function layer-1×1 convolution layer-sigmoid activation function layer. Input spatial attention into the channel attention network to obtain channel attention. Inputting the enhanced local feature enhancement map and the enhanced global feature enhancement map into the corresponding channel attention network can obtain the final enhanced feature map.
[0066] In the embodiments of the present application, a new local-global fusion parking space detection framework is designed. In terms of network design, in order to achieve better interaction and enhancement of local and global features, an interactive attention enhancement network is proposed to achieve end-to-end feature enhancement, and an interactive attention enhancement network is designed based on channel attention and spatial attention, which can improve the accuracy of parking space detection.
[0067] The enhanced feature map may include a local enhanced feature map and a global enhanced feature map.
[0068] 206, local information and global information are obtained according to the final enhanced feature map.
[0069] For example, the final enhanced feature map is passed through a local information extractor and a global information extractor to obtain local information and global information. The extractor can be used to extract feature information from two branches to obtain the required parking space information. For example, the feature maps from two branches are passed through a local information extractor and a global information extractor to obtain the final global information and local information.
[0070] The global information extractor may be composed of 6 3×3 convolutions, and the local information extractor may be composed of 5 3×3 convolutions.
[0071] Global information can be represented by a 24×24×6 tensor, which is mainly used to characterize parking spaces. For example, a picture with an image resolution of 384×384 is divided into 16×16 24×24 grids to predict the desired information in the network. The 6 in the above 24×24×6 refers to the information that needs to be predicted, which are confidence, the position of the paired corner points, whether the parking space is occupied, the number of channels occupied by 24×24×1, the number of channels occupied by 24×24×4, and the number of channels occupied by 24×24×1. The grid can also be divided in other ways, and this embodiment is just an example.
[0072] The global information can be represented by a 24×24×6 tensor. The global information can be divided into three parts: 24×24×1, 24×24×4, and 24×24×1, which represent the confidence of the parking space, the position of the paired corner points, and the parking space occupancy status, respectively. The global feature information and the true value are calculated by the L2 loss function to obtain loss1. The true value can be the real data manually labeled.
[0073] Specifically, the functions of the three parts of the tensor in the global feature information are as follows: the first part 24×24×1 is used to represent the confidence of the parking space corresponding to each cell, which is implemented using one channel; the second part 24×24×4 is used to represent the position of the paired parking space corner points, using two vectors with the corresponding cell center point as the starting point and the two entrance corner points of the parking space as the end point, which is implemented using 4 channels; the third part 24×24×1 is used to indicate whether the parking space corresponding to the cell is occupied, which is implemented using one channel.
[0074] In a specific embodiment, the global information can be used to accurately determine whether a parking space is occupied, but the precise coordinates of the corner points cannot be obtained. The reason is that when predicting the distance between paired corner points, the algorithm makes a rough prediction based on the cells contained in the parking space, which can lead to large errors. More sophisticated processing is required when predicting the position of corner points, so in addition to realizing prediction based on parking space information, this patent also realizes prediction based on corner points for each cell through another branch. In this way, more accurate parking space prediction can be achieved through global information and local information.
[0075] Another branch implements corner-based prediction for each cell, which can be determined based on local information. Local information can be represented by a 24×24×5 tensor, which is mainly used to characterize specific corner points. Local information may include the confidence, position and direction of the corner points in each grid (cell). Applying the obtained feature tensor to the original image means dividing the original image into 24×24 cells. The algorithm of this patent obtains the corresponding output result for each grid. The local information 24×24×5 tensor can be divided into three parts: 24×24×1, 24×24×2, and 24×24×2, which respectively represent the confidence, position and direction of the corner points. The local information can be calculated with the true value using the L2 loss function to obtain loss2. Here the L2 loss function is used for model training.
[0076] Specifically, the functions of the three parts of the local information tensor are as follows: the first part 24×24×1 is used to predict the confidence that there is a corner point in the cell; the second part 24×24×2 is used to predict the position of the corner point in the cell relative to the cell center, which is represented by a two-dimensional vector and implemented in the model using two channels; the third part 24×24×2 is used to predict the direction of the corresponding corner point in the cell, which is represented by a two-dimensional vector and implemented in the model using two channels.
[0077] In practice, the corner point positions obtained using local information will be more accurate than those obtained from global information. Therefore, when designing the model, local information is used to optimize the positions of parking space corner points in global information.
[0078] According to the particularity of the parking space detection task, two branches are designed. One branch is used to learn the global information based on the parking space, and the other branch is used to learn the local information based on the entrance corner points. This can realize the end-to-end training of the parking space detection model and improve the accuracy of parking space detection.
[0079] 207 , comparing the global information and the local information obtained through detection with the actual parking space information and the actual corner point information, and correcting the parameters of the network model.
[0080] As an embodiment, the error between the predicted value and the true value of the network is calculated using the L2 loss function, and the network is optimized by back propagation. The network training process can use the L2 loss function to calculate the error between the predicted value and the true value of the network, and the network is optimized by back propagation. For example, the global feature information and the true value are calculated by the L2 loss function to obtain loss1. The local information can be calculated by the L2 loss function with the true value to obtain loss2.
[0081] In one embodiment of the present application, the initial learning rate can be set to 0.1 during the training phase, and the obtained loss1 and loss2 are used to implement end-to-end training of the parking space detection model through reverse gradient propagation and the Adam algorithm. The training amount of 500 epochs can be set by default. Epoch is an important concept in the neural network training process. Generally speaking, an epoch is equal to the process of training once with all the samples in the training set. When a complete data set passes through the neural network once and returns once, that is, forward propagation and backward propagation are performed once, this process is called an epoch. In this way, designing more epochs can optimize network parameters through continuous training to achieve more accurate parking space prediction.
[0082] The predicted value can be obtained by integrating the above local information with the global information, and the true value can be obtained by manual labeling during actual training.
[0083] The above steps 201-207 are the training process of the model. The model reasoning process formally applied to the actual scenario will be described in detail below in combination with steps 208-214. The model reasoning process can be used in the testing or application process, and the model training process can be used in the training process before the model is perfected. The reasoning stage can be performed after the training stage.
[0084] 208, obtaining a picture of the parking spot.
[0085] Specifically, step 208 of the model reasoning process is the same as step 201 in the model training process and will not be described in detail here.
[0086] 209, processing the image of the parking spot to obtain a processed image.
[0087] Specifically, step 209 of the model reasoning process is the same as step 202 in the model training process and will not be described in detail here.
[0088] 210, the processed image is input into the front-end network Darknet53 to obtain a feature map.
[0089] Specifically, step 210 of the model reasoning process is the same as step 203 in the model training process, and will not be described in detail here.
[0090] 211, a local feature map and a global feature map are obtained according to the feature map.
[0091] Specifically, step 211 of the model reasoning process is the same as step 204 in the model training process and will not be described in detail here.
[0092] 212, the interactive attention enhancement network is used to interactively enhance the local feature map and the global feature map to obtain an enhanced feature map.
[0093] Specifically, step 212 of the model reasoning process is the same as step 205 in the model training process and will not be described in detail here.
[0094] 213, local information and global information are obtained according to the enhanced feature map.
[0095] Specifically, step 213 of the model reasoning process is the same as step 206 in the model training process and will not be described in detail here.
[0096] 214, the local information and the global information are integrated by non-maximum suppression (NMS) based on corner points and parking spaces to achieve the purpose of detecting parking spaces.
[0097] Specifically, first, local information and global information can be subjected to corner-based NMS to obtain corners. In this stage, if the corner point obtained based on global information (globaljunction) is near the corner point obtained based on local information (localjunction), then the corner point obtained by global information (globaljunction) will be replaced by the corner point obtained based on local information (localjunction).
[0098] Secondly, the parking spaces obtained based on the global information are merged with the corner points obtained based on the local information and the global information, thereby realizing the integration of the local information and the global information.
[0099] Further, as an embodiment, after the corner point-based NMS, the parking space-based NMS can be performed. For example, the parking spaces with corner points that have not been replaced by corner points obtained from local information are cleared. When a heavy parking space is encountered, a parking space with high confidence is selected, and the result of parking space detection is obtained by combining the corner point-based NMS and the parking space-based NMS. Specifically, the parking space-based NMS is mainly divided into two steps: 1. Delete heavy parking spaces (parking slots) where two corners obtained from global information have not been replaced by corner points obtained from local information. 2. Delete multiple heavy parking spaces where one corner obtained from global information is replaced by a corner point obtained from local information, while the other corner point is not replaced. In this case, the parking space with high confidence is retained.
[0100] NMS or non-maximum suppression is a common deduplication step for target detection. Since the neural network will detect multiple candidate targets near the same target, it is a process of screening them according to the confidence of each candidate target. Corner-based NMS can remove redundant points and replace points in global features with points in local features, so as to screen out more accurate corner point positions. The purpose of parking space-based NMS is to integrate the information of local features and global features to achieve prediction of the entire parking space. Corner-based NMS and parking space-based NMS are special processing based on NMS for parking space detection models.
[0101] Traverse each parking space-based global information, match the obtained paired parking space corner points with the corner point information in the local information, and when the corner point coordinates in the global information are near the corner point coordinates in the local information, that is, in the implementation, when the absolute value of the difference between the horizontal and vertical coordinates of the two corner points is less than 1, replace the corner point coordinates in the local information with the corner point coordinates in the global tensor, and expand the angle corresponding to the corner point. Through reasoning in large-scale data sets, not only can the parking space information be effectively detected, but also the angle corresponding to the parking space corner point can be accurately predicted.
[0102] This patent replaces all the coordinate information of the corner points in the global tensor to complete the first stage of the corner point-based NMS operation. Then, the global information based on the parking space is traversed, the parking space information with corner points that have not been replaced is deleted, and the duplicate parking space with the highest confidence is selected and retained, thereby obtaining the final parking space detection result.
[0103] Darknet53 is a convolutional neural network model that can be used for target detection and classification tasks. It is the core network model of YOLOy3, and its design idea is to improve the effect of feature extraction by stacking multiple convolutional and residual connection layers. In general, Darknet53 is a very effective convolutional neural network with high accuracy and efficiency, and is a backbone network widely used in modern computer vision tasks. This invention creatively applies Darknet53 to the parking space detection model for the first time. Compared with other deep neural networks, Darknet53 has only 53 layers. It is applied to the parking space detection model with faster training speed and higher performance. The number of parameters of Darknet53 is much smaller than that of other networks, which makes the parking space detection model faster, easier to train and optimize than other models. Darknet53 uses low-precision calculations and parallel computing technology, which makes the model's inference speed very fast and can be executed on low-power devices. The advantage of DarKknet53 is that it learns image features and can efficiently solve the classification problems of a large number of categories of objects.
[0104] In one embodiment of the present application, an interactive attention enhancement network is introduced into the method flow of parking space detection, and local features and global features are interactively enhanced and fused to achieve parking space detection, thereby optimizing model parameters and improving the accuracy of parking space detection.
[0105] Figure 3 This is a block diagram of a device for detecting parking spaces provided by an embodiment of the present application.
[0106] The parking space detection device 30 includes a memory 31, a processor 32, and a control program stored in the memory and executable on the processor. When the processor executes the control program, the parking space detection method is implemented. The processor can implement the following Figure 1 and Figure 2 The processor 32 is used to support the device to execute the above-mentioned embodiment. Figure 1 and Figure 2The operations or functions performed by the steps in the method flow shown. The device 30 for detecting parking spaces may also include a communication interface 33 and a bus 34. The processor, the communication interface and the memory may be interconnected or connected to each other through a bus 940. Among them, the communication interface is used to support the device to communicate. The memory is used to store program codes and data of the device. The processor calls the code or data stored in the memory to implement corresponding operations. The memory may be coupled to the processor or not. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which may be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules.
[0107] In one embodiment of the present application, parking space detection is achieved by introducing an interactive attention enhancement network into parking space detection, and interactively enhancing and fusing local features and global features. This can optimize model parameters and improve the accuracy of parking space detection.
[0108] Figure 4 40 may include a device 30 for detecting a parking space. The device 30 for detecting a parking space includes a memory 31, a processor 32, and a control program stored in the memory and executable on the processor. When the processor executes the control program, a method for detecting a parking space is implemented. The processor may implement the following: Figure 1 and Figure 2 The method for detecting a parking space in the present invention. The device 30 for detecting a parking space can be similar to the aforementioned Figure 3 The description in will not be repeated in detail here.
[0109] In one embodiment of the present application, parking space detection is achieved by introducing an interactive attention enhancement network into parking space detection, and interactively enhancing and fusing local features and global features. This can optimize model parameters and improve the accuracy of parking space detection.
[0110] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.
[0111] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a parking space on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0112] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0113] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.
[0114] Various aspects of the present invention are described herein with reference to the flow charts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each box of the flow chart and / or block diagram and the combination of each box in the flow chart and / or block diagram can be implemented by computer-readable program instructions.
[0115] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0116] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0117] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.
[0118] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.
Claims
1. A method for detecting a parking space, characterized in that: include: Obtain a local feature map and a global feature map of the image of the parking spot; Inputting the local feature map and the global feature map into an interactive attention enhancement network to perform interactive enhancement to obtain an enhanced local feature map and an enhanced global feature map; as well as The enhanced local feature map and the enhanced global feature map are integrated to output a parking space detection result.
2. The method according to claim 1, characterized in that: The step of integrating the enhanced local feature map and the enhanced global feature map to output a parking space detection result includes: Extracting information from the enhanced local feature map and the enhanced global feature map respectively to obtain local information and global information; and The local information and the global information are integrated based on non-maximum suppression (NMS), and the parking space detection result is output.
3. The method according to claim 2, characterized in that The integrating the local information and the global information based on non-maximum suppression (NMS) and outputting the parking space detection result comprises: Based on the NMS, obtaining corner point information according to the local information and the global information; Based on the NMS, obtaining parking space information according to the global information; and The parking space information is merged with the corner point information to output the parking space detection result.
4. The method according to claim 1, characterized in that: The step of inputting the local feature map and the global feature map into an interactive attention enhancement network and performing interactive enhancement to obtain an enhanced local feature map and an enhanced global feature map comprises: Performing element-by-element multiplication on the local feature map and the global feature map to obtain an interaction feature map; and The enhanced local feature map and the enhanced global feature map are obtained according to the interactive feature map.
5. The method according to claim 4, characterized in that The interactive feature map is obtained according to the following method: M=F g ⊙F l , where F g represents the global feature map, F l represents the local feature map, ⊙ represents the element-by-element multiplication of the matrix, and M is the interaction feature map.
6. The method according to claim 4 or 5, characterized in that: The step of obtaining the enhanced local feature map and the enhanced global feature map according to the interactive feature map comprises: F g and F l The enhanced local feature enhancement map and the enhanced global feature enhancement map are obtained by assigning values in the following manner: Among them, softmax represents the activation function, conv is a 1×1 convolution operation, Represents matrix multiplication.
7. The method according to any one of claims 1 to 5, characterized in that The step of integrating the enhanced local feature map and the enhanced global feature map to output a parking space detection result includes: Inputting the enhanced local feature enhancement map and the enhanced global feature enhancement map into the corresponding channel attention network to obtain a final enhanced feature map, wherein the channel attention network is configured as follows: convolution layer-activation function layer-convolution layer-activation function layer; Passing the final enhanced feature map through an information extractor to obtain local information and global information, wherein the local information includes information for characterizing corner points, and the global information includes information for characterizing parking spaces; and The parking space detection result is output according to the local information and the global information.
8. The method according to claim 7, characterized in that The global information includes the confidence of the parking space, the position of the paired corner point and the occupancy status of the parking space; and / or the local information includes the confidence of the corner point, the position of the corner point and the direction of the corner point.
9. The method according to claim 7 or 8, characterized in that: The method further comprises: The detected global information, the local information, the actual parking space information, and the actual corner point information are compared to correct the network parameters.
10. The method according to any one of claims 1 to 5, characterized in that The local feature map and global feature map of the parking spot image are obtained by: Inputting the image of the parking spot into the front-end network to obtain a feature map; The feature map is input into a backend network to obtain the local feature map and the global feature map.
11. The method according to claim 10, characterized in that The inputting the picture of the parking spot into the front-end network to obtain the feature map includes: performing image resolution processing on the picture of the parking spot; The image processed with the image resolution is input into the front-end network for encoding to obtain the feature map.
12. The method according to any one of claims 1, characterized in that Inputting the feature map into the backend network to obtain a local feature map and a global feature map comprises: The feature map is input into a dual-branch back-end network for decoding to obtain the local feature map and the global feature map.
13. The method according to any one of claims 1-10, wherein the network used in the method is Darknet53.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store instructions, and the instructions are loaded by a processor to execute the method according to any one of claims 1 to 13.
15. A device for detecting a parking space, characterized in that: The method comprises a memory, a processor and a control program stored in the memory and executable on the processor. When the processor executes the control program, the method for detecting a parking space according to any one of claims 1 to 13 is implemented.
16. A vehicle, characterized in that: The device comprises the device for detecting a parking space as claimed in claim 15.
Citation Information
Patent Citations
Compressed image super-division method and image compression method and system
CN115358929A
Cited By
Information fusion parking space detection method and system based on deep learning
CN119942496A
