Lightweight vacant parking space recognition algorithm based on GhostNet

Abstract

The present invention discloses a lightweight empty parking space recognition algorithm based on GhostNet, which relates to the technical field of target detection and recognition. Based on an existing parking space data set, a field-collected picture is added to the existing parking space data set to construct an empty parking space detection data set, and the data set is annotated. The annotation types are mainly divided into two types of empty parking spaces and parking space corner points. Ghost Net is used as a backbone feature extraction network. Ghost Net is used to output effective feature layers of two scales, an enhanced feature extraction network is constructed, and an SPPF and an attention module are embedded. A decoupling head is used to perform regression prediction on the effective feature layer, thereby generating a model. After the lightweight empty parking space recognition model based on GhostNet is constructed, the empty parking space is first detected, and the next step of path planning is performed according to the generated parking space corner points, and the type of the parking space is judged according to the distance between the parking space corner points.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection and recognition, and in particular to a lightweight vacant parking space recognition algorithm based on GhostNet. Background Art

[0002] With the steady growth of the national economy and the continuous development of the automobile industry, the number of cars in my country has increased sharply. As of September 2021, the number of small cars in my country has reached 390 million; the vigorous development of artificial intelligence technology has also led to the rapid rise of research on driverless technology in the automotive industry. Among them, the automatic parking assistance system, as an important part of driverless driving, has also become a hot research topic. Parking space recognition, as an important perception link in the automatic parking system, is crucial to the subsequent vehicle parking path planning and control due to its accuracy and speed; the existing vacant parking space recognition technology can be divided into parking space detection and positioning methods based on user interface, infrastructure, free space and parking space marking lines. With the development of machine vision, the method based on parking space marking lines has become a research hotspot, Jung [1] et al. applied peak detection and clustering in Hough space to detect parking spaces, and identified separated parking space marking lines through T-shaped template matching. The literature used Line Segment Detector to detect and identify parking space lines. These low-level visual detection methods based on straight lines and corners are difficult to meet real-time requirements, and their generalization and robustness are relatively low. For this reason, a lightweight empty parking space recognition algorithm based on GhostNet is proposed. Based on the low computing resource cost and high real-time requirements required by the automatic parking system, a GhostNet lightweight target detection network with Ghost Module as the core is proposed to provide conditions for the deployment and application of the model on embedded vehicle-mounted mobile terminals. Summary of the invention

[0003] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a lightweight empty parking space recognition algorithm based on GhostNet.

[0004] The purpose of the present invention can be achieved by the following technical solution: a lightweight empty parking space recognition algorithm based on GhostNet, the method comprises the following steps:

[0005] Step 1: Based on the existing parking space dataset, add field-collected images to the existing parking space dataset to construct an empty parking space detection dataset, which is annotated through labeling. The annotation types are mainly divided into two types: empty parking spaces and parking space corner points;

[0006] It needs to be further explained that, in the specific implementation process, labeling is a data annotation tool that can play the role of marking data.

[0007] Step 2: Build a GhostNet lightweight network and use GhostNet as the backbone feature extraction network. It should be further explained that in the specific implementation process, the benefit of such design and construction is to reduce the parameters and calculation amount of the model and improve the running speed of the model.

[0008] Step 3: Use the effective feature layers of 13×13 and 26×26 scales output by GhostNet to build an enhanced feature extraction network and embed SPPF and attention modules; it should be further explained that in the specific implementation process, this design is used to improve detection accuracy;

[0009] Step 4: Use Decoupled Head to perform regression prediction on the effective feature layer to generate a lightweight empty parking space recognition model based on GhostNet. It should be further explained that in the specific implementation process, the benefit of such a design is to improve accuracy.

[0010] Step 5: After the construction of the lightweight empty parking space recognition model based on GhostNet is completed, the empty parking space is first detected, and the detected parking space corner points in the empty parking space are connected as a pair of parking space corner points. The next step of path planning is performed based on the generated parking space corner points, and the type of parking space is determined based on the distance between the parking space corner points; it should be noted that, first, when the device detects an empty parking space, the empty parking space area will be set as the ROI region of interest, and the pixel coordinates of the parking space corners detected in the empty parking space will be output, and the next step of pixel coordinate to world coordinate conversion will be performed for the subsequent automatic parking path planning needs.

[0011] Furthermore, the process of constructing the empty parking space detection dataset is as follows: collect images to establish a dataset, mainly empty parking space pictures and parking space angle pictures. Use the open source software labelImg to standardize the pictures and divide them into two categories: parking (empty parking space) and hore (parking space angle). It should be further explained that there are 4896 pictures in total.

[0012] Furthermore, the backbone feature extraction network is as follows: Figure 2As shown in the figure, assuming that the input image size is (a, a, z), first use Ghost Module1 with a step size of 2 for downsampling to obtain higher semantic information, and the output image size is (a / 2, a / 2, 2z). Then a residual edge is introduced, which is directly connected to the end after being weighted by the attention mechanism senet. The trunk part will pass through Ghost Module2 with a step size of 1 for further feature extraction, and the result will be added to the residual edge introduced before. The final output feature map size is (a / 2, a / 2, 2z). At this point, a residual structure GhostResidual based on Ghost Module is constructed, which improves the accuracy while alleviating the gradient vanishing problem caused by increasing the depth in deep neural networks.

[0013] The final Ghostnet backbone network structure built with the Ghost Residual structure is shown in Table 1:

[0014]

[0015] Table 1

[0016] Furthermore, the construction process of the enhanced feature extraction network is to use the effective feature layers of 26×26 and 13×13 scales output by Ghost Net to construct an enhanced feature extraction network. First, the effective features of 13×13 are passed through the SPPF structure composed of maximum pooling, and the size of the pooling kernel is 5×5. The SPPF structure can greatly increase the receptive field and separate the most significant contextual features; the 13×13 feature layer is upsampled and overlapped with the 26×26 feature layer, and then downsampled by a maximum pooling with a step size of 2. During this period, the swish activation function is added. By repeatedly extracting features, the purpose of enhancing feature extraction is achieved. At the same time, the SENet attention module is added to allow the network to focus on the channels that need the most attention and suppress less important feature information.

[0017] The receptive field is the size of the area where the pixels on the feature map output by each layer of the convolutional neural network are mapped on the input image.

[0018] Furthermore, the Decoupled Head is used to strengthen the output of the feature extraction network. The two effective feature layers are input into the Decoupled Head for regression and prediction. For each feature layer, different convolutional layers are used to obtain three prediction results with channels of #anchor×4, #anchor×1, and #anchor×20. #anchor×4 is used to adjust the size and position of the prediction box, #anchor×1 is used to determine whether an object is included, and #anchor×20 is used for classification to determine the type of the object included. The three prediction results are stacked, and the final result is:

[0019] (H,W,#anchor×4+#anchor×1+#anchor×C).

[0020] Beneficial effects of the present invention:

[0021] During use, the present invention is based on an existing parking space data set, adds field-collected pictures to the existing parking space data set, constructs an empty parking space detection data set, and annotates it, and the annotation types are mainly divided into two types of empty parking spaces and parking space corner points; uses Ghost Net as a backbone feature extraction network; uses Ghost Net to output effective feature layers of two scales, constructs an enhanced feature extraction network, and embeds SPPF and attention modules; uses a decoupling head to perform regression prediction on the effective feature layer to generate a model; after the lightweight empty parking space recognition model based on GhostNet is built, first detect the empty parking space, perform the next step of path planning according to the generated parking space corner points, and judge the type of the parking space according to the distance between the parking space corner points; this not only improves the detection speed, but also reduces the size of the model, and can better realize deployment on an embedded development board with insufficient computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a flow chart of the present invention;

[0024] Figure 2 It is a schematic diagram of the Ghost Residual structure of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] like Figure 1-2 As shown, a lightweight empty parking space recognition algorithm based on GhostNet includes the following steps:

[0027] Step 1: Based on the existing parking space dataset, add field-collected images to the existing parking space dataset to construct an empty parking space detection dataset, which is annotated through labeling. The annotation types are mainly divided into two types: empty parking spaces and parking space corner points;

[0028] It needs to be further explained that, in the specific implementation process, labeling is a data annotation tool that can play the role of marking data.

[0029] Step 2: Build a GhostNet lightweight network and use GhostNet as the backbone feature extraction network. It should be further explained that in the specific implementation process, the benefit of such design and construction is to reduce the parameters and calculation amount of the model and improve the running speed of the model.

[0030] Step 3: Use the effective feature layers of 13×13 and 26×26 scales output by GhostNet to build an enhanced feature extraction network and embed SPPF and attention modules; it should be further explained that in the specific implementation process, this design is used to improve detection accuracy;

[0031] Step 4: Use Decoupled Head to perform regression prediction on the effective feature layer to generate a lightweight empty parking space recognition model based on GhostNet. It should be further explained that in the specific implementation process, the benefit of such a design is to improve accuracy.

[0032] Step 5: After the construction of the lightweight empty parking space recognition model based on GhostNet is completed, the empty parking space is first detected, and the detected parking space corner points in the empty parking space are connected as a pair of parking space corner points. The next step of path planning is performed based on the generated parking space corner points, and the type of parking space is determined based on the distance between the parking space corner points; it should be noted that, first, when the device detects an empty parking space, the empty parking space area will be set as the ROI region of interest, and the pixel coordinates of the parking space corners detected in the empty parking space will be output, and the next step of pixel coordinate to world coordinate conversion will be performed for the subsequent automatic parking path planning needs.

[0033] It should be further explained that in the specific implementation process, the construction process of the empty parking space detection dataset is as follows: collect images to establish a dataset, mainly empty parking space pictures and parking space angle pictures. Use the open source software labelImg to standardize the pictures and divide them into two categories: parking (empty parking space) and hore (parking space angle). It should be further explained that there are 4896 pictures in total.

[0034] It should be further explained that, in the specific implementation process, the backbone feature extraction network is as follows: Figure 2 As shown in the figure, assuming that the input image size is (a, a, z), first use Ghost Module1 with a step size of 2 for downsampling to obtain higher semantic information, and the output image size is (a / 2, a / 2, 2z). Then a residual edge is introduced, which is directly connected to the end after being weighted by the attention mechanism senet. The trunk part will pass through Ghost Module2 with a step size of 1 for further feature extraction, and the result will be added to the residual edge introduced before. The final output feature map size is (a / 2, a / 2, 2z). At this point, a residual structure Ghost Residual based on Ghost Module is constructed, which improves the accuracy while alleviating the gradient vanishing problem caused by increasing the depth in deep neural networks.

[0035] The final Ghostnet backbone network structure built with the Ghost Residual structure is shown in Table 1:

[0036]

[0037] Table 1

[0038] It needs to be further explained that, in the specific implementation process, the construction process of the enhanced feature extraction network is to use the effective feature layers of 26×26 and 13×13 scales output by Ghost Net to construct an enhanced feature extraction network. First, the effective features of 13×13 are passed through the SPPF structure composed of maximum pooling, and the size of the pooling kernel is 5×5. The SPPF structure can greatly increase the receptive field and separate the most significant contextual features; the 13×13 feature layer is upsampled and overlapped with the 26×26 feature layer, and then downsampled by a maximum pooling with a step size of 2. The swish activation function is added during this period. By repeatedly extracting features, the purpose of enhancing feature extraction is achieved. At the same time, the SENet attention module is added to allow the network to focus on the channels that need the most attention and suppress less important feature information.

[0039] The receptive field is the size of the area where the pixels on the feature map output by each layer of the convolutional neural network are mapped on the input image.

[0040] It should be further explained that in the specific implementation process, the two effective feature layers output by the Decoupled Head are input into the Decoupled Head for regression and prediction. For each feature layer, different convolutional layers are used to obtain three prediction results with channels of #anchor×4, #anchor×1, and #anchor×20. #anchor×4 is used to adjust the size and position of the prediction box, #anchor×1 is used to determine whether an object is included, and #anchor×20 is used for classification to determine the type of object included. The three prediction results are stacked, and the final result is:

[0041] (H,W,#anchor×4+#anchor×1+#anchor×C).

[0042] It needs to be further explained that, in the specific implementation process, the GhostNet is composed of CBL (DarknetConv2D+BatchNormalization+LeakyReLU) and Ghost Residual. Ghost Residual is the residual module composed of ghost modules mentioned above. When a picture is input, it first passes through two CBR modules for downsampling, and then passes through three Ghost Residual modules. It increases the network depth while alleviating the gradient disappearance problem.

[0043] Finally, GhostNet outputs two effective feature layers of 26×26 and 13×13. These two effective feature layers pass through the senet attention module, so that the channels are weighted according to the weight coefficients to suppress unimportant channel information. Then the effective feature layer of 13×13 scale is input into the SPPF structure. The SPPF structure is composed of maximum pooling, and the size of the pooling kernel is 5×5. The SPPF structure can greatly increase the receptive field and separate the most significant contextual features. The input feature layer will first undergo a maximum pooling, and the output results will be subjected to maximum pooling in turn, repeated twice, and the results of each time will be stacked and output. Then the output results are input into the enhanced feature extraction network together with the effective feature layer of 26×26 scale.

[0044] In the enhanced feature extraction network, the 13×13 feature layer is first upsampled once and then fused with the 26×26 feature layer. Senet is also added in the upsampling process. The 26×26 feature layer is downsampled after a convolution through a maximum pooling with a step size of 2 and a pooling kernel of 2×2. The Senet attention mechanism is added in agreement, and the result is fused with the 13×13 feature layer. In this way, features are repeatedly extracted, which increases feature reuse and achieves the purpose of enhanced feature extraction.

[0045] The final result is input into the Decoupled Head for regression and prediction. After the model is built, the training parameters are as follows: Batchsize is set to 8, learning rate is 0.001, training is performed for 100 epochs, and training is terminated automatically when the loss no longer changes.

[0046] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A lightweight empty parking space recognition method based on GhostNet, characterized in that: The method comprises the following steps: Step 1: Based on the existing parking space dataset, add field-collected images to the existing parking space dataset to construct an empty parking space detection dataset, which is annotated through labeling. The annotation types are divided into two categories: empty parking spaces and parking space corner points; Step 2: Build a GhostNet lightweight network and use GhostNet as the backbone feature extraction network; Step 3: Use GhostNet output , The effective feature layers of two scales build an enhanced feature extraction network and embed the SPPF and attention modules; The construction process of the enhanced feature extraction network is to use the output of GhostNet as well as The effective feature layers of these two scales build a strengthened feature extraction network. First, The effective features of are passed through the SPPF structure composed of maximum pooling, and the size of the pooling kernel is ; The SPPF structure can greatly increase the receptive field and separate the most significant contextual features. The feature layer is upsampled and The feature layer is fused, and then down-sampled by a maximum pooling with a step size of 2. The swish activation function is added during this period. The features are extracted repeatedly and the SENet attention module is added at the same time. Step 4: Use Decoupled Head to perform regression prediction on the effective feature layer to generate a lightweight empty parking space recognition model based on GhostNet; Step 5: After the lightweight empty parking space recognition model based on GhostNet is built, the empty parking spaces are detected first, and the detected parking space corner points in the empty parking spaces are connected as a pair of parking space corner points. The next step of path planning is performed based on the generated parking space corner points, and the type of parking space is determined based on the distance between the parking space corner points.

2. The lightweight vacant parking space recognition method based on GhostNet according to claim 1 is characterized in that: The process of constructing the empty parking space detection dataset is as follows: collect images to establish a dataset, which are empty parking space pictures and parking space corner pictures. Use the open source software labelImg to standardize the pictures and divide them into two categories: parking and parking.

3. The lightweight vacant parking space recognition method based on GhostNet according to claim 1 is characterized in that: The construction process of the backbone feature extraction network is as follows: the input image size is set to a, a, z, and GhostModule1 with a step size of 2 is first used for downsampling to obtain semantic information. The output image size is a / 2, a / 2, 2z, and then a residual edge is introduced. After being weighted by the attention mechanism senet, it is directly connected to the end. The backbone part will be further extracted by Ghost Module2 with a step size of 1, and the extraction result will be added with the residual edge. The final output feature map size is a / 2, a / 2, 2z, and a residual structure Ghost Residual based on Ghost Module is constructed.

4. The lightweight vacant parking space recognition method based on GhostNet according to claim 1 is characterized in that: The receptive field is the size of the area where the pixels on the feature map output by each layer of the convolutional neural network are mapped on the input image.

5. The lightweight vacant parking space recognition method based on GhostNet according to claim 1 is characterized in that: The Decoupled Head is used to strengthen the output of the feature extraction network. The two effective feature layers are input into the Decoupled Head for regression and prediction. For each feature layer, different convolutional layers are used to obtain the channels. , , The three prediction results are: Used to adjust the size and position of the prediction box. Used to determine whether an object is contained. Used for classification, determine the type of objects contained, stack these three prediction results, the final result is: 。

Images