Method for identifying electric vehicle in elevator based on improved YOLOv8

Abstract

The invention discloses a method for recognizing an electric vehicle in an elevator based on improved YOLOv8. The method comprises the following steps that an electric vehicle elevator entering image is collected; an electric vehicle recognition model is constructed, wherein the electric vehicle recognition model is composed of a feature extraction backbone network module, an aggregation neck network module, a multi-dimensional cooperative attention module and a candidate frame prediction module; a feature extraction backbone network module is adopted to extract feature maps of different resolutions of the collected images; carrying out fusion through an aggregation neck network module, inputting a fused feature map into a multi-dimensional cooperative attention module, and generating a multi-dimensional weighted feature map; inputting the multi-dimensional weighted feature map into a candidate box prediction module to generate final prediction output; by designing the local aggregation module and introducing the local aggregation module into the aggregation neck network module, the electric vehicle identification model can be better expressed in a complex scene through hierarchical feature aggregation, and the capability of identifying small electric vehicles and electric vehicle edge details in an elevator is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for identifying electric vehicles in elevators based on improved YOLOv8. Background Art

[0002] With the continuous advancement of urbanization, more and more residential buildings in communities are equipped with elevators. At the same time, the popularity of electric vehicles has also brought convenience, but safety accidents caused by electric vehicles entering elevators are frequent. After riding an electric vehicle, the battery and motor temperature are high, and the space inside the elevator is small and the air circulation is poor. When an electric vehicle enters an elevator, the vehicle is prone to collision with the elevator door or side wall, and this collision is likely to cause the electric vehicle to spontaneously combust or even explode. Therefore, in order to prevent electric vehicles from entering the elevator and causing safety accidents, it is necessary to identify electric vehicles in the elevator and take preventive measures in advance to ensure the safety of residents' travel.

[0003] To address the problem of electric vehicles entering elevators, the current traditional identification methods include physical vehicle blocking, weight sensor identification, and visual identification. Physical vehicle blocking is to manually set up a railing at the entrance of the elevator to block electric vehicles. However, this method not only restricts the entry of electric vehicles, but also blocks other means of transportation such as bicycles and strollers, which is inconvenient and too costly; weight sensors identify electric vehicles by measuring the weight in the elevator, but when there are many passengers, this method may not be able to accurately determine whether an electric vehicle has entered. In contrast, the visual recognition method uses computer vision technology and deep learning methods to capture images of electric vehicles through cameras and identify the characteristics of electric vehicles through image processing algorithms. This method has high accuracy and fast response speed, has great development potential, and will become the main means to solve the problem of electric vehicle identification in the future.

[0004] Among the current deep learning electric vehicle recognition methods, there are mainly CNN-based detection algorithms and Transformer-based models. However, the Transformer-based model cannot achieve real-time detection and the computing resources are too large. Among the CNN-based detection algorithms, Faster R-CNN and SSD algorithms have low recognition efficiency in complex occluded environments. Therefore, the CNN-based YOLO framework is the best choice for real-time electric vehicle detection. YOLO is a real-time detection framework widely used in computer vision target detection tasks, which can complete target recognition in a single reasoning process. Compared with other detection frameworks, it has a faster recognition speed and is suitable for a variety of scenarios including images, videos and real-time streaming. As the most practical version of the series, YOLOv8 further improves the detection speed and accuracy and provides a unified framework for model training. However, even so, the YOLOv8 framework has its own limitations or shortcomings when identifying electric vehicles. First, in terms of small object detection, the detection accuracy of YOLOv8 for smaller electric vehicles is insufficient. Secondly, the appearance of electric vehicles changes greatly under different angles, partial occlusion or extreme lighting conditions, which makes the traditional YOLOv8 framework difficult to recognize in these situations. In addition, if the amount of training data is insufficient or the data is too single, the model may also have overfitting problems. Therefore, in order to solve these problems in the electric vehicle entry recognition scenario, it is necessary to improve on the basis of the traditional YOLOv8 framework to further improve the robustness and efficiency of electric vehicle recognition, so as to better cope with the challenges in this scenario. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method for identifying electric vehicles in elevators based on improved YOLOv8, which aims to solve the problems mentioned in the background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a method for identifying electric vehicles in elevators based on improved YOLOv8, comprising the following steps: Step S1: constructing an electric vehicle entering an elevator image dataset, wherein the electric vehicle entering an elevator image dataset includes a number of electric vehicle entering an elevator images; Step S2: constructing an electric vehicle recognition model, wherein the electric vehicle recognition model is composed of a feature extraction backbone network module, an aggregation neck network module, a multi-dimensional collaborative attention module and a candidate box prediction module, and the feature extraction backbone network module, the aggregation neck network module, the multi-dimensional collaborative attention module and the candidate box prediction module are arranged in series; Step S3: inputting the electric vehicle entering the elevator image in the electric vehicle entering the elevator image data set into the feature extraction backbone network module for feature extraction, and obtaining a high-resolution electric vehicle entering the elevator feature map, a medium-resolution electric vehicle entering the elevator feature map, and a low-resolution electric vehicle entering the elevator feature map respectively; Step S4: inputting the high-resolution electric vehicle entering the elevator feature map, the medium-resolution electric vehicle entering the elevator feature map and the low-resolution electric vehicle entering the elevator feature map into the aggregation neck network module through parallel channels for fusion, thereby obtaining a high-resolution electric vehicle entering the elevator aggregation feature map, a medium-resolution electric vehicle entering the elevator aggregation feature map and a low-resolution electric vehicle entering the elevator aggregation feature map respectively; Step S5: inputting the high-resolution electric vehicle entry aggregate feature map, the medium-resolution electric vehicle entry aggregate feature map and the low-resolution electric vehicle entry aggregate feature map into the multidimensional collaborative attention module, extracting dimensional feature maps in three dimensions of channel, space and position from the high-resolution electric vehicle entry aggregate feature map, the medium-resolution electric vehicle entry aggregate feature map and the low-resolution electric vehicle entry aggregate feature map according to different resolutions, aggregating the dimensional feature maps through average pooling and standard deviation pooling, calculating the weights of the aggregated dimensional feature maps through excitation transformation, and generating a multidimensional weighted feature map; Step S6: Input the multi-dimensional weighted feature map into the candidate box prediction module, perform target prediction on the multi-dimensional weighted feature map through bounding box regression, target classification and confidence prediction, post-process the prediction results, and generate the final prediction output.

[0007] Further, the aggregation neck network module is composed of a first local aggregation module, a second local aggregation module, a third local aggregation module, a fourth local aggregation module, a first group exchange convolution block, a second group exchange convolution block and a third group exchange convolution block; The processing flow of the aggregation neck network module is as follows: upsampling the low-resolution electric vehicle entry feature map and splicing it with the medium-resolution electric vehicle entry feature map to obtain a medium- and low-scale fusion feature map, inputting the medium- and low-scale fusion feature map into the first local aggregation module to obtain a medium- and low-scale aggregation feature map, inputting the medium- and low-scale aggregation feature map into the first group exchange convolution block to obtain a medium- and low-scale convolution feature map, splicing the medium- and low-scale convolution feature map with the high-resolution electric vehicle entry feature map to obtain a multi-scale aggregation feature map, inputting the multi-scale aggregation feature map into the second local aggregation module to obtain a high-resolution electric vehicle entry aggregation feature map, and inputting the high-resolution electric vehicle entry feature map into the second local aggregation module. The electric vehicle entering the elevator aggregate feature map is input into the second group exchange convolution block to obtain a multi-scale convolution feature map, the multi-scale convolution feature map is spliced ​​with the medium and low-scale aggregate feature map to obtain a multi-scale spliced ​​feature map, the multi-scale spliced ​​feature map is input into the third local aggregation module to obtain a medium-resolution electric vehicle entering the elevator aggregate feature map, the medium-resolution electric vehicle entering the elevator aggregate feature map is input into the third group exchange convolution block to obtain a multi-layer convolution feature map, the multi-layer convolution feature map is spliced ​​with the low-resolution electric vehicle entering the elevator feature map to obtain a multi-layer spliced ​​feature map, the multi-layer spliced ​​feature map is input into the fourth local aggregation module to obtain a low-resolution electric vehicle entering the elevator aggregate feature map; Among them, the upsampling operation adopts a bilinear interpolation method.

[0008] Furthermore, the feature extraction backbone network module is composed of a first convolutional layer, a second convolutional layer, a first feature fusion splicing layer, a third convolutional layer, a second feature fusion splicing layer, a fourth convolutional layer, a third feature fusion splicing layer, a fifth convolutional layer, a fourth feature fusion splicing layer and a pooling layer, which are connected in sequence; wherein, the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer have the same structure; the first feature fusion splicing layer, the second feature fusion splicing layer and the third feature fusion splicing layer have the same structure.

[0009] Furthermore, the processing flow of the feature extraction backbone network module is as follows: the input image, i.e., the electric vehicle entering the elevator image, sequentially passes through the first convolution layer, the second convolution layer, the first feature fusion splicing layer, the third convolution layer, the second feature fusion splicing layer, the fourth convolution layer, the third feature fusion splicing layer, the fifth convolution layer, the fourth feature fusion splicing layer and the pooling layer, wherein the output of the second feature fusion splicing layer is the high-resolution electric vehicle entering the elevator feature map X b The output of the third feature fusion splicing layer is the medium-resolution electric vehicle entry feature map X m The output of the pooling layer is the low-resolution electric vehicle entry feature map X s .

[0010] Furthermore, the first local aggregation module, the second local aggregation module, the third local aggregation module and the fourth local aggregation module are all of the same structure, and are formed by combining the VoVNet module and the CSPNet module; The processing flow of the first local aggregation module is: VoVNet module: First, multiple convolution operations are performed on the input, i.e., the medium and low scale fusion feature maps, to obtain a multi-convolution feature map X(X 1 conv ,X 2 conv ), where X 1 conv is the feature map after the first ordinary convolution operation, X 2 conv Is X 1 conv The feature map after the second ordinary convolution operation; then, X 2 conv After the depth-separable convolution operation, the depth convolution feature map X is obtained 3 conv ; Finally, the multi-convolution feature map X and the deep convolution feature map X 3 conv Perform concatenation to get the output X of the VoVNet module 4 conv ; CSPNet module: First, the output X of the VoVNet module is split using a split operation. 4 conv Divided into Part I X part1 and Part II X part2 , then, for the second part X part2 Perform convolution operation to obtain the cross-stage convolution feature map X part2_conv , and then the first part X part1 and the cross-stage convolutional feature map X part2_conv Perform a splicing operation to obtain the output of the first local aggregation module.

[0011] Furthermore, the processing flow of the first convolutional layer is: the input image, i.e., the image of the electric vehicle entering the elevator, is represented as , Indicates the number of channels of the input image, and Represents the input image The height and width of represents a set of real numbers; the processing process of the first convolutional layer is expressed as: ; In the formula, It is the feature map output after the first convolutional layer; is the activation function SiLU; represents the convolution operation; and are the weights and biases of the first convolutional layer; Indicates the step size of each slide during the convolution operation.

[0012] Furthermore, the processing flow of the first feature fusion concatenation layer is as follows: the output of the second convolutional layer After inputting the first feature fusion concatenation layer, it is divided into the direct transfer part And the convolution processing part , expressed as: ; In the formula, Indicates the direct delivery part; Represents the convolution processing part; , and Respectively and The number of channels; represents the first feature fusion splicing layer; Convolution processing part Processing is performed, expressed as: ; In the formula, , They represent the first feature fusion concatenation layer. The weights and biases of the convolutional layers; After processing and Perform splicing to obtain splicing features , and finally the splicing features After passing through the final convolution layer of the first feature fusion splicing layer, the output of the first feature fusion splicing layer is obtained , expressed as: ; ; In the formula, Represents a splicing operation; and Represents the weights and biases of the final convolutional layer of the first feature fusion concatenation layer.

[0013] Furthermore, the processing flow of the pooling layer is as follows: the output of the fourth feature fusion splicing layer is input into the pooling layer, first through a convolutional layer with a step size of 1, and the activation feature map is obtained. , and then activate the feature map Perform the maximum pooling operation to obtain the pooling feature map , expressed as: ; In the formula, Represents the feature map generated from pooling kernels k of different sizes in the pooling layer, that is, the pooling feature map; Represents the activation feature map Perform multi-scale maximum pooling operations; Indicates the size of the pooling kernel; Represents the activation feature map The number of fills; is the specific value of the filling quantity; The feature map will be activated With pooling feature map The concatenation is performed in the channel dimension, and the multi-scale information is integrated through convolution operation, while the number of channels is compressed at the same time to obtain the output of the pooling layer, that is, the low-resolution feature map X s .

[0014] Furthermore, the first group exchange convolution block, the second group exchange convolution block, and the third group exchange convolution block have the same structure; The workflow of the first group exchange convolution block is as follows: First, the input, i.e., the output of the first local aggregation module, is first subjected to a standard convolution operation through a convolution layer to obtain the group convolution feature map X csp1 , then, the group convolution X csp1Then perform a convolution operation through a 1×1 convolution layer to generate additional ghost features and obtain the ghost feature map X ghost Then, the channel order of the ghost feature is disrupted by the channel shuffle operation to obtain the group convolution ghost feature map , which is the output of the first group exchange convolution block, is expressed as: ; In the formula, X ghost_s It is the group convolution ghost feature map obtained after the ghost feature is shuffled through the channels; Represents a channel shuffle operation.

[0015] Furthermore, the specific process of step S5 is: aggregate the output of the neck network module That is, the high-resolution electric vehicle entry aggregate feature map, the medium-resolution electric vehicle entry aggregate feature map and the low-resolution electric vehicle entry aggregate feature map are input into the multi-dimensional collaborative attention module, and the attention weights are calculated and collaboratively fused through channel dimension attention feature modeling, space dimension attention feature modeling and position dimension attention feature modeling respectively; The specific process of channel dimension attention feature modeling is as follows: First, the input is the output of the aggregation neck network module Perform global average pooling in the spatial dimension to obtain the channel weight vector f c ∈R c , expressed as: ; In the formula, Represents the average of all pixel values ​​on channel c; Respectively The row and column indexes in the range and ; By double summation Traverse all spatial positions on channel c, that is, all pixel points, accumulate the pixel values, and get X p (c,u,v),X p (c,u,v) represents input The pixel value at position (u,v) on channel c; For the channel weight vector f c Perform a linear transformation to obtain the weight W of each channel c , then W c Application Each channel output of , get the output of channel dimension attention feature modeling , expressed as: ; In the formula, is the activation function Sigmoid; W c and b c Represent the weight parameters and bias parameters of each channel respectively; The specific process of modeling spatial dimension attention features is as follows: Aggregate using average pooling and maximum pooling in the channel dimension to obtain a two-dimensional spatial feature map , expressed as: ; In the formula, Represents the input in the channel dimension Perform average pooling; Represents the input in the channel dimension Perform maximum pooling; Represents input The pixel value on all channels at position (u,v); Indicates all channels; The two-dimensional spatial feature map After learning the spatial weights through a 7×7 convolutional layer, the input Multiply them together to get the output of spatial dimension attention feature modeling , expressed as: ; In the formula, Indicates the use of a 7×7 convolutional layer for the two-dimensional spatial feature map Perform convolution operation; The specific process of modeling the position dimension attention feature is as follows: Mapping to three different spaces generates vectors, expressed as: ; In the formula, represents the query vector; is the weight matrix of the query vector; represents the key vector; represents a value vector; is the weight matrix of the key vector; is the weight matrix of the value vector; After obtaining the vector, the attention weight is calculated through the softmax function, and the attention weight is used for weighting to obtain the output of the position dimension attention feature modeling , expressed as: ; In the formula, Represents input Middle position The query vector of Represents input Middle position The key vector of ; Represents input Middle position The value vector at ; Represents the Softmax function; Indicates input All locations Perform summation; Respectively represent input The horizontal coordinate is , the vertical axis is location; Will , , Fusion to form the final multi-dimensional weighted feature map , expressed as: ; In the formula, Represents element-wise multiplication.

[0016] Compared with the existing technology, the present invention has the following beneficial effects:

[0017] (1) The present invention designs a local aggregation module and introduces it into the aggregation neck network module in the electric vehicle recognition model. Through hierarchical feature aggregation, the electric vehicle recognition model can perform better in complex scenarios and significantly improve the recognition ability of small electric vehicles in elevators and electric vehicle edge details. The present invention combines convolutional linear transformation and channel shuffling operations and introduces them into the electric vehicle recognition model together with the local aggregation module, thereby greatly improving the recognition accuracy of the electric vehicle recognition model.

[0018] (2) The present invention introduces a multi-dimensional collaborative attention module into the electric vehicle recognition model, so that the electric vehicle recognition model can take into account feature interactions at different levels and can process multiple input information at the same time. By combining this information, the electric vehicle recognition model has the ability of efficient space-scale joint processing, more accurately locates the target during the detection process, and can better identify and distinguish electric vehicles from other objects in complex environments such as lighting changes, partial occlusion or complex background environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the electric vehicle identification model structure of the present invention.

[0020] Figure 2 It is a schematic diagram of the structure of the feature extraction backbone network module of the present invention. DETAILED DESCRIPTION

[0021] like Figure 1As shown, the present invention provides a technical solution: an electric vehicle identification method in an elevator based on an improved YOLOv8, comprising the following steps:

[0022] Step S1: constructing an electric vehicle entering an elevator image dataset, wherein the electric vehicle entering an elevator image dataset includes a plurality of electric vehicle entering an elevator images.

[0023] The specific process of step S1 is as follows: due to the lack of a public dataset of images of electric vehicles in elevators, more than 6,000 images of electric vehicles entering elevators (electric vehicle entry images) were manually collected. These images were taken by cameras in various communities and residential buildings around us, as well as by humans. The images cover a variety of types of electric vehicles, including traditional electric vehicles and new electric vehicles for the elderly.

[0024] Preprocess the collected images of electric vehicles entering the elevator:

[0025] 1. Remove similar images of electric vehicles in elevators.

[0026] 2. Use the labeling tool (label-studio) to semi-automatically label the electric vehicles in the collected images of electric vehicles entering the elevator.

[0027] 3. Perform consistency check on the annotated electric vehicle entering elevator images, that is, calculate the similarity of annotation results of multiple people on the same content, and take the annotation results with high similarity to ensure the rationality of the annotation.

[0028] The preprocessed electric vehicle entry elevator images constitute an electric vehicle entry elevator image dataset.

[0029] Step S2: construct an electric vehicle recognition model, which consists of a feature extraction backbone network module, an aggregation neck network module, a multi-dimensional collaborative attention module and a candidate box prediction module, and the feature extraction backbone network module, the aggregation neck network module, the multi-dimensional collaborative attention module and the candidate box prediction module are arranged in series.

[0030] Step S3: input the electric vehicle entering the elevator image in the electric vehicle entering the elevator image data set into the feature extraction backbone network module for feature extraction, and obtain a high-resolution electric vehicle entering the elevator feature map, a medium-resolution electric vehicle entering the elevator feature map and a low-resolution electric vehicle entering the elevator feature map respectively.

[0031] like Figure 2As shown in the figure, the feature extraction backbone network module consists of the first convolutional layer (CBS), the second convolutional layer, the first feature fusion splicing layer (C2f), the third convolutional layer, the second feature fusion splicing layer, the fourth convolutional layer, the third feature fusion splicing layer, the fifth convolutional layer, the fourth feature fusion splicing layer and the pooling layer (SPPF) connected in sequence; among them, the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer have the same structure; the first feature fusion splicing layer, the second feature fusion splicing layer and the third feature fusion splicing layer have the same structure.

[0032] The processing flow of the feature extraction backbone network module is as follows: the input image (electric vehicle entry image) passes through the first convolution layer, the second convolution layer, the first feature fusion splicing layer, the third convolution layer, the second feature fusion splicing layer, the fourth convolution layer, the third feature fusion splicing layer, the fifth convolution layer, the fourth feature fusion splicing layer and the pooling layer in sequence, wherein the output of the second feature fusion splicing layer is the high-resolution electric vehicle entry feature map X b The output of the third feature fusion splicing layer is the medium-resolution electric vehicle entry feature map X m The output of the pooling layer is the low-resolution electric vehicle entry feature map X s .

[0033] The processing flow of the first convolutional layer is as follows: the input image (electric vehicle entering the elevator image) is represented as , Indicates the number of channels of the input image, and Represents the input image The height and width of represents the set of real numbers; input image First, the first convolutional layer effectively extracts image features and reduces the spatial dimension of the image so that the image can be processed better and faster. The processing process of the first convolutional layer can be expressed as: (1); In the formula, It is the feature map output after the first convolutional layer; is the activation function SiLU; represents the convolution operation; and are the weights and biases of the first convolutional layer; Indicates the step size of each slide during the convolution operation.

[0034] The processing flow of the first feature fusion splicing layer is as follows: the output of the second convolutional layer After inputting the first feature fusion concatenation layer, it is divided into the direct transfer part And the convolution processing part , expressed as: (2); In the formula, Indicates the direct transfer part without convolution operation; Represents the convolution processing part, which is used for subsequent convolution processing; , and Respectively and The number of channels; Represents the first feature fusion concatenation layer.

[0035] Convolution processing part Processing is performed, expressed as: (3); In the formula, , They represent the first feature fusion concatenation layer. The weights and biases of the convolutional layers.

[0036] After processing and Perform splicing to obtain splicing features , and finally the splicing features After passing through the final convolution layer of the first feature fusion splicing layer, the output of the first feature fusion splicing layer is obtained , expressed as: (4); (5); In the formula, Represents a splicing operation; and Represents the weights and biases of the final convolutional layer of the first feature fusion concatenation layer.

[0037] The pooling layer (SPPF) can significantly expand the receptive field by combining convolution operations and maximum pooling operations, helping the electric vehicle recognition model to more accurately identify small target groups.

[0038] The processing flow of the pooling layer (SPPF) is as follows: the output of the fourth feature fusion splicing layer is input into the pooling layer (SPPF), first through a convolutional layer with a step size of 1 (used to extract preliminary features and adjust the number of channels), and then the activation feature map is obtained. , and then activate the feature map Perform the maximum pooling operation to further improve the receptive field and obtain the pooling feature map , expressed as: (6); In the formula, Represents the feature map generated from pooling kernels k of different sizes in the pooling layer, that is, the pooling feature map; Represents the activation feature map Perform multi-scale maximum pooling operations; Indicates the size of the pooling kernel; Represents the activation feature map The number of fills; The specific value of the filling quantity.

[0039] The feature map will be activated With pooling feature map The concatenation is performed in the channel dimension, and the multi-scale information is integrated through convolution operation, while the number of channels is compressed at the same time to obtain the output of the pooling layer, that is, the low-resolution feature map X s .

[0040] Step S4: The high-resolution electric vehicle entering the elevator feature map, the medium-resolution electric vehicle entering the elevator feature map and the low-resolution electric vehicle entering the elevator feature map are input into the aggregation neck network module through parallel channels for fusion, and the high-resolution electric vehicle entering the elevator aggregation feature map, the medium-resolution electric vehicle entering the elevator aggregation feature map and the low-resolution electric vehicle entering the elevator aggregation feature map are obtained respectively.

[0041] Among them, the aggregation neck network module is composed of a first local aggregation module, a second local aggregation module, a third local aggregation module, a fourth local aggregation module, a first group exchange convolution block, a second group exchange convolution block and a third group exchange convolution block.

[0042] The processing flow of the aggregation neck network module is as follows: upsampling the low-resolution electric vehicle entry feature map and splicing it with the medium-resolution electric vehicle entry feature map to obtain a medium- and low-scale fusion feature map, inputting the medium- and low-scale fusion feature map into the first local aggregation module to obtain a medium- and low-scale aggregation feature map, inputting the medium- and low-scale aggregation feature map into the first group exchange convolution block to obtain a medium- and low-scale convolution feature map, splicing the medium- and low-scale convolution feature map with the high-resolution electric vehicle entry feature map to obtain a multi-scale aggregation feature map, inputting the multi-scale aggregation feature map into the second local aggregation module to obtain a high-resolution electric vehicle entry aggregation feature map, and inputting the high-resolution electric vehicle entry feature map into the second local aggregation module. The vehicle-entering-elevator aggregated feature map is input into the second group exchange convolution block to obtain a multi-scale convolutional feature map, the multi-scale convolutional feature map is spliced ​​with the medium and low-scale aggregated feature maps to obtain a multi-scale spliced ​​feature map, the multi-scale spliced ​​feature map is input into the third local aggregation module to obtain a medium-resolution electric vehicle entering the elevator aggregated feature map, the medium-resolution electric vehicle entering the elevator aggregated feature map is input into the third group exchange convolution block to obtain a multi-layer convolutional feature map, the multi-layer convolutional feature map is spliced ​​with the low-resolution electric vehicle entering the elevator feature map to obtain a multi-layer spliced ​​feature map, the multi-layer spliced ​​feature map is input into the fourth local aggregation module to obtain a low-resolution electric vehicle entering the elevator aggregated feature map.

[0043] Among them, the upsampling operation adopts a bilinear interpolation method to estimate the new pixel value by linearly weighting the surrounding pixels, thereby achieving a smoother spatial change response.

[0044] Among them, the first local aggregation module (VoVGSCSPG), the second local aggregation module, the third local aggregation module and the fourth local aggregation module have the same structure and are formed by combining the VoVNet module and the CSPNet module.

[0045] The processing flow of the first local aggregation module is:

[0046] The VoVNet module is used for progressive feature extraction. In order to extract the input features hierarchically, the present invention utilizes the progressive connection structure of the VoVNet module. First, multiple convolution operations are performed on the input (medium and low scale fusion feature maps) to obtain a multi-convolution feature map X(X 1 conv ,X 2 conv ), where X 1 conv is the feature map after the first ordinary convolution operation, X 2 conv Is X 1 conv The feature map after the second ordinary convolution operation; then, X 2 conv After the depth-separable convolution operation, the depth convolution feature map X is obtained 3 conv ; Finally, the multi-convolution feature map X and the deep convolution feature map X 3 conv Perform concatenation to get the output X of the VoVNet module 4 conv .

[0047] The CSPNet module is used for cross-stage partial fusion; in order to reduce feature redundancy and optimize gradient flow, the CSPNet module improves computational efficiency by feature map segmentation and partial fusion. First, the output X of the VoVNet module is segmented and 4 conv Divided into Part I X part1 and Part II X part2 , then, for the second part X part2 Perform convolution operation to obtain the cross-stage convolution feature map X part2_conv , and then the first part X part1 and the cross-stage convolutional feature map X part2_conv Perform a splicing operation to obtain the output of the first local aggregation module.

[0048] Among them, the first group exchange convolution block (GHConv), the second group exchange convolution block, and the third group exchange convolution block have the same structure.

[0049] The workflow of the first group exchange convolution block is as follows: In order to reduce the amount of computation and improve the feature fusion effect, the information flow between features is promoted by disrupting the channel order. First, the input (the output of the first local aggregation module) is first subjected to a standard convolution operation through a convolution layer to extract feature information and obtain the group convolution feature map X csp1 , then, the group convolution X csp1 Then perform a convolution operation through a 1×1 convolution layer to generate additional ghost features and obtain the ghost feature map X ghost Then, the channel order of the ghost feature is disrupted by the channel shuffle operation to improve the feature expression ability of the network and obtain the group convolution ghost feature map , which is the output of the first group exchange convolution block, is expressed as: (7); In the formula, X ghost_s It is the group convolution ghost feature map obtained after the ghost feature is shuffled through the channels, and the size remains unchanged; Represents a channel shuffle operation.

[0050] Step S5: Input the high-resolution electric vehicle entry aggregated feature map, the medium-resolution electric vehicle entry aggregated feature map and the low-resolution electric vehicle entry aggregated feature map into the multi-dimensional collaborative attention module, extract dimensional feature maps from the three dimensions of channel, space and position of the high-resolution electric vehicle entry aggregated feature map, the medium-resolution electric vehicle entry aggregated feature map and the low-resolution electric vehicle entry aggregated feature map according to different resolutions, aggregate the dimensional feature maps through average pooling and standard deviation pooling, calculate the weights of the aggregated dimensional feature maps through excitation transformation, and generate a multi-dimensional weighted feature map.

[0051] Among them, the multi-dimensional collaborative attention module focuses on feature modeling in three dimensions: channel dimension attention feature modeling, spatial dimension attention feature modeling, and position dimension attention feature modeling. By calculating the attention weights separately in these three dimensions and performing collaborative fusion, the module can effectively enhance the key information in the input feature map.

[0052] The specific process of channel dimension attention feature modeling is as follows: First, the input is the output of the aggregation neck network module (High-resolution electric vehicle entry ladder aggregation feature map, medium-resolution electric vehicle entry ladder aggregation feature map and low-resolution electric vehicle entry ladder aggregation feature map) perform global average pooling in the spatial dimension to obtain the channel weight vector f c ∈R c , expressed as: (8); In the formula, Represents the average of all pixel values ​​on channel c; Respectively The index of the row (height) and column (width) in the range and .

[0053] By double summation Traverse all spatial positions (i.e. all pixels) on channel c, accumulate the pixel values, and get X p (c,u,v),X p (c,u,v) represents input The pixel value at position (u,v) on channel c.

[0054] Get the channel weight vector f c After that, the weight W of each channel is obtained through a simple linear transformation c , then W c Application Each channel output of , get the output of channel dimension attention feature modeling , expressed as: (9); In the formula, is the activation function Sigmoid; W c and b c Represent the weight parameters and bias parameters of each channel respectively.

[0055] The specific process of modeling spatial dimension attention features is as follows: Aggregate using average pooling and maximum pooling in the channel dimension to obtain a two-dimensional spatial feature map , expressed as: (10); In the formula, Represents the input in the channel dimension Perform average pooling; Represents the input in the channel dimension Perform maximum pooling; Represents input The pixel value on all channels at position (u,v); Indicates all channels.

[0056] Then, the two-dimensional spatial feature map After learning the spatial weights through a 7×7 convolutional layer, the input Multiply them together to get the output of spatial dimension attention feature modeling , expressed as: (11); In the formula, Indicates the use of a 7×7 convolutional layer for the two-dimensional spatial feature map Perform convolution operation.

[0057] The specific process of modeling the position dimension attention feature is as follows: Mapping to three different spaces generates vectors, expressed as: (12); In the formula, represents the query vector; is the weight matrix of the query vector; represents the key vector; represents a value vector; is the weight matrix of the key vector; is the weight matrix of the value vector.

[0058] After obtaining the vector, the attention weight is calculated through the softmax function, and these attention weights are used for weighting to obtain the output of the position dimension attention feature modeling , expressed as: (13); In the formula, Represents input Middle position The query vector is used to find the input Center and position Other locations where relevant; Represents input Middle position A key vector describing the input The characteristics of each position in Represents input Middle position The value vector at ; Represents the Softmax function; Indicates input All locations Perform summation; Respectively represent input The horizontal coordinate is , the vertical axis is location.

[0059] Will , , Fusion to form the final multi-dimensional weighted feature map , expressed as: (14); In the formula, Represents element-wise multiplication.

[0060] Step S6: Input the multi-dimensional weighted feature map into the candidate box prediction module, perform target prediction on the multi-dimensional weighted feature map through bounding box regression, target classification and confidence prediction, post-process the prediction results, and generate the final prediction output.

[0061] Among them, the candidate box prediction module uses the original YOLOv8 detection head.

[0062] In the loss function of bounding box regression, the traditional CIoU loss function only focuses on the overlapping area of ​​the predicted box and the true box, and is not strong enough to optimize the aspect ratio, and cannot effectively constrain the shape and size of the predicted box. To this end, the present invention adopts the MPDIoU loss function as the loss function of bounding box regression to achieve a more comprehensive position measurement and more effective gradient information; MPDIoU loss function It is expressed as: (15); In the formula, Represents the intersection-over-union ratio of the predicted box and the true box; and is an additional weight coefficient used to balance the impact of the new distance metric and aspect ratio; In polar coordinates, considering the direction angle The influence of, the maximum distance between the predicted box and the true box; Indicates the maximum diagonal length of the predicted box and the true box; It is a term that measures the difference in aspect ratio between the predicted box and the real box. It is the weight coefficient used to balance the aspect ratio in the original CIoU loss function. Here, the weight coefficient And the term that measures the difference in aspect ratio between the predicted box and the true box Multiply it by this to achieve a more comprehensive position measure.

[0063] Among them, the electric vehicle recognition model uses the collected and processed electric vehicle entry image dataset to train the target detection model in a supervised learning manner. All parameters are learnable parameters, and are optimized through the back propagation algorithm, combined with the MPDIoU loss function, cross entropy loss function and binary cross entropy loss function; during the training process, the number of iterations of the electric vehicle recognition model is set to 100, the learning rate is 0.01, and the SGD optimizer is used to optimize the electric vehicle recognition model parameters; after all training is completed, the best performing electric vehicle recognition model parameters are saved, and finally the electric vehicle recognition prediction and performance evaluation are performed on the test set; the test set is obtained by dividing the electric vehicle entry image dataset.

[0064] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. The electric vehicle identification method in the elevator based on the improved YOLOv8 is characterized by: The steps include: Step S1: constructing an electric vehicle entering an elevator image dataset, wherein the electric vehicle entering an elevator image dataset includes a number of electric vehicle entering an elevator images; Step S2: constructing an electric vehicle recognition model, wherein the electric vehicle recognition model is composed of a feature extraction backbone network module, an aggregation neck network module, a multi-dimensional collaborative attention module and a candidate box prediction module, and the feature extraction backbone network module, the aggregation neck network module, the multi-dimensional collaborative attention module and the candidate box prediction module are arranged in series; Step S3: inputting the electric vehicle entering the elevator image in the electric vehicle entering the elevator image data set into the feature extraction backbone network module for feature extraction, and obtaining a high-resolution electric vehicle entering the elevator feature map, a medium-resolution electric vehicle entering the elevator feature map, and a low-resolution electric vehicle entering the elevator feature map respectively; Step S4: inputting the high-resolution electric vehicle entering the elevator feature map, the medium-resolution electric vehicle entering the elevator feature map and the low-resolution electric vehicle entering the elevator feature map into the aggregation neck network module through parallel channels for fusion, thereby obtaining a high-resolution electric vehicle entering the elevator aggregation feature map, a medium-resolution electric vehicle entering the elevator aggregation feature map and a low-resolution electric vehicle entering the elevator aggregation feature map respectively; Step S5: inputting the high-resolution electric vehicle entry aggregate feature map, the medium-resolution electric vehicle entry aggregate feature map and the low-resolution electric vehicle entry aggregate feature map into the multidimensional collaborative attention module, extracting dimensional feature maps in three dimensions of channel, space and position from the high-resolution electric vehicle entry aggregate feature map, the medium-resolution electric vehicle entry aggregate feature map and the low-resolution electric vehicle entry aggregate feature map according to different resolutions, aggregating the dimensional feature maps through average pooling and standard deviation pooling, calculating the weights of the aggregated dimensional feature maps through excitation transformation, and generating a multidimensional weighted feature map; Step S6: Input the multi-dimensional weighted feature map into the candidate box prediction module, perform target prediction on the multi-dimensional weighted feature map through bounding box regression, target classification and confidence prediction, post-process the prediction results, and generate the final prediction output.

2. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 1, characterized in that: The aggregation neck network module is composed of a first local aggregation module, a second local aggregation module, a third local aggregation module, a fourth local aggregation module, a first group exchange convolution block, a second group exchange convolution block and a third group exchange convolution block; The processing flow of the aggregation neck network module is as follows: upsampling the low-resolution electric vehicle entry feature map and splicing it with the medium-resolution electric vehicle entry feature map to obtain a medium- and low-scale fusion feature map, inputting the medium- and low-scale fusion feature map into the first local aggregation module to obtain a medium- and low-scale aggregation feature map, inputting the medium- and low-scale aggregation feature map into the first group exchange convolution block to obtain a medium- and low-scale convolution feature map, splicing the medium- and low-scale convolution feature map with the high-resolution electric vehicle entry feature map to obtain a multi-scale aggregation feature map, inputting the multi-scale aggregation feature map into the second local aggregation module to obtain a high-resolution electric vehicle entry aggregation feature map, and inputting the high-resolution electric vehicle entry feature map into the second local aggregation module. The electric vehicle entering the elevator aggregate feature map is input into the second group exchange convolution block to obtain a multi-scale convolution feature map, the multi-scale convolution feature map is spliced ​​with the medium and low-scale aggregate feature map to obtain a multi-scale spliced ​​feature map, the multi-scale spliced ​​feature map is input into the third local aggregation module to obtain a medium-resolution electric vehicle entering the elevator aggregate feature map, the medium-resolution electric vehicle entering the elevator aggregate feature map is input into the third group exchange convolution block to obtain a multi-layer convolution feature map, the multi-layer convolution feature map is spliced ​​with the low-resolution electric vehicle entering the elevator feature map to obtain a multi-layer spliced ​​feature map, the multi-layer spliced ​​feature map is input into the fourth local aggregation module to obtain a low-resolution electric vehicle entering the elevator aggregate feature map; Among them, the upsampling operation adopts a bilinear interpolation method.

3. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 2, characterized in that: The feature extraction backbone network module consists of a first convolutional layer, a second convolutional layer, a first feature fusion splicing layer, a third convolutional layer, a second feature fusion splicing layer, a fourth convolutional layer, a third feature fusion splicing layer, a fifth convolutional layer, a fourth feature fusion splicing layer and a pooling layer, which are connected in sequence; among them, the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer have the same structure; the first feature fusion splicing layer, the second feature fusion splicing layer and the third feature fusion splicing layer have the same structure.

4. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 3 is characterized in that: The processing flow of the feature extraction backbone network module is as follows: the input image, i.e., the electric vehicle entering the elevator image, passes through the first convolution layer, the second convolution layer, the first feature fusion splicing layer, the third convolution layer, the second feature fusion splicing layer, the fourth convolution layer, the third feature fusion splicing layer, the fifth convolution layer, the fourth feature fusion splicing layer and the pooling layer in sequence, wherein the output of the second feature fusion splicing layer is the high-resolution electric vehicle entering the elevator feature map X b The output of the third feature fusion splicing layer is the medium-resolution electric vehicle entry feature map X m The output of the pooling layer is the low-resolution electric vehicle entry feature map X s .

5. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 4 is characterized in that: The first local aggregation module, the second local aggregation module, the third local aggregation module and the fourth local aggregation module have the same structure, and are formed by combining the VoVNet module and the CSPNet module; The processing flow of the first local aggregation module is: VoVNet module: First, multiple convolution operations are performed on the input, i.e., the medium and low scale fusion feature maps, to obtain a multi-convolution feature map X(X 1 conv ,X 2 conv ), where X 1 conv is the feature map after the first ordinary convolution operation, X 2 conv Is X 1 conv The feature map after the second ordinary convolution operation; then, X 2 conv After the depth-separable convolution operation, the depth convolution feature map X is obtained 3 conv ; Finally, the multi-convolution feature map X and the deep convolution feature map X 3 conv Perform concatenation to get the output X of the VoVNet module 4 conv ; CSPNet module: First, the output X of the VoVNet module is split using a split operation. 4 conv Divided into Part I X part1 and Part II X part2 , then, for the second part X part2 Perform convolution operation to obtain the cross-stage convolution feature map X part2_conv , and then the first part X part1 and the cross-stage convolutional feature map X part2_conv Perform a splicing operation to obtain the output of the first local aggregation module.

6. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 5 is characterized in that: The processing flow of the first convolutional layer is: the input image, i.e. the image of the electric car entering the elevator, is represented as , Indicates the number of channels of the input image, and Represents the input image The height and width of represents a set of real numbers; the processing process of the first convolutional layer is expressed as: ； In the formula, It is the feature map output after the first convolutional layer; is the activation function SiLU; represents the convolution operation; and are the weights and biases of the first convolutional layer; Indicates the step size of each slide during the convolution operation.

7. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 6, characterized in that: The processing flow of the first feature fusion splicing layer is: the output of the second convolutional layer After inputting the first feature fusion concatenation layer, it is divided into the direct transfer part And the convolution processing part , expressed as: ； In the formula, Indicates the direct delivery part; Represents the convolution processing part; , and Respectively and The number of channels; represents the first feature fusion splicing layer; Convolution processing part Processing is performed, expressed as: ； In the formula, , They represent the first feature fusion concatenation layer. The weights and biases of the convolutional layers; After processing and Perform splicing to obtain splicing features , and finally the splicing features After passing through the final convolution layer of the first feature fusion splicing layer, the output of the first feature fusion splicing layer is obtained , expressed as: ； ; In the formula, Represents a splicing operation; and Represents the weights and biases of the final convolutional layer of the first feature fusion concatenation layer.

8. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 7, characterized in that: The processing flow of the pooling layer is as follows: the output of the fourth feature fusion splicing layer is input into the pooling layer, and first passes through a convolutional layer with a step size of 1 to obtain the activation feature map , and then activate the feature map Perform the maximum pooling operation to obtain the pooling feature map , expressed as: ； In the formula, Represents the feature map generated from pooling kernels k of different sizes in the pooling layer, that is, the pooling feature map; Represents the activation feature map Perform multi-scale maximum pooling operations; Indicates the size of the pooling kernel; Represents the activation feature map The number of fills; is the specific value of the filling quantity; The feature map will be activated With pooling feature map The concatenation is performed in the channel dimension, and the multi-scale information is integrated through convolution operation, while the number of channels is compressed at the same time to obtain the output of the pooling layer, that is, the low-resolution feature map X s .

9. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 8, characterized in that: The first group exchange convolution block, the second group exchange convolution block, and the third group exchange convolution block have the same structure; The workflow of the first group exchange convolution block is as follows: First, the input, i.e., the output of the first local aggregation module, is first subjected to a standard convolution operation through a convolution layer to obtain the group convolution feature map X csp1 , then, the group convolution X csp1 Then perform a convolution operation through a 1×1 convolution layer to generate additional ghost features and obtain the ghost feature map X ghost Then, the channel order of the ghost feature is disrupted by the channel shuffle operation to obtain the group convolution ghost feature map , which is the output of the first group exchange convolution block, is expressed as: ； Where, X ghost_s It is the group convolution ghost feature map obtained after the ghost feature is shuffled through the channels; Represents a channel shuffle operation.

10. The method for identifying electric vehicles in elevators based on improved YOLOv8 according to claim 9, characterized in that: The specific process of step S5 is: aggregate the output of the neck network module That is, the high-resolution electric vehicle entry aggregate feature map, the medium-resolution electric vehicle entry aggregate feature map and the low-resolution electric vehicle entry aggregate feature map are input into the multi-dimensional collaborative attention module, and the attention weights are calculated and collaboratively fused through channel dimension attention feature modeling, space dimension attention feature modeling and position dimension attention feature modeling respectively; The specific process of channel dimension attention feature modeling is as follows: First, the input is the output of the aggregation neck network module Perform global average pooling in the spatial dimension to obtain the channel weight vector f c ∈R c , expressed as: ； In the formula, Represents the average of all pixel values ​​on channel c; Respectively The row and column indexes in the range and ; By double summation Traverse all spatial positions on channel c, that is, all pixel points, accumulate the pixel values, and get X p (c,u,v),X p (c,u,v) represents input The pixel value at position (u,v) on channel c; For the channel weight vector f c Perform a linear transformation to obtain the weight W of each channel c , then W c Application Each channel output of , get the output of channel dimension attention feature modeling , expressed as: ； In the formula, is the activation function Sigmoid; W c and b c Represent the weight parameters and bias parameters of each channel respectively; The specific process of modeling spatial dimension attention features is as follows: Aggregate using average pooling and maximum pooling in the channel dimension to obtain a two-dimensional spatial feature map , expressed as: ； In the formula, Represents the input in the channel dimension Perform average pooling; Represents the input in the channel dimension Perform maximum pooling; Represents input The pixel value on all channels at position (u,v); Indicates all channels; The two-dimensional spatial feature map After learning the spatial weights through a 7×7 convolutional layer, the input Multiply them together to get the output of spatial dimension attention feature modeling , expressed as: ； In the formula, Indicates the use of a 7×7 convolutional layer for the two-dimensional spatial feature map Perform convolution operation; The specific process of modeling the position dimension attention feature is as follows: Mapping to three different spaces generates vectors, expressed as: ； In the formula, represents the query vector; is the weight matrix of the query vector; represents the key vector; represents a value vector; is the weight matrix of the key vector; is the weight matrix of the value vector; After obtaining the vector, the attention weight is calculated through the softmax function, and the attention weight is used for weighting to obtain the output of the position dimension attention feature modeling , expressed as: ； In the formula, Represents input Middle position The query vector of Represents input Middle position The key vector of ; Represents input Middle position The value vector at ; Represents the Softmax function; Indicates input All locations Perform summation; Respectively represent input The horizontal coordinate is , the vertical axis is location; Will , , Fusion to form the final multi-dimensional weighted feature map , expressed as: ； In the formula, Represents element-wise multiplication.

Images