Automobile inner and outer fuel tank cap detection method based on combination of Vmamba and YOLOv8n

Abstract

The invention discloses an automobile internal and external fuel tank cap detection method based on combination of Vmamba and YOLOv8n, which comprises the following steps: firstly, acquiring photos of an automobile internal and external fuel tank cap, constructing an initial data set, preprocessing the photos in the data set, dividing the preprocessed data set, and then improving a YOLOv8n network structure based on a Vmamba framework, so as to detect the internal and external fuel tank caps of the automobile. And constructing an automobile internal and external fuel tank cap detection model, training the model based on the data set, and finally detecting the automobile internal and external fuel tank cap by using the trained automobile internal and external fuel tank cap detection model. According to the scheme of the invention, a YOLOv8n network structure improved based on a VMama framework is adopted to construct an automobile inner and outer fuel tank cap detection model, on one hand, the center of an object is directly predicted, the detection process is simplified, on the other hand, by introducing a 2D-Selective-Scan (SS2D) module, global and local information of an image can be effectively processed, linear complexity is kept, and the detection efficiency is improved. High-precision real-time detection of the inner and outer fuel tank caps of the automobile under various illumination conditions can be realized.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and computer vision, and specifically relates to a method for detecting inner and outer fuel tank caps of a car based on the combination of Vmamba and YOLOv8n. Background Art

[0002] In the field of object detection, especially for the detection of internal and external fuel tank caps of cars, although traditional convolutional neural networks (CNNs) and Transformer architectures have achieved remarkable results, they still have some limitations. For example, CNN is limited by the local receptive field when capturing long-distance dependencies, and the self-attention mechanism of Transformer has high computational complexity and high video memory consumption when processing large-scale images. These limitations are particularly evident in small object detection tasks such as car fuel tank caps, because they are usually small in size and may be occluded, making detection more difficult.

[0003] Among them, the VMamba framework based on the state-space model (SSM) has attracted attention due to its advantages in capturing long-range dependencies and global modeling. By introducing the 2D-Selective-Scan (SS2D) module, the VMamba framework can effectively process the global and local information of the image while maintaining linear complexity, which is particularly important when dealing with small target detection tasks.

[0004] However, although the VMamba framework has many advantages in theory, its performance and efficiency still need to be further verified and optimized in practical applications, especially in the specific field of internal and external fuel tank cap detection of automobiles. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to provide a method for detecting internal and external fuel tank caps of a car based on the combination of Vmamba and YOLOv8n.

[0006] The specific technical solution for achieving the purpose of the present invention is:

[0007] A method for detecting internal and external fuel tank caps of a car based on Vmamba and YOLOv8n, comprising the following steps:

[0008] Step 1: Collect photos of the fuel tank cap inside and outside the car to build an initial data set;

[0009] Step 2: Preprocess the photos in the data set and divide the preprocessed data set;

[0010] Step 3: Based on the VMamba framework, the network structure of YOLOv8n is improved to build a detection model for the internal and external fuel tank caps of a car, and the model is trained based on the data set;

[0011] Step 4: Use the trained car inner and outer fuel tank cap detection model to detect the inner and outer fuel tank caps of the car.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] The solution of the present invention adopts the network structure of YOLOv8n improved based on VMamba framework to construct a detection model for the inner and outer fuel tank caps of automobiles. The overall network adopts the YOLOv8n structure. YOLOv8 is an anchor-free model that directly predicts the center of the object, simplifying the detection process. In the YOLOv8 series, YOLOv8n is the smallest model with the fastest speed and is easy to perform video detection. The VMamba framework can effectively process the global and local information of the image by introducing the 2D-Selective-Scan (SS2D) module, while maintaining linear complexity, which is particularly important when processing small target detection tasks such as the inner cap of the automobile fuel tank. In addition, in the model structure, the C2f structure in the Backbone backbone network is replaced with the ODSSBlock structure to reduce some network parameters of the model and ensure the real-time performance of video detection.

[0014] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic flow chart of the method for detecting internal and external fuel tank caps of a car based on the combination of Vmamba and YOLOv8n of the present invention.

[0016] Figure 2 It is a structural schematic diagram of the automobile inner and outer fuel tank cap detection model of the present invention.

[0017] Figure 3 It is a loss function curve graph of the improved YOLOv8 model of the present invention.

[0018] Figure 4 This is a diagram for identifying the inner and outer covers of a vehicle fuel tank under high light conditions in an embodiment of the present invention.

[0019] Figure 5 This is a diagram for identifying the inner and outer covers of a car fuel tank under dark lighting conditions in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] Example

[0021] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an kind" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0023] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0024] Combination Figure 1 , a method for detecting the inner and outer fuel tank caps of a car based on the combination of Vmamba and YOLOv8n, comprising the following steps:

[0025] Step 1: Collect photos of the fuel tank cap inside and outside the car to build the initial data set:

[0026] Collect photos of the internal and external fuel tank caps of cars at different angles, different lighting conditions, and different positions, and shuffle their order to form an initial data set.

[0027] In a specific operation, a camera can be used to collect photos of the inner and outer fuel tank caps of a car in an open state at different angles and under different lighting conditions, or the photos of the inner and outer fuel tank caps of a car in an open state can be collected based on the network;

[0028] In this embodiment, an industrial camera is used to collect 100 photos of the inner and outer fuel tank caps of the car in the open state at different angles and under different lighting conditions, and 1,400 photos of the inner and outer fuel tank caps of the car in the open state are collected through the Internet. The above pictures are shuffled, and the images captured under different lighting conditions, different angles, different positions, etc. are mixed to produce a high-definition initial data set of 1,500 complex situations;

[0029] Step 2: Preprocess the photos in the initial data set and divide the preprocessed data set into:

[0030] First, pre-processing is performed, including scaling the image and adjusting the image size so that its maximum size meets the model requirements while maintaining the original aspect ratio of the image. The input size of the corresponding YOLOv8 model in this embodiment is 640×640×3. The output feature maps of each scale of the model are superimposed to obtain 4 prediction boxes, 1 confidence level, and 2 category probabilities corresponding to 1500 feature maps;

[0031] Normalize the image pixel values;

[0032] Perform data enhancement and labeling on the image, where data enhancement includes mosaic enhancement, mixed enhancement, random perturbation, color perturbation, confidence filtering and non-maximum suppression;

[0033] The labeling process includes separately selecting and labeling the outer cover and the inner cover of the automobile fuel tank in the data set photo, and obtaining the category, height, width, and position information of the identification frame of the outer cover and the inner cover of the automobile fuel tank.

[0034] Finally, the preprocessed data set is divided into training set, test set and validation set according to a certain ratio.

[0035] Step 3: Based on the VMamba framework, the network structure of YOLOv8n is improved to build a detection model for the internal and external fuel tank caps of a car, and the model is trained based on the data set;

[0036] Specifically:

[0037] Combination Figure 2 , the automobile internal and external fuel tank cap detection model is constructed based on the YOLOv8n network architecture, including a feature extraction unit, a feature fusion unit and a detection unit;

[0038] The feature extraction unit is implemented based on an improved Vmamba Backbone network, including a Conv convolution layer and an ODSSBlock layer, and is used to extract image features. The specific process is as follows:

[0039] (1) Input the image into the 0th layer Conv convolution layer to extract preliminary features, and output an image with a size of 1 / 2 of the original input image. In this embodiment, the input of the 0th layer is a 640*640*3 image, and the output feature map size is 320*320*64, with a length and width of 1 / 2 of the original input image;

[0040] (2) The output image of the 0th Conv convolutional layer is used as the input of the 1st Conv convolutional layer, and further features are extracted through the 1st Conv convolutional layer, and the output is an image with a size of 1 / 4 of the original input image. In this embodiment, the 1st Conv convolutional layer has an output feature map size of 160*160*128, and the length and width are 1 / 4 of the original input image;

[0041] (3) The second ODSS Block layer receives the feature image from the first Conv convolution layer as input. It first uses depthwise separable convolution to effectively extract local spatial information. It then performs batch normalization to provide a certain degree of regularization and reduce overfitting. It also divides the input image into a series of sub-images, each of which represents a specific direction. The sub-images are scanned and expanded along four symmetrical directions, and features are extracted from the sub-images in each direction. The features in different directions are merged to form an output image of the same size as the input image.

[0042] More specifically, the ODSS Block layer receives the input feature Z l-3 , perform deep separable convolution on it, operate on each input channel independently, and then perform batch normalization to reduce overfitting; the intermediate state obtained is mixed with channel information through 1×1 convolution, and the feature expression is enhanced using the nonlinear GeLU activation function; the processed feature map is consistent with the original input Z l-3 Through residual connection fusion, the output Z of the local space block is obtained l-2 ;

[0043] The output Z of the local space block l-2 After SS2D processing, the input image is first divided into a series of sub-images using Scan Expansion. Each sub-image represents a specific direction. Scan expansion is performed along four symmetrical directions: up-down, down-up, left-right, and right-left. Then, the S6 Block is used to extract features from these sub-images. Finally, Scan Merge is used to merge these sub-images to form an output image of the same size as the input image.

[0044] After that, a 1x1 convolution is performed on the output Z of SS2D. l-1 Create a branch on and exist Depthwise separable convolution is used as the position encoding module on the branch, and residual connection is used to improve the efficiency of gradient reflux during training.

[0045] The nonlinear GeLU activation function is used to control the information flow of each layer and is multiplied by elements. The branches are merged, then fused with the global features through 1x1 convolution, and finally connected with the original input Z through residual connection l-2 Addition;

[0046] The output Z of RG Block l The input Z before SS2D processing l-3 Add the residuals to get the final output feature Z l .

[0047] The output feature map size of this layer is still 160*160*128;

[0048] (4) The output of the ODSS Block layer is used as the input of the next Conv convolutional layer, and the output is an image with a size of 1 / 8 of the original input image. In this embodiment, the output feature map size is 80*80*256.

[0049] (5) The image is then repeatedly input into the Conv convolutional layer and ODSS Block layer of the 3rd to 8th layers, and finally the SPPF layer is used in the 9th layer to complete the extraction of the final image feature information.

[0050] The feature fusion unit is implemented based on the Neck network, and performs feature fusion through the feature pyramid network;

[0051] The detection unit is implemented based on the Head network, and through the regression of the bounding box and the classification of the category, EIoU is used as the positioning loss function of YOLOv8 as the basis for the final recognition.

[0052] The process of Head network recognition is as follows:

[0053] The detection unit outputs the predicted bounding box of the fuel tank cap, determines the area of ​​the intersection and union of the predicted bounding box of the fuel tank cap and the true bounding box, and determines the IoU loss based on this, thereby determining the EIoU loss L:

[0054] L=L IoU +L asp +L dis

[0055]

[0056] L IoU =1-IoU

[0057]

[0058]

[0059] Among them, w, h, b are the width, height and center point of the predicted bounding box respectively; w gt and h gt is the width and height of the ground-truth bounding box; w c and h c is the width and height of the smallest enclosing rectangle containing the two boxes, A i Represents the intersection area of ​​the predicted bounding box and the true bounding box, A u represents the union area of ​​the predicted bounding box and the true bounding box, and ρ represents the center point b of the predicted bounding box and the center point b of the true bounding box. gt The Euclidean distance of

[0060] After obtaining the predicted box, the model uses the EIOU loss function to calculate the difference between the predicted box and the corresponding true box. This includes calculating the IOU, center point distance loss, and aspect ratio loss, and adding them together to form the EIOU loss.

[0061] After the loss function is calculated, back propagation is performed next. The back propagation algorithm calculates the gradient of the loss function with respect to the model parameters. Using the calculated gradient, the model parameters are updated through the optimization algorithm to reduce the value of the loss function, that is, to reduce the difference between the predicted box and the true box.

[0062] Through multiple iterations of training, the model gradually learns how to predict bounding boxes more accurately. At each iteration, the model updates its parameters according to the guidance of the EIOU loss function. After a sufficient number of training iterations, the model will converge on the loss function.

[0063] That is, based on the value of the EIoU loss L, the parameters of the automobile inner and outer fuel tank cap detection model are continuously optimized during the model training process until, after multiple iterations of training, the automobile inner and outer fuel tank cap detection model meets the set requirements and the model training is completed.

[0064] like Figure 3 The figure shows the curve diagrams of the loss function in this embodiment during training and verification, which are the bounding box loss, classification loss, feature point loss, precision and recall rate.

[0065] Figure 4 This is a recognition diagram of the inner and outer covers of a car fuel tank under high light conditions in this embodiment. Figure 5 This is a diagram for identifying the inner and outer covers of a car fuel tank under dark lighting conditions in this embodiment. It can be seen that the improved inner and outer fuel tank cover detection model of the present invention can produce very accurate detection results under both high lighting and dark lighting conditions.

[0066] Step 4: Use the trained car inner and outer fuel tank cap detection model to detect the inner and outer fuel tank caps of the car.

[0067] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0068] Step 1: Collect photos of the fuel tank cap inside and outside the car to build an initial data set;

[0069] Step 2: Preprocess the photos in the initial data set and divide the preprocessed data set;

[0070] Step 3: Based on the VMamba framework, the network structure of YOLOv8n is improved to build a detection model for the internal and external fuel tank caps of a car, and the model is trained based on the data set;

[0071] Step 4: Use the trained car inner and outer fuel tank cap detection model to detect the inner and outer fuel tank caps of the car.

[0072] A computer storable medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the following steps are implemented:

[0073] Step 1: Collect photos of the fuel tank cap inside and outside the car to build an initial data set;

[0074] Step 2: Preprocess the photos in the initial data set and divide the preprocessed data set;

[0075] Step 3: Based on the VMamba framework, the network structure of YOLOv8n is improved to build a detection model for the internal and external fuel tank caps of a car, and the model is trained based on the data set;

[0076] Step 4: Use the trained car inner and outer fuel tank cap detection model to detect the inner and outer fuel tank caps of the car.

[0077] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting internal and external fuel tank caps of automobiles based on Vmamba and YOLOv8n, characterized in that: The following steps are involved: Step 1: Collect photos of the fuel tank cap inside and outside the car to build an initial data set; Step 2: Preprocess the photos in the initial data set and divide the preprocessed data set; Step 3: Based on the VMamba framework, the network structure of YOLOv8n is improved to build a detection model for the internal and external fuel tank caps of a car, and the model is trained based on the data set; Step 4: Use the trained car inner and outer fuel tank cap detection model to detect the inner and outer fuel tank caps of the car.

2. The method for detecting the inner and outer fuel tank caps of a car based on the combination of Vmamba and YOLOv8n according to claim 1 is characterized in that: The initial data set in step 1 is specifically: Collect photos of the internal and external fuel tank caps of cars at different angles, different lighting conditions, and different positions, and shuffle their order to form an initial data set.

3. The method for detecting the inner and outer fuel tank caps of a car based on the combination of Vmamba and YOLOv8n according to claim 1 is characterized in that: The preprocessing and division of the photos in the data set in step 2 is specifically as follows: Resize the image so that its largest dimension fits the model requirements while maintaining the image’s original aspect ratio; Normalize the image pixel values; Perform data enhancement and labeling processing on the image, where data enhancement processing includes mosaic enhancement, mixed enhancement, random perturbation, and color perturbation processing; Finally, the preprocessed data set is divided into training set, test set and validation set according to a certain ratio.

4. The method for detecting the inner and outer fuel tank caps of a vehicle based on the combination of Vmamba and YOLOv8n according to claim 3 is characterized in that: The labeling process includes separately selecting and labeling the outer cover and the inner cover of the automobile fuel tank in the data set photo, and obtaining the category, height, width, and position information of the identification frame of the outer cover and the inner cover of the automobile fuel tank.

5. The method for detecting the inner and outer fuel tank caps of a car based on the combination of Vmamba and YOLOv8n according to claim 1 is characterized in that: The vehicle inner and outer fuel tank cap detection model in step 3 is specifically: The vehicle internal and external fuel tank cap detection model is built based on the YOLOv8n network architecture, including a feature extraction unit, a feature fusion unit and a detection unit; The feature extraction unit is implemented based on an improved Vmamba Backbone network, including a Conv convolution layer and an ODSSBlock layer, for extracting image features; The feature fusion unit is implemented based on the Neck network, and performs feature fusion through the feature pyramid network; The detection unit is implemented based on the Head network, and through the regression of the bounding box and the classification of the category, EIoU is used as the positioning loss function of YOLOv8 as the basis for the final recognition.

6. The method for detecting the inner and outer fuel tank caps of a vehicle based on the combination of Vmamba and YOLOv8n according to claim 5 is characterized in that: The process of extracting image features is as follows: (1) Input the image into the 0th Conv convolution layer to extract preliminary features and output an image with a size of 1 / 2 of the original input image; (2) The output image of the 0th Conv convolutional layer is used as the input of the 1st Conv convolutional layer, and further features are extracted through the 1st Conv convolutional layer. The output is an image with a size of 1 / 4 of the original input image. (3) The second ODSS Block layer receives the feature image from the first Conv convolution layer as input. It first uses depthwise separable convolution to effectively extract local spatial information. It then performs batch normalization to provide a certain degree of regularization and reduce overfitting. It also divides the input image into a series of sub-images, each of which represents a specific direction. The sub-images are scanned and expanded along four symmetrical directions, and features are extracted from the sub-images in each direction. The features in different directions are merged to form an output image of the same size as the input image. (4) The output of the ODSS Block layer is used as the input of the next Conv convolutional layer, and the output is an image with a size of 1 / 8 of the original input image; (5) The image is then repeatedly input into the Conv convolution layer and the ODSS Block layer, and finally the SPPF layer is used to extract the final image feature information.

7. The method for detecting the internal and external fuel tank caps of a vehicle based on the combination of Vmamba and YOLOv8n according to claim 5 is characterized in that: The process of Head network recognition is as follows: The detection unit outputs the predicted bounding box of the fuel tank cap, determines the area of ​​the intersection and union of the predicted bounding box of the fuel tank cap and the true bounding box, and determines the IoU loss based on this, thereby determining the EIoU loss L: L=L IoU +L asp +L dis L IoU =1-IoU Among them, w, h, b are the width, height and center point of the predicted bounding box respectively; w gt and h gt is the width and height of the ground-truth bounding box; w c and h c is the width and height of the smallest enclosing rectangle containing the two boxes, A i Represents the intersection area of ​​the predicted bounding box and the true bounding box, A u represents the union area of ​​the predicted bounding box and the true bounding box, and ρ represents the center point b of the predicted bounding box and the center point b of the true bounding box. gt The Euclidean distance of Based on the value of EIoU loss L, the parameters of the automobile inner and outer fuel tank cap detection model are continuously optimized during the model training process until, after multiple iterations of training, the automobile inner and outer fuel tank cap detection model meets the set requirements and the model training is completed.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to claims 1 to 7 are implemented.

9. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Images