Marker detection method based on SSIM and YOLOV8 algorithms and electronic equipment

Abstract

The invention discloses a marker detection method based on SSIM and YOLOV8 algorithms. The method comprises the following steps: collecting an original picture A and a marker-containing picture B of a to-be-detected site; performing graying processing on the original picture A and the picture B containing the marker to respectively obtain a picture A1 without the marker and a picture B1 containing the marker; calculating the structural similarity of the marker-free picture A1 and the marker-containing picture B1 to obtain an SSIM difference graph C; and inputting the SSIM difference graph C into the trained YOLOV8 neural network model to obtain the position information of the marker. According to the invention, the detection precision can be effectively improved, and cheating behaviors in a subject 2 examination can be effectively prevented.

Description

Technical Field

[0001] The present invention belongs to the field of driver training and driving test, and specifically relates to a marker detection technology based on SSIM and YOLOV8 algorithms. Background Art

[0002] As an important part of the driving skills certification process, the subject 2 test is designed to evaluate and confirm whether the driver has mastered the necessary operating skills and safety rules. The subject 2 test covers multiple key skill points such as reversing into a garage and parallel parking, which are operations that are frequently used in daily driving and are critical to safety. However, in reality, in order to increase the pass rate, there are many cheating behaviors such as placing markers in the test site in advance. This behavior seriously undermines the fairness of the test and also poses a potential threat to road traffic safety.

[0003] Currently, most monitoring methods still rely on manual observation or conventional video monitoring systems, which can only be applied to very obvious cheating behaviors. In some complex examination environments, existing monitoring methods are particularly difficult to identify small target markers, making it difficult to effectively prevent cheating behaviors. Summary of the invention

[0004] To solve the above problems, the present invention provides a marker detection method based on SSIM and YOLOV8, which can effectively improve the detection accuracy and effectively prevent cheating in the subject two examination.

[0005] The first aspect of the present invention discloses a marker detection method based on SSIM and YOLOV8 algorithm, comprising:

[0006] Collect the original image A of the site to be tested and the image B containing the markers;

[0007] The obtained original image A and the image B containing the marker are grayed out to obtain an image A1 without the marker and an image B1 containing the marker respectively;

[0008] Calculate the structural similarity between the image A1 without the marker and the image B1 with the marker to obtain an SSIM difference graph C;

[0009] The SSIM difference graph C is input into the trained YOLOV8 neural network model to obtain the position information of the marker.

[0010] The marker detection method according to claim 1, characterized in that the calculation formula of the structural similarity between the non-marker image A1 and the marker image B1 is as follows:

[0011]

[0012] Where SSIM(x,y) represents the difference between the image A1 without markers and the image B1 with markers at the same pixel position x,y; μ x and μ y are the mean values ​​of the unmarked image A1 and the marked image B1 at x and y, respectively, x and σ y is the variance of the unmarked image A1 and the marked image B1 at x,y, σ xy is the covariance of the unmarked image A1 and the marked image B1 at x, y; c 1 and c 2 is a constant.

[0013] As an optional solution, the YOLOV8 neural network model includes an Input network module, a Backbone network module, a Neck network module and a Head network module; the Input network module is used to receive the input SSIM difference map C; the Backbone network module is a backbone network, used to extract image features of the SSIM difference map C; the Neck network module is used to fuse the image features output by the Backbone network module; the Head network module is used to output the fused features to obtain multi-scale target detection information of the SSIM difference map, and the multi-scale target detection information includes the location information of the marker.

[0014] As an optional solution, the multi-scale target detection information includes the category of the marker and its probability, as well as the location information of the marker; the marker is calibrated by a bounding box, and the multi-scale target detection information also includes the coordinates of the center point of the rectangular bounding box, and the width and height of the bounding box.

[0015] As an optional solution, the loss function of the YOLOV8 neural network model includes a classification loss branch and a regression loss branch;

[0016] The classification loss uses VFL Loss to calculate the global category loss. VFL Loss uses an asymmetric weighted operation. The loss function calculation formula of VFL Loss is defined as follows:

[0017]

[0018] Where p is the category confidence predicted by the model, with a value between 0 and 1; q is the category label of the detection box, which is generally between 0 and 1; γ is the focus factor; ∝ is the balance factor, with a value between [0, 1];

[0019] The regression loss is calculated as CIOU_Loss+DFL Loss. CIOU_Loss is used to calculate the IOU between the prediction box and the target box. The calculation formula of CIOU_Loss is as follows:

[0020]

[0021] in,

[0022] Where ρ is the distance between the center point of the annotation box A and the center point of the prediction box B, c is the diagonal length of the minimum enclosing rectangle of the annotation box A and the prediction box B, v is the similarity of the aspect ratio of the annotation box A and the prediction box B, α is the influencing factor of v, wl, hl are the width and height of the annotation box A, respectively, and wp, hp are the width and height of the prediction box B, respectively;

[0023] The calculation formula for defining DFL Loss is as follows:

[0024] DFL Loss = DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))(7)

[0025] In the formula, S i , S i+1 are the predicted value and the near-predicted value output by the network respectively; y, y i ,y i+1 They are the actual value of the label, the label integral value, and the adjacent label integral value.

[0026] The second aspect of the present invention discloses an electronic device, comprising a processor and a memory; the processor is used to call a computer program stored in the memory to execute the marker detection method based on the SSIM and YOLOV8 algorithms described in the first aspect of the present invention and any optional scheme.

[0027] The third aspect of the present invention discloses a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute the marker detection method based on SSIM and YOLOV8 algorithms described in the first aspect of the present invention and any optional scheme.

[0028] The present invention has the following beneficial effects:

[0029] The present invention uses the SSIM structural similarity algorithm and the deep learning YOLOV8 target detection algorithm, and uses a differential image for detection instead of the original image, which can better highlight the markers and improve the detection accuracy. The YOLOV8 target detection algorithm used has a small number of parameters and has high accuracy and running speed for small target detection tasks. Therefore, by combining the advantages of the SSIM and YOLOV8 algorithms, real-time and efficient detection of small target markers (such as plastic bottles, small stones, bricks, etc.) in the examination area can be achieved, the recognition rate of cheating behavior can be further improved, such cheating can be effectively prevented, and the fairness and security of the examination can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the original image without markers placed;

[0031] Figure 2 It is the original image where the markers are placed;

[0032] Figure 3 Yes Figure 1 The result after grayscale processing;

[0033] Figure 4 Yes Figure 2 The result after grayscale processing;

[0034] Figure 5 Yes Figure 3 and Figure 4 The calculated SSIM difference map;

[0035] Figure 6 This is a schematic diagram of the network structure of YOLOV8;

[0036] Figure 7 It is a schematic diagram of IOU;

[0037] Figure 8 is an indication that a marker is detected on the difference map. DETAILED DESCRIPTION

[0038] In order to better understand the present invention, the technical solution of the present invention is further introduced below in conjunction with the accompanying drawings and specific embodiments.

[0039] The embodiment of the present invention discloses a marker detection method based on SSIM and YOLOV8 algorithm. The method is used in the driving skill subject II test. Compared with the prior art, the method uses a differential image for detection instead of an original image for detection, which can more accurately identify the cheating behavior of the examinee and improve the fairness and security of the test. The method specifically includes the following steps:

[0040] Step S1: Collect the original image A and the image B containing the markers of the subject 2 test site.

[0041] In actual application, the original image A and the image B containing markers of the subject 2 test site can be collected through the test site camera. The original image A does not contain markers. The markers in the image B containing markers refer to objects artificially placed in the subject 2 test site to assist the subject 2 test, such as plastic bottles, small stones, bricks and other markers of different sizes and shapes.

[0042] Step S2: grayscale the acquired original image A and the image B containing the marker.

[0043] By transforming the original image A( Figure 1 shown) and the marker image B ( Figure 2 As shown in the figure, grayscale processing is performed to obtain the image A1 ( Figure 3 shown) and the marker-containing image B1 ( Figure 4 to simplify the subsequent image processing and analysis process and reduce the amount of data and computational complexity.

[0044] Step S3: Calculate the SSIM (structural similarity) of the grayscale processed image A1 without markers and the image B1 with markers to obtain the SSIM difference graph C ( Figure 5 shown).

[0045] Among them, SSIM, namely Structural Similarity Index, is an indicator used to measure the similarity between two images. The core idea of ​​SSIM is to evaluate the similarity of images by comparing three key features: brightness, contrast and structure. Among them: brightness is measured by calculating the average pixel value of the image; contrast is measured by calculating the variance of the image; structure is measured by calculating the covariance between image pixels.

[0046] SSIM is calculated for the grayscale image A1 without markers and the image B1 with markers. The calculation formula of SSIM is as follows:

[0047]

[0048] In the formula, SSIM(x,y) represents the difference between the image A1 without markers and the image B1 with markers at the same pixel position x,y. x and μ y are the mean values ​​of the unmarked image A1 and the marked image B1 at x and y, respectively, x and σ y is the variance of the unmarked image A1 and the marked image B1 at x,y, σ xyis the covariance of the unmarked image A1 and the marked image B1 at x, y. 1 and c 2 is a constant used to maintain stability and prevent the denominator from being zero. For the entire image, SSIM is the difference image between the image A1 without the marker and the image B1 with the marker. The above formula can be used to obtain the difference value of each pixel position in the image, thereby obtaining the SSIM difference image of the two images, that is, the SSIM difference image C.

[0049] MSSIM is expressed as the average difference value obtained by averaging the difference values ​​at each pixel position x, y in the difference map SSIM. The calculation formula of MSSIM is as follows:

[0050]

[0051] Where M represents the number of pixels in the difference image.

[0052] It can be understood that the value range of MSSIM is [-1, 1]. When two pictures are exactly the same, the MSSIM value is 1. The larger the MSSIM value, the more similar the two pictures are. The present invention uses the SSIM difference map C as the basis for target detection, making the marker more visually obvious and easier to detect.

[0053] Step S4: Use the trained YOLOV8 neural network model to perform target detection on the difference image C and locate the position of the marker.

[0054] like Figure 6 As shown in the figure, the neural network detection model constructed based on the YOLOV8 network (referred to as "YOLOV8 neural network model") mainly includes: Input network module, Backbone network module, Neck network module and Head network module, among which, the Input network module is mainly used to receive the input SSIM difference map C; the Backbone network module is the backbone network, which is mainly used to extract the image features of the SSIM difference map C; the Neck network module is mainly used to fuse the image features output by the Backbone network module; the Head network module is mainly used to output the fused features and obtain the multi-scale target detection information of the SSIM difference map.

[0055] Among them, the Input network module is the input part of the model. In the training stage, the RGB three-channel picture and label information are input; in the inference test stage, only the picture is input, and the model predicts the information of the corresponding target. The input picture resolution is 640*640, and the label is all the annotation box information in the current input picture, including the category of each annotation box and the coordinate position of the box. The label can be specifically expressed as [cls_id, xc, yc, w, h], where cls_id represents the category number of the annotation box. The current model only detects one category of markers, so the category number is 0, xc and yc represent the coordinate values ​​of the center point of the annotation box, and w and h represent the width and height of the annotation box.

[0056] Among them, the Backbone network module mainly adopts convolutional layer, C2f residual structure and SPPF pooling structure, and downsamples the input image five times to obtain five layers of feature maps, namely P1, P2, P3, P4 and P5. Among them, the feature maps of P3, P4 and P5 layers are used for subsequent multi-scale feature fusion, and the sizes of feature maps are 80*80*256, 40*40*512 and 20*20*1024 respectively. The C2f residual structure mainly splits the feature map, one part is directly transmitted, and the other part is convolved, normalized and activated through a series of Bottleneck blocks. The final generated feature map is concat spliced ​​with the part of the feature map that is directly transmitted, thereby further enhancing the key information extraction of the feature map. The SPPF pooling structure splices pooling features of different scales to efficiently capture multi-scale information, which helps to enhance the model's detection ability for targets of different sizes.

[0057] Among them, the Neck network module uses the Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN) to perform multi-scale feature fusion on the P3, P4, and P5 layer feature maps output by the Backbone module. Specifically, first, a top-down feature pyramid is constructed through FPN to achieve the initial fusion of multi-scale features; then, based on FPN, the bottom-up path of PAN is introduced to pass the bottom-level features to the upper layers layer by layer to further enrich the multi-scale features; finally, through lateral connections, features of different scales are fused so that the feature maps contain richer contextual information and semantic information, enhancing the model's ability to detect objects of different scales.

[0058] Among them, the Head network module is responsible for processing the three scale features output by the Neck network module respectively to generate the final detection results. The output head consists of two parts: the bounding box regressor and the classifier. The output dimension of the bounding box regressor is 4*reg_max. reg_max is a hyperparameter, which is 16. 4 represents the predicted coordinate values ​​x, y, w, h of the bounding box, which are the horizontal coordinate, vertical coordinate, width and height of the center point of the box respectively. The output dimension of the classifier is cls_num. cls_num is the number of all categories, representing the probability value of each category to which the target in the box belongs. The category corresponding to the maximum probability value is the target category. Here there is only the marker category, so cls_num = 1. The corresponding target detection result output by the model can be obtained through the predicted coordinates and category probability values. Plotting the predicted rectangular bounding box coordinates on the graph can obtain the specific position of the model predicted target on the graph; based on the output results of the classifier, the corresponding target category (Mark) and probability value (for example: 0.98) can be obtained.

[0059] The loss function of the YOLOV8 neural network model mainly includes two branches, namely: classification loss branch and regression loss branch. The classification loss uses VFL Loss to calculate the global category loss. Among them, VFL Loss uses an asymmetric weighted operation, and the loss function calculation formula of VFL Loss is defined as follows:

[0060]

[0061] Where p is the category confidence predicted by the model, which is generally between 0 and 1; q is the category label of the detection box, which is generally between 0 and 1; γ is the focus factor, which is usually set to 2; ∝ is the balance factor, which is between [0, 1].

[0062] The regression loss is calculated as CIOU_Loss + DFL Loss. CIOU_Loss is used to calculate the IOU between the predicted box and the target box. Figure 7 As shown, blue represents the annotation box A (target box), whose area is S1, gray represents the prediction box B, whose area is S2, and red represents the intersection of the prediction box B and the annotation box A, whose area is S3. IOU is the ratio of the intersection to the union, and the calculation formula is as follows:

[0063]

[0064] The calculation formula of IOU_Loss is as follows:

[0065] IOU_Loss = 1 - IOU (4)

[0066] CIOU_Loss adds the overlapping area, center point distance, and aspect ratio into the calculation. The calculation formula of CIOU is as follows:

[0067]

[0068] in,

[0069] The calculation formula of CIOU_Loss is as follows:

[0070]

[0071] Where ρ is the distance between the center point of the annotation box A and the center point of the prediction box B, c is the diagonal length of the minimum enclosing rectangle of the annotation box A and the prediction box B, v is the similarity of the aspect ratio of the annotation box A and the prediction box B, α is the influencing factor of v, wl, hl are the width and height of the annotation box A, respectively, and wp, hp are the width and height of the prediction box B, respectively.

[0072] The value range of the arctan function is 0 to Π / 2, so the value range of v is 0 to 1. When the aspect ratios of the labeled box A and the predicted box B are equal, v is 0; when the aspect ratios of the labeled box A and the predicted box B differ infinitely, v is 1. The smaller the loss function, the closer the category of the predicted box and the predicted box are to the true box.

[0073] The calculation formula of DFL Loss (Distribution Focal Loss) is defined as follows:

[0074] DFL Loss = DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 ))(7)

[0075] In the formula, S i , S i+1 are the predicted value and the near-predicted value output by the network respectively; y, y i ,y i+1 They are the actual value of the label, the label integral value, and the adjacent label integral value.

[0076] The DFL loss optimizes the probabilities of the two positions closest to the label, one on the left and one on the right, by learning the loss of the positions around the label, so that the network can focus on the distribution of the target position and the surrounding area more quickly.

[0077] Iterative training is performed based on the calculated total network loss. When the network parameter learning process converges, the training ends and a usable YOLOV8 neural network model is obtained. Specifically, during the training process, the input data reaches the output end from the model input end through forward calculation, and the current loss value is calculated by combining the loss function and label information (position coordinates, categories). Then, through the back propagation of the neural network, the gradient of each parameter to be updated is calculated according to the loss value, and the network parameter value is updated. After looping the above training steps and iterating a certain number of times, the network parameter learning process converges, and the corresponding weight file is saved for the model to use in the subsequent inference prediction mode.

[0078] It is understandable that during the network training process, the loss value decreases significantly in the early stage. After a certain number of generations of training, it almost stops decreasing and stabilizes in a small range, that is, convergence. For example, at the beginning, the loss value is 3. Before 80 generations, the loss value decreases significantly, from 3 to 0.02. After 80 generations, the loss value is almost stable and fluctuates between 0.03-0.01. Therefore, after 80 generations, it is considered convergence. These numerical standards will be different for different training data.

[0079] The embodiment of the present invention is based on the above method. Figure 1 and Figure 2 They are the original images without markers and with markers placed respectively; Figure 3 and Figure 4 They are Figure 1 and Figure 2 The grayscale processing result; through the SSIM structural similarity algorithm, Figure 3 and Figure 4 On the basis of , the similarity of the two grayscale images in terms of brightness, contrast and structure is calculated to obtain the SSIM difference image, that is, Figure 5 From the SSIM difference map, we can see that thanks to the SSIM structural similarity algorithm, the area with the marker is clearly distinguished from other areas, which provides a target feature with obvious distinction for the YOLOV8 model to detect the marker in the second stage, and plays a key role in improving the final marker detection accuracy. Use the trained YOLOV8 neural network model to detect the target on the SSIM difference map and locate the markers such as Figure 8 As shown, the position information of the marker is finally output.

[0080] Furthermore, an electronic device is disclosed in an embodiment of the present invention, which mainly includes a processor and a memory. The processor is mainly used to call a computer program stored in the memory and execute the marker detection method based on the SSIM and YOLOV8 algorithms provided in the above embodiment.

[0081] Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the marker detection method based on SSIM and YOLOV8 algorithms provided by the present invention.

[0082] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic device and program code in the computer-readable storage medium described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0083] Finally, it should be noted that although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments and application fields, and the above-mentioned specific embodiments are only illustrative and instructive, rather than restrictive. Under the guidance of this specification, a person of ordinary skill in the art can also make many forms without departing from the scope of protection of the claims of the present invention, all of which belong to the scope of protection of the present invention.

Claims

1. A marker detection method based on SSIM and YOLOV8 algorithm, characterized in that: include: Collect the original image A of the site to be tested and the image B containing the markers; The obtained original image A and the image B containing the marker are grayed out to obtain an image A1 without the marker and an image B1 containing the marker respectively; Calculate the structural similarity between the image A1 without the marker and the image B1 with the marker to obtain an SSIM difference graph C; The SSIM difference graph C is input into the trained YOLOV8 neural network model to obtain the position information of the marker.

2. The marker detection method according to claim 1, characterized in that: The calculation formula for the structural similarity between the non-marker image A1 and the marker image B1 is as follows: Where SSIM(x,y) represents the difference between the image A1 without markers and the image B1 with markers at the same pixel position x,y; μ x and μ y are the mean values ​​of the unmarked image A1 and the marked image B1 at x and y, respectively, x and σ y is the variance of the unmarked image A1 and the marked image B1 at x,y, σ xy is the covariance of the unmarked image A1 and the marked image B1 at x, y; c1 and c2 are constants.

3. The marker detection method according to claim 1 or 2, characterized in that: The YOLOV8 neural network model includes an Input network module, a Backbone network module, a Neck network module and a Head network module; the Input network module is used to receive an input SSIM difference map C; the Backbone network module is a backbone network, used to extract image features of the SSIM difference map C; the Neck network module is used to fuse the image features output by the Backbone network module; the Head network module is used to output the fused features to obtain multi-scale target detection information of the SSIM difference map, and the multi-scale target detection information includes the location information of the marker.

4. The marker detection method according to claim 3, characterized in that: The multi-scale target detection information includes the category of the marker and its probability, and the position information of the marker; the marker is calibrated by a bounding box, and the multi-scale target detection information also includes the coordinates of the center point of the rectangular bounding box, and the width and height of the bounding box.

5. The marker detection method according to claim 1 or 2, characterized in that: The loss function of the YOLOV8 neural network model includes a classification loss branch and a regression loss branch; The classification loss uses VFL Loss to calculate the global category loss. VFL Loss uses an asymmetric weighted operation. The loss function calculation formula of VFL Loss is defined as follows: Where p is the category confidence predicted by the model, with a value between 0 and 1; q is the category label of the detection box, which is generally between 0 and 1; γ is the focus factor; ∝ is the balance factor, with a value between [0, 1]; The regression loss is calculated as CIOU_Loss+DFL Loss. CIOU_Loss is used to calculate the IOU between the prediction box and the target box. The calculation formula of CIOU_Loss is as follows: in, Where ρ is the distance between the center point of the annotation box A and the center point of the prediction box B, c is the diagonal length of the minimum enclosing rectangle of the annotation box A and the prediction box B, v is the similarity of the aspect ratio of the annotation box A and the prediction box B, α is the influencing factor of v, wl, hl are the width and height of the annotation box A, respectively, and wp, hp are the width and height of the prediction box B, respectively; The calculation formula for defining DFL Loss is as follows: DFL Loss=DFL(S i ,S i+1 )=-((y i+1 -y)log(S i )+(y-y i )log(S i+1 ))(7) In the formula, S i , S i+1 are the predicted value and the near-predicted value output by the network respectively; y, y i ,y i+1 They are the actual value of the label, the label integral value, and the adjacent label integral value.

6. An electronic device, characterized in that: It comprises a processor and a memory; the processor is used to call the computer program stored in the memory to execute the marker detection method based on the SSIM and YOLOV8 algorithms described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the marker detection method based on SSIM and YOLOV8 algorithm as described in any one of claims 1 to 4.

Images