Target identification method of vehicle-to-vehicle OCC system based on improved YOLOv8

Abstract

The invention provides a vehicle-to-vehicle OCC system target identification method based on an improved YOLOv8, and the method comprises the steps: firstly carrying out the preprocessing of original image data, and carrying out the improvement on the basis of the conventional YOLOv8; the original SPPF is replaced by CAIFI in a backbone network, and an EC2f module is introduced to extract feature information of an OCC system target; a BIFPN module is introduced and a weighting function of the BIFPN module is changed to carry out feature fusion and delete a feature network which contributes little to an OCC system target, so that the purposes of simplifying the network and combining information of high-level features and information of a bottom layer are achieved; a ShapeIoU function is adopted as a frame loss function, and the boundary of the LED array is positioned by considering the shape and the scale of the target LED array of the OCC system; according to the method, high recognition precision can be achieved, high-accuracy recognition can be carried out on the LED array, the moving LED can be well recognized, and people are helped to better decode the subsequent LED array and extract information, such as license plate information and fault information, contained in the LED array.

Description

Technical Field

[0001] The present invention belongs to the technical field of visible light communication and intelligent transportation, and in particular relates to a vehicle-to-vehicle OCC system target recognition method based on an improved YOLOv8. Background Art

[0002] With the popularity of the Internet of Things and the growth of digital services such as social media and video content, the demand for mobile data communications is growing excessively. To meet this exponentially growing demand, bandwidth should be increased or spectral efficiency should be improved. However, the improvement of spectral efficiency is slow and cannot meet this insatiable demand. Exploiting new spectrum becomes a unique solution, and a wider bandwidth of the Terahertz (THz) spectrum is required. In this context, Optical Wireless Communication (OWC) has attracted great research interest from researchers in recent years due to its many ideal properties, including a large available spectrum (from 350nm to 1550nm), high energy efficiency, independent regulation, and well-controlled communication security.

[0003] Visible Light Communications (VLC), as an optical wireless communication technology, has been widely studied for its application scenarios and methods. Optical Camera Communication (OCC) is a type of VLC that uses a camera with an image sensor as a receiver. OCC has several unique advantages. First, the system is simple to implement; second, cameras are used in most smart devices such as smartphones, cars, and smart home kits, so general devices with cameras can be manufactured as receivers for OCC systems; in addition, through image processing, coupled with the spatial separation capability of the information system, the noise light source in the receiver can be effectively suppressed, which makes OCC a viable option for outdoor communications such as vehicle-to-infrastructure, vehicle-to-vehicle, and vehicle-to-pedestrian communication scenarios.

[0004] In recent years, intelligent transportation systems have become a research hotspot. Accurate vehicle positioning is the key to supporting safe driving, emergency braking, autonomous navigation systems, etc. Self-driving cars are intelligent cars that use computer systems to achieve unmanned driving. Self-driving is the main direction of the development of intelligence and networking in the field of automobiles and transportation in the world. OCC technology uses LED as the transmitter and camera as the receiver, with low deployment cost. It also has the advantages of low latency, no spectrum certification, no electromagnetic radiation, no radio frequency interference, and green environmental protection. In underground parking lots or other areas without GPS signals, OCC technology can be used to transmit vehicle information.

[0005] OCC technology is considered to be a powerful supplement to the current communication technology for autonomous driving, but the current research on OCC is not mature enough, and the existing research on OCC system target recognition methods can be divided into two categories. One is based on traditional image recognition methods, such as connected domain analysis algorithm, FAST corner detection algorithm, edge detection algorithm, etc. These algorithms identify the target by simply judging the shape, position, and brightness of the system target; the other is to use deep learning methods, using the advantages of deep learning such as strong learning ability, wide coverage, good adaptability, data-driven, high upper limit, and good portability, to comprehensively extract features for system target recognition, such as Mask Rcnn, YOLO, etc. The image recognition part uses traditional image processing algorithms, which are easily affected by the binary threshold and cannot adapt to complex environmental backgrounds.

[0006] The YOLO algorithm is a single-stage target detection algorithm in the field of deep learning, which is widely used in target detection and recognition. YOLOv8 mainly draws on the design advantages of models such as YOLOv5, YOLOv6, and YOLOX. YOLOv8 is a SOTA model. It is built on the basis of historical versions of the YOLO series and introduces new features and improvements to further improve performance and flexibility, making it the best choice for tasks such as target detection, image segmentation, and pose estimation. Its specific innovations include a new backbone network, a new Ancher-Free detection head, and a new loss function, which can run on a variety of hardware platforms from CPU to GPU. Although YOLOv8 has good recognition effects in OCC system target recognition, its recognition ability still has room for improvement.

[0007] In order to further promote the practical application of OCC in autonomous vehicle scenarios, it is necessary to improve the target recognition method of the vehicle-to-vehicle OCC system of YOLOv8 to solve the accurate recognition of LED arrays in complex outdoor environments and complete relevant technical verification on the actual measurement system. Summary of the invention

[0008] In order to overcome the problems existing in the prior art, the present invention provides a vehicle-to-vehicle OCC system target recognition method based on an improved YOLOv8.

[0009] The present invention is achieved by adopting the following technical solutions:

[0010] An improved YOLOv8 vehicle-to-vehicle OCC system target recognition method obtains original image data, preprocesses the original image data, obtains preprocessed image data, and modifies YOLOv8: CAIFI replaces SPPF in the backbone network, combines the head network with BIFPN, changes the BIFPN weight function, and uses the Shape_IoU loss function as the head network to build an improved YOLOv8 LED array detection model; the preprocessed image is input into the improved YOLOv8 OCC system target detection model for calculation, and the LED array detection result is obtained to achieve the purpose of accurately identifying the LED array. The specific steps of this method are as follows:

[0011] Step 1: Build LED array dataset:

[0012] Process (1) Collecting LED array data set in outdoor environment;

[0013] During the data collection process, the position of the camera remains unchanged, and the LED array moves within the range of 0m-100m from the camera. At the same time, one or some LED lights in the LED array that can light up and turn off independently will light up or turn off randomly, and the LED array data set under different distances and lighting conditions is obtained. During the collection process, the LED array is tilted during the movement to simulate the bumpy phenomenon when the actual car is driving, and the LED data sets at different angles are obtained;

[0014] Process (2) uses the LabelImg platform to manually annotate the LED array dataset. The annotation information consists of two parts: category and position. The category of the LED array is marked as "Led array"; the real position coordinate information of the LED array is marked as (x min ,y min ,x max ,y max ), draw the real position coordinate information in the form of a rectangular frame on the collected image, and record the rectangular frame as the real target frame, where (x min ,y min ) represents the coordinate of the upper left corner of the LED array, (x max ,y min ) represents the coordinate of the upper right corner of the LED array, (x min ,y max ) represents the coordinate of the lower left corner of the LED array, (x max ,y max ) represents the coordinate of the lower right corner of the LED array;

[0015] Process (3) divides the labeled data set into three parts: training set, validation set, and test set for model training and testing. The training set accounts for 70% of the entire data set and is used to train the model and learn the features in the data. The validation set accounts for 20% of the entire data set and is used to verify the performance and generalization ability of the model and adjust and optimize the model. The test set accounts for 10% of the entire data set and is used to evaluate the final performance and accuracy of the model.

[0016] Step 2: Change the YOLOv8 feature extraction module

[0017] Three different methods are used to extract features from feature maps at different locations:

[0018] (1) Basic C2f module for feature extraction

[0019] The C2f module consists of a CBS convolutional layer, a Split layer, and a BottleNeck. BottleNeck uses a residual connection to directly add the input LED array features to the output LED array features, alleviating the gradient vanishing and gradient exploding problems, and allowing information transfer between layers to avoid deep network degradation. The C2f module is used to extract low-level features of the target LED array of the car-to-car OCC system.

[0020] (2) EC2f module for feature extraction

[0021] The EC2f module consists of the C2f module and the ECA attention mechanism. The EC2f module performs global average pooling on the feature map processed by the C2f module, compresses the spatial dimension to 1x1, and obtains a channel-dimensional vector. The output is activated by the Sigmoid function:

[0022]

[0023] Where x is the input feature vector after global average pooling. The compressed vector is processed by the Sigmoid activation function to generate a weight vector between 0 and 1. Each channel of the original input vehicle-to-vehicle OCC system target LED array feature map is multiplied by the corresponding weight to obtain the weighted feature map f E :

[0024]

[0025] where y c For the feature map after C2f processing, the ECA module avoids the complex dimensionality reduction and dimensionality increase process through global average pooling and weighted processing, thereby achieving high efficiency and lightweight characteristics, and uses the EC2f module to extract mid-level features;

[0026] (3) CAIFI module for feature extraction

[0027] Replace the SPPF of the original YOLOv8 model with CAIFI. CAIFI reads the width and height of the feature map (image features) of the input LED array image C and converts the input feature map f E Flatten the image and transpose it using the permute function to get the flattened input feature map and record it as srcf. CAIFI will generate positional encoding information (positional encoding) and use srcf and pos_embed as the input of the encoder layer (Encoder layer). The encoder layer includes multi-head self-attention, linear layer, and random inactivation layer (Dropout). The specific workflow is as follows:

[0028] v=srcf

[0029] q=k=src+pos_emb

[0030] v, q, k are used as Multi-Head Self-Attention inputs and are processed through self-attention to obtain srca. After srca is subjected to dropout processing, it is added (ADD) to srcf to obtain srcl. Srcl is normalized (Norm), passes through FFN (fully connected layer, activation layer, pooling layer), and then subjected to dropout processing. Cross-layer connection is performed, and srcl and srca are subjected to ADD&Norm (addition and normalization) to obtain the result of Encoderlayer output. The output feature map has the same size as the input feature map. The result is permuted and processed with contiguous function to obtain [B, C, H, W], and then passes through GELU activation function;

[0031] f G =GELU(x)=x*P(X≤x)=x*Φ(x)

[0032]

[0033]

[0034] or:

[0035] GELU(x)=x*σ(1.072x)

[0036] Among them, Φ(x) is the cumulative distribution of Gaussian normal distribution, σ(.) is the Sigmoid function;

[0037] The feature map f obtained after GELU activation function processing G As input, f G After the convolutional layer processing, the number of channels is converted from 512 to 128, which is suitable for subsequent feature fusion operations, and the CAIFI module is used for high-level feature extraction;

[0038] Step 3: Change the YOLOv8 feature fusion module

[0039] The BIFPN module is introduced into the Neck network. The BIFPN module consists of multiple repeated network layers, each of which contains multiple feature map nodes and adopts a multi-scale fusion approach.

[0040] The input features of different LED arrays have different resolutions, and they usually contribute unequally to the output features; the basic YOLOv8 algorithm simply sums up the feature maps, such as using concat or shortcut connections, without distinguishing the feature maps added at the same time. The introduction of BIFPN adds an additional weight to each input and allows the network to learn the importance of each input feature;

[0041] Modify the weight function of the basic BIFPN, the expression of weight O:

[0042]

[0043] Where w is a learnable weight parameter, which is initialized to a vector of all 1s. t is an introduced learnable smoothing parameter, which is initially set to 2. j is the input category with a value of 2. i is the input feature vector dimension with a value of 3. ε is set to 0.0001 to avoid numerical instability. I i is the input feature map after feature extraction;

[0044] Step 4: Change the YOLOv8 prediction loss function

[0045] The input OCC system target LED array feature map is passed through the Neck network to obtain 80*80, 40*40 and 20*20 feature maps, which are sent to the detection head (Detect) for detection processing. After processing, the location information, type and confidence of the OCC system target LED array are obtained; IoU (Intersection over Union) is used to measure the accuracy of the bounding box; the basic YOLOv8 uses DFL Loss+CIOU Loss as regression loss, which can enhance the generalization and flexibility of the YOLOv8 model in complex situations, such as occlusion and moving objects; however, the regression loss function currently used only considers the overlap between bounding boxes, and does not pay enough attention to the shape and scale of the bounding box;

[0046] Shape_IoU is used as the new loss function, and its loss function is:

[0047] L Shape-IoU =1-IoU+distance shape +P×Ω shape

[0048]

[0049] distance shape =hh×(x c -x c gt ) 2 / c 2 +ww×(y c -y c gt ) 2 ) / c 2

[0050]

[0051] Wherein, B is the real position information of the target LED array of the OCC system marked in step 1 process (2), B gt The position information of the target LED array of the OCC system predicted by the improved YOLOv8. The gt boxes are B and B gt The overlapping area between the two is used to set the parameters for the vehicle-to-vehicle OCC system target recognition. The scale is set to 0.2, P is set to 0.4, ww is the weight coefficient in the horizontal direction, and hh is the weight coefficient in the vertical direction. The values ​​of ww and hh have the following relationship with the gt box:

[0052]

[0053]

[0054] The width of B is w1 and the height is h1; gt The width is w2 and the height is h2; w and h are calculated based on the width w1 and w2 and the height h1 and h2 of the bounding box, using the following formula:

[0055]

[0056]

[0057] Although the recognized OCC target array is approximately a square, it will be bumpy in the actual process, causing the aspect ratio to change. Therefore, more influencing factors need to be considered. By introducing ww, hh and scale, the influence of the shape and scale of the bounding box on the recognition effect is reduced;

[0058] Step 5: Input the training set images into the improved YOLOv8 model for training and optimization until the model converges:

[0059] Input the training set images into the improved YOLOv8 model for training and optimizing the model until convergence:

[0060] Process (1) Build a target recognition model based on the improved YOLOv8 network to extract image feature matrix and predict target position. The target recognition model built based on the improved YOLOv8 network includes Backbone network and Neck network; Backbone network: Feature Extractor extracts features, and its function is to extract information from the image for use by the subsequent network; Neck network: placed between the Backbone network and the Head network to better utilize the features extracted by the Backbone, playing the role of "feature fusion"; Head network: uses the previously extracted features to make recognition;

[0061] First, the training set data (Input) is fed into the Backbone network, and the input module uses the Mosaic method to enhance the image data:

[0062] Randomly select the coordinates of the image stitching reference point (x c ,y c ), where x c ∈(0,1280),y c ∈(0,1280); select four images from the training set and randomly scale them to 0.5-2.0 times the original image size; (x c ,y c) is the center, and the four images are stitched into an image A with a size of 1280×1280 in the order of upper left, upper right, lower left, and lower right; image A is adaptively scaled to fix the size of image A to 640×640; the Mosaic operation is turned off for the last 10 epochs;

[0063] Performing Letterbox image data enhancement on the image data enhanced by the Mosaic method, scaling the long side of the image data enhanced by the Mosaic method to a specified size, and filling the short side with black pixels to obtain the preprocessed image;

[0064] The convolution module in the feature extraction of the Neck network uses CBS and C2f. CBS consists of a two-dimensional convolution + two-dimensional BatchNorm + SiLU activation function, and C2f consists of a CBS convolution layer + a split + n Bottleneck series. BottleNeck uses a residual connection to directly add the input LED array and the output LED array features to alleviate the gradient vanishing and gradient exploding problems, and allows information transfer between layers to avoid the degradation of deep networks.

[0065] The operation of extracting features from the input image is performed in the Backbone network; it extracts low-level to high-level features of the image layer by layer, gradually reduces the spatial resolution and increases the number of channels to obtain a more abstract and semantically rich feature representation, and sends the input LED array image C to the backbone network. Image C passes through multiple convolutional layers and 4 C2f feature extraction modules in sequence; the specific process is: after passing through two convolutional layers and a C2f structure, it passes through a convolutional layer and a C2f structure in sequence to extract the feature matrix a1 in image C, a1 continues to be transmitted downward, passes through a convolutional layer and a C2f structure in sequence to extract the feature matrix a2; a2 continues to be transmitted downward, passes through a convolutional layer, a C2f structure in sequence to extract the feature matrix, and the CAIFI module extracts the feature matrix a3;

[0066] Then, the feature matrices a1, a2 and a3 extracted by the feature extraction module are sent to the Neck network for feature fusion processing. The feature fusion module uses the BIFPN module to fuse the extracted feature matrices; the feature matrix of a3 after a convolution layer is recorded as d1, d1 is upsampled (Upsample), and then processed with a2 through the BIFPN module to obtain the feature matrix b1, b1 is passed through the C2f module to obtain c1, c1 is passed through a convolution layer to obtain d2, d2 is upsampled and fused with a1 through the BIFPN module to obtain the feature matrix b2, b2 is passed through the C2f module to obtain c2, and then passed through a convolution layer to obtain d3, d3 and c1 are processed through the BIFPN module to obtain the feature matrix b3, b3 After passing through the C2f module, c3 is obtained. After passing through a convolution layer, c3 is obtained to obtain d4. d4 and a3 are processed by the BIFPN module to obtain the feature matrix b4. b4 is processed by the C2f module to obtain c4. Finally, the feature matrices c2, c3 and c4 fused by the feature fusion module are sent to the Head network. c4 is sent to the Conv layer to output k1 prediction results on the scale of 80×80. c3 is sent to the Conv layer to output k2 prediction results on the scale of 40×40. c2 is sent to the Conv layer to output k3 prediction results on the scale of 20×20. The prediction results include the predicted position coordinate information of the LED array (x1, y1, w1, h1), (x2, y2, w2, h2), (x3, y3, w3, h3), ..., (x k ,y k ,w k ,h k ), confidence p1, p2, p3, …, p k and predicted categories φ1, φ2, φ3, …, φ k , where k = k1 + k2 + k3, the predicted position coordinate information is drawn on the image in the form of a rectangular box, and the rectangular box is recorded as the predicted target box;

[0067] Establish a loss function L, use the loss function L to calculate the prediction results output by the model, and get the error of the prediction results. The calculation formula is:

[0068] L=L cls +L bbox

[0069] Where, L cls represents the classification loss function, L bbox represents the bounding box loss function;

[0070] Classification loss L cls The calculation formula is:

[0071]

[0072] The classification loss is used to evaluate the quality of the model prediction results. In the formula, N is the number of samples collected in the data set in step 1 (1); M is the number of categories, which is 2; y ic is the value of the label, y ic Indicates whether the predicted category in the predicted target box is "Led array", that is When the predicted category in the predicted target box is "Led array", y ic =1, when the prediction category in the prediction target box is not "Led array", y ic =0; p ic is the probability obtained after the Softmax function;

[0073]

[0074] where z k and z m is a set of feature vectors obtained after processing in steps 2 and 3. is the sum of all eigenvectors;

[0075] The bounding box loss uses Shape_IoU as a new loss function, and its loss function is:

[0076] L Shape-IoU =1-IoU+distance shape +P×Ω shape

[0077]

[0078] distance shape =hh×(x c -x c gt ) 2 / c 2 +ww×(y c -y c gt ) 2 ) / c 2

[0079]

[0080] Wherein, B is the real position information of the target LED array of the OCC system marked in step 1 process (2), B gt The location information predicted by the improved YOLOv8, gt boxes are B and B gt The overlapping area between them is used to set the parameters for the vehicle-to-vehicle OCC system target recognition, with scale set to 0.2 and P set to 0.4;

[0081] Although the recognized OCC target array is approximately a square, it will be bumpy in the actual process, causing the aspect ratio to change. Therefore, more influencing factors need to be considered. By introducing ww, hh and scale, the influence of the shape and scale of the bounding box on the recognition effect is reduced;

[0082] Through the process of step 5, we can obtain the prediction results at three different scales: (x1, y1, w1, h1), (x2, y2, w2, h2), (x3, y3, w3, h3), ..., (x k ,y k ,w k ,h k )、p1,p3,p3,…,p k and φ1, φ2, φ3, …, φ k After that, the error loss function L of the prediction result is calculated by the previous loss function L i , which is the loss on the training set; back propagation applies the chain rule to calculate L i The gradient value of the model parameters is updated according to the gradient value, and the loss function L is calculated again in step 5. i+1 , if L i+1 <L i , then save the updated YOLOv8 model parameters, L i+1 >L i , the YOLOv8 model parameters are not updated, iterate 300 times, and save the last updated YOLOv8 model parameters, which are recorded as the recognition model;

[0083] The recognition model is used to recognize the test set images in step 1 process (3). The size of the test set image T is fixed to 640×640 by the Mosaic method; after C2f in step 2, a feature map of 80×80 is obtained. EC2f obtains a 40×40 feature map And the CAIFI feature extraction module to obtain a 20×20 feature map After the BIFPN feature fusion in step 3, a 128×128 feature map is obtained. 256×256 feature map And a 512×512 feature map Will After step five, the loss function is obtained; the output value of the loss function is the recognition result.

[0084] Compared with the prior art, the present invention has the following beneficial effects:

[0085] The invention is innovatively based on the YOLOv8 target detection algorithm; Step 2 introduces the C2f module, EC2f module and CAIFI module into YOLOv8 as feature extraction modules, and utilizes multiple modules to perform more effective feature extraction; Step 3 introduces the BIFPN module and changes its weight function for feature fusion, and improves the feature fusion effect of the target feature map of the vehicle-to-vehicle OCC system by changing the weight function; Step 4 changes the YOLOv8 prediction boundary box loss function, taking into account the influence of the shape and scale of the vehicle-to-vehicle OCC system target boundary box on the recognition effect; and is applied to the vehicle-to-vehicle OCC system target LED array recognition, and a large number of experiments on the data set of the recognition model constructed in step 5 show that the improved YOLOv8 algorithm has good performance in the recognition of the vehicle-to-vehicle OCC system target LED array. Using the same picture for prediction, the improved Yolov8 algorithm has a certain improvement at different angles and distances, and the recognition accuracy improvement is particularly obvious when the distance is far, which can meet the needs of long-distance accurate recognition and has a wider range of applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 A schematic diagram of a hardware scenario based on an improved YOLOv8 vehicle-to-vehicle OCC system target recognition method provided by the present invention.

[0087] Figure 2 Schematic diagram of the original YOLOv8 structure in the prior art.

[0088] Figure 3 A schematic diagram of the structure of YOLOv8 of a vehicle-to-vehicle OCC system target recognition method based on an improved YOLOv8 provided by the present invention.

[0089] Figure 4 A flowchart of a vehicle-to-vehicle OCC system target recognition method based on an improved YOLOv8 provided by the present invention.

[0090] Figure 5 This is a flow chart of the C2f feature extraction module introduced in step 2 of this method.

[0091] Figure 6 This is a flowchart of the EC2f feature extraction module introduced in step 2 of this method.

[0092] Figure 7 This is a flow chart of the CAIFI feature extraction module introduced in step 2 of this method. DETAILED DESCRIPTION

[0093] like Figure 1The figure shows a schematic diagram of a hardware scenario based on an improved YOLOv8 vehicle-to-vehicle OCC system target recognition method provided by the present invention. The front vehicle uses the LED array in the taillight to send light signals as the transmitting end, and the rear vehicle uses the sCMOS camera to receive light signals as the receiving end to realize communication between the front vehicle and the rear vehicle. The front vehicle modulates its own speed, position, brake and other information through coding and modulation, and then controls the LED array to send a light-off signal through the driving amplifier circuit, and sends it to the optical channel for transmission. After the rear vehicle uses the sCMOS camera to collect the light signal sent by the LED array, it restores the image. The improved YOLOv8 vehicle-to-vehicle OCC system target recognition method can still achieve high-precision recognition in the presence of interference. According to the image of the identified transmitting end, the information in the LED array is restored to obtain the speed, position, brake and other information of the front vehicle.

[0094] like Figure 2 The following is a schematic diagram of the original YOLOv8 structure. Figure 3 The figure shows the structure diagram of the target recognition method of the vehicle-to-vehicle OCC system of the improved YOLOv8 provided by the present invention. Figure 2 The EC2f feature extraction module was introduced, CAIFI replaced the original SPPF, and the BIFPN module was introduced.

[0095] like Figure 4 The figure shows a flow chart of the vehicle-to-vehicle OCC system target recognition method of the improved YOLOv8 provided by the present invention, and a vehicle-to-vehicle OCC system target recognition model is constructed by improving feature extraction and feature fusion to perform OCC system target recognition.

[0096] The present invention is described in detail below in conjunction with the accompanying drawings:

[0097] The present invention provides a vehicle-to-vehicle OCC system target recognition method for improving YOLOv8, obtains original image data, preprocesses the original image data, obtains preprocessed image data, and modifies YOLOv8: replaces SPPF in the backbone network with CAIFI, combines the head network with BIFPN, and uses the Shape_IoU loss function as the head network to construct an improved YOLOv8 LED array detection model; inputs the preprocessed image into the improved YOLOv8 OCC system target detection model for calculation, obtains the LED array detection result, and achieves the purpose of accurately identifying the LED array. The specific steps of the method are as follows:

[0098] Step 1: Build LED array dataset:

[0099] Process (1) Collecting LED array data set in outdoor environment;

[0100] During the data collection process, the position of the camera remains unchanged, and the LED array moves within the range of 0m-100m from the camera. At the same time, one or some LED lights in the LED array that can light up and turn off independently will light up or turn off randomly, and the LED array data set under different distances and lighting conditions is obtained. During the collection process, the LED array is tilted during the movement to simulate the bumpy phenomenon when the actual car is driving, and the LED data sets at different angles are obtained;

[0101] Process (2) uses the LabelImg platform to manually annotate the LED array dataset. The annotation information consists of two parts: category and position. The category of the LED array is marked as "Led array"; the real position coordinate information of the LED array is marked as (x min ,y min ,x max ,y max ), draw the real position coordinate information in the form of a rectangular frame on the collected image, and record the rectangular frame as the real target frame, where (x min ,y min ) represents the coordinate of the upper left corner of the LED array, (x max ,y min ) represents the coordinate of the upper right corner of the LED array, (x min ,y max ) represents the coordinate of the lower left corner of the LED array, (x max ,y max ) represents the coordinate of the lower right corner of the LED array;

[0102] Process (3) divides the labeled data set into three parts: training set, validation set, and test set for model training and testing. The training set accounts for 70% of the entire data set and is used to train the model and learn the features in the data. The validation set accounts for 20% of the entire data set and is used to verify the performance and generalization ability of the model and adjust and optimize the model. The test set accounts for 10% of the entire data set and is used to evaluate the final performance and accuracy of the model.

[0103] Step 2: Change the YOLOv8 feature extraction module

[0104] Three different methods are used to extract features from feature maps at different locations:

[0105] (1) Basic C2f module for feature extraction

[0106] like Figure 5The C2f module shown in the figure is composed of CBS convolutional layer, Split layer and BottleNeck. BottleNeck uses residual connection to directly add the input LED array and output LED array features, which alleviates the gradient vanishing and gradient exploding problems and allows information transfer between layers to avoid the deep network degradation phenomenon. The C2f module is used to extract low-level features of the target LED array of the car-to-car OCC system.

[0107] (2) EC2f module for feature extraction

[0108] like Figure 6 The EC2f module shown is composed of a C2f module and an ECA attention mechanism. The EC2f module performs global average pooling on the feature map processed by the C2f module, compresses the spatial dimension to 1x1, and obtains a channel-dimensional vector. The obtained output is activated by the Sigmoid function:

[0109]

[0110] Where x is the input feature vector after global average pooling. The compressed vector is processed by the Sigmoid activation function to generate a weight vector between 0 and 1. Each channel of the original input vehicle-to-vehicle OCC system target LED array feature map is multiplied by the corresponding weight to obtain the weighted feature map f E :

[0111]

[0112] where y c For the feature map after C2f processing, the ECA module avoids the complex dimensionality reduction and dimensionality increase process through global average pooling and weighted processing, thereby achieving high efficiency and lightweight characteristics, and uses the EC2f module to extract mid-level features;

[0113] (3) CAIFI module for feature extraction

[0114] Replace the SPPF of the original YOLOv8 model with CAIFI, such as Figure 7 The CAIFI shown reads the width and height of the feature map of the input LED array image C and converts the input feature map f E Flatten the image and transpose it using the permute function to get the flattened input feature map and record it as srcf. CAIFI will generate position encoding information and use srcf and pos_embed as the input of the encoding layer. The encoding layer includes multi-head attention, linear layer, and random inactivation layer. The specific workflow is as follows:

[0115] v=srcf

[0116] q=k=src+pos_emb

[0117] v, q, k are used as Multi-Head Self-Attention inputs and are processed through self-attention to obtain srca. After srca is subjected to dropout processing, it is added to srcf to obtain srcl. Srcl is normalized, passes through the fully connected layer, activation layer, and pooling layer, and then subjected to dropout processing. Cross-layer connection is performed, srcl is added to srca, and then normalized to obtain the result of Encoderlayer output. The output feature map has the same size as the input feature map. The result is permuted and processed with the contiguous function to obtain [B, C, H, W], and then passes through the GELU activation function;

[0118] f G =GELU(x)=x*P(X≤x)=x*Φ(x)

[0119]

[0120]

[0121] or:

[0122] GELU(x)=x*σ(1.072x)

[0123] Among them, Φ(x) is the cumulative distribution of Gaussian normal distribution, σ(.) is the Sigmoid function;

[0124] The feature map f obtained after GELU activation function processing G As input, f G After the convolutional layer processing, the number of channels is converted from 512 to 128, which is suitable for subsequent feature fusion operations, and the CAIFI module is used for high-level feature extraction;

[0125] Step 3: Change the YOLOv8 feature fusion module

[0126] The BIFPN module is introduced into the Neck network. The BIFPN module consists of multiple repeated network layers, each of which contains multiple feature map nodes and adopts a multi-scale fusion approach.

[0127] The input features of different LED arrays have different resolutions, and they usually contribute unequally to the output features. The introduction of BIFPN adds an additional weight to each input and allows the network to learn the importance of each input feature.

[0128] Modify the weight function of the basic BIFPN, the expression of weight O:

[0129]

[0130] Where w is a learnable weight parameter, which is initialized to a vector of all 1s. t is an introduced learnable smoothing parameter, which is initially set to 2. j is the input category with a value of 2. i is the input feature vector dimension with a value of 3. ε is set to 0.0001 to avoid numerical instability. I i is the input feature map after feature extraction;

[0131] Step 4: Change the YOLOv8 prediction loss function

[0132] The input OCC system target LED array feature map is passed through the Neck network to obtain 80*80, 40*40 and 20*20 feature maps, which are sent to the detection head for detection processing. After processing, the location information, type and confidence of the OCC system target LED array are obtained; IoU is used to measure the accuracy of the bounding box; the currently used regression loss function only considers the overlap between bounding boxes, and does not pay enough attention to the shape and scale of the bounding box;

[0133] Shape_IoU is used as the new loss function, and its loss function is:

[0134] L Shape-IoU =1-IoU+distance shape +P×Ω shape

[0135]

[0136] distance shape =hh×(x c -x c gt ) 2 / c 2 +ww×(y c -y c gt ) 2 ) / c 2

[0137]

[0138] Wherein, B is the real position information of the target LED array of the OCC system marked in step 1 process (2), B gt The position information of the target LED array of the OCC system predicted by the improved YOLOv8. The gt boxes are B and B gtThe overlapping area between the two is used to set the parameters for the vehicle-to-vehicle OCC system target recognition. The scale is set to 0.2, P is set to 0.4, ww is the weight coefficient in the horizontal direction, and hh is the weight coefficient in the vertical direction. The values ​​of ww and hh have the following relationship with the gt box:

[0139]

[0140]

[0141] The width of B is w1 and the height is h1; gt The width is w2 and the height is h2; w and h are calculated based on the width w1 and w2 and the height h1 and h2 of the bounding box, using the following formula:

[0142]

[0143]

[0144] Although the recognized OCC target array is approximately a square, it will be bumpy in the actual process, causing the aspect ratio to change. Therefore, more influencing factors need to be considered. By introducing ww, hh and scale, the influence of the shape and scale of the bounding box on the recognition effect is reduced;

[0145] Step 5: Input the training set images into the improved YOLOv8 model for training and optimization until the model converges:

[0146] Input the training set images into the improved YOLOv8 model for training and optimizing the model until convergence:

[0147] Process (1) Build a target recognition model based on the improved YOLOv8 network to extract image feature matrix and predict target position. The target recognition model built based on the improved YOLOv8 network includes Backbone network and Neck network; Backbone network: Feature Extractor extracts features, and its function is to extract information from the image for use by the subsequent network; Neck network: placed between the Backbone network and the Head network to better utilize the features extracted by the Backbone, playing the role of "feature fusion"; Head network: uses the previously extracted features to make recognition;

[0148] First, the training set data is fed into the Backbone network, and the input module uses the Mosaic method to enhance the image data:

[0149] Randomly select the coordinates of the image stitching reference point (x c ,y c ), where x c∈(0,1280),y c ∈(0,1280); select four images from the training set and randomly scale them to 0.5-2.0 times the original image size; (x c ,y c ) is the center, and the four images are stitched into an image A with a size of 1280×1280 in the order of upper left, upper right, lower left, and lower right; image A is adaptively scaled to fix the size of image A to 640×640; the Mosaic operation is turned off for the last 10 epochs;

[0150] Performing Letterbox image data enhancement on the image data enhanced by the Mosaic method, scaling the long side of the image data enhanced by the Mosaic method to a specified size, and filling the short side with black pixels to obtain the preprocessed image;

[0151] The convolution module in the feature extraction of the Neck network uses CBS and C2f. CBS consists of a two-dimensional convolution + two-dimensional BatchNorm + SiLU activation function, and C2f consists of a CBS convolution layer + a split + n Bottleneck series. BottleNeck uses a residual connection to directly add the input LED array and the output LED array features to alleviate the gradient disappearance and gradient explosion problems, and allows information transfer between layers to avoid the degradation of deep networks.

[0152] The operation of extracting features from the input image is performed in the Backbone network; it extracts low-level to high-level features of the image layer by layer, gradually reduces the spatial resolution and increases the number of channels to obtain a more abstract and semantically rich feature representation, and sends the input LED array image C to the backbone network. Image C passes through multiple convolutional layers and 4 C2f feature extraction modules in sequence; the specific process is: after passing through two convolutional layers and a C2f structure, it passes through a convolutional layer and a C2f structure in sequence to extract the feature matrix a1 in image C, a1 continues to be transmitted downward, passes through a convolutional layer and a C2f structure in sequence to extract the feature matrix a2; a2 continues to be transmitted downward, passes through a convolutional layer, a C2f structure in sequence to extract the feature matrix, and the CAIFI module extracts the feature matrix a3;

[0153] Then, the feature matrices a1, a2 and a3 extracted by the feature extraction module are sent to the Neck network for feature fusion processing. The feature fusion module uses the BIFPN module to fuse the extracted feature matrices; the feature matrix of a3 after a convolution layer is recorded as d1, d1 is upsampled, and processed with a2 through the BIFPN module to obtain the feature matrix b1, b1 is passed through the C2f module to obtain c1, c1 is passed through a convolution layer to obtain d2, d2 is upsampled, and feature fused with a1 through the BIFPN module to obtain the feature matrix b2, b2 is passed through the C2f module to obtain c2, and then passed through a convolution layer to obtain d3, d3 and c1 are processed through the BIFPN module to obtain the feature matrix b3, b3 is passed through the C2f module After the module, c3 is obtained. After c3 passes through a convolution layer, d4 is obtained. d4 and a3 are processed by the BIFPN module to obtain the feature matrix b4. b4 passes through the C2f module to obtain c4. Finally, the feature matrices c2, c3 and c4 fused by the feature fusion module are sent to the Head network; c4 is sent to the Conv layer to output k1 prediction results on the scale of 80×80; c3 is sent to the Conv layer to output k2 prediction results on the scale of 40×40; c2 is sent to the Conv layer to output k3 prediction results on the scale of 20×20; the prediction results include the predicted position coordinate information of the LED array (x1, y1, w1, h1), (x2, y2, w2, h2), (x3, y3, w3, h3), ..., (x k ,y k ,w k ,h k ), confidence p1, p2, p3, …, p k and predicted categories φ1, φ2, φ3, …, φ k , where k = k1 + k2 + k3, the predicted position coordinate information is drawn on the image in the form of a rectangular box, and the rectangular box is recorded as the predicted target box;

[0154] Establish a loss function L, use the loss function L to calculate the prediction results output by the model, and get the error of the prediction results. The calculation formula is:

[0155] L=L cls +L bbox

[0156] Where, L cls represents the classification loss function, L bbox represents the bounding box loss function;

[0157] Classification loss L cls The calculation formula is:

[0158]

[0159] The classification loss is used to evaluate the quality of the model prediction results. In the formula, N is the number of samples collected in the data set in step 1 (1); M is the number of categories, which is 2; y ic is the value of the label, y ic Indicates whether the predicted category in the predicted target box is "Led array", that is When the predicted category in the predicted target box is "Led array", y ic =1, when the prediction category in the prediction target box is not "Led array", y ic =0; p ic is the probability obtained after the Softmax function;

[0160]

[0161] where z k and z m is a set of feature vectors obtained after processing in steps 2 and 3. is the sum of all eigenvectors;

[0162] The bounding box loss uses Shape_IoU as a new loss function, and its loss function is:

[0163] L Shape-IoU =1-IoU+distance shape +P×Ω shape

[0164]

[0165] distance shape =hh×(x c -x c gt ) 2 / c 2 +ww×(y c -y c gt ) 2 ) / c

[0166]

[0167] Wherein, B is the real position information of the target LED array of the OCC system marked in step 1 process (2), B gt The location information predicted by the improved YOLOv8, gt boxes are B and B gt The overlapping area between them is used to set the parameters for the vehicle-to-vehicle OCC system target recognition, with scale set to 0.2 and P set to 0.4;

[0168] Although the recognized OCC target array is approximately a square, it will be bumpy in the actual process, causing the aspect ratio to change. Therefore, more influencing factors need to be considered. By introducing ww, hh and scale, the influence of the shape and scale of the bounding box on the recognition effect is reduced;

[0169] Through the process of step 5, we can obtain the prediction results at three different scales: (x1, y1, w1, h1), (x2, y2, w2, h2), (x3, y3, w3, h3), ..., (x k ,y k ,w k ,h k )、p1,p3,p3,…,p k and φ1, φ2, φ3, …, φ k After that, the error loss function L of the prediction result is calculated by the previous loss function L i , which is the loss on the training set; back propagation applies the chain rule to calculate L i The gradient value of the model parameters is updated according to the gradient value, and the loss function L is calculated again in step 5. i+1 , if L i+1 <L i , then save the updated YOLOv8 model parameters, L i+1 >L i , the YOLOv8 model parameters are not updated, iterate 300 times, and save the last updated YOLOv8 model parameters, which are recorded as the recognition model;

[0170] The recognition model is used to recognize the test set images in step 1 process (3). The size of the test set image T is fixed to 640×640 by the Mosaic method; after C2f in step 2, a feature map of 80×80 is obtained. EC2f obtains a 40×40 feature map After CAIFI feature extraction module, a 20×20 feature map is obtained After the BIFPN feature fusion in step 3, a 128×128 feature map is obtained. 256×256 feature map 512×512 feature map Will After step five, the loss function is obtained; the output value of the loss function is the recognition result.

[0171] Improved YOLOv8 for performance evaluation:

[0172] Precision, recall, average precision (AP) and mean average precision (mAP) are used as evaluation indicators of the convergence model.

[0173] Precision is the proportion of true positive samples among all predicted positive samples, and is calculated as:

[0174] P=TP / (TP+FP)

[0175] Among them, TP is the number of positive samples predicted by the model as positive samples, and FP is the number of negative samples predicted by the model as positive samples.

[0176] The recall rate is the proportion of all true positive samples that are predicted to be positive samples. The formula is as follows:

[0177] R=TP / (TP+FN)

[0178] Among them, TP is the number of positive samples predicted by the model as positive samples, and FN is the number of positive samples predicted by the model as negative samples.

[0179] AP is often used to measure the detection effect of the model on a certain type of target. The formula is as follows:

[0180]

[0181] P(r) is the accuracy of target recognition in the vehicle-to-vehicle OCC system, and mAP refers to the average AP calculated over all categories. It provides a comprehensive performance evaluation of the YOLOv8 model over all categories:

[0182]

[0183] The value of n is 2, P m The IoU threshold affects the mAP value of the model. The commonly used mAP indicators are mAP@0.5 and mAP@0.5:0.95. mAP@0.5 means the average accuracy of all categories when the IoU threshold is 0.5. mAP@0.5:0.95 is the result of averaging the IoU thresholds from 0.5 to 0.95 with a step size of 0.05. The formula is as follows:

[0184]

[0185] The training results of each evaluation index are shown in Table 1:

[0186] Table 1

[0187]

[0188] A large number of experiments on the data set show that the improved YOLOv8 algorithm has good performance in identifying the target LED array in the car-to-car OCC system. Using the same picture for prediction, the improved Yolov8 algorithm has certain improvements at different angles and distances. The improvement in recognition accuracy is particularly obvious when the distance is far, which can meet the needs of long-distance accurate recognition and has a wider range of applications.

Claims

1. A vehicle-to-vehicle OCC system target recognition method for improving YOLOv8, obtaining original image data, preprocessing the original image data, obtaining preprocessed image data, and modifying YOLOv8: replacing SPPF in the backbone network with CAIFI, combining the head network with BIFPN, and using the Shape_IoU loss function as the head network to construct an improved YOLOv8 LED array detection model; inputting the preprocessed image into the improved YOLOv8 OCC system target detection model for calculation, obtaining LED array detection results, and achieving the purpose of accurately identifying LED arrays, characterized in that, The specific steps of this method are as follows: Step 1: Build LED array dataset: Process (1) Collecting LED array data set in outdoor environment; During the data collection process, the position of the camera remains unchanged, and the LED array moves within the range of 0m-100m from the camera. At the same time, one or some LED lights in the LED array that can light up and turn off independently will light up or turn off randomly, and the LED array data set under different distances and lighting conditions is obtained. During the collection process, the LED array is tilted during the movement to simulate the bumpy phenomenon when the actual car is driving, and the LED data sets at different angles are obtained; Process (2) uses the LabelImg platform to manually annotate the LED array dataset. The annotation information consists of two parts: category and position. The category of the LED array is marked as "Led array"; the real position coordinate information of the LED array is marked as (x min ,y min ,x max ,y max ), draw the real position coordinate information in the form of a rectangular frame on the collected image, and record the rectangular frame as the real target frame, where (x min ,y min ) represents the coordinate of the upper left corner of the LED array, (x max ,y min ) represents the coordinate of the upper right corner of the LED array, (x min ,y max ) represents the coordinate of the lower left corner of the LED array, (x max ,y max ) represents the coordinate of the lower right corner of the LED array; Process (3) divides the labeled data set into three parts: training set, validation set, and test set for model training and testing. The training set accounts for 70% of the entire data set and is used to train the model and learn the features in the data. The validation set accounts for 20% of the entire data set and is used to verify the performance and generalization ability of the model and adjust and optimize the model. The test set accounts for 10% of the entire data set and is used to evaluate the final performance and accuracy of the model. Step 2: Change the YOLOv8 feature extraction module Three different methods are used to extract features from feature maps at different positions: (1) Basic C2f module for feature extraction The C2f module consists of a CBS convolutional layer, a Split layer, and a BottleNeck. BottleNeck uses a residual connection to directly add the input LED array features to the output LED array features, alleviating the gradient vanishing and gradient exploding problems, and allowing information transfer between layers to avoid deep network degradation. The C2f module is used to extract low-level features of the target LED array of the car-to-car OCC system. (2) EC2f module for feature extraction The EC2f module consists of the C2f module and the ECA attention mechanism. The EC2f module performs global average pooling on the feature map processed by the C2f module, compresses the spatial dimension to 1x1, and obtains a channel-dimensional vector. The output is activated by the Sigmoid function: Where x is the input feature vector after global average pooling. The compressed vector is processed by the Sigmoid activation function to generate a weight vector between 0 and 1. Each channel of the original input vehicle-to-vehicle OCC system target LED array feature map is multiplied by the corresponding weight to obtain the weighted feature map f E : where y c For the feature map after C2f processing, the ECA module avoids the complex dimensionality reduction and dimensionality increase process through global average pooling and weighted processing, thereby achieving high efficiency and lightweight characteristics, and uses the EC2f module to extract mid-level features; (3) CAIFI module for feature extraction Replace the SPPF of the original YOLOv8 model with CAIFI. CAIFI reads the width and height of the feature map of the input LED array image C and converts the input feature map f E Flatten the image and transpose it using the permute function to get the flattened input feature map and record it as srcf. CAIFI will generate position encoding information and use srcf and pos_embed as the input of the encoding layer. The encoding layer includes multi-head attention, linear layer, and random inactivation layer. The specific workflow is as follows: v=srcf q=k=src+pos_emb v, q, k are used as Multi-Head Self-Attention inputs and are processed through self-attention to obtain srca. After srca is subjected to dropout processing, it is added to srcf to obtain srcl. Srcl is normalized, passes through the fully connected layer, activation layer, and pooling layer, and then subjected to dropout processing. Cross-layer connection is performed, srcl is added to srca, and then normalized to obtain the result of Encoderlayer output. The output feature map has the same size as the input feature map. The result is permuted and processed with the contiguous function to obtain [B, C, H, W], and then passes through the GELU activation function; f G =GELU(x)=x*P(X≤x)=x*Φ(x) or: GELU(x)=x*σ(1.072x) Among them, Φ(x) is the cumulative distribution of Gaussian normal distribution, σ(.) is the Sigmoid function; The feature map f obtained after GELU activation function processing G As input, f G After the convolutional layer processing, the number of channels is converted from 512 to 128, which is suitable for subsequent feature fusion operations, and the CAIFI module is used for high-level feature extraction; Step 3: Change the YOLOv8 feature fusion module The BIFPN module is introduced into the Neck network. The BIFPN module consists of multiple repeated network layers, each of which contains multiple feature map nodes and adopts a multi-scale fusion approach. The input features of different LED arrays have different resolutions, and they usually contribute unequally to the output features. The introduction of BIFPN adds an additional weight to each input and allows the network to learn the importance of each input feature. Modify the weight function of the basic BIFPN, the expression of weight O: Where w is a learnable weight parameter, which is initialized to a vector of all 1s. t is an introduced learnable smoothing parameter, which is initially set to 2. j is the input category with a value of 2. i is the input feature vector dimension with a value of 3. ε is set to 0.0001 to avoid numerical instability. I i is the input feature map after feature extraction; Step 4: Change the YOLOv8 prediction loss function The input OCC system target LED array feature map is passed through the Neck network to obtain 80*80, 40*40 and 20*20 feature maps, which are sent to the detection head for detection processing. After processing, the location information, type and confidence of the OCC system target LED array are obtained; IoU is used to measure the accuracy of the bounding box; the currently used regression loss function only considers the overlap between bounding boxes, and does not pay enough attention to the shape and scale of the bounding box; Shape_IoU is used as the new loss function, and its loss function is: L Shape-IoU =1-IoU+distance shape +P×Ω shape distance shape =hh×(x c -x c gt ) 2 / c 2 +ww×(y c -y c gt ) 2 ) / c 2 Wherein, B is the real position information of the target LED array of the OCC system marked in step 1 process (2), B gt The position information of the target LED array of the OCC system predicted by the improved YOLOv8. The gt boxes are B and B gt The overlapping area between the two is used to set the parameters for the vehicle-to-vehicle OCC system target recognition. The scale is set to 0.2, P is set to 0.4, ww is the weight coefficient in the horizontal direction, and hh is the weight coefficient in the vertical direction. The values ​​of ww and hh have the following relationship with the gt box: The width of B is w1 and the height is h1; gt The width is w2 and the height is h2; w and h are calculated based on the width w1 and w2 and the height h1 and h2 of the bounding box, using the following formula: Although the recognized OCC target array is approximately a square, it will be bumpy in the actual process, causing the aspect ratio to change. Therefore, more influencing factors need to be considered. By introducing ww, hh and scale, the influence of the shape and scale of the bounding box on the recognition effect is reduced; Step 5: Input the training set images into the improved YOLOv8 model for training and optimization until the model converges: Input the training set images into the improved YOLOv8 model for training and optimizing the model until convergence: Process (1) Build a target recognition model based on the improved YOLOv8 network to extract image feature matrix and predict target position. The target recognition model built based on the improved YOLOv8 network includes Backbone network and Neck network; Backbone network: Feature Extractor extracts features, and its function is to extract information from the image for use by the subsequent network; Neck network: placed between the Backbone network and the Head network to better utilize the features extracted by the Backbone, playing the role of "feature fusion"; Head network: uses the previously extracted features to make recognition; First, the training set data is fed into the Backbone network, and the input module uses the Mosaic method to enhance the image data: Randomly select the coordinates of the image stitching reference point (x c ,y c ), where x c ∈(0,1280),y c ∈(0,1280); select four images from the training set and randomly scale them to 0.5-2.0 times the original image size; (x c ,y c ) is the center, and the four images are stitched into an image A with a size of 1280×1280 in the order of upper left, upper right, lower left, and lower right; image A is adaptively scaled to fix the size of image A to 640×640; the Mosaic operation is turned off for the last 10 epochs; Performing Letterbox image data enhancement on the image data enhanced by the Mosaic method, scaling the long side of the image data enhanced by the Mosaic method to a specified size, and filling the short side with black pixels to obtain the preprocessed image; The convolution module in the feature extraction of the Neck network uses CBS and C2f. CBS consists of a two-dimensional convolution + two-dimensional BatchNorm + SiLU activation function, and C2f consists of a CBS convolution layer + a split + n Bottleneck series. BottleNeck uses a residual connection to directly add the input LED array and the output LED array features to alleviate the gradient vanishing and gradient exploding problems, and allows information transfer between layers to avoid the degradation of deep networks. The operation of extracting features from the input image is performed in the Backbone network; it extracts low-level to high-level features of the image layer by layer, gradually reduces the spatial resolution and increases the number of channels to obtain a more abstract and semantically rich feature representation, and sends the input LED array image C to the backbone network. Image C passes through multiple convolutional layers and 4 C2f feature extraction modules in sequence; the specific process is: after passing through two convolutional layers and a C2f structure, it passes through a convolutional layer and a C2f structure in sequence to extract the feature matrix a1 in image C, a1 continues to be transmitted downward, passes through a convolutional layer and a C2f structure in sequence to extract the feature matrix a2; a2 continues to be transmitted downward, passes through a convolutional layer, a C2f structure in sequence to extract the feature matrix, and the CAIFI module extracts the feature matrix a3; Then, the feature matrices a1, a2 and a3 extracted by the feature extraction module are sent to the Neck network for feature fusion processing. The feature fusion module uses the BIFPN module to fuse the extracted feature matrices; the feature matrix of a3 after a convolution layer is recorded as d1, d1 is upsampled, and processed with a2 through the BIFPN module to obtain the feature matrix b1, b1 is passed through the C2f module to obtain c1, c1 is passed through a convolution layer to obtain d2, d2 is upsampled, and feature fused with a1 through the BIFPN module to obtain the feature matrix b2, b2 is passed through the C2f module to obtain c2, and then passed through a convolution layer to obtain d3, d3 and c1 are processed through the BIFPN module to obtain the feature matrix b3, b3 is passed through the C2f module After the module, c3 is obtained. After c3 passes through a convolution layer, d4 is obtained. d4 and a3 are processed by the BIFPN module to obtain the feature matrix b4. b4 passes through the C2f module to obtain c4. Finally, the feature matrices c2, c3 and c4 fused by the feature fusion module are sent to the Head network; c4 is sent to the Conv layer to output k1 prediction results on the scale of 80×80; c3 is sent to the Conv layer to output k2 prediction results on the scale of 40×40; c2 is sent to the Conv layer to output k3 prediction results on the scale of 20×20; the prediction results include the predicted position coordinate information of the LED array (x1, y1, w1, h1), (x2, y2, w2, h2), (x3, y3, w3, h3), ..., (x k ,y k ,w k ,h k ), confidence p1, p2, p3, …, p k and predicted categories φ1, φ2, φ3, …, φ k , where k = k1 + k2 + k3, the predicted position coordinate information is drawn on the image in the form of a rectangular box, and the rectangular box is recorded as the predicted target box; Establish a loss function L, use the loss function L to calculate the prediction results output by the model, and get the error of the prediction results. The calculation formula is: L=L cls +L bbox Where, L cls represents the classification loss function, L bbox represents the bounding box loss function; Classification loss L cls The calculation formula is: The classification loss is used to evaluate the quality of the model prediction results. In the formula, N is the number of samples collected in the data set in step 1 (1); M is the number of categories, which is 2; y ic is the value of the label, y ic Indicates whether the predicted category in the predicted target box is "Led array", that is When the predicted category in the predicted target box is "Led array", y ic =1, when the prediction category in the prediction target box is not "Led array", y ic =0; p ic is the probability obtained after the Softmax function; where z k and z m is a set of feature vectors obtained after processing in steps 2 and 3. is the sum of all eigenvectors; The bounding box loss uses Shape_IoU as a new loss function, and its loss function is: L Shape-IoU =1-IoU+distance shape +P×Ω shape distance shape =hh×(x c -x c gt ) 2 / c 2 +ww×(y c -y c gt ) 2 ) / c 2 Wherein, B is the real position information of the target LED array of the OCC system marked in step 1 process (2), B gt The location information predicted by the improved YOLOv8, gt boxes are B and B gt The overlapping area between them is used to set the parameters for the vehicle-to-vehicle OCC system target recognition, with scale set to 0.2 and P set to 0.4; Although the recognized OCC target array is approximately a square, it will be bumpy in the actual process, causing the aspect ratio to change. Therefore, more influencing factors need to be considered. By introducing ww, hh and scale, the influence of the shape and scale of the bounding box on the recognition effect is reduced; Through the process of step 5, we can obtain the prediction results at three different scales: (x1, y1, w1, h1), (x2, y2, w2, h2), (x3, y3, w3, h3), ..., (x k ,y k ,w k ,h k )、p1,p3,p3,…,p k and φ1, φ2, φ3, …, φ k After that, the error loss function L of the prediction result is calculated by the previous loss function L i , which is the loss on the training set; back propagation applies the chain rule to calculate L i The gradient value of the model parameters is updated according to the gradient value, and the loss function L is calculated again in step 5. i+1 , if L i+1 <L i , then save the updated YOLOv8 model parameters, L i+1 >L i , the YOLOv8 model parameters are not updated, iterate 300 times, and save the last updated YOLOv8 model parameters, which are recorded as the recognition model; The recognition model is used to recognize the test set images in step 1 (3). The size of the test set image T is fixed to 640×640 by the Mosaic method. After C2f in step 2, a feature map of 80×80 is obtained. EC2f obtains a 40×40 feature map And the CAIFI feature extraction module to obtain a 20×20 feature map After the BIFPN feature fusion in step 3, a 128×128 feature map is obtained. 256×256 feature map And a 512×512 feature map Will After step five, the loss function is obtained; the output value of the loss function is the recognition result.

Images