Method for detecting small target foreign matter constructed by fusing multi-scale sequence based on yolk

Abstract

The invention discloses a yolk-based small target foreign matter detection method constructed by fusing a multi-scale sequence. The method comprises the following steps: preprocessing an obtained image; obtaining a target input image, and extracting original features of the target input image to obtain original depth features; performing hybrid convolution on the extracted original depth features to form a new total feature amount for subsequent detection training; performing multi-scale fusion on the extracted original depth features to obtain three feature maps with different scales; and according to the three feature maps with different scales, inputting a yolov5 detection head to obtain a prediction result. According to the yolk-based small target foreign matter detection method constructed by fusing the multi-scale sequence, YOLOv5 is selected as a basic network according to image characteristics, a network structure is redesigned, fine loss is reduced, light weight is guaranteed, richer gradient flow information is obtained, and therefore the detection precision and speed are guaranteed while the light weight of the network is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a small target foreign body detection method based on YOLO fusion multi-scale sequence construction. Background Art

[0002] Looking at the development history of target detection, it is actually a process from traditional target detection to deep learning-based target detection. Traditional target detection methods are suitable for some simple objects and have very limited processing capabilities. Compared with traditional methods, deep learning-based target detection methods can automatically learn features and classifications, and to a certain extent solve the problems of traditional methods. However, due to the small pixels of small targets, it is difficult to extract effective information, resulting in huge difficulties and challenges in small target detection. Existing deep learning-based small target detection methods are all improved on the mainstream target detection model to improve the detection performance of small targets.

[0003] Image detection algorithms based on deep learning are mainly divided into single-stage detectors and two-stage detectors. Two-stage detectors, such as the R-CNN series of algorithms, first generate a series of candidate regions, and then classify and regress these regions. Single-stage detectors, such as the YOLO and SSD series of algorithms, perform target detection directly on the image without generating candidate regions first, so they are faster.

[0004] YOLO is a real-time, end-to-end object detection method proposed in 2016. Unlike traditional object detection methods, YOLO treats object detection as a regression problem and directly predicts bounding boxes and category probabilities in a single network. The main advantages of YOLO are its fast speed, real-time object detection, and high accuracy.

[0005] In the field of target detection, small target detection faces many challenges, such as small pixels in small target images, difficulty in extracting effective information, lack of semantic information in underlying features, and differences between the backbone network used by the detection model and the detection task. Therefore, improving the accuracy of small target detection has become a research hotspot in the field of target detection.

[0006] At present, there are three main methods for small target detection: one-stage, two-stage, and multi-stage. As the research on small target detection progresses rapidly, new methods for small target detection based on deep learning continue to emerge. For example, in the small target detection stage based on machine learning, image pyramid is the mainstream method for constructing multi-scale features. MTCNN uses this idea. First, it constructs an image pyramid, and then uses CNN to extract facial features on each layer of the image, thereby detecting facial targets of different resolutions. In this way, the features extracted on each layer of resolution image contain rich semantics, which is conducive to the detection of small targets. The small target detection method based on contextual information improves the performance of small target detection by utilizing the connection between the target and the surrounding pixels and the target, and mining and utilizing the relationship between objects. For example, some literature also introduces contextual information. First, a context-assisted method based on prior frames is proposed. For a target face, there will be a series of prior frames related to the face. These prior frames contain more context information on the feature map with a large receptive field. Using these prior frames as auxiliary information will be conducive to supervised learning of small-scale, blurred and partially occluded context features. This method won the championship in the 2018 WIDER FACE face detection competition. In addition, a small target detection method based on a new backbone network and training strategy is developed. By analyzing the differences between classification data sets and detection data sets, a backbone network and training strategy specifically for target detection tasks are proposed to improve the detection performance of small targets. Another document proposes a new target detection training method and model, combining the pre-trained model with the method of training from scratch, using a pre-trained SSD network as the backbone network, and using a lightweight auxiliary network LSN trained from scratch. The role of LSN is to make up for the loss of the backbone network in the feature extraction process, extract more accurate mid- and bottom-level features, and provide more detailed contour information for target detection. In addition, there are ideas such as improving feature resolution and performing data enhancement to improve the detection performance of small targets. However, compared with the detection performance of large and medium-sized targets, there is still a big gap in the performance of small target detection. In the future, there are still many problems to be solved, such as how to explore better and more comprehensive multi-scale feature fusion strategies to further improve the detection performance of small targets. Summary of the invention

[0007] In view of the above defects of the prior art, the technical problem to be solved by the present invention is that the feature layer of the existing small target foreign body detection is not semantically rich, easily disturbed by the background, and easily causes false detection and missed detection in small target detection; the small target training samples are insufficient, so that the small target cannot be fully learned during the model training process; the existing YOLO algorithm is prone to lose key detail information in processing small target foreign body detection, resulting in low performance of small target foreign body detection. Therefore, the present invention provides a small target foreign body detection method based on YOLO fusion multi-scale sequence construction, selects YOLOv5 as the basic network according to the image characteristics, redesigns the network structure, reduces surprise loss, ensures lightweight while obtaining richer gradient flow information, thereby ensuring lightweight network while ensuring detection accuracy and speed.

[0008] To achieve the above object, the present invention provides a small target foreign body detection method based on YOLO fusion multi-scale sequence construction, comprising the following steps:

[0009] Preprocess the acquired images;

[0010] Acquire a target input image, extract original features from the target input image, and obtain original depth features;

[0011] Perform mixed convolution on the extracted original deep features to form a new feature set for subsequent detection training;

[0012] The extracted original depth features are fused at multiple scales to obtain feature maps of three different scales.

[0013] According to the feature maps of three different scales, the yolov5 detection head is input to obtain the prediction results.

[0014] Furthermore, the acquired image is preprocessed, specifically including:

[0015] The training set images are obtained, the training set images are cropped and normalized, the input image size of the model is adjusted, the samples are input into the lightweight small target foreign body detection model, and the small target foreign body detection model is trained.

[0016] Furthermore, before training the model, the images of the training samples need to be cropped and normalized, and the images are uniformly cropped to a size of 640x640. In the case of insufficient training samples, random flipping, random cropping, and random brightness adjustment are used to ensure that there are enough training samples.

[0017] Further, a target input image is obtained, and original features are extracted from the target input image to obtain original depth features, specifically including: obtaining a target input image, and extracting original features from the target input image using an input feature pre-extraction network to obtain original depth features of the target input image;

[0018] Furthermore, the first 10 layers of the New CSP-Darknet53 network are used as the backbone framework of the pre-extraction feature network to extract the original deep features from the input image.

[0019] Furthermore, the extracted original depth features are mixed and convolved to form a new feature set for subsequent multi-scale feature fusion, including:

[0020] The original deep features are mixed convolved, and a feature splicing module is used after the convolution module to fuse the previous layer and shallow layer features to form a multi-scale fused feature map.

[0021] Then, the obtained feature map is used to extract features of different levels and abstraction levels through the C2f module; the C2f module first passes the multi-scale fused feature map through the first convolution layer cv1, and then divides the multi-scale fused feature map into two parts; one part is directly passed to the output, and the other part is processed through multiple Bottleneck modules. Finally, the results of the two parts are spliced ​​in the channel dimension and passed through the second convolution layer cv2 to obtain a new feature output for subsequent model training;

[0022] The feature map processed by the C2f module is then processed by hybrid convolution. At the same time, the SEnet attention mechanism is added after the hybrid convolution module to optimize these features and fuse multi-level features, focusing perception on the important parts of the features and suppressing other useless features.

[0023] Furthermore, the extracted original depth features are subjected to multi-scale fusion, specifically including:

[0024] First, the feature map obtained by each C3 module in the New CSP-Darknet53 backbone network is input into the scale sequence feature fusion module. The scale sequence fusion module first adjusts the number of channels of the image size feature map to be consistent through 1x1 convolution, and then uses bilinear interpolation to adjust the size of all feature maps to the same size;

[0025] Next, the unsqueeze method is used to increase the dimension of each feature layer, changing it from a 3D tensor [height, width, channel] to a 4D tensor [depth, height, width, channel] and then concatenating the 4D feature maps along the depth dimension to form a 3D feature map for subsequent convolution;

[0026] Finally, 3D convolution, 3D batch normalization and SiLU activation function are used to complete scale sequence feature extraction.

[0027] Furthermore, the 3D feature map is added to the 20th layer feature to obtain more feature information of different sizes, and the feature map D1 is obtained;

[0028] Use Bi_Concat to concatenate the 22nd layer features with the 6th and 16th layers to obtain more feature information of different scales; then input the C2f module to further extract features and obtain feature map D2;

[0029] The feature map D2 is concatenated with the feature map of the 11th layer to achieve multi-scale fusion. The concatenated and fused feature map of D2 and the 11th layer is input into the C2f module for final feature extraction. The final feature map is recorded as D3.

[0030] Furthermore, according to the feature maps of three different scales, the yolov5 detection head is input to obtain the prediction results, which specifically include three feature maps of different scales D1, D2, and D3 input into the final detection head. Then the model uses the grid-based anchor to predict the type and position information within the network, and then performs prediction and post-processing after positive sample matching and loss calculation based on the detection information; finally, the model prediction results are output, including the category of each target and its corresponding bounding box coordinates and other information.

[0031] Technical Effects

[0032] The present invention discloses a small target foreign body detection method based on yolo fused multi-scale sequence construction, which redesigns the feature extraction network for small target data sets, is more in line with the needs of small target detection, can accurately detect small targets, has high accuracy, short detection time, and strong anti-interference ability; more scale features are combined in the process of feature fusion, and the detection is more accurate.

[0033] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the operation flow of a small target foreign body detection method constructed by fusion multi-scale sequence based on YOLO in a preferred embodiment of the present invention;

[0035] Figure 2 It is a schematic diagram of a C2f module of a small target foreign body detection method constructed by fusion multi-scale sequence based on YOLO in a preferred embodiment of the present invention;

[0036] Figure 3It is a schematic diagram of a triple feature splicing module of a small target foreign body detection method constructed by fusion multi-scale sequence based on YOLO in a preferred embodiment of the present invention;

[0037] Figure 4 It is a schematic diagram of a multi-scale sequence feature fusion module of a small target foreign body detection method constructed by fusion multi-scale sequences based on YOLO in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0039] In the following description, specific details such as specific internal procedures and techniques are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0040] The present invention provides a small target foreign body detection method based on YOLO fusion multi-scale sequence construction, comprising the following steps:

[0041] S1: preprocess the acquired image;

[0042] S2: Obtain a target input image, extract original features from the target input image, and obtain original depth features;

[0043] S3: Perform mixed convolution on the extracted original deep features to form a new feature set for subsequent detection training;

[0044] S4: Perform multi-scale fusion on the extracted original depth features to obtain feature maps of three different scales;

[0045] S5: According to the feature maps of three different scales, input the yolov5 detection head to get the prediction results.

[0046] Wherein, S1: preprocessing the acquired image, specifically including:

[0047] The training set images are obtained, cropped and normalized, the input image size of the model is adjusted, the samples are input into the lightweight small target foreign body detection model, and the small target foreign body detection model is trained. In order to perform small target foreign body detection and obtain key foreign body information, the images are cropped and normalized according to the hyperparameter settings.

[0048] In the example of the present invention, in order to train small foreign bodies with high detection accuracy, a large number of training samples are needed to prepare training set samples. The pork images collected by the samples are obtained by X-ray imaging. The training set samples include pork of different shapes and sizes and different types of foreign bodies, such as missing pork, broken bones, and rotten meat.

[0049] In this example, the data is divided into images and labels, which represent image samples and corresponding labels respectively. Images and labels are subdivided into train, test, and val subfolders, which are used for training, testing, and validation, respectively, with a ratio of 7:2:1. Before training the model, in order to ensure the consistency of the image size of the training set samples to meet the model's requirements for input images, the images of the training samples need to be cropped and normalized, and the images are uniformly cropped to a size of 640x640. In addition, when there are not enough training samples, random flipping, random cropping, and random brightness adjustment are used to ensure that there are enough training samples.

[0050] S2: Obtain a target input image, extract original features from the target input image, and obtain original depth features, specifically including obtaining a target input image, extracting original features from the target input image using an input feature pre-extraction network, and obtaining original depth features of the target input image. In this embodiment, the first 10 layers of the New CSP-Darknet53 network are specifically used as the backbone framework of the pre-extraction feature network to extract original depth features from the input image.

[0051] S3: Perform mixed convolution on the extracted original deep features to form a new total feature set for subsequent fusion operations of features at different scales and adding an attention mechanism, including:

[0052] The original deep features are mixed convolved, and a feature splicing module is used after the convolution module to fuse the previous layer and shallow layer features to form a multi-scale fused feature map.

[0053] Then, the obtained feature map is used to extract features of different levels and abstraction levels through the C2f module; the C2f module first passes the multi-scale fused feature map through the first convolution layer cv1, and then divides the multi-scale fused feature map into two parts; one part is directly passed to the output, and the other part is processed through multiple Bottleneck modules. Finally, the results of the two parts are spliced ​​in the channel dimension and passed through the second convolution layer cv2 to obtain a new feature output for subsequent model training;

[0054] The feature map processed by the C2f module is then processed by hybrid convolution. At the same time, the SEnet attention mechanism is added after the hybrid convolution module to optimize these features and fuse multi-level features, focusing perception on the important parts of the features and suppressing other useless features.

[0055] In order to obtain more small target detail features, the SEnet attention mechanism is added after the hybrid convolution module to extract channel information in the features, focus perception on the important parts of the features, and suppress other useless features, such as focusing on the detailed texture information of broken bones in pork, while suppressing other background information. Specifically, the feature map obtained by the hybrid convolution is averaged and pooled (nn.AdaptiveAvgPool2d) to compress the two-dimensional features (H*W) of each channel into 1 real number, and change the feature map dimension from [h,w,c] to [1,1,c] to obtain the channel-level global features. The correlation between channels is constructed through two fully connected layers. The SEnet attention mechanism downsamples each channel and operates the fully connected layer. The weight of each channel is adaptively learned through the relationship between each channel, and the weights of different channels are obtained. Subsequently, the learned channel attention weight is multiplied by the feature map obtained by the hybrid convolution to adjust the feature value between each channel, emphasize important channel information, and suppress unimportant channel information. The feature map processed by the SEnet attention mechanism is used in the subsequent multi-scale splicing module.

[0056] S4: Multi-scale fusion of the extracted original depth features, including:

[0057] First, the feature map obtained by each C3 module in the New CSP-Darknet53 backbone network is input into the scale sequence feature fusion module. The scale sequence fusion module first adjusts the number of channels of the image size feature map to be consistent through 1x1 convolution, and then uses bilinear interpolation to adjust the size of all feature maps to the same size;

[0058] Next, the unsqueeze method is used to increase the dimension of each feature layer, changing it from a 3D tensor [height, width, channel] to a 4D tensor [depth, height, width, channel] and then concatenating the 4D feature maps along the depth dimension to form a 3D feature map for subsequent convolution;

[0059] Finally, 3D convolution, 3D batch normalization and SiLU activation function are used to complete the scale sequence feature extraction. The feature map obtained by the above operation is added to the 20th layer feature to effectively obtain more feature information of different scales. The obtained feature map is recorded as D1 for the final detection.

[0060] Furthermore, the 3D feature map is added to the 20th layer feature to obtain more feature information of different sizes, and the feature map D1 is obtained;

[0061] Use Bi_Concat to concatenate the 22nd layer features with the 6th and 16th layers to obtain more feature information of different scales; then input the C2f module to further extract features and obtain feature map D2;

[0062] For the D2 feature map, the C2f module is used to further reduce the width and height and increase the number of channels, and the SEnet attention mechanism is used to enhance the feature extraction again. The feature map D2 is tensor-concatenated with the 11th layer feature map to achieve multi-scale fusion. The feature map after the concatenation and fusion of D2 and the 11th layer is input into the C2f module for the final feature extraction. The final feature map is recorded as D3.

[0063] S5: The three feature maps D1, D2, and D3 of different scales obtained in S3 and S4 are input into the final detection head. The model then uses the grid-based anchor to predict the type and location information in the network, and then performs prediction and post-processing after positive sample matching and loss calculation based on the detection information. Finally, the model predicts the results, including the category of each target and its corresponding bounding box coordinates.

[0064] When calculating the loss, an imbalance in the number of positive and negative samples often leads to unsatisfactory training results. The extreme value of the width and height of the annotation box and the width and height of the anchor is calculated. As long as the extreme value is between 1 / 4 and 4, the anchor is set as a positive sample. The process of adding positive samples is actually adding labels, which can be understood as generating many more annotation boxes for the same target. If there are more annotation boxes for the same target, the number of prediction results for this annotation box will also increase. However, compared with the case where no positive samples are added, the difference in the output labels is not too large. After the positive and negative samples are matched, the number of boxes is greatly reduced, and then the loss function is calculated.

[0065] The YOLOv5 loss function is used in the present invention, which is mainly composed of classification loss, bbox regression loss (error between the predicted box and the real box), and confidence loss. The total loss function is the sum of the three. Different weight coefficients can also be added to the three losses. Usually, the confidence loss takes the maximum weight, and the box loss and classification loss are weighted second, such as 0.4, 0.3, and 0.3 respectively, to achieve the effect of different proportions. The bbox regression loss used in the present invention is CIoU Loss. The minimum normalized center point distance in this loss can make the convergence speed faster and the regression error smaller in any scenario. When the distance between box A and box B is infinite and the aspect ratio difference is infinite, DIOU takes -1, v takes 1, and alpha takes 0.5. At this time, CIOU takes -1-0.5=-1.5; when box A and box B completely overlap, DIOU takes 1, v takes 0, α takes 0, and CIOU takes 1-0=1. Therefore, the value range of CIOU is -1.5~1. The specific definition function is:

[0066]

[0067] The confidence loss is calculated by the sample pairs obtained by matching the positive samples. One is the target confidence score p0 in the prediction box, and the other is the iou value p between the prediction box and the corresponding target box. iou , which is used as ground-truth. The value range of CIOU is -1.5~1, while the value range of confidence label is 0~1, so CIOU needs to be truncated: when CIOU is less than 0, directly take 0 as the label. The two calculate the binary cross entropy to get the final target confidence loss. At the same time, the dimension of the label should be consistent with the output dimension of the neural network. Taking the 80*80 grid as an example, the confidence label is also a matrix of dimension 3*80*80. First, assign values ​​to the confidence label matrix. Assuming that the confidence label is matrix L and the predicted confidence is matrix P, the calculation formula of BCE loss for each value in the matrix is ​​as follows:

[0068] Loss BCE (z,x,y)=-L(z,x,y)*logP(z,x,y)-(1-L(z,x,y))*log((1-P(z,x,y))

[0069] 0≤z<3

[0070] 0≤x<80

[0071] 0≤y<80

[0072] BCE loss requires that the value range of the input data must be between 0 and 1. Thus, the confidence loss is obtained:

[0073]

[0074] Category loss is similar to confidence loss. It is calculated by the category score of the prediction box and the one-hot performance of the target box category. The neural network predicts three prediction boxes for each grid, and the prediction information of each prediction box contains N classification probabilities. N is the total number of categories. For example, the COCO dataset has 80 categories, so N is 80. Therefore, for the COCO dataset, each prediction box has 80 classification probabilities of 0 to 1, so the neural network predicts a total of 3*80*80*80 classification probabilities to form a prediction probability matrix. The target confidence loss and category loss use the binary cross entropy function with sigmoid.

[0075]

[0076] In S5, prediction and post-processing are performed after positive sample matching and loss calculation. The specific processing flow is as follows: For each prior frame, the detection head predicts four bounding box parameters (center point coordinates, width and height), confidence score (whether the prior frame contains the target) and multiple category probabilities (the category of the target in the prior frame). The prediction results are processed and filtered, and background targets are removed according to the preset confidence, and some targets that are too large or too small are removed according to the width and height. Non-maximum suppression (NMS) is performed. Suppressing prediction frames with low scores involves two indicators: IOU (intersection over union: the ratio of the overlapping area of ​​two frames to the area of ​​the union of the two frames) and the prediction frame score. If multiple prediction frames predict the same object, if the IOU value of the two frames is greater than the threshold (generally the threshold is 0.5 or 0.7), then the NMS algorithm will discard the prediction frame with a low score. The specific implementation process of NMS non-maximum suppression is to first sort the prediction results of the category in descending order according to the confidence, calculate the IOU of the largest confidence box with the rest, and remove those with IOU greater than the threshold. Record the prediction box with the highest score, and then repeat the above steps from the remaining prediction boxes until only one box is left, and finally get the prediction result.

[0077] Next, we combine the lightweight small target foreign body detection model based on yolo and Figure 1 The network structure diagram shown is used to illustrate the model training process implemented by the present invention:

[0078] like Figure 1 As shown, the training set images are obtained, the training set pork images are cropped and normalized, the input image size of the model is adjusted to 640x640x3, where 3 represents the number of three channels of the pork image, and the samples are input into the lightweight small target foreign body detection model.

[0079] The original deep features are extracted through the feature pre-extraction network. The first 10 layers of the New CSP-Darknet53 network are used as the backbone of the pre-extraction feature network to extract the original deep features from the input image. The input dimension of this module is 640x640x3, and the final output is 20x20x1024.

[0080] The original features obtained by the C3 module are subjected to multi-level feature fusion. Taking the features output by the fourth-layer C3 module as an example, the multi-level features with a feature dimension of 80x80x256 output by the C3 module are input into the hybrid convolution for calculation, and the results are passed into the SEnet attention module for feature optimization. The optimized features are adjusted to the same dimension size as the features of the sixth and tenth layers and passed into the Zoom_cat module for multi-scale fusion. Finally, the multi-scale fused features are passed into the C2f module to extract features of different levels and abstraction levels. Specifically, the input feature dimension of the hybrid convolution here is 80x80x256, and the output feature is 80x80x256. The SEnet attention module does not change the feature dimension, and the output dimension is still 80x80x256. When performing multi-scale fusion with the features of the sixth and tenth layers, the feature dimension of the medium-scale feature map is first obtained as 40x40x512, and the large-scale features are adaptively max pooled and adaptively average pooled. The results of the two are added together, and the output feature dimension is 40x40x512. For small-scale features, the nearest neighbor interpolation method is used to upsample to the dimension of the medium-scale feature map. Finally, the resized large, medium, and small feature maps are spliced ​​along the channel dimension, and the returned spliced ​​feature map dimension is 40x40x512. Finally, the multi-scale fused features are output to the C2f module to extract features at different levels and output feature maps with a feature dimension of 40x40x512.

[0081] The features output by the feature enhancement network are passed to the detection head for prediction. First, the background targets are removed according to the confidence, and then the prediction results of this category are sorted using non-maximum suppression (NMS). The results with larger IoU are removed by calculating the IoU of the maximum confidence and other prediction results, and the prediction result with the largest score is retained and removed from the list. The above steps are repeated for the remaining prediction boxes until only one box is left, and the prediction result is finally obtained. Finally, the output prediction result and the annotation value of the input sample image are combined with the loss function, and the Adam optimizer algorithm is used, with the learning rate set to 1x10 -3 , the model parameters are updated by minimizing the loss function until the model converges.

[0082] S2: Collect and process the pork to be tested, obtain an image by imaging the pork through an X-ray machine, and process the pork X-ray image using step 1 to obtain the image to be tested.

[0083] In the example of the present invention, the user only needs to put the pork into the conveyor belt and wait for the X-ray machine to collect the pork image. The user does not need to do any other processing on the pork in advance. The example of the present invention adopts this method to ensure high detection accuracy.

[0084] S3: Input the pork X-ray image to be tested into the trained small target foreign body detection model to obtain the final foreign body detection. If foreign body is found in the test result, the model will mark and display the type and location information of the foreign body. If the test result is normal and there is no foreign body, it will pass normally without performing other operations.

[0085] A pre-trained small target foreign body detection model is used to detect the image to be verified. Specifically, it is determined whether the image to be detected contains foreign bodies based on a pre-set threshold. If there are foreign bodies, relevant information is output on the image and the user is prompted. Otherwise, the image is output normally.

[0086] S4: Feedback the test results to the user, and the user performs subsequent processing based on the test results.

[0087] In specific use, first obtain the training set image, obtain the pork image through the X-ray machine, and perform cropping and normalization processing. Then, extract the original features of the target input image through the feature pre-extraction network, filter out the influence of some irrelevant information, and obtain the original depth features of the target input image. Then, use different convolution blocks to perform multi-level feature extraction on the original depth features to extract information with rich context, and then perform feature optimization based on the attention mechanism on the multi-level features, focus the model's perception on areas with rich discriminant information, suppress irrelevant areas, adjust the optimized features to the same dimension for multi-scale feature fusion, obtain the spliced ​​feature map, and then further extract features through the C2f module. After the feature extraction network is enhanced, the final feature map is input to the detection head for prediction, and the model is continuously optimized according to the loss function to obtain the trained small target foreign body detection model. The pork to be inspected is collected and processed, and the collected pork X-ray image is inspected. The X-ray pork image is preprocessed using step 1. If there is no foreign matter, it is normal and the next image is inspected. Otherwise, the input image contains foreign matter and is not allowed to pass. Finally, the inspection result is fed back to the user, and the user performs subsequent processing according to the inspection result. The small target foreign matter detection model proposed in the present invention can effectively detect foreign matter information in pork, and has excellent detection performance and generalization ability in actual production applications.

[0088] The present invention provides a small target foreign body detection method based on YOLO fusion multi-scale sequence construction. According to the image characteristics, YOLOv5 is selected as the basic network, and the network structure is redesigned: SEnet attention mechanism is embedded in SPPF and Head parts; C2f structure is used to replace the C3 module in the original network; feature pyramid module is improved, and Bi-FPN is used to fuse the 6th, 16th and 22nd layer features to reduce information loss.

[0089] The C3 module mainly extracts gradient flow information through the idea of ​​CSPNtet extraction and diversion combined with the residual structure, while the C2f structure is designed based on the C3 module and combined with the idea of ​​ELAN to obtain richer gradient flow information while ensuring lightweight.

[0090] In view of the characteristics of the small target data set, the 2nd, 4th, 6th, and 8th layers are stacked with different scale feature maps by using the ScalSeq module, and the effective feature maps of different resolutions are adjusted to the same resolution for splicing, and then their scale sequence features are extracted. The information contained in the feature maps of different scales is crucial for small target detection. At the same time, Zoom_cat is used to split large, medium, and small features, and then the features are enlarged to improve the detailed feature information to better identify small targets. Finally, the feature fusion is added and spliced ​​with the result after processing the small target feature map, and the hybrid convolution is used to replace the traditional convolution after the NewCSP-Darknet53 backbone network of the model to ensure the detection accuracy and speed while ensuring the lightweight network.

[0091] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A small target foreign body detection method based on yolo fusion multi-scale sequence construction, characterized in that: The following steps are involved: Preprocess the acquired images; Acquire a target input image, extract original features from the target input image, and obtain original depth features; Perform mixed convolution on the extracted original deep features to form a new feature set for subsequent detection training; The extracted original depth features are fused at multiple scales to obtain feature maps of three different scales. According to the feature maps of three different scales, the yolov5 detection head is input to obtain the prediction results.

2. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 1, characterized in that: Preprocess the acquired image, including: The training set images are obtained, the training set images are cropped and normalized, the input image size of the model is adjusted, the samples are input into the lightweight small target foreign body detection model, and the small target foreign body detection model is trained.

3. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 2, characterized in that: Before training the model, the images of the training samples need to be cropped and normalized, and the images are uniformly cropped to a size of 640x640. If there are not enough training samples, random flipping, random cropping, and random brightness adjustment are used to ensure that there are enough training samples.

4. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 1, characterized in that: Acquire a target input image, extract original features from the target input image, and obtain original depth features, specifically including: acquire a target input image, extract original features from the target input image using an input feature pre-extraction network, and obtain original depth features of the target input image.

5. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 4, characterized in that: The first 10 layers of the New CSP-Darknet53 network are used as the backbone framework of the pre-extraction feature network to extract original deep features from the input image.

6. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 1, characterized in that: The extracted original depth features are mixed and convolved to form a new total feature for subsequent multi-scale feature fusion, including: The original deep features are mixed convolved, and a feature splicing module is used after the convolution module to fuse the previous layer and shallow layer features to form a multi-scale fused feature map. Then, the obtained feature map is used to extract features of different levels and abstraction levels through the C2f module; the C2f module first passes the multi-scale fused feature map through the first convolution layer cv1, and then divides the multi-scale fused feature map into two parts; one part is directly passed to the output, and the other part is processed through multiple Bottleneck modules. Finally, the results of the two parts are spliced ​​in the channel dimension and passed through the second convolution layer cv2 to obtain a new feature output for subsequent model training; The feature map processed by the C2f module is then processed by hybrid convolution. At the same time, the SEnet attention mechanism is added after the hybrid convolution module to optimize these features and fuse multi-level features, focusing perception on the important parts of the features and suppressing other useless features.

7. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 1, characterized in that: The extracted original depth features are fused at multiple scales, including: First, the feature map obtained by each C3 module in the New CSP-Darknet53 backbone network is input into the scale sequence feature fusion module. The scale sequence fusion module first adjusts the number of channels of the image size feature map to be consistent through 1x1 convolution, and then uses bilinear interpolation to adjust the size of all feature maps to the same size; Next, the unsqueeze method is used to increase the dimension of each feature layer, changing it from a 3D tensor [height, width, channel] to a 4D tensor [depth, height, width, channel] and then concatenating the 4D feature maps along the depth dimension to form a 3D feature map for subsequent convolution; Finally, 3D convolution, 3D batch normalization and SiLU activation function are used to complete scale sequence feature extraction.

8. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 7, characterized in that: Add the 3D feature map to the 20th layer feature to obtain more feature information of different sizes, and obtain feature map D1; Use Bi_Concat to concatenate the 22nd layer features with the 6th and 16th layers to obtain more feature information of different scales; then input the C2f module to further extract features and obtain feature map D2; The feature map D2 is concatenated with the feature map of the 11th layer to achieve multi-scale fusion. The concatenated and fused feature map of D2 and the 11th layer is input into the C2f module for final feature extraction. The final feature map is recorded as D3.

9. A small target foreign body detection method based on YOLO fusion multi-scale sequence construction as claimed in claim 1, characterized in that: According to the feature maps of three different scales, the yolov5 detection head is input to obtain the prediction results, which specifically include three feature maps of different scales D1, D2, and D3 input into the final detection head. Then the model uses the grid-based anchor to predict the type and position information within the network, and then performs prediction and post-processing after positive sample matching and loss calculation based on the detection information; finally, the model prediction results are output, including the category of each target and its corresponding bounding box coordinates and other information.

Images