Composite plate quality inspection model, method and system based on improved YOLOv framework

Abstract

The present invention relates to the technical field of prefabricated structures and large computer models in civil engineering, and in particular to a composite plate quality inspection model, method and system based on an improved YOLOv framework. The present invention proposes a composite plate quality inspection model construction method based on an improved YOLOv framework. First, the model framework is improved on the basis of YOLOv8n, and an additional feature fusion line is added to the shallow network to increase the proportion of small targets in the feature fusion information. In the neck network structure of the improved model, the four layers of information extracted by the backbone network are first convolved to adjust the feature map size for feature fusion. After fusion, the information is output from the shallow layer to the deep layer to four size segmentation heads for target segmentation. In this way, the components in the composite plate are detected by using an instance segmentation method, and an accurate target contour can be obtained, which is convenient for obtaining target area, size and other information in subsequent research to meet the actual engineering application, filling the gap in image recognition methods suitable for quality inspection of composite plates with complex structures.

Description

Technical Field

[0001] The present invention relates to the technical field of civil engineering prefabricated structures and computer large-scale models, and in particular to a composite plate quality inspection model, method and system based on an improved YOLOv framework. Background Art

[0002] With the reform of the traditional construction industry, prefabricated components are increasingly used around the world. Such components are mainly prefabricated in factories or off-site, and then transported to the construction site for assembly. This off-site production and assembly method brings convenience and efficiency to on-site construction. But on the other hand, there is a lack of coordination and collaboration between different operations such as the production and on-site assembly of prefabricated components. As for the widely used prefabricated composite panels, due to the characteristics of short production cycle, simple template production, and large construction demand, it is already a horizontal component with a very high production volume among prefabricated components, and it generally has the problem of uncertainty in quality control. The component layout in the produced composite panels may not meet the requirements of the drawings, such as the wrong number and position layout of each embedded part, and the offset of the truss reinforcement position. The layout of these components plays a decisive role in whether the composite panels can be safely and effectively used for on-site assembly, so the factory needs to perform real-time detection when producing composite panels.

[0003] After pouring concrete, the composite slab components are relatively simple and clear, and the inspection is not difficult. Therefore, most of the quality inspections in the factory are still set after pouring concrete. On the one hand, this inspection timing cannot observe the reinforcement buried in the concrete, which poses a certain quality control risk; on the other hand, if the composite slab after pouring concrete is inspected, it still needs to be reworked or scrapped, wasting manpower and material resources. Considering the above factors, the factory also began to set the inspection of composite slab components before pouring concrete, but at this stage, the horizontal and vertical distribution steel bars, truss steel bars, and various embedded parts of the composite slab are staggered, and there are mutual obstructions, and multiple targets need to be inspected at the same time. This task is very difficult to inspect according to the current manual visual inspection method. It is not only inefficient and requires a lot of manpower, but the inspection accuracy is also affected by subjective factors, which affects the production process of the composite slab assembly line. With the development of artificial intelligence technology, image recognition technology has become mature and has begun to be used for multi-target quality inspection in engineering. However, the current image recognition technology performs better on images with fewer features, and the inspection effect is poor for composite slab components with staggered parts and complex features. Summary of the invention

[0004] In order to overcome the problem in the above-mentioned prior art of lack of image recognition methods suitable for quality inspection of composite panels with complex structures, the present invention proposes a method for constructing a composite panel quality inspection model based on an improved YOLOv framework, and realizes a deep learning model for high-precision and high-efficiency detection of the distribution of embedded parts of composite panels, which is conducive to promoting intelligent inspection of composite panels.

[0005] The present invention proposes a method for constructing a composite plate quality inspection model based on an improved YOLOv framework:

[0006] Firstly, the YOLOv8n network framework is improved to obtain the laminated plate quality inspection model. The improvement method is as follows: the neck network of the YOLOv8n network is replaced by the SOBI network module; the SOBI network module includes two weighted feature fusion modules and four multi-scale feature fusion modules connected sequentially;

[0007] Let the four feature extractors of the backbone network of the YOLOv8n network from the shallow layer to the deep layer be respectively recorded as the first feature extractor, the second feature extractor, the third feature extractor and the fourth feature extractor;

[0008] In the SOBI network module, the first weighted feature fusion module performs weighted fusion on the convolution of the output data of the third feature extractor in the backbone network and the output data of the fourth feature extractor, and the second weighted feature fusion module performs weighted fusion on the convolution of the output data of the first weighted feature fusion module and the output data of the second feature extractor in the backbone network; the first multi-scale feature fusion module performs multi-scale fusion on the convolution of the output data of the second weighted feature fusion module and the output data of the first feature extractor in the backbone network; the second multi-scale feature fusion module performs multi-scale fusion on the convolution of the output data of the first multi-scale feature fusion module, the output data of the second weighted feature fusion module and the output data of the second feature extractor in the backbone network; the third multi-scale feature fusion module performs multi-scale fusion on the convolution of the output data of the second multi-scale feature fusion module, the output data of the first weighted feature fusion module and the output data of the third feature extractor in the backbone network; the fourth multi-scale feature fusion module performs multi-scale fusion on the convolution of the output data of the third multi-scale feature fusion module and the output data of the fourth feature extractor in the backbone network;

[0009] Then, a learning data set is constructed, which is used to store images of composite panels marked with various types of embedded parts and embedded holes; and a composite panel quality inspection model is trained on the learning data set to obtain a composite panel quality inspection model for identifying various types of embedded parts and embedded holes.

[0010] Preferably, the SOBI network module includes a second convolutional network, a third convolutional network, a fourth convolutional network, a fifth convolutional network, and a first sampling splicing module, a second sampling splicing module, a third sampling splicing module, a first convolution splicing module, a second convolution splicing module, and a third convolution splicing module connected in sequence;

[0011] The first sampling splicing module, the second sampling splicing module and the third sampling splicing module are collectively referred to as sampling splicing modules, and the sampling splicing modules are composed of a sampling network, a splicing network and a C2f network connected in sequence; the first convolution splicing module, the second convolution splicing module and the third convolution splicing module are collectively referred to as convolution splicing modules, and the convolution splicing modules are composed of a convolution network, a splicing network and a C2f network connected in sequence; the input end of the second convolution network, the input end of the third convolution network, the input end of the fourth convolution network and the input end of the fifth convolution network are used as the four input ends of the SOBI network module, which are used to connect the four feature extractors of the backbone network from shallow to deep;

[0012] The input of the first sampling splicing module is connected to the output of the fifth convolutional network, and the input of the splicing network in the first sampling splicing module is also connected to the output of the fourth convolutional network; the input of the splicing network in the second sampling splicing module is also connected to the output of the third convolutional network, and the input of the splicing network in the third sampling splicing module is also connected to the output of the second convolutional network; the input of the splicing network in the first convolutional splicing module is also connected to the output of the third convolutional network and the output of the second sampling splicing module, the input of the splicing network in the second convolutional splicing module is also connected to the output of the fourth convolutional network and the output of the first sampling splicing module, and the input of the splicing network in the third convolutional splicing module is also connected to the output of the fifth convolutional network;

[0013] The output end of the third sampling splicing module, the output end of the first convolution splicing module, the output end of the second convolution splicing module and the output end of the third convolution splicing module serve as the four output ends of the SOBI network module, which are used to connect the corresponding segmentation heads of the head network.

[0014] Preferably, the splicing network in the third sampling splicing module, the splicing network in the first convolution splicing module, the splicing network in the second convolution splicing module and the splicing network in the third convolution splicing module all use SDI modules;

[0015] The SDI module includes: a spatial attention unit, a channel attention unit, a sampling unit, a smoothed convolutional network and a channel stitching unit; the multiple data input to the SDI module are sequentially passed through the spatial attention unit and the channel attention unit to extract spatial attention signs and channel attention features; the data output by the channel attention unit is processed by the sampling unit so that the multiple data output by the sampling unit maintain the same dimension; the multiple data of the same dimension output by the sampling unit are convolved through a smoothed convolutional network and then sent to the channel stitching unit for dimensional stitching as the output data of the SDI module.

[0016] Preferably, the splicing network of the first sampling splicing module and the splicing network of the second sampling splicing module both adopt Bi-FPN networks.

[0017] Preferably, in the SOBI network module, the sampling network adopts the Upsample network; and in the sampling splicing module and the convolution splicing module, the back end of the splicing network is connected to a 3-layer C2f network.

[0018] Preferably, in the SOBI network module, the splicing network adopts a Concat network.

[0019] The present invention proposes a composite plate quality inspection method based on an improved YOLOv framework. First, the composite plate quality inspection model is obtained by using the composite plate quality inspection model construction method based on the improved YOLOv framework; the captured image of the composite plate to be inspected is input into the composite plate quality inspection model, and the composite plate quality inspection model marks various types of embedded parts and embedded holes; and whether the composite plate is qualified is judged according to the marking results.

[0020] Preferably, the quality inspection model of the composite board marks the pixel positions of various types of embedded parts and embedded holes on the composite board; when judging whether the composite board is qualified, first traverse the category labels to determine whether the number of various types of embedded parts and embedded holes is correct; if not, the composite board is judged to be unqualified; if yes, compare whether the overlap between the pixel position annotation value of each annotated object and the pixel position design value reaches the set threshold; if no, the composite board is judged to be unqualified; if yes, the composite board is judged to be qualified; the annotated objects are the various types of embedded parts and embedded holes.

[0021] The present invention proposes a composite plate quality inspection system based on an improved YOLOv framework, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is connected to the memory, and the processor is used to execute the computer program to implement the composite plate quality inspection method based on the improved YOLOv framework.

[0022] The present invention proposes a storage medium storing a computer program, which, when executed, is used to implement the composite plate quality inspection method based on the improved YOLOv framework.

[0023] The advantages of the present invention are:

[0024] (1) The present invention proposes a method for constructing a composite plate quality inspection model based on an improved YOLOv framework. First, the model framework is improved on the basis of YOLOv8n, and an additional feature fusion line is added to the shallow network to increase the proportion of small targets in the feature fusion information. In the neck network structure of the improved model, the four layers of information extracted by the backbone network are first convolved to adjust the feature map size for feature fusion. After fusion, the information is output from the shallow layer to the deep layer to the four size segmentation heads for target segmentation. In this way, the components in the composite plate are detected by instance segmentation, and an accurate target contour can be obtained, which is convenient for obtaining target area, size and other information in subsequent research to meet practical engineering applications.

[0025] (2) SDI processes the feature maps of each layer through spatial and channel attention mechanisms before adding them together. This attention mechanism improves the accuracy of feature map information by integrating local spatial information and global channel information in the feature map. SDI mainly addresses the problem of information fusion weights of feature maps with a large number of layers. Replacing the information fusion module in the deep layer of the neck network with the SDI module improves the detection accuracy while maintaining a high detection speed.

[0026] (3) The weighted feature fusion module BiFPN is set at the shallow fusion module of the neck network. It introduces a learnable weight for each connection edge, allowing the model to adaptively adjust the fusion method according to the importance of different features. This weighted method optimizes the fusion effect of multi-scale feature fusion and improves the accuracy of feature representation.

[0027] (4) The present invention proposes a combination of SDI and Bi-FPN modules. The SDI and Bi-FPN modules are introduced into the neck network, and a suitable combination and configuration position are found to greatly improve the model accuracy.

[0028] (5) The present invention uses images of laminated panels with various types of embedded parts and embedded holes to train the improved model to meet the research needs. The dataset is constructed by combining field sampling and data enhancement, which improves the robustness of the model and the instance segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is the YOLOv8n network structure diagram;

[0030] Figure 2 This is the network structure diagram of the network module SOBI;

[0031] Figure 3 This is the network structure diagram of the model YOLOv8_SOBI;

[0032] Figure 4 This is the network structure diagram of module SDI;

[0033] Figure 5 This is the network structure diagram of the model YOLOv8_SBS;

[0034] Figure 6 Advance network architectures for different models;

[0035] Figure 7 Build a method flow chart for the model;

[0036] Figure 8Schematic diagram of the detection results of different models; where a1 is the original image taken by the first superimposed board, b1 is the original image taken by the second superimposed board, and c1 is the original image taken by the third superimposed board; a2 is the detection result of the model YOLOv_SBS for image a1, b2 is the detection result of the model YOLOv_SBS for image b1, and c2 is the detection result of the model YOLOv_SBS for image c1; a3 is the detection result of the model YOLOv5 for image a1, b3 is the detection result of the model YOLOv5 for image b1, and c3 is the detection result of the model YOLOv5 for image c1. test results; a4 is the detection result of model YOLOv7 on image a1, b4 is the detection result of model YOLOv7 on image b1, and c4 is the detection result of model YOLOv7 on image c1; a5 is the detection result of model YOLOv8n on image a1, b5 is the detection result of model YOLOv8n on image b1, and c5 is the detection result of model YOLOv8n on image c1; a6 is the detection result of model YOLOv9 on image a1, b6 is the detection result of model YOLOv9 on image b1, and c6 is the detection result of model YOLOv9 on image c1. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The method for constructing a composite plate quality inspection model based on an improved YOLOv framework is proposed in this paper. Figure 1 The neck network of the YOLOv8n framework shown is improved to form a model framework that is more suitable for overlapping plate detection.

[0039] Figure 1 The YOLOv8n framework shown is a commonly used deep learning model framework in this field and will not be described in detail here. It is worth noting that Figure 1 The convolutional network uses the Conv network as an example. In the specific implementation, other convolutional networks such as CNN can also be used; the C2f network and SPPF module are inherent network modules of the YOLOv8n framework and are not explained; Upsample is an existing sampling network, and the Concat network is an existing dimensional splicing network.

[0040] The following mainly introduces two improved model frameworks proposed in the present invention.

[0041] The first improved model is recorded as the composite plate quality inspection model YOLOv_SOBI

[0042] The original YOLOv8n neck network adopts the PAN-FPN (Path Aggregation Network-Feature Pyramid Network) structure. The PAN (Path Aggregation Network) structure extracts three-layer feature pyramid information in the backbone network, while the FPN (Feature Pyramid Network) structure performs data feature fusion, but the two operations are performed independently. The transmission of the three-layer feature pyramid information to the FPN structure will cause some information loss, especially for the idea of ​​the floor in the data set.

[0043] The model YOLOv_SOBI proposed in this paper refers to the skip connection of the bidirectional connection pyramid to the neck network downsampling structure in BiFPN to perform feature fusion and realize Figure 2 The SOBI (small object Bi-directional) module shown.

[0044] When the traditional BiFPN concept is used in the YOLO network, the last three layers of information of the feature pyramid are mostly used for fusion. However, the target size span in the composite plate data set is large, and it is difficult to identify small targets such as reinforcement ribs. Only fusing three layers of information cannot fully analyze the multi-scale target features, and the small target information is insufficient and easy to be ignored. In the process of feature pyramid information processing, the shallow network receptive field is relatively small and the image resolution is high, so the small target information contained is richer and contains the contextual relationship between the small target and the surrounding information. The present invention adds an additional feature fusion line to the shallow network to increase the proportion of small targets in the feature fusion information.

[0045] Finally, in the neck network structure, the four layers of information extracted by the backbone network are first convolved to adjust the size of the feature map for feature fusion. After fusion, the information is output from shallow to deep layers to four size segmentation heads for target segmentation. The segmentation head of the model is designed after the feature extraction module of the neck network from shallow to deep. Generally, the lower the number of layers, the more small-size target information is contained. Therefore, most models set three layers of segmentation heads at different positions to meet the requirements of detecting multi-scale targets. However, the target size span of the superimposed plate dataset is too large and the sizes are different. Therefore, compared with the three segmentation heads of the original YOLOv8n model, the four segmentation heads designed by the improved model have more advantages in predicting the superimposed plate dataset.

[0046] Reference Figure 3, the YOLOv_SOBI proposed in this embodiment includes a backbone network, a neck network and a head network.

[0047] The backbone network maintains the original network structure of YOLOv8n; the backbone network includes the first convolutional network, the first convolutional module, the second convolutional module, the third convolutional module, the fourth convolutional module and the SPPF network connected in sequence;

[0048] The first convolution module includes a sequentially arranged convolution network and a 3-layer C2f network; the second convolution module includes a sequentially arranged convolution network and a 6-layer C2f network; the third convolution module includes a sequentially arranged convolution network and a 6-layer C2f network; the fourth convolution module includes a sequentially arranged convolution network and a 3-layer C2f network.

[0049] A neck network SOBI network module, including a second convolutional network, a third convolutional network, a fourth convolutional network, a fifth convolutional network, a first sampling splicing module, a second sampling splicing module, a third sampling splicing module, a first convolution splicing module, a second convolution splicing module and a third convolution splicing module;

[0050] The first sampling splicing module, the second sampling splicing module and the third sampling splicing module have the same structure, collectively referred to as sampling splicing modules, which are composed of a sampling network, a Concat network and a C2f network connected in sequence; and the back end of the Concat network is connected to a 3-layer C2f network. The sampling splicing module takes the input of the sampling network as input and the output of the C2f network as output.

[0051] The first convolutional splicing module, the second convolutional splicing module and the third convolutional splicing module have the same structure and are collectively referred to as convolutional splicing modules. The convolutional splicing module is composed of a sequentially connected convolutional network, a Concat network and a C2f network; and the back end of the Concat network is connected to a 3-layer C2f network. The convolutional splicing module takes the input of the convolutional network as input and the output of the C2f network as output.

[0052] The sampling network may specifically adopt an Upsample network.

[0053] The input of the second convolutional network is connected to the output of the first convolutional module in the backbone network, the input of the third convolutional network is connected to the output of the second convolutional module in the backbone network, the input of the fourth convolutional network is connected to the output of the third convolutional module in the backbone network, and the input of the fifth convolutional network is connected to the output of the SPPF module in the backbone network;

[0054] The input of the first sampling splicing module is connected to the output of the fifth convolutional network, and the input of the splicing network in the first sampling splicing module is also connected to the output of the fourth convolutional network; the input of the splicing network in the second sampling splicing module is also connected to the output of the third convolutional network, and the input of the splicing network in the third sampling splicing module is also connected to the output of the second convolutional network;

[0055] The input of the splicing network in the first convolutional splicing module is also connected to the output of the third convolutional network and the output of the second sampling splicing module, the input of the splicing network in the second convolutional splicing module is also connected to the output of the fourth convolutional network and the output of the first sampling splicing module, and the input of the splicing network in the third convolutional splicing module is also connected to the output of the fifth convolutional network.

[0056] The head network includes four segmentation heads, and the inputs of the four segmentation heads are respectively connected to the output of the third sampling splicing module, the output of the first convolution splicing module, the output of the second convolution splicing module and the output of the third convolution splicing module.

[0057] The second improved model is recorded as the composite plate quality inspection model YOLOv_SBS

[0058] The improvement of the neck network of the model YOLOv_SOBI solves the multi-scale problem of the target in the data set by adding multiple layers of information fusion. However, adding multiple layers of information also makes it more difficult to distinguish information of each scale. In order to better distinguish information of different scales and perform targeted fusion, the present invention improves the feature fusion module. First, the weighted feature fusion module BiFPN is introduced, which is set at the shallow fusion module of the neck network. Compared with the original Concat module, Bi-FPN introduces a learnable weight for each connecting edge, allowing the model to adaptively adjust the fusion method according to the importance of different features. This weighted method optimizes the fusion effect of multi-scale feature fusion and improves the accuracy of feature representation.

[0059] In addition, a multi-scale feature fusion module SDI is introduced. The SDI structure is as follows Figure 4As shown in the figure, similar to the original model YOLOv8n feature fusion module Concat, both make the multi-layer feature map information pass through the module processing and perform addition operations in the dimension. But the difference is that SDI first processes the feature maps of each layer through the spatial and channel attention mechanism before adding the multi-layer feature maps. This attention mechanism improves the accuracy of the feature map information by integrating the local spatial information and the global channel information in the feature map. For example, when the information is fused in the composite plate data set, the local concentrated information of the small target of the reinforcement rib can be sorted with the overall information around the target, and then the target information will be clearer when it is input into the segmentation head detection after feature extraction. Therefore, this module SDI mainly targets the problem of the fusion weight of the feature map information with a large span of layers. The feature information fusion layer span of the first two layers of the neck network is small, so only the information fusion module of the subsequent neck network is replaced by the SDI module, while improving the detection accuracy and maintaining a high detection speed.

[0060] Based on the model YOLOv_SOBI, the Concat network in the first sampling splicing module and the Concat network in the second sampling splicing module in the model YOLOv_SBS are replaced by Bi-FPN networks.

[0061] The Concat network in the third sampling splicing module of the model YOLOv_SBS, the Concat network in the first convolution splicing module, the Concat network in the second convolution splicing module, and the Concat network in the third convolution splicing module are all replaced with the SDI module

[0062] The SDI module includes: spatial attention unit, channel attention unit, sampling unit, smooth convolution network (SmoothConv) and channel stitching unit; multiple data input to the SDI module are sequentially passed through the spatial attention unit and the channel attention unit to extract spatial attention signs and channel attention features; the data output by the channel attention unit is processed by the sampling unit, the sampling unit downsamples the high-dimensional data, downsamples the low-dimensional data, and outputs the intermediate-dimensional data with the original dimension, so that the multiple data output by the sampling unit maintain the same dimension; the multiple data of the same dimension output by the sampling unit are convolved through the smooth convolution network, and then sent to the channel stitching unit for dimensional stitching as the output data of the SDI module.

[0063] Obviously, in the YOLOv_SBS model, the SDI module in the third sampling splicing module and the third convolution splicing module has two inputs. For each input data, the spatial attention feature is first extracted, and then the channel attention feature is extracted. The channel attention feature is sampled by the sampling unit and then smoothed by convolution. The smoothed convolution corresponding to the two input data is spliced ​​by the dimensional splicing unit as the output of the SDI module.

[0064] The SDI module in the first convolutional splicing module and the second convolutional splicing module has three inputs. For each input data, the spatial attention feature is first extracted, and then the channel attention feature is extracted. The channel attention feature is sampled by the sampling unit and then smoothed convolution is performed; the smoothed convolution corresponding to the three input data is spliced ​​by the dimensional splicing unit as the output of the SDI module.

[0065] The sampling unit of the SDI module samples and processes the two input data and outputs two data of the same dimension; samples and processes the three input data and outputs three data of the same dimension.

[0066] It is worth noting that when the sampling module processes two or three input data, the dimension of the input data determines whether to upsample, downsample, or maintain the original dimension output. If the dimensions of the two input data are different, the input data can be sampled by upsampling and downsampling respectively, so that the dimensions of the sampled output data are equal; one channel of data can also be output with the original dimension, and the other channel of data can be adjusted to the dimension of the data output with the original dimension through upsampling or downsampling. If the dimensions of the two input data are the same, the sampling method of the two input data is the same, so that the dimensions of the sampled output data are also equal. The same is true for three-channel input, ensuring that the three channels of data have the same dimension after passing through the sampling unit.

[0067] In order to verify the performance of the improved model framework of the present invention in the detection of laminated plates, the above two laminated plate quality inspection models YOLOv_SOBI and YOLOv_SBS were compared and verified with multiple comparison models in the same experimental environment.

[0068] The comparison models selected in this embodiment include the existing models YOLOv5, YOLOv7, YOLOv8n and YOLOv9; and also include the deformed models YOLOv_A and YOLOv_B.

[0069] Reference Figure 6 Compared with YOLOv_SOBI, the Concat network in the first sampling splicing module of YOLOv_A adopts Bi-FPN, and the Concat network in the second sampling splicing module, the Concat network in the third sampling splicing module, the Concat network in the first convolution splicing module, the Concat network in the second convolution splicing module, and the Concat network in the third convolution splicing module all adopt SDI modules;

[0070] Reference Figure 6Compared with YOLOv_SOBI, the Concat network in the first sampling splicing module, the Concat network in the second sampling splicing module, and the Concat network in the third sampling splicing module of YOLOv_B all use Bi-FPN, and the Concat network in the first convolution splicing module, the Concat network in the second convolution splicing module, and the Concat network in the third convolution splicing module all use SDI modules.

[0071] In this embodiment, in the models YOLOv_SOBI, YOLOv_SBS and all the comparison models, the first convolutional network, the second convolutional network, the third convolutional network, the fourth convolutional network and the fifth convolutional network all adopt Conv convolutional networks.

[0072] The method for constructing the data set in this embodiment is:

[0073] First, the researchers went to the prefabricated component factory to take field photos, and obtained 307 1024*768 pixel images of the composite slab containing components but without concrete as the original images, and manually annotated the outlines of the components, including metal junction boxes (MJB), plastic junction boxes (PJB), reserved holes (RH), truss bars (TB), hoisting reinforcement bars (RR) and composite slab frames (FL);

[0074] The 307 original images were augmented to obtain 1535 labeled samples with component outlines. The data augmentation included exposure, darkening, noise, and sharpening processing. The labeled samples were divided into training set, validation set, and test set according to the ratio of 7:2:1. The results are shown in Table 1.

[0075] Table 1: Dataset information

[0076] ;

[0077] Reference Figure 7 All models obtained in this embodiment are machine-learned on the training set and the validation set, and the model performance is tested on the test set after the model training is completed. During the model training process, the maximum number of training rounds is 300, and if there is no improvement in 50 rounds, the training is terminated in advance to prevent the training model from overfitting. That is, during the model training process, the convergence condition of the model is judged as follows: the model training rounds reach 300 rounds, or the accuracy of the model has not been improved in the last 50 rounds of training.

[0078] The training process specifically includes the following steps:

[0079] The first step is to extract 16 samples from the training set as training samples, and let the model perform machine learning on the training samples;

[0080] The second step is to extract 16 validation samples from the validation set and calculate the model loss on the validation samples;

[0081] The third step is to determine whether the number of training rounds has reached 300; or whether the model accuracy of the last 50 rounds has converged, that is, whether the model accuracy range of the last 50 rounds tested on the test set is less than or equal to the set threshold; if yes, fix the model; if not, return to the first step.

[0082] All experiments in this embodiment are conducted on the Pytorch supercomputing platform of the Linux system, and the model training environment is Python-3.9.15 torch-2.1.0. Because GPU computing is more efficient than CPU computing and can perform deep learning tasks faster, this experiment uses the NVIDIA A100-SXM4-80GB computing model. Hyperparameters are set to default settings, such as the initial learning rate is 0.01, the weight decay is 0.0005, etc.

[0083] In this embodiment, the test results of the present invention models YOLOv_SOBI, YOLOv_SBS and the comparison models YOLOv5, YOLOv7, YOLOv8n, YOLOv9, YOLOv_A and YOLOv_B on the test set are shown in Table 2 below.

[0084] Table 2: Model performance test

[0085] ;

[0086] Combined with the test results in Table 2, it can be seen that YOLOv8n has higher accuracy than other models YOLOv5, YOLOv7 and YOLOv9, and the average accuracy of the model on all labels has reached 87.8% mAP, and it is the most lightweight. It can be seen that it is more reasonable for the present invention to select YOLOv8 as the baseline model. Although the overall accuracy of the YOLOv9 model is slightly higher than that of YOLOv8n, the detection speed is slow. The improved model YOLOv_SBS is improved based on the YOLOv8n baseline model, which not only retains and surpasses the advantage of YOLOv8n in detecting the high accuracy of reinforcement ribs, but also surpasses the advantage of YOLOv9 in detecting other targets with high accuracy. The average accuracy of the YOLOv_SBS model reached 92.3%, which is much higher than other models, and the FPS is also higher than other models, far exceeding other series models, and maintaining lightweight.

[0087] By comparing the YOLOv_SBS model with the YOLOv_A and YOLOv_B models, we can see that both the SDI module and the Bi-FPN module adjust the feature map before feature fusion, so as to better screen and predict the large and small target information. However, the correct configuration position of the Bi-FPN and SDI modules affects the accuracy of information distinction.

[0088] In order to more intuitively analyze the difference between YOLOv_SBS and other models in the segmentation of composite board components, the segmentation visualization results are organized as follows: Figure 8 . The first row of the figure shows three randomly selected original images of the superimposed plate, which contain all the component categories of the superimposed plate, to show the segmentation effect of various models on different components. The second row of images shows the segmentation effect of the model YOLOv_SBS on the components. In these three images, all kinds of components are completely segmented by YOLOv_SBS, without missed detection or multiple detection. Models YOLOv5, YOLOv7, YOLOv8n and YOLOv9 can only roughly find the corresponding component targets, but they all miss many segmentation areas to varying degrees, especially the reinforcement ribs. Models YOLOv5, YOLOv7, YOLOv8n and YOLOv9 did not identify all the components in these three images.

[0089] Of course, it is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any reference numerals in the claims should not be regarded as limiting the claims involved.

[0090] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0091] The techniques, shapes, and structural parts not described in detail in the present invention are all well-known techniques.

Claims

1. A composite board quality inspection method based on an improved YOLOv framework, characterized in that: First, the laminated board quality inspection model is obtained; the image of the laminated board to be inspected is input into the laminated board quality inspection model, and the laminated board quality inspection model is marked with various types of embedded parts and embedded holes; whether the laminated board is qualified is determined according to the marked results; The quality inspection model of the laminated board marks the pixel positions of various types of embedded parts and embedded holes on the laminated board; when judging whether the laminated board is qualified, first traverse the category labels to determine whether the number of various embedded parts and embedded holes is correct; if not, the laminated board is judged to be unqualified; if yes, compare whether the overlap between the pixel position annotation value of each annotated object and the pixel position design value reaches the set threshold; If not, the composite board is judged to be unqualified; if yes, the composite board is judged to be qualified; the marked objects are various embedded parts and embedded holes; The laminated plate quality inspection model is obtained by improving the YOLOv8n network framework. The improvement method is as follows: the neck network of the YOLOv8n network is replaced by the SOBI network module; The SOBI network module includes a second convolutional network, a third convolutional network, a fourth convolutional network, a fifth convolutional network, and a first sampling splicing module, a second sampling splicing module, a third sampling splicing module, a first convolution splicing module, a second convolution splicing module, and a third convolution splicing module connected in sequence; The first sampling splicing module, the second sampling splicing module and the third sampling splicing module are collectively referred to as sampling splicing modules, and the sampling splicing modules are composed of a sampling network, a splicing network and a C2f network connected in sequence; the first convolution splicing module, the second convolution splicing module and the third convolution splicing module are collectively referred to as convolution splicing modules, and the convolution splicing modules are composed of a convolution network, a splicing network and a C2f network connected in sequence; The splicing network in the third sampling splicing module, the splicing network in the first convolution splicing module, the splicing network in the second convolution splicing module and the splicing network in the third convolution splicing module all use the SDI module; The SDI module includes: spatial attention unit, channel attention unit, sampling unit, smoothed convolutional network and channel splicing unit; The multiple data input to the SDI module are sequentially passed through the spatial attention unit and the channel attention unit to extract the spatial attention features and the channel attention features; the data output by the channel attention unit is processed by the sampling unit so that the multiple data output by the sampling unit maintain the same dimension; the multiple data of the same dimension output by the sampling unit are convolved through a smoothed convolutional network and then sent to the channel splicing unit for dimensional splicing as the output data of the SDI module.

2. The laminated board quality inspection method based on the improved YOLOv framework according to claim 1, characterized in that: The input end of the second convolutional network, the input end of the third convolutional network, the input end of the fourth convolutional network and the input end of the fifth convolutional network are used as the four input ends of the SOBI network module, which are used to connect the four feature extractors of the backbone network from shallow to deep; The input of the first sampling splicing module is connected to the output of the fifth convolutional network, and the input of the splicing network in the first sampling splicing module is also connected to the output of the fourth convolutional network; the input of the splicing network in the second sampling splicing module is also connected to the output of the third convolutional network, and the input of the splicing network in the third sampling splicing module is also connected to the output of the second convolutional network; the input of the splicing network in the first convolutional splicing module is also connected to the output of the third convolutional network and the output of the second sampling splicing module, the input of the splicing network in the second convolutional splicing module is also connected to the output of the fourth convolutional network and the output of the first sampling splicing module, and the input of the splicing network in the third convolutional splicing module is also connected to the output of the fifth convolutional network; The output end of the third sampling splicing module, the output end of the first convolution splicing module, the output end of the second convolution splicing module and the output end of the third convolution splicing module serve as the four output ends of the SOBI network module, which are used to connect the corresponding segmentation heads of the head network.

3. The laminated board quality inspection method based on the improved YOLOv framework according to claim 1, characterized in that: The splicing network of the first sampling splicing module and the splicing network of the second sampling splicing module both adopt Bi-FPN networks.

4. The laminated board quality inspection method based on the improved YOLOv framework according to claim 1, characterized in that: In the SOBI network module, the sampling network uses the Upsample network; and in the sampling splicing module and the convolution splicing module, the back end of the splicing network is connected to a 3-layer C2f network.

5. The laminated board quality inspection method based on the improved YOLOv framework according to claim 1, characterized in that: In the SOBI network module, the concatenation network uses the Concat network.

6. A laminated plate quality inspection system based on an improved YOLOv framework, characterized in that: It includes a memory and a processor, the memory stores a computer program, the processor is connected to the memory, and the processor is used to execute the computer program to implement the composite plate quality inspection method based on the improved YOLOv framework as described in any one of claims 1-5.

7. A storage medium, characterized in that: A computer program is stored, and when the computer program is executed, it is used to implement the composite plate quality inspection method based on the improved YOLOv framework as described in any one of claims 1 to 5.

Images