Construction quality prediction model training and detection method and device, and storage medium

Abstract

The invention discloses a construction quality prediction model training and detection method and device, and a storage medium, and the training method comprises the steps: obtaining a sample image set, wherein the sample image set comprises positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene; training a YOLOv9 model based on the positive sample and the negative sample in the sample image set, wherein the YOLOv9 model comprises a main prediction branch and an auxiliary reversible branch; based on the first loss of the main prediction branch and the second loss of the auxiliary reversible branch, determining the prediction result of the input sample of the YOLOv9 model and the model loss between the corresponding construction quality category labels; parameters of the YOLOv9 model are adjusted based on model loss, and the positive samples and the negative samples in the sample image set continue to be used for training the YOLOv9 model; and taking the model loss function converged YOLOv9 model as a trained construction quality detection model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method and detection method, device, and readable storage medium for a construction quality prediction model. Background Art

[0002] Traditional construction sites are usually in complex environmental conditions, and there may be factors such as insufficient lighting, bad weather, and dust, which affect the image quality and the accuracy of the algorithm. The objects on the construction site may be diverse, blocked, and deformed, which increases the difficulty of object recognition. At the construction site, the detection of safe construction quality usually requires real-time performance, requiring the algorithm to be able to detect and feedback quickly and accurately, which poses a challenge to the performance and efficiency of the algorithm.

[0003] How to improve the accuracy and real-time performance of safety construction quality inspection is a technical problem that needs to be solved at present. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a training method and a detection method, device, and readable storage medium for a construction quality prediction model, so as to solve the problems of low accuracy and real-time performance of construction quality detection.

[0005] In order to solve the above technical problems, this specification is implemented as follows: In a first aspect, a training method for a construction quality detection model is provided, comprising: Acquire a sample image set, wherein the sample image set includes positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene, and the sample of the target construction scene includes the main features of the target object in the target construction scene; The YOLOv9 model is trained based on the positive samples and negative samples in the sample image set, wherein the YOLOv9 model includes a main prediction branch and an auxiliary reversible branch, wherein the main prediction branch is used to predict the bounding box and the construction quality category of the target object in the input sample, and the auxiliary reversible branch is used to predict the bounding box and the construction quality category of the target object in the input samples of different scales based on multiple levels, and integrate the gradient information of each level and back-propagate it to the main prediction branch for updating the parameters of the YOLOv9 model, wherein one level corresponds to one scale; Determine a model loss between a prediction result of an input sample of the YOLOv9 model and a corresponding construction quality category label based on a first loss of the main prediction branch and a second loss of the auxiliary reversible branch, wherein the first loss is determined by a loss of a bounding box prediction of the target object by the main prediction branch and a loss of a construction quality category prediction by the main prediction branch, and the second loss is determined by a total loss of a bounding box prediction of the target object at each level and a total loss of a construction quality category prediction at each level; Adjusting the parameters of the YOLOv9 model based on the model loss and continuing to train the YOLOv9 model using the positive samples and negative samples in the sample image set; The YOLOv9 model with converged model loss function is used as the trained construction quality inspection model.

[0006] Optionally, before training the YOLOv9 model based on the positive samples and negative samples in the sample image set, the method further includes: Using the Sobel operator to perform a convolution operation on the target sample image to detect the edge of the target sample image; Determine the clarity of the target sample image based on the spatial complexity and noise complexity of the target sample image; By converting the target sample image from the spatial domain to the frequency domain, the global information of the target sample image is extracted; If the edge, clarity and global information of the target sample image do not meet the preset conditions, the target sample image is deleted from the sample image set; The bounding box and the construction quality category of the target object in each sample image in the sample image set are labeled to obtain positive samples and negative samples.

[0007] Optionally, the main prediction branch includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract multiple key image features of the input sample, the neck network is used to fuse the multiple key image features, and the head network is used to predict the bounding box and construction quality category of the input sample based on the fused features.

[0008] Optionally, the auxiliary reversible branch includes a feature pyramid network and a path aggregation network. The feature pyramid network includes multiple levels of feature pyramids, which are respectively used to extract multiple key image features of the input samples of different scales and predict bounding boxes and construction quality categories of targets in the input samples of different sizes; The path aggregation network is arranged between the feature pyramids of each level of the feature pyramid network and the main prediction branch, and is used to integrate the gradient information of the feature pyramids of each level and back-propagate to the main prediction branch and calculate the corresponding prediction loss of the feature pyramids of each level on the path aggregation network.

[0009] Optionally, the second loss is the total loss of the bounding box prediction of the target object by multiple levels of the feature pyramid network, the total loss of the construction quality category prediction by multiple levels of the feature pyramid network, and the sum of the corresponding prediction losses of the feature pyramids of each level calculated by the path aggregation network on the path aggregation network.

[0010] Optionally, after using the YOLOv9 model with converged model loss function as the trained construction quality detection model, the method further includes: Determine a first sample in the sample image set having an erroneous prediction result by comparing the labels of the positive samples and the negative samples in the sample image set with the prediction results of each sample output by the YOLOv9 model; Adjust the image corresponding to the first sample based on the position change between the predicted bounding box in the prediction result of the first sample output by the YOLOv9 model and the real bounding box marked in the label of the first sample, and the change between the predicted construction quality category in the prediction result of the first sample output by the YOLOv9 model and the construction quality category marked in the label of the first sample; The samples used to train the YOLOv9 model are updated based on the adjusted first samples and the corresponding labels.

[0011] Optionally, adjusting the image corresponding to the first sample based on a position change between a predicted bounding box in a prediction result of the first sample output by the YOLOv9 model and a labeled bounding box in a label of the first sample, and a change between a predicted construction quality category in a prediction result of the first sample output by the YOLOv9 model and a labeled construction quality category in the label of the first sample, comprises: Calculate the intersection-over-union ratio of the predicted bounding box in the prediction result of the first sample output by the YOLOv9 model and the labeled bounding box in the label of the first sample to determine whether the predicted bounding box of the first sample is offset; If a shift occurs, the labeled bounding box of the first sample is rotated by a preset angle; Determine whether the predicted construction quality category in the prediction result of the first sample output by the YOLOv9 model is different from the labeled construction quality category in the label of the first sample; If they are different, the first sample is enlarged and the image clarity of the first sample is adjusted.

[0012] In a second aspect, a construction quality detection method is provided, comprising: Acquire an image of a target construction scene, wherein the image contains main features of a target object in the target construction scene; The image is input into a pre-trained YOLOv9 model for prediction to obtain a bounding box and a construction quality category of the target object in the image. The YOLOv9 model is trained based on a sample image set including positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene. The YOLOv9 model includes a main prediction branch and an auxiliary reversible branch. The main prediction branch is used to predict the bounding box and the construction quality category of the target object in the input sample. The auxiliary reversible branch is used to predict the bounding box and the construction quality category of the target object in the input samples of different scales based on multiple levels, and integrate the gradient information of each level and back-propagate it to the main prediction branch for updating the parameters of the YOLOv9 model, where one level corresponds to one scale.

[0013] According to a third aspect, an electronic device is provided, comprising a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and wherein the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect or the second aspect.

[0014] In a fourth aspect, a readable storage medium is provided, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect or the second aspect are implemented.

[0015] In an embodiment of the present application, a sample image set is obtained, wherein the sample image set includes positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene, and the sample of the target construction scene includes the main features of the target object in the target construction scene; the YOLOv9 model is trained based on the positive samples and negative samples in the sample image set, and the YOLOv9 model includes a main prediction branch and an auxiliary reversible branch, wherein the main prediction branch is used to predict the bounding box and construction quality category of the target object in the input sample, and the auxiliary reversible branch is used to predict the bounding box and construction quality category of the target object in the input sample of different scales based on multiple levels, and integrate the gradient information of each level to back-propagate to the main prediction branch for updating the parameters of the YOLOv9 model, where one level corresponds to one scale; the YOLO is determined based on the first loss of the main prediction branch and the second loss of the auxiliary reversible branch. v9 model and the model loss between the prediction result of the input sample of the main prediction branch and the corresponding construction quality category label, the first loss is determined by the loss of the bounding box prediction of the target object by the main prediction branch and the loss of the construction quality category prediction of the main prediction branch, and the second loss is determined by the total loss of the bounding box prediction of the target object at each level and the total loss of the construction quality category prediction at each level; the parameters of the YOLOv9 model are adjusted based on the model loss and the positive and negative samples in the sample image set are continued to be used to train the YOLOv9 model; the YOLOv9 model with converged model loss function is used as the trained construction quality detection model, thereby being able to solve the problems in safe construction quality detection with higher accuracy, faster speed and stronger robustness, improve the automation and accuracy of the construction process, effectively reduce human errors and improve construction quality, and realize more intelligent and efficient safe construction detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 It is a flowchart of the training method of the construction detection model of the embodiment of the present application.

[0017] Figure 2 It is a structural block diagram of the YOLOv9 model of an embodiment of the present application.

[0018] Figure 3 It is a flow chart of the construction quality detection method of the embodiment of the present application.

[0019] Figure 4 It is a flow chart of the construction quality detection method of the embodiment of the present application.

[0020] Figure 5 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. The numbering of the drawings in this application is only used to distinguish the various steps in the scheme, and is not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0022] In order to solve the problems existing in the prior art, the present application embodiment provides a training method for a construction quality detection model, such as Figure 1 As shown, the following steps are included: Step 102, obtaining a sample image set, wherein the sample image set includes positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene, and the sample of the target construction scene includes the main features of the target object in the target construction scene; Step 104, training a YOLOv9 model based on positive samples and negative samples in the sample image set, wherein the YOLOv9 model includes a main prediction branch and an auxiliary reversible branch, wherein the main prediction branch is used to predict a bounding box and a construction quality category of a target object in an input sample, and the auxiliary reversible branch is used to predict a bounding box and a construction quality category of a target object in input samples of different scales based on multiple levels, and integrate the gradient information of each level and back-propagate it to the main prediction branch for updating the parameters of the YOLOv9 model, wherein one level corresponds to one scale; Step 106, determining a model loss between a prediction result of an input sample of the YOLOv9 model and a corresponding construction category label based on a first loss of the main prediction branch and a second loss of the auxiliary reversible branch, wherein the first loss is determined by a loss of a bounding box prediction of the target object by the main prediction branch and a loss of a construction category prediction by the main prediction branch, and the second loss is determined by a total loss of a bounding box prediction of the target object at each level and a total loss of a construction category prediction at each level; Step 108, adjusting the parameters of the YOLOv9 model based on the model loss and continuing to train the YOLOv9 model using the positive samples and negative samples in the sample image set; Step 110, using the YOLOv9 model with converged model loss function as the trained construction quality inspection model.

[0023] Based on the solution provided in the above embodiment, specifically, in the above step 102, when obtaining the sample image set, construction pictures of different construction scenes are collected by a camera device. For example, for tower construction, the pictures include indoor antenna installation pictures, tower base secondary pouring pictures, feeder laying pictures, etc.

[0024] The shooting distance should not be too far, and the target object should occupy at least 1 / 3 of the picture. The camera should face the target object of each construction scene and shoot horizontally. The main features of the target object should be obvious, and the object itself should not be overly blocked, such as green plants, irrelevant personnel, nearby buildings, etc., so as not to affect recognition. The image pixel should not be less than 640*640, and the image data format should support jpg and png images. The image of each construction scene must contain positive and negative samples.

[0025] In order to improve the accuracy of the trained YOLOv9 model, it is necessary to pre-process each sample before inputting it into the YOLOv9 model for training.

[0026] Optionally, before training the YOLOv9 model based on the positive samples and negative samples in the sample image set, the method further includes: performing a convolution operation on the target sample image using a Sobel operator to detect the edge of the target sample image; determining the clarity of the target sample image based on the spatial complexity and noise complexity of the target sample image; extracting the global information of the target sample image by converting the target sample image from the spatial domain to the frequency domain; if the edge, clarity and global information of the target sample image do not meet preset conditions, deleting the target sample image from the sample image set; and labeling the bounding box and construction quality category of the target object in each sample image in the sample image set to obtain positive and negative samples.

[0027] Data pre-processing is a modular image pre-processing capability packaged according to actual needs, which is used to detect whether the sample image is clear, whether the shooting angle is qualified, whether the image is blocked by light, whether there are non-related obstacles, etc. In the embodiment of the present application, pre-processing is performed in combination with edge detection, image clarity evaluation and frequency domain analysis.

[0028] Specifically, edge detection uses the Sobel operator to perform convolution operations on each sample image to detect the edges in the sample image. The Sobel operator contains two sets of 3x3 matrices, which are used for horizontal and vertical edge detection respectively. If A represents the original sample image, Gx and Gy represent the grayscale values ​​of the sample image after horizontal and vertical edge detection respectively, and G represents the gradient amplitude, the corresponding formula can be expressed as: Gx = |-1 0 1| |-2 0 2| |-1 0 1|*A Gy = |-1 -2 -1| |0 0 0| |1 2 1|*A G=sqrt(Gx^2+Gy^2).

[0029] By comparing the gradient amplitude G with the preset threshold, it can be determined whether the edge detection of each sample image is qualified.

[0030] The clarity of the sample image can be evaluated by the following formula: Clarity = (SpatialComplexity - NoiseComplexity) / SpatialComplexity Among them, Clarity represents clarity, SpatialComplexity represents spatial complexity, and NoiseComplexity represents noise complexity. By comparing the clarity with the preset threshold, it can be determined whether the clarity detection of each sample image is qualified.

[0031] Frequency domain analysis is to perform Fourier transform on the sample image, converting the sample image from the spatial domain to the frequency domain. Frequency domain analysis can be used to extract global information of the sample image, such as texture, periodicity, etc. The Fourier transform formula is:

[0032] Where t represents the time variable, f(t) represents the real function in the time domain, represents the original sample image, ω represents the angular frequency, and j represents the imaginary unit.

[0033] After the sample images have undergone the above pre-processing, unqualified sample images are directly discarded, and qualified sample images are then annotated to obtain labels corresponding to each sample image for model training.

[0034] Model training is based on a large amount of labeled data, which provides information about the location, size, and construction quality category of the target in the sample image. Therefore, in order to improve the performance and accuracy of the model, the above information can be accurately labeled for safe construction quality detection with the help of a predetermined image annotation tool, thereby providing high-quality training data for the algorithm. The labels of the sample images corresponding to the construction quality category are divided into two categories: quality (qualified) and unquality (unqualified).

[0035] After obtaining a set of sample images including positive samples and negative samples, the YOLOv9 model can be trained.

[0036] Specifically, the main prediction branch includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract multiple key image features of the input sample, the neck network is used to fuse multiple key image features, and the head network is used to predict the bounding box and construction quality category of the input sample based on the fused features.

[0037] The auxiliary reversible branch includes a feature pyramid network and a path aggregation network. The feature pyramid network includes multiple levels of feature pyramids, which are respectively used to extract multiple key image features of the input samples of different scales and predict the bounding boxes and construction quality categories of targets in the input samples of different sizes; the path aggregation network is arranged between the feature pyramids of each level of the feature pyramid network and the main prediction branch, and is used to integrate the gradient information of the feature pyramids of each level and back-propagate to the main prediction branch and calculate the corresponding prediction loss of the feature pyramids of each level on the path aggregation network.

[0038] like Figure 2 As shown, the YOLOv9 model includes a main prediction branch 100 and an auxiliary reversible branch 200. The input sample 10 is input into the main prediction branch 100 and the auxiliary reversible branch 200 respectively. The main prediction branch 100 is used to predict the bounding box and construction quality category of the target object in the input sample 10. The auxiliary reversible branch 200 is used to predict the bounding box and construction quality category of the target object in the input samples of different scales based on multiple levels, and integrate the gradient information of each level to back-propagate to the main prediction branch 200 for updating the parameters of the YOLOv9 model.

[0039] The backbone network (Backbone) of the main prediction branch 100 mainly extracts image features from the input sample image. Backbone extracts key image feature information from the image through multi-layer convolution operations. The neck network (Neck) of the main prediction branch 100 mainly performs multi-scale feature fusion on the extracted key image features and passes these features to the head network (Head) of the prediction layer. The head network performs the final regression prediction based on the fused features. It is mainly responsible for predicting the position, size and construction quality category of the objects appearing in the input sample image, and uses appropriate loss functions (such as Linear Rectification Function (ReLU)) to optimize the prediction results.

[0040] The loss function of the main prediction branch 100 is: , where λ box is the weight coefficient of the loss function for predicting the bounding box of the target object, L box is the loss function for the bounding box prediction of the target object;cls is the weight coefficient of the loss function for predicting the construction quality category of the target object, L cls The loss function for predicting the construction quality category of the target object.

[0041] Loss function L box For regression prediction of the position and size of the bounding box, the SmoothL1 loss function is usually used, and the corresponding formula is:

[0042] in, is the number of positive samples, and are the true bounding box and predicted bounding box marked in the label of the i-th positive sample at the j coordinate.

[0043] Loss function L cls For multi-class classification of construction quality, the binary cross entropy (BCE) loss function is usually used. For each predicted bounding box, the corresponding formula is:

[0044] in, is the construction quality category (0 or 1) marked in the label of the sample corresponding to the i-th predicted bounding box, The predicted probabilities for the construction quality categories.

[0045] The loss function of the auxiliary reversible branch 200 uses multi-level auxiliary supervision information, Feature Pyramid Network (FPN) and Path Aggregation Network (PAN).

[0046] The feature pyramid network includes multiple levels of feature pyramids, which are used to extract multiple key image features of input samples of different scales and predict the bounding boxes and construction quality categories of targets in input samples of different sizes.

[0047] The second loss of the auxiliary reversible branch is the total loss of the bounding box prediction of the target object by multiple levels of the feature pyramid network, the total loss of the construction quality category prediction by multiple levels of the feature pyramid network, and the sum of the corresponding prediction losses of the feature pyramids of each level calculated by the path aggregation network on the path aggregation network.

[0048] Assuming that the feature pyramid network has a total of l levels, the loss function of the feature pyramid network is The formula is as follows:

[0049] in, and are the loss functions for bounding box prediction and construction quality category prediction at the lth level, respectively, and are calculated in the same way as the corresponding loss functions of the main prediction branch.

[0050] The path aggregation network is set between the feature pyramids of each level of the feature pyramid network and the main prediction branch. It is used to integrate the gradient information of the feature pyramids of each level, back-propagate to the main prediction branch, and calculate the corresponding prediction loss of the feature pyramids of each level on the path aggregation network.

[0051] The calculation method of the loss function of the path aggregation network is similar to that of the feature pyramid network. The path aggregation network fuses multiple key image features extracted from input samples of different scales through bottom-up and top-down path aggregation. The loss function formula of the prediction loss corresponding to each level of the feature pyramid calculated by the path aggregation network on the path aggregation network is as follows:

[0052] in, and They are respectively the loss function for predicting the bounding box corresponding to the fused features at the lth level and the loss function for predicting the construction quality category.

[0053] The FPN loss function and the PAN loss function are loss functions calculated at each level corresponding to the feature pyramid network and the path aggregation network, and are used together with the loss function of the main prediction branch as auxiliary supervision signals.

[0054] The loss function of the main prediction branch and the loss functions of the feature pyramid network and path aggregation network of the auxiliary reversible branch are added according to certain weights to obtain the overall loss function of the YOLOv9 model: + + .

[0055] Based on the overall loss function of the above model, the parameters of the YOLOv9 model are adjusted and the positive and negative samples in the sample image set are continued to be used to train the YOLOv9 model until the overall loss function of the model converges. The trained YOLOv9 model is obtained and used for construction quality detection.

[0056] During or after training, you can save the weights and configuration files of the YOLOv9 model for subsequent reasoning or continued training.

[0057] In order to further improve the training accuracy of the construction quality inspection model, the prediction results of the trained YOLOv9 model can be combined, the samples corresponding to the wrong prediction results can be post-processed, the samples can be adjusted, and the trained YOLOv9 model can be trained again as training samples.

[0058] Optionally, after using the YOLOv9 model with converged model loss function as the trained construction quality detection model, it also includes: determining the first sample in the sample image set with incorrect prediction results by comparing the labels of positive samples and negative samples in the sample image set with the prediction results of each sample output by the YOLOv9 model; adjusting the image corresponding to the first sample based on the position change between the predicted bounding box in the prediction result of the first sample output by the YOLOv9 model and the real bounding box marked in the label of the first sample, and the change between the predicted construction quality category in the prediction result of the first sample output by the YOLOv9 model and the construction quality category marked in the label of the first sample; and updating the sample used to train the YOLOv9 model based on the adjusted first sample and the corresponding label.

[0059] Specifically, by comparing the prediction results of the samples output by the YOLOv9 model with the labels of the corresponding samples, it is possible to determine whether the prediction results of the input samples are correct. If not, it is necessary to compare the predicted bounding box of the incorrectly predicted sample image with the true bounding box marked in the sample label, including the position and size of the bounding box.

[0060] In the embodiment of the present application, the intersection over union (IoU) is used for comparison. IoU is an indicator to measure the degree of overlap between the real bounding box and the predicted bounding box. It calculates the ratio of the intersection area of ​​the real bounding box and the predicted bounding box to the union area. The closer the IoU value is to 1, the higher the degree of overlap between the two bounding boxes and the more accurate the prediction. Assuming that the area of ​​the predicted bounding box is A and the area of ​​the real bounding box is B, the intersection area represents the area of ​​the overlapping part of the predicted bounding box and the real bounding box, recorded as A∩B.

[0061] The union area (Union) represents the area after the predicted bounding box and the true bounding box are merged, denoted as A∪B. The union area is equal to the area A of the predicted bounding box plus the area B of the true bounding box, and then subtracted from their intersection area A∩B, that is, A∪B = A + B - A∩B.

[0062] The calculation formula for the intersection over union (IoU) is: IoU = (A∩B) / (A + B - A∩B) Through the comparison analysis of the intersection over union ratio, we can find out the possible causes of prediction errors, such as low IoU values, and determine whether the predicted bounding box has horizontal or vertical offsets. By comparing the predicted construction quality category with the annotated construction quality category in the corresponding sample label, if they are different, it means that the classification is wrong, which may be caused by the low clarity of the sample image.

[0063] If an offset occurs, the annotated bounding box of the corresponding sample is rotated by a preset angle; if the clarity is low, the corresponding sample is enlarged and the image clarity is improved. The adjusted sample and the corresponding label are updated to the sample image set as a sample for retraining the YOLOv9 model.

[0064] In addition, analysis reports can be generated based on the causes of errors, providing richer information for the automated detection process of safe construction.

[0065] To explain in detail the training method of the construction quality detection model provided by the embodiment of the present invention, the following will be combined with Figure 3 It is understood that some steps in this embodiment are the same as those in the above embodiment, and the relevant description is appropriately omitted to avoid repetition. Figure 3 As shown, this embodiment includes the following steps: Step 202, pre-processing the sample images corresponding to the construction scene; Step 204, determine whether the pre-processed image is qualified, if so, proceed to step 204, otherwise proceed to step 206; Step 206, discard unqualified pictures; Step 208, labeling qualified images to obtain labels; Step 210, training a YOLOv9 model using the annotated labels and image samples; Step 212, the YOLOv9 model outputs a prediction result; Step 214, combining the prediction result and the label to determine whether the image detection is correct, if so, proceed to step 218, otherwise proceed to step 216; Step 216, post-processing the image with detection error; Step 218, returning the prediction result.

[0066] In one embodiment, after combining the prediction results and the label to determine that the image detection is incorrect in step 214, a vector database-assisted optimization model, i.e., post-processing, is introduced for the image that the YOLOv9 model detects incorrectly. The specific steps of post-processing are: 1. Build a vector database: Use Milvus to build a vector database to store vectors of erroneous images. Milvus can efficiently store and retrieve high-dimensional vectors, providing support for subsequent error analysis and model optimization. 2. Image encoding and storage: When images are passed into the YOLOv9 model for target detection, if the model detects erroneous images, it calls the Milvus vector database to encode these erroneous images into high-dimensional vectors and store them in the database. In this way, the data of erroneous images will be continuously accumulated. 3. Error image processing: When a sufficient number of erroneous images are stored in the Milvus vector database, these images can be exported for pre-processing. The pre-processing step includes quality inspection of the images and filtering out unqualified images, thereby ensuring that high-quality data is processed in subsequent steps. 4. Error analysis: During the processing process, an error analysis report can be generated through the Milvus vector database retrieval function. If quality problems are detected in the images, specifications are generated for these problems to improve the image shooting and data collection standards. 5. Model retraining: If there is no problem with the image quality, it indicates that the error may be in the YOLOv9 model itself. At this point, the already labeled error pictures are combined with the previous training data to build a new data set, and the YOLOv9 model is retrained to continuously optimize the detection effect.

[0067] Specifically, the process of generating an error analysis report from the Milvus vector database is as follows: When the YOLOv9 model detects an erroneous image, the image is converted into a high-dimensional vector and the detected erroneous image vector is stored in, for example, the Milvus vector database. Each vector represents the features of the image, including the visual information of the image and the detection error. Each image vector can be accompanied by metadata, such as image ID, detection result, label, shooting conditions, etc., for subsequent analysis.

[0068] Milvus vector database is a database designed specifically for processing high-dimensional data (such as pictures, text, sound and video, etc.). It can convert unformatted data into support and perform efficient storage and search through embedding algorithms. It extracts the features of the picture through a deep learning model and converts the picture into a high-dimensional vector representation.

[0069] Milvus vector database provides efficient similarity retrieval function, which can perform cluster analysis on error image vectors. Through similarity algorithms (such as Euclidean distance or cosine similarity), images with detected errors can be divided into different categories or groups to identify common error types. For example, images may cause consistent errors in the YOLOv9 model due to certain similar features (such as occlusion, insufficient light, etc.). The clustering results can be presented through graphical tools, which display error images by category and mark the factors that may cause errors.

[0070] Perform image quality analysis on the erroneous images stored in the Milvus vector database to check whether there are errors caused by the quality of the image itself (such as blur, uneven lighting, low resolution, etc.). By extracting these features and comparing them with similar images stored in the full image vector database, the Milvus vector database can generate statistical reports on image quality issues. For example, if occlusion or lighting problems frequently occur in a large number of images, the clustering function of the Milvus vector database will be able to identify these patterns and generate analysis reports on these features, indicating that this is the main cause of the error.

[0071] In addition, Milvus vector database can find similar images to a certain erroneous image based on vector retrieval, so as to analyze the common patterns of model detection errors. For example, it may be found that the model frequently misjudges certain objects in specific environments (such as low light at night) or for certain specific objects (such as those that are too small or complex in shape). This analysis can help generate detailed statistics on error types and list common categories of misjudgment.

[0072] Then, based on the results of cluster analysis and quality inspection, a report is automatically generated (the report can be presented in the form of tables, graphs or text to help users more intuitively understand the characteristics of erroneous images and analyze the reasons why the model may be wrong). The content includes: statistics and suggestions on image quality issues, such as lighting, resolution, shooting angle, etc., analysis of common patterns of detection errors, listing the circumstances under which the model is prone to errors, and image annotation problems (such as incorrect labels or missing labels) can also be analyzed by retrieving similar images.

[0073] The Milvus vector database is a database designed specifically for processing high-dimensional data. It can convert unformatted data into support and perform efficient storage and search through embedding algorithms. The features of the image are extracted through a deep learning model and converted into a high-dimensional vector representation. These vectors are then stored through the Milvus vector database and retrieved using its efficient indexing algorithms, such as the Inverted File Index (IVF) or Hierarchical Navigable Small World Graphs (HNSW). Next, after detecting an image that the model misjudged, its feature vector is extracted, and the vector retrieval function of the Milvus vector database is used to find the most similar image to it, and the cause of the misdetection is analyzed, which may be due to image occlusion, lighting problems, or inaccurate feature extraction. The Milvus vector database can be used to quickly locate errors and analyze them, and a post-processing report can be generated to optimize model performance.

[0074] In an embodiment of the present application, a sample image set is obtained, wherein the sample image set includes positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene, and the sample of the target construction scene includes the main features of the target object in the target construction scene; the YOLOv9 model is trained based on the positive samples and negative samples in the sample image set, and the YOLOv9 model includes a main prediction branch and an auxiliary reversible branch, wherein the main prediction branch is used to predict the bounding box and construction quality category of the target object in the input sample, and the auxiliary reversible branch is used to predict the bounding box and construction quality category of the target object in the input sample of different scales based on multiple levels, and integrate the gradient information of each level to back-propagate to the main prediction branch for updating the parameters of the YOLOv9 model, where one level corresponds to one scale; the YOLO is determined based on the first loss of the main prediction branch and the second loss of the auxiliary reversible branch. v9 model and the model loss between the prediction result of the input sample of the main prediction branch and the corresponding construction quality category label, the first loss is determined by the loss of the bounding box prediction of the target object by the main prediction branch and the loss of the construction quality category prediction of the main prediction branch, and the second loss is determined by the total loss of the bounding box prediction of the target object at each level and the total loss of the construction quality category prediction at each level; the parameters of the YOLOv9 model are adjusted based on the model loss and the positive and negative samples in the sample image set are continued to be used to train the YOLOv9 model; the YOLOv9 model with converged model loss function is used as the trained construction quality detection model, thereby being able to solve the problems in safe construction quality detection with higher accuracy, faster speed and stronger robustness, improve the automation and accuracy of the construction process, effectively reduce human errors and improve construction quality, and realize more intelligent and efficient safe construction detection.

[0075] Alternatively, if Figure 4 As shown, the embodiment of the present application also provides a construction quality detection method, which specifically includes the following steps: Step 302, acquiring an image of a target construction scene, wherein the image includes main features of a target object in the target construction scene; Step 304: input the image into a pre-trained YOLOv9 model for prediction to obtain a bounding box and a construction quality category of the target object in the image. The YOLOv9 model is trained based on a sample image set including positive samples belonging to safe construction and negative samples belonging to unsafe construction in various construction scenes. The YOLOv9 model includes a main prediction branch and an auxiliary reversible branch. The main prediction branch is used to predict the bounding box and the construction quality category of the target object in the input sample. The auxiliary reversible branch is used to predict the bounding box and the construction quality category of the target object in the input sample of different scales based on multiple levels, and integrate the gradient information of each level and back-propagate it to the main prediction branch for updating the parameters of the YOLOv9 model, where one level corresponds to one scale.

[0076] Through the above-mentioned model based on YOLOv9 training, the accuracy and real-time performance of construction quality detection can be improved, and more intelligent and efficient safety construction detection can be achieved.

[0077] The pre-trained YOLOv9 model can be based on the above Figures 1 to 3 The training method of the construction quality inspection model is obtained. To avoid repetition, it will not be repeated here.

[0078] Alternatively, if Figure 5 As shown, an embodiment of the present application further provides an electronic device 2000, including a processor 2400 and a memory 2200, wherein the memory 2200 stores programs or instructions that can be executed on the processor 2400, and when the program or instructions are executed by the processor 2400, the training method of the above-mentioned construction quality inspection model or each step of the construction quality inspection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0079] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements the training method of any of the above-mentioned construction quality detection models or the various processes of the construction quality detection method embodiment, and can achieve the same technical effect. To avoid repetition, it is not repeated here. Among them, the readable storage medium includes a computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0080] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program is operable to enable a computer to implement any of the above-mentioned training methods for construction quality inspection models or various processes of the construction quality inspection method embodiments when executed, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0081] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0083] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A training method for a construction quality detection model, characterized in that: include: Acquire a sample image set, wherein the sample image set includes positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene, and the sample of the target construction scene includes the main features of the target object in the target construction scene; The YOLOv9 model is trained based on the positive samples and negative samples in the sample image set, wherein the YOLOv9 model includes a main prediction branch and an auxiliary reversible branch, wherein the main prediction branch is used to predict the bounding box and the construction quality category of the target object in the input sample, and the auxiliary reversible branch is used to predict the bounding box and the construction quality category of the target object in the input samples of different scales based on multiple levels, and integrate the gradient information of each level and back-propagate it to the main prediction branch for updating the parameters of the YOLOv9 model, wherein one level corresponds to one scale; Determine a model loss between a prediction result of an input sample of the YOLOv9 model and a corresponding construction quality category label based on a first loss of the main prediction branch and a second loss of the auxiliary reversible branch, wherein the first loss is determined by a loss of a bounding box prediction of the target object by the main prediction branch and a loss of a construction quality category prediction by the main prediction branch, and the second loss is determined by a total loss of a bounding box prediction of the target object at each level and a total loss of a construction quality category prediction at each level; Adjusting the parameters of the YOLOv9 model based on the model loss and continuing to train the YOLOv9 model using the positive samples and negative samples in the sample image set; The YOLOv9 model with converged model loss function is used as the trained construction quality inspection model.

2. The method according to claim 1, characterized in that Before training the YOLOv9 model based on the positive samples and negative samples in the sample image set, the method further includes: Using the Sobel operator to perform a convolution operation on the target sample image to detect the edge of the target sample image; Determine the clarity of the target sample image based on the spatial complexity and noise complexity of the target sample image; By converting the target sample image from the spatial domain to the frequency domain, the global information of the target sample image is extracted; If the edge, clarity and global information of the target sample image do not meet the preset conditions, the target sample image is deleted from the sample image set; The bounding box and the construction quality category of the target object in each sample image in the sample image set are labeled to obtain positive samples and negative samples.

3. The method according to claim 1, characterized in that The main prediction branch includes a backbone network, a neck network and a head network connected in sequence, the backbone network is used to extract multiple key image features of the input sample, the neck network is used to fuse the multiple key image features, and the head network is used to predict the bounding box and construction quality category of the input sample based on the fused features.

4. The method according to claim 1, characterized in that: The auxiliary reversible branch includes a feature pyramid network and a path aggregation network. The feature pyramid network includes multiple levels of feature pyramids, which are respectively used to extract multiple key image features of the input samples of different scales and predict bounding boxes and construction quality categories of targets in the input samples of different sizes; The path aggregation network is arranged between the feature pyramids of each level of the feature pyramid network and the main prediction branch, and is used to integrate the gradient information of the feature pyramids of each level and back-propagate to the main prediction branch and calculate the corresponding prediction loss of the feature pyramids of each level on the path aggregation network.

5. The method according to claim 4, characterized in that The second loss is the sum of the total loss of the bounding box prediction of the target object by multiple levels of the feature pyramid network, the total loss of the construction quality category prediction by multiple levels of the feature pyramid network, and the corresponding prediction losses of the feature pyramids of each level calculated by the path aggregation network on the path aggregation network.

6. The method according to claim 1, characterized in that After using the YOLOv9 model with converged model loss function as the trained construction quality detection model, the method further includes: Determine a first sample in the sample image set having an erroneous prediction result by comparing the labels of the positive samples and the negative samples in the sample image set with the prediction results of each sample output by the YOLOv9 model; Adjust the image corresponding to the first sample based on the position change between the predicted bounding box in the prediction result of the first sample output by the YOLOv9 model and the real bounding box marked in the label of the first sample, and the change between the predicted construction quality category in the prediction result of the first sample output by the YOLOv9 model and the construction quality category marked in the label of the first sample; The samples used to train the YOLOv9 model are updated based on the adjusted first samples and the corresponding labels.

7. The method according to claim 6, characterized in that The adjusting the image corresponding to the first sample based on the position change of the predicted bounding box in the prediction result of the first sample output by the YOLOv9 model and the labeled bounding box in the label of the first sample, and the change of the predicted construction quality category in the prediction result of the first sample output by the YOLOv9 model and the labeled construction quality category in the label of the first sample, includes: Calculate the intersection-over-union ratio of the predicted bounding box in the prediction result of the first sample output by the YOLOv9 model and the labeled bounding box in the label of the first sample to determine whether the predicted bounding box of the first sample is offset; If a shift occurs, the labeled bounding box of the first sample is rotated by a preset angle; Determine whether the predicted construction quality category in the prediction result of the first sample output by the YOLOv9 model is different from the labeled construction quality category in the label of the first sample; If they are different, the first sample is enlarged and the image clarity of the first sample is adjusted.

8. A construction quality detection method, characterized in that: include: Acquire an image of a target construction scene, wherein the image contains main features of a target object in the target construction scene; The image is input into a pre-trained YOLOv9 model for prediction to obtain a bounding box and a construction quality category of the target object in the image. The YOLOv9 model is trained based on a sample image set including positive samples belonging to safe construction and negative samples belonging to unsafe construction in each construction scene. The YOLOv9 model includes a main prediction branch and an auxiliary reversible branch. The main prediction branch is used to predict the bounding box and the construction quality category of the target object in the input sample. The auxiliary reversible branch is used to predict the bounding box and the construction quality category of the target object in the input samples of different scales based on multiple levels, and integrate the gradient information of each level and back-propagate it to the main prediction branch for updating the parameters of the YOLOv9 model, where one level corresponds to one scale.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 7 or the steps of the method according to claim 8 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the method according to any one of claims 1 to 7 or the steps of the method according to claim 8 are implemented.

Images