Wood counting method in mobile scene based on DeblurGAN-v2 and YOLOv5

Abstract

The invention relates to a wood counting method in a mobile scene based on DeblurGAN-v2 and YOLOv5, and belongs to the technical field of information. The problem that in the prior art, motion blurred images cannot be effectively processed, so that wood counting precision is low, and efficiency is low is solved. According to the method, an industrial camera is used for collecting a motion blurred image of wood in a moving scene, and a visual image calibration tool is used for marking the boundary of the wood in the image to construct an original data set; the method comprises the following steps: inputting an original data set into a DeblurGAN-v2 model for preprocessing operation, generating a clear image, and constructing a deblurring data set; and training the deblurred data set, establishing a wood detection model based on YOLOv5, and performing model evaluation. And inputting the images in the deblurring data set into the trained YOLOv5 model for prediction, and identifying and counting woods in the images. The accuracy and efficiency of wood counting in the mobile environment are remarkably improved.

Description

Technical Field

[0001] The invention belongs to the field of information technology and relates to a wood counting method in a mobile scene based on DeblurGAN-v2 and YOLOv5. Background Art

[0002] With the rapid growth of timber demand, efficiency and safety issues in timber transportation have become increasingly prominent. Counting large amounts of timber during transportation is extremely complex and time-consuming, and traditional manual methods can no longer meet the needs of efficiency and accuracy. In mobile scenarios, such as transport vehicles and loading and unloading equipment, rapid timber counting faces even more severe challenges, and blurred images in motion further increase the difficulty of accurate counting.

[0003] As a typical scenario for timber transportation, docks need to handle a large amount of timber every day, and the counting work is usually done manually. This traditional method is inefficient and prone to errors, and there are certain safety risks. For example, workers check the timber one by one, which consumes a lot of manpower and delays the work process. Compared with docks, counting may be more difficult in other mobile scenarios, such as transport vehicles or temporary storage sites, and an efficient and reliable automation solution is urgently needed.

[0004] The rapid development of computer vision and deep learning has provided a new solution for wood counting, especially in mobile scenarios.

[0005] Chinese patent publication number CN116862871A discloses a wood counting method based on hybrid features. The method uses an instance segmentation network to process adhered wood and irregularly shaped wood, thereby improving the generalization ability and detection accuracy of counting.

[0006] A Chinese patent with publication number CN112686872A discloses a wood counting method based on deep learning. The method uses the Mask RCNN model to segment and count wood, and improves robustness by using overlap judgment and false detection processing.

[0007] However, both methods can only process static data images, and the processing effect on motion blurred image data is not ideal.

[0008] In summary, the existing wood counting detection methods have the following disadvantages:

[0009] (1) Many existing wood detection methods rely on the detection and segmentation of wood shape contours in images. However, in mobile scenes, wood is often in motion, and the captured images are often accompanied by motion blur. Traditional methods often have difficulty accurately counting wood in blurred images due to insufficient description and modeling capabilities of complex shapes, resulting in low accuracy.

[0010] (2) Low efficiency of manual processing: In mobile environments, the post-processing steps in existing methods often rely on the subjective judgment and manual screening of operators, which makes the algorithm processing inefficient. Manual processing requires a lot of time and effort, and is easily affected by subjective factors, resulting in inconsistent and unstable results. Summary of the invention

[0011] In view of this, the purpose of the present invention is to provide a wood counting method in a mobile scene based on DeblurGAN-v2 and YOLOv5. The Chinese name of DeblurGAN-v2 is "Deblurring Generative Adversarial Network V2 Version"; the Chinese name of YOLOv5 is "Single-stage Object Detection Algorithm V5 Version", and the English full name is "You Only Look Once version 5".

[0012] In order to cope with the demand for rapid counting of a large number of timber in mobile scenarios such as docks, and to solve the difficulty of image recognition caused by motion blur during transportation, the present invention has developed a method for counting timber in mobile scenarios based on DeblurGAN-v2 and YOLOv5. The prior art lacks an algorithm for effectively processing motion blurred images, and the blurring effect on timber during transportation has made it difficult to accurately count. To this end, the present invention proposes a method combining DeblurGAN-v2 deblurring technology and YOLOv5 target detection model. By training and testing the timber data set, it can effectively remove image blur, accurately identify and quickly count timber. This innovation not only greatly improves counting accuracy and efficiency, but also reduces manual intervention, improves the overall efficiency and reliability of timber transportation management in mobile environments, and fills the gap in the prior art.

[0013] In order to achieve the above object, the present invention provides the following technical solutions:

[0014] A wood counting method in a mobile scene based on DeblurGAN-v2 and YOLOv5 includes the following steps:

[0015] S1: Take motion blurred pictures of wood in a moving scene, mark the wood boundaries in the image, and construct the original dataset;

[0016] S2: Input the original data set into the DeblurGAN-v2 model for preprocessing, use the pre-trained model provided by DeblurGAN-v2 to perform image deblurring operations, generate deblurred images, and construct a deblurred data set;

[0017] S3: Train the deblurred dataset and build a wood detection model based on YOLOv5;

[0018] S4: Input the images in the deblurred dataset into the trained YOLOv5 model for prediction, identifying and counting the wood in the image.

[0019] Further, the S1 includes:

[0020] B1: Motion blurred image data of wood is collected by industrial cameras as the initial sample data set;

[0021] B2: Use the visual image calibration tool Labelimg to annotate the acquired data set, mark the target in the image with a rectangular box and add the target label.

[0022] Further, the S2 includes:

[0023] B1: Based on the original wood dataset, a padding method is used on each image to ensure that the size and format of all images are consistent;

[0024] B2: Load the pre-trained DeblurGAN-v2 model and perform deblurring on the normalized image dataset;

[0025] B3: Save the deblurred image generated by DeblurGAN-v2 in PNG format to the specified path.

[0026] Further, the S3 includes:

[0027] C1: Based on the image data of S2, the original dataset is expanded using data augmentation technology, and the expanded dataset is divided into a training set and a test set;

[0028] C2: Configure the dataset configuration file wood.yaml of the wood image dataset and load the corresponding YOLOv5 network model for deep training;

[0029] C3: Based on the convergence of the average accuracy of each category and the model loss value, determine whether it is necessary to adjust the hyperparameters and retrain until the model converges;

[0030] C4: Load the generated weight file into the YOLOv5 model prediction network test.py, make predictions on the wood test set, annotate the detected wood information in the picture, and output the relevant prediction picture.

[0031] Furthermore, the YOLOv5 model consists of an input layer, a backbone network, a neck network and a prediction network.

[0032] Furthermore, the YOLOv5 model uses the GIoU loss function to optimize the bounding box positioning, and the loss function includes positioning loss and confidence loss;

[0033] The loss function formula is as follows:

[0034]

[0035] L G =0.5·L p +0.006·L X +0.01 L adv (4)

[0036] Wherein, in formula (1), I s The resulting deblurred image, I GT is a real clear image, N is the total number of pixels;

[0037] In formula (2), φ represents the feature map extracted from the VGG19 network;

[0038] In formula (3), z~p z (z) represents a random noise distribution, p z The noise z is sampled out from It represents the pursuit of expectations;

[0039] In formula (4), L G It is the total loss function that combines pixel loss, perceptual loss and adversarial loss to optimize deblurring.

[0040] Further, the S4 includes:

[0041] D1: Input the images in the deblurred dataset into the trained YOLOv5 model for prediction and identify the wood in the image;

[0042] D2: Count the number of wood in the image based on the prediction results of the YOLOv5 model.

[0043] A wood counting device in a mobile scene based on deep learning, comprising:

[0044] An image acquisition module, used to acquire motion blurred images of wood in a moving scene;

[0045] Image processing module, used for preprocessing, deblurring and wood detection of collected images;

[0046] The counting module is used to count the number of wood in the image according to the output results of the wood detection module.

[0047] A computer-readable storage medium stores a computer program, which implements the method when executed by a processor.

[0048] The beneficial effects of the present invention are as follows: the present invention combines deep learning technologies such as DeblurGAN-v2 and YOLOv5, aiming to realize wood target detection and counting in mobile scenarios. First, based on the powerful deblurring capability of DeblurGAN-v2, the present invention can effectively eliminate the influence of motion blur on image quality and restore the original wood image. By removing the blurred area in the wood image, the accuracy of subsequent image processing is improved. Secondly, the target detection model of YOLOv5 is used to accurately identify and count the wood images after deblurring. YOLOv5 has the advantages of excellent detection capability and multi-target recognition, and can quickly and efficiently identify wood in different mobile scenarios. The present invention adopts an end-to-end deep learning framework, organically combines deblurring with target detection tasks, and significantly improves the accuracy and efficiency of wood counting in mobile environments.

[0049] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0051] Figure 1 A flow chart of a wood counting method for a mobile scene based on DeblurGAN-v2 and YOLOv5 provided in an embodiment of the present invention;

[0052] Figure 2 An architecture diagram of a wood counting method for mobile scenes based on DeblurGAN-v2 and YOLOv5 provided in an embodiment of the present invention;

[0053] Figure 3 A comparison diagram of moving scene samples before and after deblurring provided by an embodiment of the present invention;

[0054] Figure 4 This is a comparison chart of the recognition effects of the moving scene wood detection model before and after deblurring provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0056] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0057] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0058] See also Figure 1 to Figure 4 The purpose of the present invention is to provide a wood counting method in a mobile scene based on DeblurGAN-v2 and YOLOv5, so as to effectively process motion blurred images and improve the accuracy of existing wood counting in mobile scenes, and realize deep learning rapid detection and machine automated counting.

[0059] A wood counting method in mobile scenes based on DeblurGAN-v2 and YOLOv5, the specific steps are as follows:

[0060] S1: Take a certain number of motion blurred pictures of wood in moving scenes and mark the wood boundaries in the images to form the original dataset.

[0061] S2: The original dataset is input into the DeblurGAN-v2 model for preprocessing, and the image deblurring operation is performed using the pre-trained model provided by DeblurGANv2, and finally a deblurred image is generated to form a deblurred dataset.

[0062] S3: Train the deblurred dataset and establish a wood detection model based on YOLOv5 to verify the detection effect of the model.

[0063] S4: View the test result images after testing and evaluation to verify the performance and accuracy of the model in the wood target recognition and counting tasks.

[0064] The S1 includes:

[0065] B1: The motion blurred image data of wood is collected by an industrial camera as the initial sample data set. The visual image calibration tool Labelimg is used to annotate the acquired data set. The target in the image is marked with a rectangular frame and labeled.

[0066] The sample dataset image obtained in B1 above is in PNG image format. The Labelimg annotation software used generates a txt label file that follows the YOLO format. The original image and the annotated txt file generate the original wood image dataset required for training the neural network.

[0067] The S2 includes:

[0068] B1: Based on the original wood dataset, the padding method is used on each image to fill the specified value around the image to ensure that the size and format of all images are consistent, all of which are 256×256 in size.

[0069] B2: Load the pre-trained DeblurGAN-v2 model. No need to retrain the model. Use the model to deblur the normalized image dataset directly.

[0070] B3: Save the deblurred image generated by DeblurGAN-v2 in PNG format to the specified path. The file name of each output image is consistent with its corresponding original input image.

[0071] DeblurGAN-v2 in B2 above consists of a generator and a discriminator. The generator receives noise input and generates images, while the discriminator distinguishes between real samples and generated samples. The goal of the generator is to generate indistinguishable images until the two reach Nash equilibrium. The generator uses the MobileNet structure. The loss function formula is as follows:

[0072]

[0073] L G =0.5·L p +0.006·L X +0.01 L adv (4)

[0074] Wherein, in formula (1), I s The resulting deblurred image, I GT is a real clear image, and N is the total number of pixels. In formula (2), φ represents the feature map extracted from the VGG19 network. In formula (3), z~p z (z) represents a random noise distribution, p z The noise z is sampled out from represents the expected value. In formula (4), L G This formula combines pixel loss, perceptual loss and adversarial loss to optimize deblurring.

[0075] The S3 includes:

[0076] C1: Based on the image data of S2, the original data set is expanded using data enhancement technology, including image flipping, rotation, scaling and other operations. Then, the expanded data set is divided into a training set and a test set in a ratio of 8:2 by random sampling. The processed original data and the labeled data are stored in the set locations respectively;

[0077] C2: Configure the dataset configuration file wood.yaml of the wood image dataset, fill in the training set and validation set paths, the number of categories, and the category labels. According to the configured wood.yaml and the set hyperparameters, load the corresponding YOLOv5 network model for deep training.

[0078] C3: Based on the convergence of the average accuracy of each category and the model loss value, determine whether it is necessary to adjust the hyperparameters and retrain until the model converges. When the number of training rounds increases, the model loss curve and the average accuracy curve gradually converge, and the training can be ended and the corresponding weight file can be generated.

[0079] C4: Load the generated weight file into the YOLOv5 model prediction network test.py, make predictions on the wood test set, annotate the detected wood information in the picture, and output the relevant prediction picture.

[0080] In the above C2, the main hyperparameters include pre-trained weights, network model, configuration file, training rounds, batch size, device type, input image size, number of threads, optimizer, and learning rate. These hyperparameters should be adjusted accordingly according to the hardware configuration and model convergence to achieve the best results.

[0081] In the above C3, the YOLOv5 network structure built using the PyTorch framework consists of an input layer, a backbone network (backbone), a neck network (neck), and a prediction network (head). The backbone network includes CBP modules, C3 modules, and SPP modules; the neck network includes CBS modules, C3 modules, and PaNet modules; and the prediction network consists of a detection head. The C3 modules in the backbone and neck networks include branch 1 and branch 2, where branch 1 includes a convolutional layer, a batch normalization layer, and a SiLU activation function, and branch 2 adds multiple Bottleneck structures on this basis to improve feature extraction capabilities.

[0082] In YOLOv5, the input layer preprocesses the image, and randomly scales, crops, and arranges the image through Mosaic data enhancement technology to make the distribution of small targets more uniform, and adds adaptive anchor box calculation and adaptive image scaling to improve the detection speed and accuracy of the model. The backbone network (Backbone) includes Conv module, C3 module and spatial pyramid pooling (SPPF). The CSP module divides the feature map into two parts in the channel dimension, one part enters the regular network path, and the other part is directly spliced ​​to the feature map, reducing gradient redundancy and reducing the amount of calculation while ensuring detection accuracy. In the Neck structure, the feature pyramid network (FPN) and path aggregation network (PAN) are used. FPN transmits feature information from the top layer to the bottom, and PAN transmits spatial information from the bottom to the top, finally realizing parameter aggregation, which effectively improves the fusion ability of multi-scale features. In the prediction stage, YOLOv5 uses the GIoU loss function to optimize the bounding box positioning. The loss function includes positioning loss (box_loss) and confidence loss (obj_loss), and retains the optimal target box through weighted non-maximum suppression (weightedNMS), thereby improving the accuracy of detection. The loss function formula is as follows:

[0083]

[0084] Loss = L CIoU +L conf (9)

[0085] In formula (5), IoU is the ratio of the intersection area of ​​the real box and the predicted box to the union area. 2 (b,b gt ) represents the Euclidean distance between the center point of the predicted box and the true box, b and b gtRepresent the center points of the predicted box and the real box respectively, and c represents the diagonal length of the minimum closed box covering the predicted box and the real box. The parameter α is used as a weight coefficient to balance the influence of different terms in the loss function, and v represents the aspect ratio difference between the predicted box and the real box. In formula (6), α is a weight parameter used to balance the influence of IoU and aspect ratio on CIoU. In formula (7), it is used to measure the consistency of aspect ratio between the predicted box and the real box. Among them, w gt and h gt Represent the width and height of the real box, w and h represent the width and height of the predicted box. Formula (8) represents the confidence loss. Here, S is the side length of the feature map, and B is the number of anchor points of each unit in the feature map. Indicates whether the jth anchor in the i-th unit is responsible for the probability of falling into the bounding box of the region. If so, otherwise The definition and value of On the contrary, C ij is the confidence score of the jth anchor in the ith unit. λ is the balance coefficient, which is usually set to 0.5 during training. Formula (9) is the total loss function, including the CIoU loss L CIoU and confidence loss L conf .

[0086] In order to evaluate the performance of this algorithm, the present invention uses blurred wood images collected in a mobile environment as the experimental image set. The input wood image size is 256*256, the batch size is set to 8, and a total of 100 epochs are trained. In the experiment, the DeblurGAN-v2 model is first used to deblur the blurred image to generate a clear image. The deblurred image is then input into the YOLOv5 model for wood detection and counting.

[0087] Figure 4 The comparison of detection effects before and after deblurring is shown, and Table 1 lists the differences in detection effects of different YOLO models on blurred and deblurred images.

[0088] Table 1

[0089]

[0090] Experimental results show that YOLOv7 and YOLOv8 perform well when processing high-quality images, with detection accuracies of 83.61% and 82.54% for blurred images, respectively, but their performance under deblurred images has declined. YOLOv9 shows a greater decline in accuracy (the average accuracy of blurred images is only 76.32%). The combination of DeblurGAN-v2 and YOLOv5 of the present invention has higher detection accuracy and robustness in mobile environments. The accuracy rate on deblurred images is as high as 91.90%, the highest among all models, and it also shows significant advantages in recall rate and average accuracy, reaching 82.12% and 88.75% respectively. The CSP module and SPP module of YOLOv5 can not only effectively extract multi-scale features, but also maintain good detection robustness in mobile scenes, which is why it is superior to other versions of YOLO models in mobile scenes.

[0091] Therefore, the method proposed in the present invention is more suitable for wood counting in mobile scenarios.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A wood counting method in mobile scenarios based on DeblurGAN-v2 and YOLOv5, characterized by: The following steps are involved: S1: Take motion blurred pictures of wood in a moving scene, mark the wood boundaries in the image, and construct the original dataset; S2: Input the original data set into the DeblurGAN-v2 model for preprocessing, use the pre-trained model provided by DeblurGAN-v2 to perform image deblurring operations, generate deblurred images, and construct a deblurred data set; S3: Train the deblurred dataset and build a wood detection model based on YOLOv5; S4: Input the images in the deblurred dataset into the trained YOLOv5 model for prediction, identifying and counting the wood in the image.

2. The wood counting method in mobile scenarios based on DeblurGAN-v2 and YOLOv5 according to claim 1, characterized in that: The S1 includes: B1: Motion blurred image data of wood is collected by industrial cameras as the initial sample data set; B2: Use the visual image calibration tool Labelimg to annotate the acquired data set, mark the target in the image with a rectangular box and add the target label.

3. The wood counting method in mobile scene based on DeblurGAN-v2 and YOLOv5 according to claim 1, characterized in that: The S2 includes: B1: Based on the original wood dataset, a padding method is used on each image to ensure that the size and format of all images are consistent; B2: Load the pre-trained DeblurGAN-v2 model and perform deblurring on the normalized image dataset; B3: Save the deblurred image generated by DeblurGAN-v2 in PNG format to the specified path.

4. The wood counting method in mobile scene based on DeblurGAN-v2 and YOLOv5 according to claim 1, characterized in that: The S3 includes: C1: Based on the image data of S2, the original dataset is expanded using data augmentation technology, and the expanded dataset is divided into a training set and a test set; C2: Configure the dataset configuration file wood.yaml of the wood image dataset and load the corresponding YOLOv5 network model for deep training; C3: Based on the convergence of the average accuracy of each category and the model loss value, determine whether it is necessary to adjust the hyperparameters and retrain until the model converges; C4: Load the generated weight file into the YOLOv5 model prediction network test.py, make predictions on the wood test set, annotate the detected wood information in the picture, and output the relevant prediction picture.

5. The wood counting method in mobile scene based on DeblurGAN-v2 and YOLOv5 according to claim 1, characterized in that: The YOLOv5 model consists of an input layer, a backbone network, a neck network, and a prediction network.

6. The wood counting method in a mobile scene based on DeblurGAN-v2 and YOLOv5 according to claim 1, characterized in that: The YOLOv5 model uses the GIoU loss function to optimize the bounding box positioning, and the loss function includes positioning loss and confidence loss; The loss function formula is as follows: Wherein, in formula (1), I s The resulting deblurred image, I GT is a real clear image, N is the total number of pixels; In formula (2), φ represents the feature map extracted from the VGG19 network; In formula (3), z~p z (z) represents a random noise distribution, p z The noise z is sampled out from It represents the pursuit of expectations; In formula (4), L G It is the total loss function that combines pixel loss, perceptual loss and adversarial loss to optimize deblurring.

7. The wood counting method in mobile scene based on DeblurGAN-v2 and YOLOv5 according to claim 1, characterized in that: The S4 includes: D1: Input the images in the deblurred dataset into the trained YOLOv5 model for prediction and identify the wood in the image; D2: Count the number of wood in the image based on the prediction results of the YOLOv5 model.

8. A wood counting device in a mobile scenario based on deep learning, characterized in that: include: An image acquisition module, used to acquire motion blurred images of wood in a moving scene; Image processing module, used for preprocessing, deblurring and wood detection of collected images; The counting module is used to count the number of wood in the image according to the output results of the wood detection module.

9. A computer-readable storage medium having a computer program stored thereon, wherein the program implements the method of claim 1 when executed by a processor.

Images