Log counting method based on Yolov9-UNet in complex environment

Abstract

The invention relates to a log counting method based on Yolov9-UNet in a complex environment, and belongs to the technical field of image processing. The method comprises the steps that a Transform is embedded into a UNet, and a Transs-UNet module is formed; the method comprises the following steps of: combining a Trans-UNet module with Yolov9 to form a Yolov9-UNet model; the method comprises the following steps: acquiring training data to train a Yov9-UNet model, and respectively training a Trans-UNet module and Yov9 by referring to a training process of a GAN network; and acquiring an image containing the end face of the log, performing normalization processing on the image, inputting the normalized image into the trained Yolov9-UNet model to obtain a log detection frame, and performing statistics on the detection frame to obtain the log number. According to the invention, effective recovery of images in different scenes can be realized, and log counting in a complex environment is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and relates to restoration of a log image in a complex environment and a method for counting logs based on the restored image. Background Art

[0002] As the global demand for wood resources continues to grow, accurate log counting has become an important issue in forest management and the wood industry. Traditional manual counting methods are not only inefficient, but also easily affected by human factors, resulting in inaccurate counting results. Due to the rise of deep learning and its widespread application in computer vision tasks, the use of image processing and deep learning technology to achieve automatic log counting has become a hot topic in current research.

[0003] Currently, log counting methods can be divided into two categories: traditional counting methods and deep learning counting methods.

[0004] (1) Traditional counting methods mainly rely on image processing and traditional machine learning techniques to count logs. These methods are generally divided into three categories: counting methods based on edge detection, counting methods based on morphological analysis, and counting methods based on feature extraction. The basic principles and shortcomings of these methods are briefly described as follows:

[0005] 1) The counting method based on edge detection mainly uses edge detection algorithms, such as the Canny operator, to identify the log boundary; then uses a high-pass filter to remove background noise, enhance the log outline, and extract clear edge information; finally, Hough transform is used for line detection to determine the length and number of logs. This method performs well when dealing with simple backgrounds, but the accuracy will be greatly affected in complex backgrounds and overlapping wood.

[0006] 2) The counting method of morphological analysis is based on the geometric shape characteristics of logs. Usually, morphological operations (such as expansion, corrosion, opening and closing operations) are first used to effectively separate overlapping wood and identify their shape characteristics, and then the machine learning regression model is used for counting. This method has high computational efficiency and is suitable for processing large amounts of wood, but when the lighting changes and the wood size varies greatly, the recognition rate will decrease, thus affecting the counting accuracy.

[0007] 3) Feature extraction methods focus on quantifying the color, texture and shape characteristics of logs. Log classification and counting are achieved by extracting local features (such as HOG features, LBP features, etc.) and combining them with machine learning algorithms (such as SVM or KNN). The advantage of this method is that it can achieve robustness in various environments, but model training requires a large number of labeled samples.

[0008] (2) Deep learning technology mainly focuses on the characteristics of wood by constructing a convolutional neural network (CNN). Currently, the most widely used deep learning counting method is to use the YOLO series of models to perform multi-target detection on logs, and then calculate the number of logs based on the number of detections. Compared with pure image processing methods, deep learning models can improve counting accuracy in complex scenarios and reduce reliance on manually adjusted parameters.

[0009] Although the accuracy of log counting methods based on deep learning has been greatly improved compared to traditional algorithms, especially for high-quality images. However, the accuracy of log counting in actual complex environments using the current deep learning counting method needs to be improved. For example, in outdoor environments such as docks and forests, the old cameras will cause the captured log images to contain a lot of noise. At the same time, the intensity of light and rain and fog will also affect the quality of the captured log images, thereby affecting the accuracy of log counting. At present, the existing deep learning technology methods cannot solve the above problems well. Summary of the invention

[0010] In view of this, the purpose of the present invention is to provide a log counting method based on Yolov9-UNet in a complex environment. The original log image is restored by combining UNet with Transformer to form a Trans-UNet module to improve its image quality, and then Yolov9 is used to perform target detection on the restored image, so as to achieve the purpose of improving the log counting accuracy of low-quality real images.

[0011] In order to achieve the above object, the present invention provides the following technical solutions:

[0012] A log counting method based on Yolov9-UNet in a complex environment, the method includes:

[0013] S1. Embed Transformer into UNet to form a Trans-UNet module. Combine the Trans-UNet module with Yolov9 to form a Yolov9-UNet model.

[0014] S2. Collect training data to train the Yolov9-UNet model. During the training process, first train the Trans-UNet module while freezing the Yolov9 parameters. After the Trans-UNet module is trained, freeze the Trans-UNet module parameters and use the Trans-UNet output images to train Yolov9.

[0015] S3. Collect images containing the end faces of logs and perform normalization processing, input the normalized images into the trained Yolov9-UNet model to obtain log detection frames, and count the detection frames to obtain the number of logs.

[0016] Furthermore, in step S1, a Trans-UNet module is formed by embedding multiple Transformers into UNet; wherein, a Transformer is connected between the convolution blocks of the encoder and decoder of UNet, specifically, the corresponding convolution blocks in the U-shaped structure of UNet are connected by a Transformer; at the same time, Transformers are embedded between the convolution blocks in the UNet encoder, and Transformers are embedded between the convolution blocks in the UNet decoder.

[0017] Furthermore, in step S2, the training process of the Yolov9-UNet model includes: first, collecting image training data, and normalizing the size of the image training data through opencv; then freezing the parameters of Yolov9, inputting the normalized image training data into the Trans-UNet module for training, and optimizing the parameters of the Trans-UNet module through the loss function; then inputting the image data optimized by the Trans-UNet module into Yolov9, freezing the parameters of the Trans-UNet module, training Yolov9, and adjusting the parameters of Yolov9 through the loss function.

[0018] Among them, during the training process of the Yolov9-UNet model, it is necessary to alternately train the Trans-UNet module and the Yolov9 part, and optimize the parameters through their respective loss functions during the training process, so that the loss values ​​of the Trans-UNet module and Yolov9 are finally minimized, thereby completing the training.

[0019] Furthermore, in step S3, the process of counting logs through the Yolov9-UNet model includes: collecting images containing the end faces of logs, normalizing the images through opencv, and inputting the normalized images into the trained Yolov9-UNet model; first optimizing the images through the Trans-UNet module to improve the image quality; then performing target detection on the optimized images through Yolov9 to obtain the detection frames of the logs in the image, and obtaining the number of logs after counting the detection frames.

[0020] The beneficial effects of the present invention are as follows: based on the excellent feature extraction capability of Transformer and the good image generation capability of UNet, the present invention combines the two to form a Trans-UNet module, and uses the important feature information in the complex environment image learned in the downsampling stage to guide the generation of high-quality log images. The module can achieve effective restoration of images with different illumination, noisy images, and raindrop blurred images. In addition, the present invention uses the idea of ​​alternating learning to cross-train Trans-UNet and Yolov9, achieving an end-to-end task combining image restoration and target detection, so that Yolov9-UNet can better achieve log counting in complex environments.

[0021] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0023] Figure 1 A schematic diagram of a flow chart of a log counting method proposed in one embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the structure of the Yolov9-UNet module;

[0025] Figure 3 It is a schematic diagram of the structure of the Trans-UNet module;

[0026] Figure 4 This is the image restoration effect diagram of the Trans-UNet module;

[0027] Figure 5 This is a comparison chart of log testing results using various methods under different environments. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] Aiming at the problem that the quality of log images taken by cameras in outdoor complex environments such as docks and forest farms is low, which affects the accuracy of log counting, the present invention proposes a log counting method based on the combination of Yolov9, UNet and Transformer. The method first combines UNet with Transformer to form a Trans-UNet module to restore the original log image to improve its image quality, and then uses Yolov9 to perform target detection on the restored image, ultimately achieving the purpose of improving the log counting accuracy of low-quality real images.

[0030] See also Figure 1 to Figure 3 , an embodiment of the present invention provides a log counting method based on Yolov9-UNet in a complex environment, and the method is specifically as follows:

[0031] 1. First, Transformer and UNet are combined to form a Trans-UNet module, which is used to restore the log pile images collected in complex environments to improve image quality and facilitate subsequent log detection, recognition and counting.

[0032] The downsampling of UNet is mainly to extract image features, and then upsampling is to restore the extracted features. However, if the image contains severe noise, when only downsampling is performed, the model will think that the noise is important information of the image, thereby losing the original important information of the image. Therefore, adding Transformer allows the downsampling process to consider global information and local information and retain the important information of the image. Based on the above improved downsampling structure and the structure of UNet, the upsampling part adopts the same addition method. On the other hand, the original jump connection of UNet is to ensure that the restored image contains the information of the original image, but because the original image contains severe noise information, if the jump connection is directly performed, noise may be introduced in the upsampling process. By introducing Transformer, important features of the image can be extracted and the introduction of noise can be avoided to a limited extent. Therefore, by introducing Transformer, the improved UNet can effectively extract important feature information of the image, eliminate other redundant information such as noise, and thus make the restored image higher in quality.

[0033] Based on the above principles, in this embodiment, the Transformer is embedded into the UNet network to form the following Figure 3 The Trans-UNet module shown in the figure, in which Transformer is embedded in both the encoder and decoder of the UNet network, and in each feature extraction layer (convolutional block) of the encoder and decoder. Transformer can consider both global and local information at the same time. Compared with simply using convolutional blocks for denoising, adding Transformer has a better denoising effect on the image.

[0034] 2. After the Trans-UNet module is built, it is combined with Yolov9 to obtain the following Figure 2 The Yolov9-UNet model shown is used to detect and count logs.

[0035] 3. Prepare training data for training the constructed Trans-UNet module. During the training process, you need to freeze the parameters of Yolov9 and train the Trans-UNet module separately.

[0036] In this embodiment, since the labeled log dataset is small in size, a public steel bar detection dataset is used for training, and after the training is completed, the log dataset is used for verification.

[0037] In the training phase of the Trans-UNet module, the original steel bar image is first input and its size is normalized using OpenCV. After freezing the Yolov9 parameters, the normalized images are batch-sent to the Trans-UNet module for training, and the parameters of Trans-UNet are optimized according to the corresponding loss function, where the loss function is shown in the following formula:

[0038]

[0039] 4. Use the log end images restored by the Trans-UNet module to train Yolov9.

[0040] After the Trans-UNet module data is verified, the end face image of the log is collected and input into Trans-UNet to obtain the restored image. Use with Train Yolov9 and fine-tune its parameters based on Yolov9’s object detection loss.

[0041] It should be noted that during the Yolov9 training process, the Trans-UNet parameters also need to be frozen.

[0042] After Yolov9 training is completed, it is tested using test data.

[0043] 5. Collect images containing the end faces of logs in the working scene and input them into the Yolov9-UNet model for log counting.

[0044] The user collects images containing the end faces of logs in the work scene and uses OpenCV to normalize the image size. The normalized image is sent to the Yolov9-UNet model to obtain the target detection result, and the target detection frame is counted to obtain the number of logs.

[0045] Among them, the image quality improvement effect of the Yolov9-UNet model is as follows: Figure 4 As shown, from Figure 4 It can be seen that for log images under different light intensities, noises and raindrops, the Trans-UNet model proposed in the present invention can effectively restore the original images, that is, the Trans-UNet model can greatly improve the image quality of log images in complex scenes.

[0046] In another embodiment of the present invention, the Yolov9-UNet model is used to compare the detection effects with several mainstream Yolo models. Figure 5 As can be seen from Table 1, for high-quality images, both the yolo series and Yolov9-UNet can achieve high-precision detection and counting, but in complex environments (such as light enhancement and reduction, noise and raindrops), the detection and counting accuracy of the yolo series will decrease. The Yolov9-UNet proposed in the present invention has better robustness for different complex environments, and the detection and counting accuracy of logs is higher (more than 90%). It can be seen that the Yolov9-UNet proposed in the present invention is more suitable for log detection and counting tasks in real complex environments.

[0047] Table 1

[0048]

[0049]

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A log counting method based on Yolov9-UNet in complex environment, characterized by: Embed Transformer into UNet to form a Trans-UNet module; combine the Trans-UNet module with Yolov9 to form a Yolov9-UNet model; Collect training data to train the Yolov9-UNet model; during the training process, first train the Trans-UNet module while freezing the Yolov9 parameters; after the Trans-UNet module is trained, freeze the Trans-UNet module parameters and use the images output by Trans-UNet to train Yolov9; An image containing the end face of a log is collected and normalized, the normalized image is input into the trained Yolov9-UNet model to obtain a log detection frame, and the detection frame is counted to obtain the number of logs.

2. The log counting method according to claim 1, characterized in that: The Trans-UNet module is formed by embedding multiple Transformers into UNet, wherein Transformers are connected between the convolution blocks of the encoder and decoder of UNet; Transformers are embedded between the convolution blocks in the UNet encoder, and Transformers are embedded between the convolution blocks in the UNet decoder.

3. The log counting method according to claim 1, characterized in that: The training process of the Yolov9-UNet model includes: firstly collecting image training data and normalizing the size of the image training data through opencv; then freezing the parameters of Yolov9, inputting the normalized image training data into the Trans-UNet module for training, and optimizing the parameters of the Trans-UNet module through the loss function; then inputting the image data optimized by the Trans-UNet module into Yolov9, freezing the parameters of the Trans-UNet module, training Yolov9, and adjusting the parameters of Yolov9 through the loss function.

4. The log counting method according to claim 1 or 3, characterized in that: During the training process of the Yolov9-UNet model, it is necessary to alternately train the Trans-UNet module and the Yolov9 part, and optimize the parameters through their respective loss functions during the training process, so that the loss values ​​of the Trans-UNet module and Yolov9 are minimized and the training is completed.

5. The log counting method according to claim 1, characterized in that: The process of counting logs by the Yolov9-UNet model includes: collecting images containing the end faces of logs, normalizing the images by opencv, and inputting the normalized images into the trained Yolov9-UNet model; first optimizing the images by the Trans-UNet module to improve the image quality; then performing target detection on the optimized images by Yolov9 to obtain the detection frames of the logs in the images, and obtaining the number of logs by counting the detection frames.

Images