Walnut kernel grading detection method based on YOLOv5
Through the Walnut kernel grading detection method based on YOLOv5, the problems of high cost, low accuracy and low efficiency of walnut kernel grading detection are solved, and efficient and accurate walnut kernel grading detection are achieved, especially the grading detection effect of overlapping kernel images is significant. promote.
Patent Information
- Application Number
- CN202411063049.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-05
- Publication Date
- 2024-11-19
AI Technical Summary
Walnut kernel grade grading testing has problems of high cost, low precision and low efficiency in my country, which seriously restricts the development of the walnut industry.
Using the Walnut kernel grading detection method based on YOLOv5, by collecting and labeling walnut kernel images, building the YOLOv5 target detection model, image enhancement processing and feature optimization, and combining bounding box calculation and non-maximum suppression technology, a walnut kernel grading detection interface is constructed .
The accuracy and efficiency of walnut kernel grading detection are significantly improved, especially the grading detection effect of complex targets and overlapping kernel images, the model's attention to key features is improved, and efficient walnut kernel grading detection is achieved.
Smart Images

Figure CN118982827A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of walnut kernel grading detection, and more specifically, to a walnut kernel grading detection method based on YOLOv5. Background Art
[0002] Walnut is an important economic tree species and woody oil tree species in my country, and occupies an important position in agricultural production and economic development. According to data from the Food and Agriculture Organization of the United Nations, my country has become the world's largest walnut producing country. With the improvement of walnut varieties, breakthroughs in cultivation and management techniques and grafting techniques, walnut planting has become an industry that my country strongly supports, and walnut planting provides important support for regional economic development. Although the walnut industry has achieved remarkable development, my country started late in walnut processing, and the degree of mechanization and intelligence still needs to be improved. In particular, in the grading and detection of walnut kernels, there are problems of high cost, low precision and low efficiency, which seriously restricts the further development of the walnut industry. Summary of the invention
[0003] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a walnut kernel grading detection method based on YOLOv5 to solve the problems raised in the above-mentioned background technology.
[0004] To achieve the above object, the present invention provides the following technical solutions:
[0005] The walnut kernel grading detection method based on YOLOv5 includes the following steps:
[0006] Step S1: collecting walnut kernel images, enhancing and annotating the walnut kernel images, and dividing them into training sets and verification sets;
[0007] Step S2: Build a YOLOv5 target detection model and input the walnut kernel image dataset into the YOLOv5 target detection model for training;
[0008] Step S3: modify the channel, size, format and dimension of the walnut kernel image to be detected, normalize it and increase its dimension, call the inference session, and obtain the prediction result of the walnut kernel image to be detected;
[0009] Step S4: construct a bounding box calculation function, obtain the bounding box through coordinate transformation and non-maximum suppression, and draw the image;
[0010] Step S5: Use the PyQT5 interface toolkit to establish a walnut kernel grading detection interface.
[0011] In a preferred embodiment, step S1 comprises the following steps:
[0012] Step S101: collecting walnut kernel images, including whole kernels, half kernels, broken kernels, worm-eaten kernels, moldy kernels, and overlapping kernels;
[0013] Step S102: performing image enhancement processing on the collected original walnut kernel image, the processing method includes random brightness adjustment and adding Gaussian noise;
[0014] Step S103: using software to annotate the walnut kernel image data obtained by image enhancement processing, the annotation format is YOLO, and the annotated data set is divided into a training set and a validation set.
[0015] In a preferred embodiment, step S2 comprises the following steps:
[0016] Add a convolution layer between the feature maps of different scales output by the backbone network and the neck network, and use 1x1 convolution to adjust the number of channels and unify the feature depth; use bilinear interpolation to upsample the deeper feature maps so that the size of the deeper feature maps matches that of the previous layer, and then fuse them with the previous layer feature maps by element addition; perform downsampling operations to pass high-level features downward, and after each downsampling, fuse the results with the output feature maps of the backbone network at the corresponding level by element addition;
[0017] For each layer of fused feature maps, a 3x3 convolution layer is added to further optimize the feature maps to eliminate the aliasing effect introduced by upsampling and the discontinuity in the fusion process;
[0018] Before the output of the neck network, insert SEB lock, use global average pooling on the feature map output by the neck network, generate a 1x1 xC vector, where C is the number of channels, to achieve compression of global information; the generated vector passes through two fully connected layers, the first layer uses the ReLU activation function, and the second layer uses the Sigma id activation function; the two fully connected layers have a dimension reduction and then dimension increase structure to achieve nonlinear excitation of features; the output of the excitation layer is used as the weight factor of the channel, and is dot-multiplied with the original feature map to achieve re-weighting of the features;
[0019] According to the above YOLOv5 target detection model, set the training parameters, load the labeled walnut kernel image dataset into the training script, save the trained YOLOv5 model parameters as a pt format file, and use PyTorch's ONNX export function to convert the trained pt format model into an ONNX format model.
[0020] In a preferred embodiment, step S3 comprises the following steps:
[0021] Read the walnut kernel image to be tested, check the shape attributes of the image, obtain the number of channels, use the image processing library to read the walnut kernel image to be tested, and obtain the shape of the image matrix, including three dimensions: height, width, and number of channels; when the walnut kernel image to be tested is a two-channel grayscale image, use the cv2.cvtColor function to convert the grayscale image into an RGB color image;
[0022] Get the original size of the walnut kernel image to be detected. The YOLOv5 model requires the input of an image of a fixed size. The walnut kernel image to be detected needs to be scaled, the aspect ratio of the walnut kernel image to be detected needs to be calculated, the target size of the walnut kernel image to be detected after scaling is determined, the scaling ratio is calculated, and the cv2.res i ze function is used to scale the walnut kernel image to be detected so that the length and width of the walnut kernel image to be detected meet the target size; in order to ensure that the size of the walnut kernel image to be detected is consistent with the target size, the scaled image needs to be filled, the height and width to be filled are calculated, and the scaled image needs to be filled using the cv2.copyMakeBorder function;
[0023] The image read by the image processing library is in BGR format. The YOLOv5 target detection model requires the input image to be in RGB format. Use the cv2.cvtColor function to convert the image from BGR format to RGB format.
[0024] The original layout of image data in memory is height H, width W, and number of channels C. The YOLOv5 target detection model requires the layout of input image data to be number of channels C, height H, and width W. The NumPy library is used to adjust the order of image dimensions.
[0025] In order to make the image data adapt to the input requirements of the model, the image data needs to be normalized. Each element in the image array is divided by the maximum value of the image pixel value so that the pixel value is converted to the range of 0 to 1.
[0026] In the YOLOv5 object detection model, the layout of the input image data is required to use the NumPy library to add a dimension. At this time, the walnut kernel image data to be detected will have a batch dimension;
[0027] Use the ONNX Runt ime library to load the ONNX model file converted in step 2 above, create an inference session, use the walnut kernel image data to be detected as the input of the inference session, call the run function of the ONNX inference session, execute model inference, and obtain the prediction result.
[0028] In a preferred embodiment, step S4 comprises the following steps:
[0029] The input of the bounding box calculation function is the output array of the YOLOv5 target detection model, including the bounding box coordinates, category confidence and category label. The integrity of the detection result is confirmed based on the bounding box coordinates, category confidence and category label. According to the preset confidence threshold, the bounding boxes with confidence less than or equal to the threshold are filtered out, and the bounding boxes with confidence greater than the threshold are retained.
[0030] For bounding boxes with confidence greater than the threshold, extract the category label with the highest confidence. In the output of the YOLOv5 target detection model, the 5th column is the confidence score and the 6th column is the category label. Replace the category label of the bounding box with confidence greater than the threshold with the category subscript to improve the readability of the detection result.
[0031] The bounding box coordinates output by the YOLOv5 target detection model are in the format of x, y, w, h, where x and y are the horizontal and vertical coordinates of the center point of the bounding box, respectively, and w and h are the width and height of the bounding box, respectively. The bounding box coordinate format is converted to the format of x1, y1, x2, y2 through the conversion formula, where x1, y1 are the coordinates of the upper left corner of the bounding box, and x2, y2 are the coordinates of the lower right corner of the bounding box;
[0032] Non-maximum suppression is performed on bounding boxes with confidence greater than a threshold, bounding boxes with high overlap are filtered, and bounding boxes exceeding their intersection-over-union threshold are deleted; a combination of distance intersection-over-union and non-maximum suppression is used, wherein the combination of distance intersection-over-union and non-maximum suppression simultaneously considers the intersection-over-union ratio and the center distance between the predicted boundary frame and the true boundary box, which is beneficial to the grading detection of walnut kernels, and the distance intersection-over-union ratio is an improved intersection-over-union ratio;
[0033] Define a color list, draw detection boxes of different categories with different colors, reversely adjust the bounding box coordinates after the distance intersection and union ratio and non-maximum suppression, map the bounding box coordinates to the original unfilled and unscaled image space, and use the drawing functions cv2.rectang le and cv2.putText to draw the bounding box, confidence category and confidence score on the original walnut kernel image.
[0034] In a preferred embodiment, in step S5:
[0035] The walnut kernel grading detection interface includes: image recognition and camera recognition, as well as custom confidence thresholds and IOU thresholds through sliders or numeric boxes, statistical detection results and display in text form; image recognition completes the detection by inputting the walnut kernel image data to the ONNX model reasoning through the user uploading the walnut kernel image to be detected; camera recognition calls the user's default camera to capture a frame of image within a preset time period, and each frame of image captured by the camera is input into the ONNX model reasoning to complete the detection.
[0036] Technical effects and advantages of the method of the present invention:
[0037] 1. By adding convolutional layers between the feature maps of different scales output by the backbone network Backbone and the neck network Neck, the feature information of different levels is integrated, so that the high-level features can better capture the global information and detailed features of the target, thereby enhancing the model's detection ability for complex targets; SEBlock is inserted before the output end of the neck network Neck to adaptively adjust the weight of each channel, enhance the feature expression ability, enable the model to better capture the key details in the walnut kernel image, and significantly improve the model's attention to key features, especially the hierarchical detection effect of overlapping kernel images has been significantly improved.
[0038] 2. By modifying the channel, size, RGB format and dimension, as well as normalizing and adding dimensions, the perfect match between the input image data and the YOLOv5 model is ensured, and the detection performance of the model is maximized; by constructing a bounding box calculation function, combined with confidence threshold filtering, coordinate transformation and non-maximum suppression technology, the accuracy of walnut kernel grading detection is improved, and by drawing the detection results, an intuitive image display method is provided, which allows users to quickly understand the grading detection of walnut kernels. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 This is a flowchart of a walnut kernel grading detection method based on YOLOv5 in the present invention; DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] Reference Figure 1 .
[0042] The embodiment provides a bogie analysis and design method of the present invention, which comprises the following steps:
[0043] Step S1: collecting walnut kernel images, enhancing and annotating the walnut kernel images, and dividing them into training sets and verification sets;
[0044] Step S2: Build a YOLOv5 target detection model and input the walnut kernel image dataset into the YOLOv5 target detection model for training;
[0045] Step S3: modify the channel, size, format and dimension of the walnut kernel image to be detected, normalize it and increase its dimension, call the inference session, and obtain the prediction result of the walnut kernel image to be detected;
[0046] Step S4: construct a bounding box calculation function, obtain the bounding box through coordinate transformation and non-maximum suppression, and draw the image;
[0047] Step S5: Use the PyQT5 interface toolkit to establish a walnut kernel grading detection interface.
[0048] Wherein step S1 is specifically implemented by the following steps:
[0049] Step S101: Collect walnut kernel images, including whole kernels, half kernels, broken kernels, worm-eaten kernels, moldy kernels, and overlapping kernels, specifically:
[0050] Representative walnut varieties were selected, including whole kernel, broken kernel, worm-eaten kernel and moldy kernel from Xinfeng, and half kernel and overlapping kernel from 185 varieties, which covered different walnut kernel morphologies and quality grades.
[0051] Prepare professional camera equipment and a stable shooting environment, ensure uniform lighting during shooting, avoid the influence of shadows and reflections on the image quality of walnut kernels, use a high-resolution camera to ensure clear image details of walnut kernels; use a background cloth with strong contrast to highlight the outline and features of walnut kernels, and avoid background colors that are close to the color of walnut kernels to improve image contrast.
[0052] Whole kernel image acquisition: Select complete and undamaged walnut kernels for shooting, ensuring that there is only one walnut kernel in each image.
[0053] Broken kernel image acquisition: Select broken walnut kernels for photography. Each image can contain multiple fragments, but the outline of each fragment must be clearly visible.
[0054] Image acquisition of worm-infested kernels: Select walnut kernels infested with insects for photographing, and ensure that the infested parts are clearly visible to facilitate marking of the infested areas.
[0055] Moldy kernel image acquisition: Select moldy walnut kernels for photography to ensure that the characteristics of the moldy area are obvious.
[0056] Half-kernel image acquisition: select half-kernel for shooting, ensuring that there is only one half-kernel in each image.
[0057] Overlapping kernel image acquisition: Select multiple walnut kernels and overlap them to shoot, ensuring that there is a clear overlap in each image.
[0058] Step S102: performing image enhancement processing on the collected original walnut kernel image, the processing method includes random brightness adjustment and adding Gaussian noise, specifically:
[0059] Use image processing tools to read the original walnut kernel image, convert the original walnut kernel image to HSV color space, randomly increase or decrease the value of the brightness channel to ensure that it does not exceed the maximum value 255 or is not lower than the minimum value 0, and convert the original walnut kernel image back to RGB color space.
[0060] The Gaussian noise is a noise that obeys a Gaussian distribution and is simulated by adding random Gaussian distribution noise to an image; the original walnut kernel image is read, a Gaussian noise matrix of the same size as the original walnut kernel image is generated, the mean and standard deviation of the noise matrix are set according to actual conditions, and the Gaussian noise matrix is added to the original walnut kernel image to ensure that the pixel value is in the range of 0 to 255.
[0061] Step S103: Use software to annotate the walnut kernel image data obtained by image enhancement processing, the annotation format is YOLO, and the annotated data set is divided into a training set and a verification set, specifically:
[0062] Open Label Img software, load the walnut kernel image to be annotated, and annotate the whole kernel, half kernel, broken kernel, worm-eaten kernel, moldy kernel, and overlapping kernel in the walnut kernel image respectively. For each walnut kernel target, select the boundary and add the corresponding label to ensure that the bounding box accurately covers the target area. Select the YOLO format for the annotation format; divide the annotated dataset into a training set and a validation set; the training set is used for model training, and the validation set is used for model performance evaluation; the division method is random selection to ensure that each type of walnut kernel appears in both the training set and the validation set.
[0063] Wherein step S2 is specifically implemented by the following steps:
[0064] A convolution layer is added between the feature maps of different scales output by the backbone network and the neck network, and 1x1 convolution is used to adjust the number of channels and unify the feature depth. Bilinear interpolation is used to upsample the deeper feature maps so that the size of the deeper feature maps matches that of the previous layer, and then they are fused with the previous layer feature maps by element addition. The fusion process starts from the deepest layer and gradually merges from the deepest layer to the shallower layer. Downsampling operations are performed using the top-down fused feature map as the basis to pass high-level features downward. After each downsampling, the result is fused with the output feature map of the backbone network at the corresponding level by element addition.
[0065] For each layer of feature maps after fusion, a 3x3 convolution layer is added to further optimize the feature maps to eliminate the aliasing effect introduced by upsampling and the discontinuity in the fusion process.
[0066] Before the output of the neck network, SEB lock is inserted, and global average pooling is used on the feature map output by the neck network to generate a 1x1 xC vector, where C is the number of channels, to achieve compression of global information; the generated vector passes through two fully connected layers, the first layer uses the ReLU activation function, and the second layer uses the Sigma id activation function; these two fully connected layers have a dimensionality reduction and then dimensionality increase structure, for example, from C to C / 16, and then from C / 16 back to C, to achieve nonlinear excitation of features; the output of the excitation layer is used as the weight factor of the channel, and point multiplication is performed with the original feature map to achieve feature re-weighting.
[0067] According to the above YOLOv5 target detection model, set the training parameters: initial learning rate is 0.01, iteration round is 100, batch size is 16, number of target categories is 4, optimizer is SGD, load the labeled walnut kernel image dataset into the training script, save the trained YOLOv5 model parameters as a pt format file, and use PyTorch's ONNX export function to convert the trained pt format model into an ONNX format model.
[0068] Wherein step S3 is specifically implemented by the following steps:
[0069] Perform channel number detection on the walnut kernel image to be detected, convert the two-channel grayscale image into a three-channel color image to detect the number of channels of the input image, and ensure that it meets the input requirements of the model: read the walnut kernel image to be detected, check the shape attributes of the image, obtain the channel number information, use the image processing library to read the walnut kernel image to be detected, and obtain the shape of the image matrix, which includes three dimensions: height, width, and number of channels; when the walnut kernel image to be detected is a two-channel grayscale image, use the cv2.cvtColor function to convert the grayscale image into an RGB color image.
[0070] Modify the size of the walnut kernel image to be detected: Get the original size of the walnut kernel image to be detected. The YOLOv5 model requires the input of an image of a fixed size. The walnut kernel image to be detected needs to be scaled, the aspect ratio of the walnut kernel image to be detected is calculated, the target size of the walnut kernel image to be detected after scaling is determined, the scaling ratio is calculated, and the cv2.res i ze function is used to scale the walnut kernel image to be detected so that the length and width of the walnut kernel image to be detected meet the target size; in order to ensure that the size of the walnut kernel image to be detected is consistent with the target size, the scaled image needs to be filled, the height and width to be filled are calculated, and the scaled image is filled using the cv2.copyMakeBorder function.
[0071] Convert the walnut kernel image to be detected from BGR format to RGB format: The image read by the image processing library is in BGR format. The YOLOv5 target detection model requires the input image to be in RGB format. Use the cv2.cvtColor function to convert the image from BGR format to RGB format.
[0072] Adjust the dimensions of the walnut kernel image to be detected: The original layout of the image data in the memory is height H, width W, and number of channels C. The YOLOv5 target detection model requires the layout of the input image data to be number of channels C, height H, and width W. Use the NumPy library to adjust the order of image dimensions.
[0073] Use the minimum-maximum normalization method to normalize the walnut kernel image to be tested: In order to make the image data adapt to the input requirements of the model, the image data needs to be normalized. Each element in the image array is divided by the maximum value of the image pixel value so that the pixel value is converted to the range of 0 to 1.
[0074] Increase the dimension of the walnut kernel image to be detected: In the YOLOv5 target detection model, the layout of the input image data is required to use the NumPy library to increase a dimension. At this time, the walnut kernel image data to be detected will have a batch dimension.
[0075] Use the ONNX Runt ime library to load the ONNX model file converted in step 2 above, create an inference session, use the walnut kernel image data to be detected as the input of the inference session, call the run function of the ONNX inference session, execute model inference, and obtain the prediction result.
[0076] Step S4: constructing a bounding box calculation function, and obtaining the final result of the bounding box through coordinate transformation and non-maximum suppression is specifically implemented by the following steps:
[0077] The bounding box calculation function is used to process the detection results of the YOLOv5 target detection model, including the extraction of bounding box coordinates and confidence filtering. The input of the bounding box calculation function is the output array of the YOLOv5 target detection model, including information such as bounding box coordinates, category confidence and category labels. The integrity of the detection results is confirmed according to the bounding box coordinates, category confidence and category labels. According to a preset confidence threshold, the bounding boxes with confidence less than or equal to the threshold are filtered, and the bounding boxes with confidence greater than the threshold are retained.
[0078] For bounding boxes with confidence greater than the threshold, extract the category label with the highest confidence. In the output of the YOLOv5 target detection model, the 5th column is the confidence score and the 6th column is the category label. Replace the category label of the bounding box with confidence greater than the threshold with the category subscript to improve the readability of the detection results.
[0079] The bounding box coordinates output by the YOLOv5 target detection model are in the format of x, y, w, h, where x and y are the horizontal and vertical coordinates of the center point of the bounding box, respectively, and w and h are the width and height of the bounding box, respectively. The bounding box coordinate format is converted to x1, y1, x2, y2 format through the conversion formula x1=xw / 2, y1=yh / 2, x2=x+w / 2, y2=y+h / 2, where x1, y1 are the coordinates of the upper left corner of the bounding box, and x2, y2 are the coordinates of the lower right corner of the bounding box.
[0080] Non-maximum suppression is performed on bounding boxes with confidence greater than a threshold to filter out bounding boxes with high overlap, and bounding boxes exceeding their intersection-over-union threshold are deleted, and the bounding box with the highest confidence is selected through iteration; in actual walnut kernel grading detection, overlapping kernels are composed of multiple pieces of walnut kernels overlapping each other, and the overlapping area is large. At this time, the intersection-over-union of overlapping kernels is likely to exceed the threshold, resulting in missed detection. The combination of distance intersection-over-union and non-maximum suppression is used. The combination of distance intersection-over-union and non-maximum suppression simultaneously considers the intersection-over-union, the center distance of the predicted boundary frame and the real bounding box, which is conducive to the grading detection of overlapping kernels. The distance intersection-over-union is an improved intersection-over-union.
[0081] Define a color list, draw detection boxes of different categories with different colors, reversely adjust the bounding box coordinates after the combination of distance intersection and non-maximum suppression, map the bounding box coordinates to the original unfilled and unscaled image space, and use the drawing functions cv2.rectang le and cv2.putText to draw the bounding box, confidence category and confidence score on the original walnut kernel image.
[0082] Among them, step S5 is specifically implemented through the following steps: the walnut kernel grading detection interface includes: image recognition and camera recognition, as well as customizing the confidence threshold and IOU threshold through a slider or a digital box, statistical detection results and displaying them in text form; image recognition realizes the input of the walnut kernel image data to be detected into the ONNX model reasoning through the user uploading the walnut kernel image to be detected, and completes the detection; camera recognition captures a frame of image within a preset time period by calling the user's default camera, and each frame of image captured by the camera is input into the ONNX model reasoning to complete the detection, and the preset time period is 30 milliseconds.
[0083] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0084] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0085] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0086] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0087] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0088] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0089] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0090] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0091] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A walnut kernel grading detection method based on YOLOv5, It is characterized in that The following steps are involved: Step S1: collecting walnut kernel images, enhancing and annotating the walnut kernel images, and dividing them into training sets and verification sets; Step S2: Build a YOLOv5 target detection model and input the walnut kernel image dataset into the YOLOv5 target detection model for training; Step S3: modify the channel, size, format and dimension of the walnut kernel image to be detected, normalize it and increase its dimension, call the inference session, and obtain the prediction result of the walnut kernel image to be detected; Step S4: construct a bounding box calculation function, obtain the bounding box through coordinate transformation and non-maximum suppression, and draw the image; Step S5: Use the PyQT5 interface toolkit to establish a walnut kernel grading detection interface.
2. A walnut kernel grading detection method based on YOLOv5 according to claim 1, It is characterized in that Step S1 includes the following steps: Step S101: collecting walnut kernel images, including whole kernels, half kernels, broken kernels, worm-eaten kernels, moldy kernels, and overlapping kernels; Step S102: performing image enhancement processing on the collected original walnut kernel image, the processing method includes random brightness adjustment and adding Gaussian noise; Step S103: using software to annotate the walnut kernel image data obtained by image enhancement processing, the annotation format is YOLO, and the annotated data set is divided into a training set and a validation set.
3. A walnut kernel grading detection method based on YOLOv5 according to claim 2, Features: Add a convolution layer between the feature maps of different scales output by the backbone network and the neck network, and use 1x1 convolution to adjust the number of channels and unify the feature depth; use bilinear interpolation to upsample the deeper feature maps so that the size of the deeper feature maps matches that of the previous layer, and then fuse them with the previous layer feature maps by element addition; perform downsampling operations to pass high-level features downward, and after each downsampling, fuse the results with the output feature maps of the backbone network at the corresponding level by element addition; For each layer of fused feature maps, a 3x3 convolution layer is added to further optimize the feature maps to eliminate the aliasing effect introduced by upsampling and the discontinuity in the fusion process; Before the output of the neck network, a channel attention mechanism SEBlock is inserted, and global average pooling is used for the feature map output by the neck network to generate a 1x1 xC vector, where C is the number of channels, to achieve compression of global information; the generated vector passes through two fully connected layers, the first layer uses the ReLU activation function, and the second layer uses the Sigmoid activation function; the two fully connected layers have a dimension reduction and then dimension increase structure to achieve nonlinear excitation of features; the output of the excitation layer is used as the channel weight factor, and point multiplication is performed with the original feature map to achieve feature re-weighting; According to the above YOLOv5 target detection model, set the training parameters, load the labeled walnut kernel image dataset into the training script, save the trained YOLOv5 model parameters as a pt format file, and use PyTorch's ONNX export function to convert the trained pt format model into an ONNX format model.
4. A walnut kernel grading detection method based on YOLOv5 according to claim 3, Features: Read the walnut kernel image to be tested, check the shape attributes of the image, obtain the number of channels, use the image processing library to read the walnut kernel image to be tested, and obtain the shape of the image matrix, including the three dimensions of height, width, and number of channels; when the walnut kernel image to be tested is a two-channel grayscale image, use the cv2.cvtColor function to convert the grayscale image into an RGB color image; Get the original size of the walnut kernel image to be detected. The YOLOv5 model requires the input of an image of a fixed size. The walnut kernel image to be detected needs to be scaled, the aspect ratio of the walnut kernel image to be detected needs to be calculated, the target size of the walnut kernel image to be detected after scaling is determined, the scaling ratio is calculated, and the cv2.resize function is used to scale the walnut kernel image to be detected so that the length and width of the walnut kernel image to be detected meet the target size. In order to ensure that the size of the walnut kernel image to be detected is consistent with the target size, the scaled image needs to be filled, the height and width to be filled are calculated, and the scaled image needs to be filled using the cv2.copyMakeBorder function. The image read by the image processing library is in BGR format. The YOLOv5 target detection model requires the input image to be in RGB format. Use the cv2.cvtColor function to convert the image from BGR format to RGB format. The original layout of image data in memory is height H, width W, and number of channels C. The YOLOv5 target detection model requires the layout of input image data to be number of channels C, height H, and width W. The NumPy library is used to adjust the order of image dimensions. In order to make the image data adapt to the input requirements of the model, the image data needs to be normalized. Each element in the image array is divided by the maximum value of the image pixel value so that the pixel value is converted to the range of 0 to 1. In the YOLOv5 object detection model, the layout of the input image data is required to use the NumPy library to add a dimension. At this time, the walnut kernel image data to be detected will have a batch dimension; Use the ONNX Runtime library to load the ONNX model file converted in step 2 above, create an inference session, use the walnut kernel image data to be detected as the input of the inference session, call the run function of the ONNX inference session, execute model inference, and obtain the prediction result.
5. A walnut kernel grading detection method based on YOLOv5 according to claim 4, Features: The input of the bounding box calculation function is the output array of the YOLOv5 target detection model, including the bounding box coordinates, category confidence and category label. The integrity of the detection result is confirmed based on the bounding box coordinates, category confidence and category label. According to the preset confidence threshold, the bounding boxes with confidence less than or equal to the threshold are filtered out, and the bounding boxes with confidence greater than the threshold are retained. For bounding boxes with confidence greater than the threshold, extract the category label with the highest confidence. In the output of the YOLOv5 target detection model, the 5th column is the confidence score and the 6th column is the category label. Replace the category label of the bounding box with confidence greater than the threshold with the category subscript to improve the readability of the detection result. The bounding box coordinates output by the YOLOv5 target detection model are in the format of x, y, w, h, where x and y are the horizontal and vertical coordinates of the center point of the bounding box, respectively, and w and h are the width and height of the bounding box, respectively. The bounding box coordinate format is converted to the format of x1, y1, x2, y2 through the conversion formula, where x1, y1 are the coordinates of the upper left corner of the bounding box, and x2, y2 are the coordinates of the lower right corner of the bounding box; Non-maximum suppression is performed on bounding boxes with confidence greater than a threshold, bounding boxes with high overlap are filtered, and bounding boxes exceeding their intersection-over-union threshold are deleted; a combination of distance intersection-over-union and non-maximum suppression is used, wherein the combination of distance intersection-over-union and non-maximum suppression simultaneously considers the intersection-over-union ratio and the center distance between the predicted boundary frame and the true boundary box, which is beneficial to the grading detection of walnut kernels, and the distance intersection-over-union ratio is an improved intersection-over-union ratio; Define a color list, draw detection boxes of different categories with different colors, reversely adjust the bounding box coordinates after the distance intersection and union ratio and non-maximum suppression, map the bounding box coordinates to the original unfilled and unscaled image space, and use the drawing functions cv2.rectangle and cv2.putText to draw the bounding box, confidence category and confidence score on the original walnut kernel image.
6. A walnut kernel grading detection method based on YOLOv5 according to claim 5, Features: The walnut kernel grading detection interface includes: image recognition and camera recognition, as well as custom confidence thresholds and intersection-over-union thresholds through sliders or numeric boxes, statistical detection results and display in text form; image recognition completes the detection by inputting the image data of the walnut kernel to be detected into the ONNX model reasoning through the user uploading the image of the walnut kernel to be detected; camera recognition calls the user's default camera to capture a frame of image within a preset time period, and each frame of image captured by the camera is input into the ONNX model reasoning to complete the detection.
Citation Information
Cited By
Carya cathayensis kernel defect identification method based on improved ConvNeXtV2
CN121884014A
