Image recognition method and device based on YOLO-V10 deformable convolutional neural network
By applying the optimized YOLO-V10 deformable convolutional neural network in fruit sorting equipment, the problem of low automation of existing fruit sorting equipment is solved, the rapid accuracy of fruit color recognition is achieved, and the sorting efficiency is improved. Accuracy.
Patent Information
- Application Number
- CN202411419246.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-02-14
Smart Images
Figure CN119445363A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to an image recognition method, device and computer equipment based on a YOLO-V10 deformable convolutional neural network. Background Art
[0002] my country is a large agricultural country, and its fruit production ranks among the top in the world, but the proportion of fruit sorting is very low, while the proportion of post-harvest processing of fruit in developed countries is as high as 80% to 90%. The size, shape and color of fruits have an important impact on their sales in the market. The main indicators for grading according to the external quality of fruits are size, shape, color and surface defects. At present, the degree of automation of fruit grading in my country is low. The existing fruit sorting equipment in China is only based on mechanical engineering technology, and sorts according to the size and weight of fruits, while the detection of shape, color and surface defects is still done manually. The main disadvantages of manual sorting are: it requires a lot of labor, high labor intensity, low efficiency, and high inspection cost; the grading standard is difficult to achieve manually, and the grading accuracy is unstable, because it is difficult to achieve the detection of quantitative indicators by human vision alone, and workers use their eyes for a long time and perform monotonous operations, which will cause physical and mental fatigue and emotional fluctuations, which will cause the grading error to fluctuate with people's emotions.
[0003] Computer vision technology provides an automatic, non-destructive, efficient and objective method for external quality inspection of fruits. The market has increasingly higher requirements for product sorting, requiring that fruits be classified by quality and maturity according to their color after picking, and that the classified fruits be sold directly according to their different qualities. In this case, the sorting device is required to have a stronger visual inspection system. In the visual inspection system, the inference delay of the low-version YOLO convolutional neural network algorithm is relatively long. The number of calculation steps in this inference process increases, and the efficiency and responsiveness are slow. Summary of the invention
[0004] In view of the above problems, an image recognition method, device and computer equipment based on YOLO-V10 deformable convolutional neural network are provided. The optimized YOLOv10 adopts a new NMS-free training method, fine-tunes its various parts to improve performance, and achieves a good balance between speed and accuracy, making the model easier to deploy, with stronger performance, faster speed and more rapid response.
[0005] To solve the problems in the prior art, the present invention provides an image recognition method based on a YOLO-V10 deformable convolutional neural network, comprising: S1, receiving input images to be trained, and training a deformable convolutional neural network model based on optimized YOLO-V10 in the collected images to be trained; S2, learn a new offset based on the image trained in step S1; S3, recalculating the parameter value of the new offset in step S2; S4, performing deformable convolution on the image trained in step S1 according to the offset recalculated in step S3 and the pixel value after the offset to obtain the latest training feature image; S5. Recognize the image to be recognized according to the deformable convolutional neural network after multiple trainings and feed back to the client.
[0006] Preferably, in step S4, the process of deformable convolution is as follows: A convolution kernel of n*n is used for the image trained in step S1, and the pixel at the original image coordinate P0(n, n) is marked as (0, 0) as the center coordinate, so as to calculate the offset of each coordinate pixel in the R domain and the final offset pixel value.
[0007] Preferably, the calculation is performed for any coordinate value in the R domain, denoted as P n (m, k), then the position coordinates of (m, k) in the entire image are P 0 +P n =(m+n,k+n).
[0008] Preferably, in step S2, the new offset, denoted as △Pn, is equal to the original coordinate (P 0 +P n ) are added to get a new pixel point coordinate of △Pn+P 0 +P n .
[0009] Preferably, bilinear interpolation is used to process the pixel values of the offset pixel points according to the offset in the R domain to obtain all the offset positions and the offset pixel values.
[0010] Preferably, after obtaining all the shifted positions and shifted pixel values, the pixel at each position is multiplied by the convolution kernel weight to obtain the output pixel value: ; Among them, the first matrix represents the pixel value of each position after the offset, and the second matrix represents the weight of the convolution kernel.
[0011] An image recognition device for recognizing and grading the color of fruits, comprising: The detection box has an inlet and an outlet. A conveyor belt for conveying fruits from the inlet to the outlet is provided in the detection box. An image acquisition component for photographing fruits on the conveyor belt is also provided in the detection box. A client, electrically connected to the image acquisition component, for executing the image recognition method based on the YOLO-V10 deformable convolutional neural network as described in any one of claims 1 to 7, and outputting a recognition result; A fruit conveying device is arranged at the entrance side of the detection box and is used to convey the fruits to be graded into the detection box one by one; The grading execution device is arranged at the exit side of the detection box and is used to classify and store the fruits according to the recognition results output by the client.
[0012] Preferably, the image acquisition component is an infrared spectrum camera.
[0013] Preferably, there are at least two image acquisition components, and the two image acquisition components are arranged at intervals along the conveying direction of the conveyor belt.
[0014] Preferably, the detection box is provided with a brush assembly located at the entrance.
[0015] A computer device comprises a memory, a processor and a computer program. When the processor executes the computer program, an image recognition method based on a YOLO-V10 deformable convolutional neural network is implemented.
[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. The optimized YOLOv10 adopts a new NMS-free training method, fine-tunes its parts to improve performance, and strikes a good balance between speed and accuracy, making the model easier to deploy, more powerful, faster, and more responsive.
[0017] 2. The fruit quality inspection equipment based on image recognition of YOLO-V10 deformable convolutional neural network can accurately grade and sort fruits according to their colors, with fast recognition response speed and high sorting accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a flow chart of the image recognition method based on the YOLO-V10 deformable convolutional neural network of the present invention.
[0019] Figure 2 It is a schematic diagram of an output channel processed using an n*n convolution kernel in an image recognition method based on a YOLO-V10 deformable convolutional neural network of the present invention.
[0020] Figure 3 It is a schematic diagram of a three-dimensional result of an image recognition device of the present invention.
[0021] Figure 4 It is a structural schematic diagram of a detection box in an image recognition device of the present invention.
[0022] In the figure: 100, detection box; 101, conveyor belt; 102, image acquisition component; 103, brush component; 200, fruit conveying device; 300, grading execution device. DETAILED DESCRIPTION
[0023] In order to further understand the features, technical means, specific objectives and functions of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0024] See also Figure 1 As shown, the image recognition method based on the YOLO-V10 deformable convolutional neural network includes: S1. Receive input images to be trained, and train a deformable convolutional neural network model based on the optimized YOLO-V10 in the collected images to be trained.
[0025] S2. Based on the image trained in step S1, learn a new offset.
[0026] S3. Recalculate the parameter value of the new offset in step S2.
[0027] S4. According to the offset recalculated in step S3 and the pixel value after the offset, deformable convolution is performed on the image trained in step S1 to obtain the latest training feature image.
[0028] S5. Recognize the image to be recognized according to the deformable convolutional neural network after multiple trainings and feed back to the client.
[0029] YOLOV10 is a non-NMS design and an overall model design. The non-NMS design refers to the use of consistent dual allocation to eliminate the need for NMS, thereby reducing inference latency. The overall model design comprehensively optimizes various components from the perspective of efficiency and accuracy, including lightweight classification heads, spatial channel decoupling down sampling, and level guidance block design.
[0030] In step S4, the process of deformable convolution is as follows: A convolution kernel of n*n is used for the image trained in step S1, and the pixel at the original image coordinate P0(n, n) is marked as (0, 0) as the center coordinate, so as to calculate the offset of each coordinate pixel in the R domain and the final offset pixel value.
[0031] Calculate any coordinate value in the R domain, denoted as P n (m, k), then the position coordinates of (m, k) in the entire image are P 0 +P n =(m+n,k+n).
[0032] In step S2, the new offset, denoted as △Pn, is equal to the original coordinate (P 0 +P n ) are added to get a new pixel point coordinate of △Pn+P 0 +P n .
[0033] It should be noted that the calculation of the offset uses an n*n convolution kernel for processing, and the output channels are 2*kernel_size*kernel_size. The number of convolution output channels out_channel for calculating the offset is different from the number of output channels of the deformable convolution. Because the offset is performed on both the x-axis and the y-axis, the number of out_channel is the square of twice kernel_size.
[0034] Using an n*n convolution kernel for processing (the step size and padding are 1), the number of input channels is n, and the number of output channels is 2*n 2 .like Figure 2 As shown, the number of convolution pixel channels of any selected small square is 2*n 2 , and then reshape the corresponding channel, we will get nine in the R domain (because it is an n*n convolution kernel, so the reshape here is n 2 ) on the x-axis and y-axis.
[0035] According to the offset in the R domain, bilinear interpolation is used to process the pixel values of the offset pixels to obtain all the offset positions and offset pixel values.
[0036] The bilinear interpolation processing steps are as follows: If you want to get the value of the unknown function f at point p=(x, y), assume that the values of the function f at four points Q11=(x1, y1), Q12=(x1, y2), Q21=(x2, y1) and Q22=(x2, y2) are known, and f is the pixel value of the pixel under normal circumstances.
[0037] First, perform linear interpolation in the x direction and obtain: , .
[0038] Then perform linear interpolation in the y direction and get: .
[0039] Bilinear interpolation is performed on the offset sampling points to obtain the feature information near each sampling point. In other words, the weights of the distances between the four nearest neighboring pixels around the point are calculated; the pixel values of the four nearest neighboring pixels are weighted and averaged to obtain the pixel value of the offset position.
[0040] After obtaining all the offset positions and offset pixel values, the pixel at each position is multiplied by the convolution kernel weight to obtain the output pixel value. The calculation formula for the output pixel value is as follows: ; Among them, the first matrix represents the pixel value of each position after the offset, and the second matrix represents the weight of the convolution kernel.
[0041] like Figure 3 and Figure 4 As shown, an image recognition device recognizes and grades fruits by color, comprising: a detection box 100, having an entrance and an exit, a conveyor belt 101 for conveying fruits from the entrance direction to the exit direction is provided in the detection box 100, and an image acquisition component 102 for taking photos of fruits on the conveyor belt 101 is also set up in the detection box 100; a client is electrically connected to the image acquisition component 102, and is used to execute the image recognition method based on the YOLO-V10 deformable convolutional neural network as described in any one of claims 1 to 7, and output a recognition result; a fruit conveying device 200 is arranged at the entrance side of the detection box 100, and is used to convey the fruits to be graded into the detection box 100 one by one; a grading execution device 300 is arranged at the exit side of the detection box 100, and is used to classify and store the fruits according to the recognition results output by the client.
[0042] More specifically, the image acquisition component 102 is an infrared spectroscopy camera. The images captured by a near-infrared spectroscopy camera have higher definition.
[0043] More specifically, there are at least two image acquisition components 102 , and the two image acquisition components 102 are arranged at intervals along the conveying direction of the conveyor belt 101 .
[0044] Specifically, the detection box 100 is provided with a brush assembly 103 at the entrance. The brush can clean the residues on the fruit that affect the detection before the fruit enters the detection box 100, so as to avoid affecting the detection.
[0045] A computer device comprises a memory, a processor and a computer program. When the processor executes the computer program, an image recognition method based on a YOLO-V10 deformable convolutional neural network is implemented.
[0046] The above embodiments only express one or several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.
Claims
1. An image recognition method based on a YOLO-V10 deformable convolutional neural network, characterized in that: include: S1, receiving input images to be trained, and training a deformable convolutional neural network model based on optimized YOLO-V10 in the collected images to be trained; S2, learn a new offset based on the image trained in step S1; S3, recalculating the parameter value of the new offset in step S2; S4, performing deformable convolution on the image trained in step S1 according to the offset recalculated in step S3 and the pixel value after the offset to obtain the latest training feature image; S5. Recognize the image to be recognized according to the deformable convolutional neural network after multiple trainings and feed back to the client.
2. The image recognition method based on the YOLO-V10 deformable convolutional neural network according to claim 1, characterized in that: In step S4, the process of deformable convolution is as follows: A convolution kernel of n*n is used for the image trained in step S1, and the pixel at the original image coordinate P0(n, n) is marked as (0, 0) as the center coordinate, so as to calculate the offset of each coordinate pixel in the R domain and the final offset pixel value.
3. The image recognition method based on the YOLO-V10 deformable convolutional neural network according to claim 2, characterized in that: Calculate any coordinate value in the R domain, denoted as P n (m, k), then the position coordinates of (m, k) in the entire image are P0+P n =(m+n,k+n).
4. The image recognition method based on the YOLO-V10 deformable convolutional neural network according to claim 3, characterized in that: In step S2, the new offset, denoted as △Pn, is equal to the original coordinate (P0+P n ) are added to get a new pixel point coordinate of △Pn+P0+P n .
5. The image recognition method based on the YOLO-V10 deformable convolutional neural network according to claim 4, characterized in that: According to the offset in the R domain, bilinear interpolation is used to process the pixel values of the offset pixels to obtain all the offset positions and offset pixel values.
6. The image recognition method based on the YOLO-V10 deformable convolutional neural network according to claim 5, characterized in that: After obtaining all the offset positions and offset pixel values, the pixel at each position is multiplied by the convolution kernel weight to obtain the output pixel value: ； Among them, the first matrix represents the pixel value of each position after the offset, and the second matrix represents the weight of the convolution kernel.
7. An image recognition device for identifying and grading the color of fruits, characterized in that: include: The detection box (100) has an inlet and an outlet. A conveyor belt (101) for conveying fruit from the inlet direction to the outlet direction is provided in the detection box (100). An image acquisition component (102) for taking a photo of the fruit on the conveyor belt (101) is also provided in the detection box (100); A client, electrically connected to the image acquisition component (102), configured to execute the image recognition method based on the YOLO-V10 deformable convolutional neural network as claimed in any one of claims 1 to 7, and output a recognition result; A fruit conveying device (200) is arranged at the entrance side of the detection box (100) and is used to convey fruits to be graded into the detection box (100) one by one; The grading execution device (300) is arranged at the exit side of the detection box (100) and is used to classify and store the fruits according to the recognition results output by the client.
8. An image recognition device according to claim 7, characterized in that: The image acquisition component (102) is an infrared spectrum camera.
9. An image recognition device according to claim 8, characterized in that: The image acquisition components (102) include at least two, and the two image acquisition components (102) are arranged at intervals along the conveying direction of the conveyor belt (101).
10. An image recognition device according to claim 7, characterized in that: The detection box (100) is provided with a brush assembly (103) located at the entrance.
11. A computer device comprising a memory, a processor and a computer program, characterized in that: When the processor executes the computer program, the image recognition method based on the YOLO-V10 deformable convolutional neural network is implemented as described in any one of claims 1 to 7.