Establishment method of walnut kernel quality detection model and walnut kernel sorting automatic conveying production line based on visual detection

Abstract

The invention discloses an establishment method of a walnut kernel quality detection model and a walnut kernel sorting and automatic conveying production line based on visual detection, and belongs to the field of walnut kernel processing equipment. The automatic conveying production line comprises a feeding conveying mechanism. A discharging port of the feeding conveying mechanism communicates with a feeding port of a visual detection conveying assembly. The walnut kernel sorting mechanism is mounted on the side, away from the feeding conveying mechanism, of the visual detection conveying assembly; the visual detection transmission assembly is used for driving walnut kernels to move along a first preset direction, and the upper computer calls a walnut kernel quality detection model to detect pictures shot by the visual detection assembly so as to determine whether the state of the walnut kernels is a first state or a second state; the walnut kernels in the first state are guided to a first state walnut kernel recovery mechanism; and the walnut kernels in the second state are sorted to the blank conveying and recycling mechanism through the walnut kernel sorting mechanism. According to the walnut kernel sorting and conveying device, automatic sorting of walnut kernels in two different states and conveying and collecting after automatic sorting can be achieved.

Description

Technical Field

[0001] The invention relates to a method for establishing a walnut kernel quality detection model and a walnut kernel sorting automatic conveying production line based on visual detection, and belongs to the field of walnut kernel processing equipment based on computer vision. Background Art

[0002] Walnuts are a popular nut food, and the market demand for them is increasing year by year. In the traditional walnut kernel sorting process, quality grading is mostly dependent on manual vision and manual operation, including judging the size, shape, color, and physical damage of the walnut kernels. However, this traditional method has the problems of low efficiency, high cost, and difficulty in maintaining sorting consistency and accuracy.

[0003] Due to the limitations of manual sorting, such as high labor intensity, limited efficiency, and difficulty in adapting to large-scale processing needs during peak periods, walnut processing companies are in urgent need of an automated solution that can both improve sorting efficiency and ensure product quality. With the rapid development of image processing and machine vision technology, how to combine artificial intelligence algorithms to achieve visual inspection and sorting of walnut kernels has become a problem that needs to be solved. Summary of the invention

[0004] The present invention provides a method for establishing a walnut kernel quality detection model and an automatic walnut kernel sorting and conveying production line based on visual detection, which can realize automatic sorting of walnut kernels in two different states and conveying and collecting after automatic sorting, which not only significantly improves the sorting speed and accuracy, but also reduces the production cost, reduces the dependence on manual labor, and improves the market competitiveness of walnut kernel products.

[0005] The technical solution of the present invention is:

[0006] According to a first aspect of the present invention, a method for establishing a walnut kernel quality detection model is provided. The walnut kernel quality detection model takes yolov8 as a framework, includes a backbone network, a neck network, and a head network, wherein the original C2f module of the backbone network is replaced by a CSPPC module, and the original C2f module in the neck network is replaced by a Dual convolution module; a picture taken according to a visual detection component is used as the input of the backbone network to propose features of each layer, and at the same time, the backbone network enhances the extraction of detail features in a walnut kernel feature map through the CSPPC module, and a walnut kernel surface image feature map is obtained through an SPPF operation after the CSPPC module of the last layer; each layer of features and the walnut kernel surface image feature map are used as the input of the neck network, and secondary processing is performed through the new Dual convolution module in the Neck network to obtain a multi-scale feature preprocessing map; the multi-scale feature preprocessing map is used as the input of the head network to predict the state category and confidence of each walnut kernel in the map.

[0007] Furthermore, the CSPPS module includes a first convolution layer, a feature segmentation layer, a dual-part convolution layer, a feature splicing layer and a second convolution layer, and the dual-part convolution layer is composed of two partial convolutions; wherein, the first convolution layer performs batch normalization on the height, width and number of channels of the input image as the input of the feature segmentation layer, and the output is divided into two parts through the feature segmentation layer: the number of channels of each part is changed to half of the original, the first part of the output enters the dual-part convolution layer, and the partial convolution of the dual-part convolution layer is repeated and the feature map is spliced ​​with all dimensions unchanged. The spliced ​​feature map and the second part of the output enter the feature splicing layer together for secondary splicing, and finally the number of channels of the feature map output by the feature splicing layer is restored through the second convolution layer.

[0008] According to a second aspect of the present invention, there is provided a walnut kernel sorting automated conveying production line based on visual inspection, comprising a feeding conveying mechanism, a visual inspection mechanism, a walnut kernel sorting mechanism, a walnut kernel transmission and recovery mechanism, a first walnut kernel recovery mechanism 30, and a host computer, wherein the visual inspection mechanism comprises a visual inspection transmission component 12 and a visual inspection component;

[0009] The discharge port of the feeding conveying mechanism is communicated with the feeding port of the visual inspection transmission component 12, and is used to convey the walnut kernels to the visual inspection transmission component 12; the visual inspection component and the walnut kernel sorting mechanism are installed on the side of the visual inspection transmission component 12 away from the feeding conveying mechanism;

[0010] The visual inspection transmission component 12 is used to drive the walnut kernel to move along a first preset direction, the visual inspection component is used to take pictures of the walnut kernel moving in the first preset direction, and the host computer calls the walnut kernel quality inspection model to detect the pictures taken by the visual inspection component to determine whether the state of the walnut kernel is the first state or the second state;

[0011] When the walnut kernel is in the first state, the visual detection transmission component 12 is used to guide the walnut kernel in the first state to the first state walnut kernel recovery mechanism 30 through the movement of the visual detection transmission component 12 along the first preset direction;

[0012] When the walnut kernels are in the second state, the feeding conveying mechanism and the visual inspection transmission component 12 stop conveying, and the walnut kernel sorting mechanism moves along the first and second preset directions to sort the walnut kernels in the second state to the bad material transmission and recovery mechanism; the walnut kernel transmission and recovery mechanism is used to move the walnut kernels in the second state along the second preset direction to guide them to the second state walnut kernel recovery mechanism 29.

[0013] Furthermore, the feeding conveying mechanism includes a trapezoidal lower hopper 6 connected to the top of the primary feeding support frame 1, the outlet of the trapezoidal lower hopper 6 is communicated with the inlet of the primary receiving tray 3, and below the primary receiving tray 3 there is a primary vibrating feeder 4 installed above the primary vibrating feeding bottom plate 32, the outlet of the primary receiving tray 3 is communicated with the inlet of the primary feeding slide 5 installed on the primary slide support frame 31, the outlet of the primary feeding slide 5 is communicated with the inlet of the secondary receiving tray 9, and below the secondary receiving tray 9 there is a secondary vibrating feeder 8 installed above the secondary vibrating feeding bottom plate 33, The outlet of the secondary material receiving tray 9 is communicated with the inlet of the secondary feeding slide 46 installed on the secondary slide support frame 34, and the outlet of the secondary feeding slide 46 is communicated with the inlet of the visual inspection transmission component 12; wherein, the working surfaces of the primary material receiving tray 3 and the secondary material receiving tray 9 are parallel to the conveying direction of the visual inspection transmission component 12, and the working surfaces of the primary material receiving tray 3, the secondary material receiving tray 9, and the working plane of the visual inspection transmission component 12 are arranged in sequence up and down, and the primary feeding slide 5 and the secondary feeding slide 46 are arranged obliquely downward from the inlet to the outlet.

[0014] Furthermore, the visual inspection component includes a visual inspection conveyor belt support frame 11, a visual inspection light source mounting plate 56, a camera connecting frame 57, and a visual inspection camera 21; the visual inspection platform support frame 11 is connected with a visual inspection light source mounting plate 56 and a camera connecting frame 57, the visual inspection light source mounting plate 56 is installed with a visual inspection light source 55, the camera connecting frame 57 is installed with a visual inspection camera 21, and the camera head of the visual inspection camera 21 extends from the center position of the visual inspection light source 55, and is used to take pictures of walnut kernels on the first conveyor belt in the inspection area of ​​the visual inspection transmission component 12.

[0015] Furthermore, the walnut kernel sorting mechanism includes two sorting mechanism mounting base plates 22, a first preset direction stepper motor 25, a second preset direction synchronous belt module 26, a second preset direction servo motor 27, a first preset direction synchronous belt module 41, a double-axis cylinder 2, and a flexible clamp 51; the two sorting mechanism mounting base plates 22 are installed at intervals on the side of the conveyor belt frame of the visual detection transmission component 12 away from the feeding conveying mechanism; the second preset direction synchronous belt modules 26 are respectively installed above the two sorting mechanism mounting base plates 22, and the power is transmitted to the second preset direction synchronous belt module 26 through the second preset direction servo motor 27; the two second preset direction synchronous belt modules 26 drive the first preset direction synchronous belt module 41 to move along the second preset direction; the power is transmitted to the first preset direction synchronous belt module 41 through the first preset direction stepper motor 25 ; The Z-axis connecting plate 35 is driven to move along the first preset direction by the first preset direction synchronous belt module 41; the first preset direction synchronous belt module 41 is equipped with a first preset direction upper limit photoelectric switch 15, a first preset direction origin photoelectric switch 16 and a first preset direction lower limit photoelectric switch 17, and the second preset direction synchronous belt module 26 is equipped with a second preset direction upper limit photoelectric switch 18, a second preset direction origin photoelectric switch 19 and a second preset direction lower limit photoelectric switch 20; the Z-axis connecting plate 35 is located between the first preset direction upper limit photoelectric switch 15 and the first preset direction lower limit photoelectric switch 17, and the Z-axis connecting plate 35 is equipped with a double-axis cylinder 2, and the double-axis cylinder 2 is equipped with an upper limit magnetic reed sensor 23 and a lower limit magnetic reed sensor 24, and the end of the double-axis cylinder 2 is connected with a flexible clamp 51.

[0016] Furthermore, the walnut kernel transmission and recovery mechanism includes a second-state walnut kernel recovery mechanism 29 and a bad material transmission component 43. The bad material transmission component 43 is used to drive the second-state walnut kernels sorted out by the walnut kernel sorting mechanism to move along a second preset direction. The walnut kernels in the second state are guided to the second-state walnut kernel recovery mechanism 29 through the movement of the bad material transmission component 43 along the second preset direction.

[0017] The beneficial effects of the present invention are:

[0018] First, the walnut kernel sorting automated conveying production line based on visual inspection provided by the present invention has high flexibility and intelligence. The feeding and conveying mechanism, visual inspection mechanism, walnut kernel sorting mechanism, and walnut kernel transmission and recovery mechanism on the production line adopt a modular design. The modular design not only facilitates the maintenance and upgrading of the machine, but also can be flexibly adjusted according to actual production needs or the replacement of the detection object, thereby improving the adaptability and long-term use value of the equipment and having a wide range of applications.

[0019] Second, the walnut kernel quality detection model introduced by the present invention based on the training set under the real sample can accurately distinguish between walnut kernels that meet the standards and those that do not, providing strong technical support for the industry; because a software-based deep learning solution is adopted, the function of the equipment can be improved by upgrading the algorithm instead of replacing the hardware; this not only reduces the cost of equipment upgrades, but also enables the equipment to quickly adapt to market and technological changes. At the same time, compared with traditional visual systems, the walnut kernel quality detection model based on deep learning can learn more details from complex data, effectively reducing misjudgments caused by manual operations or limitations of traditional machine vision settings; the repeatability and reliability of the production line have been greatly improved, ensuring stability and consistency for long-term operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the overall structure of a preferred embodiment provided by the present invention;

[0021] Figure 2 It is a structural schematic diagram of the feeding and conveying mechanism;

[0022] Figure 3 It is a structural schematic diagram of the feeding conveying mechanism and the visual detection transmission component;

[0023] Figure 4 It is a structural diagram of a visual inspection mechanism;

[0024] Figure 5 It is a schematic diagram of the visual inspection component;

[0025] Figure 6 It is a structural diagram of a walnut kernel sorting mechanism;

[0026] Figure 7 It is a structural diagram of the walnut kernel transmission and recovery mechanism;

[0027] Figure 8 It is a structural diagram of the electrical cabinet;

[0028] Fig. 9 This is a schematic diagram of the structure of the walnut kernel quality detection model of the present invention;

[0029] Fig.10 Examples for data augmentation;

[0030] Fig.11 It is a schematic diagram of the CSPPC module structure;

[0031] Fig.12 It is a schematic diagram of the Dual convolution module structure;

[0032] The numbers in the figure are: 1. First-level feeding support frame; 2. Double-axis cylinder; 3. First-level receiving tray; 4. First-level vibrating feeder; 5. First-level feeding slide; 6. Trapezoidal lower hopper; 7. Second-level feeding support frame; 8. Second-level vibrating feeder; 9. Second-level receiving tray; 10. Second preset direction synchronous belt; 11. Visual inspection platform support frame; 12. Visual inspection transmission component; 13. Feeding baffle; 14. Slot-type photoelectric sensor; 15. First preset direction upper limit photoelectric switch; 16. First preset direction origin photoelectric switch; 17. First preset direction lower limit photoelectric switch; 18. Second preset direction upper limit photoelectric switch; 19. Second preset direction 1. Photoelectric switch for origin of direction; 20. Photoelectric switch for lower limit of second preset direction; 21. Visual inspection camera; 22. Installation base plate of sorting mechanism; 23. Upper limit magnetic reed sensor; 24. Lower limit magnetic reed sensor; 25. Stepper motor for first preset direction; 26. Synchronous belt module for second preset direction; 27. Servo motor for second preset direction; 28. Second driving device; 29. ​​Walnut kernel recovery mechanism for second state; 30. Walnut kernel recovery mechanism for first state; 31. First slide bucket support frame; 32. First vibration feeding base plate; 33. Second vibration feeding base plate; 34. Second slide bucket support frame; 35. Z-axis connecting plate; 36. First Preset direction synchronous belt; 37, connecting shaft; 38, slot type photoelectric sensor induction sheet; 39, first driving device; 40, flexible clamping claw connecting frame; 41, first preset direction synchronous belt module; 42, transmission support frame; 43, bad material transmission assembly; 44, transmission belt baffle; 45, reduction reduction motor fixing plate; 46, secondary feeding slide; 47, loading photoelectric sensor; 48, primary vibration feeder mounting base; 49, secondary feeding vibrator mounting base; 50, slide; 51, flexible clamping claw; 52, loading photoelectric sensor fixing plate; 53, HMI touch screen; 54, visual detection transmission belt transmission shaft; 55, visual detection light Source; 56. Visual detection light source mounting plate; 57. Camera connecting frame; 58. Electrical cabinet; 59. First state walnut kernel recovery hopper; 60. Second state walnut kernel recovery hopper; 61. European standard 40 type angle code; 62. Start button; 63. Stop button; 64. Emergency stop button; 65. Manual-automatic switching knob; 66. Feeding mechanism and detection mechanism connecting plate; 67. First preset direction synchronous belt coupling; 68. Second preset direction synchronous belt coupling; 69. Flexible baffle; 70. Groove-type photoelectric sensor fixing plate; 71. Transmission component support frame; 72. Feed baffle support frame; 73. Flexible baffle support frame; 74. Boss. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other arbitrarily without conflict.

[0034] Example 1: Figure 1-Figure 12 As shown, according to the first aspect of an embodiment of the present invention, there is provided an automated conveying production line for walnut kernel sorting based on visual inspection, comprising a feeding conveying mechanism, a visual inspection mechanism, a walnut kernel sorting mechanism, a walnut kernel transmission and recovery mechanism, a first walnut kernel recovery mechanism 30, and a host computer, wherein the visual inspection mechanism comprises a visual inspection transmission component 12 and a visual inspection component; the discharge port of the feeding conveying mechanism is communicated with the feed port of the visual inspection transmission component 12, and is used to convey the walnut kernel to the visual inspection transmission component 12; the visual inspection component and the walnut kernel sorting mechanism are installed on the side of the visual inspection transmission component 12 away from the feeding conveying mechanism; the visual inspection transmission component 12 is used to drive the walnut kernel to move along a first preset direction, and the visual inspection component is used to detect the movement in the first preset direction The walnut kernels are photographed, and the upper computer calls the walnut kernel quality detection model to detect the pictures taken by the visual detection component to determine whether the state of the walnut kernel is the first state or the second state; when the walnut kernel is in the first state, the visual detection transmission component 12 is used to guide the walnut kernels in the first state to the first state walnut kernel recovery mechanism 30 through the movement of the visual detection transmission component 12 along the first preset direction; when the walnut kernel is in the second state, the feeding conveying mechanism and the visual detection transmission component 12 stop conveying, and the walnut kernels in the second state are sorted to the bad material transmission and recovery mechanism through the movement of the walnut kernel sorting mechanism along the first and second preset directions; the walnut kernel transmission and recovery mechanism is used to move the walnut kernels in the second state along the second preset direction to guide them to the second state walnut kernel recovery mechanism 29. Exemplarily, the first state indicates that the walnut kernel is a qualified walnut kernel that meets the quality requirements; otherwise, it is a substandard walnut kernel that does not meet the quality requirements.

[0035] Furthermore, if Figure 2 , Figure 3As shown, the feeding conveying mechanism includes a trapezoidal lower hopper 6 connected to the top of the primary feeding support frame 1, the outlet of the trapezoidal lower hopper 6 is communicated with the inlet of the primary receiving tray 3, and the primary receiving tray 3 is provided with a primary vibrating feeder 4 installed above the primary vibrating feeding bottom plate 32, the outlet of the primary receiving tray 3 is communicated with the inlet of the primary feeding slide 5 installed on the primary slide support frame 31, the outlet of the primary feeding slide 5 is communicated with the inlet of the secondary receiving tray 9, and the secondary receiving tray 9 is provided with a secondary vibrating feeder 8 installed above the secondary vibrating feeding bottom plate 33, the outlet of the secondary receiving tray 9 is communicated with the secondary vibrating feeder 8 installed on the secondary slide support frame 34. The inlet of the feeding slide 46 is communicated, the outlet of the secondary feeding slide 46 is communicated with the inlet of the visual detection mechanism, and a loading photoelectric sensor 47 installed on the loading photoelectric sensor fixing plate 52 is provided at the junction of the outlet of the secondary feeding slide 46 and the inlet of the visual detection mechanism; wherein, the working surfaces of the primary receiving tray 3 and the secondary receiving tray 9 are parallel to the conveying direction of the visual detection transmission component 12, and the working surfaces of the primary receiving tray 3, the secondary receiving tray 9, and the working plane of the visual detection transmission component 12 are arranged in sequence up and down, and the primary feeding slide 5 and the secondary feeding slide 46 are arranged obliquely downward from the inlet to the outlet.

[0036] Furthermore, feed baffles 13 are installed on both sides of the inlet and outlet of the secondary receiving tray 9 through a feed baffle support frame 72, and a flexible baffle 69 is installed at the outlet end of the secondary receiving tray 9 through a flexible baffle support frame 73.

[0037] Furthermore, the working surfaces of the first-level receiving tray 3 and the second-level receiving tray 9 are provided with protruding columns 74 arranged perpendicular to the working surfaces, so as to block the linear movement of the walnut kernels and slow down the overall moving speed of the walnut kernels.

[0038] By applying the above technical solution, it can be known that a two-stage vibration feeding mode is adopted in the present invention. The first-level receiving tray 3 and the second-level receiving tray 9 are rectangular receiving trays with an opening on the top surface. The vibration frequency of the first-level vibration feeder 4 is greater than the vibration frequency of the second-level vibration feeder 8. Based on on-site practical operation, this feeding mode can realize uniformly scattered feeding of walnut kernels to a large extent (for example, the sizes of the first-level receiving tray 3 and the second-level receiving tray 9 are 350mm*190mm*40mm and 260mm*160mm*40mm, respectively. The vibration frequencies of the first-stage vibration feeder 4 and the second-stage vibration feeder 8 are set to 180-220 Hz and 100-140 Hz, respectively. Field verification shows that this design can achieve uniform feeding of walnut kernels and effectively reduce the stacking of walnut kernels; if the vibration frequency is higher than this interval, the linear motion speed of the walnut kernels is faster, which may cause splashing and overlapping of materials; if the vibration frequency is lower than this interval, the linear motion speed of the walnut kernels is slower, which may cause a large amount of stacking of walnut kernels and affect subsequent work efficiency).

[0039] Furthermore, if Figure 4 As shown, the visual detection transmission component 12 is installed on the transmission component support frame 71, and the visual detection transmission component 12 includes a first conveyor belt frame, a first driving device 39, and a first conveyor belt; the first driving device 39 installed on the first conveyor belt frame is used to drive the first conveyor belt to be movably arranged along a first preset direction.

[0040] Furthermore, if Figure 4 , Figure 5 As shown, the visual inspection component includes a visual inspection conveyor belt support frame 11, a visual inspection light source mounting plate 56, a camera connecting frame 57, and a visual inspection camera 21; the visual inspection platform support frame 11 is connected with a visual inspection light source mounting plate 56 and a camera connecting frame 57, the visual inspection light source mounting plate 56 is mounted with a visual inspection light source 55, the camera connecting frame 57 is mounted with a visual inspection camera 21, and the camera head of the visual inspection camera 21 extends from a small hole with a diameter of 32 mm opened at the center position of the visual inspection light source 55, and is used to take pictures of walnut kernels on the first conveyor belt in the inspection area of ​​the visual inspection transmission component 12; the inspection area of ​​the camera matches the maximum travel area of ​​the flexible clamp 51 in the walnut kernel sorting mechanism.

[0041] Furthermore, if Figure 4As shown, a slot-type photoelectric sensor fixing plate 70 is installed on the side of the first conveyor belt frame close to the walnut kernel recovery mechanism, and a slot-type photoelectric sensor 14 is installed on the slot-type photoelectric sensor fixing plate 70. A slot-type photoelectric sensor sensing sheet 38 is installed on the visual detection conveyor belt transmission shaft 54 ​​of the visual detection transmission assembly 12. The photoelectric sensor sensing sheet 38 cooperates with the slot-type photoelectric sensor 14 to initialize the first conveyor belt. The driving device 39 adopts a stepping motor.

[0042] Furthermore, if Figure 6As shown, the walnut kernel sorting mechanism includes two sorting mechanism mounting base plates 22, a first preset direction stepping motor 25, a second preset direction synchronous belt module 26, a second preset direction servo motor 27, and a first preset direction synchronous belt module 41; the two sorting mechanism mounting base plates 22 are installed at intervals on the side of the conveyor belt frame of the visual detection transmission component 12 away from the feeding conveying mechanism; the second preset direction synchronous belt modules 26 are respectively installed above the two sorting mechanism mounting base plates 22, and the two second preset direction synchronous belt modules 26 are connected by the second preset direction synchronous belt coupling 68 and the connecting shaft 37, and the second The preset direction servo motor 27 is installed at the end of a second preset direction synchronous belt module 26, and the power is transmitted to the second preset direction synchronous belt 10 of the second preset direction synchronous belt module 26 through the second preset direction synchronous belt coupling 68 and the connecting shaft 37 by the second preset direction servo motor 27; the second preset direction synchronous belts 10 of the two second preset direction synchronous belt modules 26 rotate to drive the slide 50 installed on the second preset direction synchronous belt module 26 to move along the second preset direction; the movement of the slide 50 drives the first preset direction synchronous belt module 41 installed above the two sets of slides 50 to move along the second preset direction. The first preset direction synchronous belt module 41 is provided with a first preset direction stepping motor 25 at its end, and the first preset direction stepping motor 25 transmits power to the first preset direction synchronous belt 36 of the first preset direction synchronous belt module 41 through the first preset direction synchronous belt coupling 67; the first preset direction synchronous belt 36 rotates to drive the Z-axis connecting plate 35 installed on the first preset direction synchronous belt module 41 to move along the first preset direction; the first preset direction synchronous belt module 41 is provided with a first preset direction upper limit photoelectric switch 15, a first preset direction origin photoelectric switch 16 and a first preset direction The downward limit photoelectric switch 17, the second preset direction synchronous belt module 26 is equipped with a second preset direction upper limit photoelectric switch 18, a second preset direction origin photoelectric switch 19 and a second preset direction lower limit photoelectric switch 20; the Z-axis connecting plate 35 is located between the first preset direction upper limit photoelectric switch 15 and the first preset direction lower limit photoelectric switch 17, the Z-axis connecting plate 35 is equipped with a double-axis cylinder 2, the double-axis cylinder 2 is equipped with an upper limit reed sensor 23 and a lower limit reed sensor 24, and the double-axis cylinder 2 is connected to a flexible clamp 51 through a flexible clamp connecting frame 40.

[0043] Furthermore, if Figure 7As shown, the walnut kernel transmission and recovery mechanism includes a second-state walnut kernel recovery lower hopper 60, a second-state walnut kernel recovery mechanism 29, and a bad material transmission component 43. The bad material transmission component 43 is used to drive the second-state walnut kernels sorted out by the walnut kernel sorting mechanism to move along a second preset direction. The movement of the bad material transmission component 43 along the second preset direction guides the walnut kernels in the second state to the second-state walnut kernel recovery lower hopper 60 to the second-state walnut kernel recovery mechanism 29.

[0044] Specifically, the bad material transmission assembly 43 is installed on the transmission support frame 42, and the bad material transmission assembly 43 includes a second conveyor belt frame, a second driving device 28, and a second conveyor belt; the second driving device 28 installed on the second conveyor belt frame through the motor fixing plate 45 is used to drive the second conveyor belt to be movably arranged along the second preset direction; the second conveyor belt frame is installed with a conveyor belt baffle 44 on the side close to the visual sorting mechanism; the transmission support frame 42 is installed with a second state walnut kernel recovery lower hopper 60 on the side away from the visual sorting mechanism, the outlet end of the second conveyor belt is arranged in communication with the second state walnut kernel recovery lower hopper 60, and the outlet side of the second state walnut kernel recovery lower hopper 60 is installed with a second state walnut kernel recovery mechanism 29, and the second state walnut kernel recovery mechanism 29 can be directly placed on the ground. The second driving device 28 adopts a reduction motor, and the first preset direction is perpendicular to the second preset direction. The present invention can achieve the maximum extent of smoothly transporting the walnut kernels sorted by the visual sorting mechanism to the second conveyor belt by cleverly installing the conveyor belt baffle 44 on the side of the second conveyor belt frame close to the visual sorting mechanism.

[0045] Furthermore, it also includes a first-state walnut kernel recovery lower hopper 59 connected to the transmission component support frame 71 in the visual inspection mechanism, and the entrance of the first-state walnut kernel recovery lower hopper 59 is connected to the outlet of the visual inspection mechanism; the first-state walnut kernel recovery lower hopper 59 is used to guide the walnut kernels in the first state to the first-state walnut kernel recovery mechanism 30, and the first-state walnut kernel recovery mechanism 30 can be directly placed on the ground.

[0046] Furthermore, if Figure 8As shown, an S7-1200 PLC installed inside the electrical cabinet 58 is also provided, and is equipped with a variety of input and output ports. Its digital input port is connected to the feeding photoelectric sensor 47, the slot photoelectric sensor 14, and the limit switches of each mechanism, and the output port is connected to the drive device and each motor and vibrator; the top of the electrical cabinet 58 is equipped with a start button 62, a stop button 63, a manual-automatic switching knob 65, an emergency stop button 64, and an HMI touch screen 53 controlled by the PLC; through the PLC and HMI touch screen, the operator can monitor the entire detection process in real time (for example, using the indicator light to distinguish the state of the walnut kernel, setting the first state to be displayed as light on, and the second state to be displayed as light off), so that the monitoring personnel can find and solve problems in time, which greatly improves production efficiency and safety.

[0047] In the above, the first-level feeding support frame 1, the first-level sliding bucket support frame 31, the second-level feeding support frame 7, the second-level sliding bucket support frame 34, the visual inspection platform support frame 11, the transmission support frame 42, the transmission component support frame 71 and other support frames are constructed using European standard 40 profiles and European standard 40 angle codes 61.

[0048] According to a second aspect of an embodiment of the present invention, a method for establishing a walnut kernel quality detection model is provided. The walnut kernel quality detection model is based on yolov8 as a framework, including a backbone network, a neck network, and a head network. The original C2f module of the backbone network is replaced by a CSPPC module, and the original C2f module in the neck network is replaced by a Dual convolution module; the complete network structure diagram is shown in FIG. Fig. 9 As shown; the pictures taken according to the visual detection component are used as the input of the backbone network to propose each layer of features, and at the same time, the CSPPC module is used in the backbone network to enhance the extraction of detail features in the walnut kernel feature map, and the SPPF operation is performed after the last layer of the CSPPC module to obtain the walnut kernel surface image feature map; the features of each layer and the walnut kernel surface image feature map are used as the input of the neck network, and the new Dual convolution module in the Neck network is used for secondary processing to obtain a multi-scale feature preprocessing map; the multi-scale feature preprocessing map is used as the input of the head network to predict the state category and confidence of each walnut kernel in the image.

[0049] like Fig.11As shown, the CSPPS module includes a first convolutional layer, a feature segmentation layer, a dual-part convolutional layer, a feature splicing layer and a second convolutional layer. The dual-part convolutional layer consists of two partial convolutions. Convolution); the first convolution layer batch normalizes the height, width and number of channels of the input image as the input of the feature segmentation layer, and the output is divided into two parts through the feature segmentation layer (Split): the number of channels of each part is reduced to half of the original number, the first part of the output enters the double-part convolution layer, and the partial convolution of the double-part convolution layer is repeated and the feature map is spliced ​​with all dimensions unchanged. The spliced ​​feature map and the second part of the output enter the feature splicing layer (Concat) for secondary splicing (that is, the feature splicing layer simultaneously receives the half-channel feature map output by the feature segmentation layer and the feature map after splicing by the double-part convolution layer, and then performs secondary splicing. This splicing is performed on the channel dimension, the spatial dimension remains unchanged, and the feature map after splicing is (N+2) times the original number of channels). Finally, the second convolution layer restores the number of channels of the (N+2) times feature map output by the feature splicing layer. This module performs multi-level processing and information fusion on the input features to achieve the purpose of improving the performance of the detection model and enhancing the feature extraction capability.

[0050] like Fig.12 As shown in the figure, the dual convolution module is composed of N convolution filters; the N convolution filters are divided into G groups, and the M / G channels of each group are processed by 3×3 and 1×1 convolutions at the same time; the 3×3 convolution kernel uses its larger size to capture the spatial information of the local area, and performs convolution operation on the feature map by sliding in space to achieve the purpose of obtaining spatial information. The 1×1 convolution kernel linearly transforms the channels of the feature map, adjusts and combines its features, extracts more detailed features, and realizes the fusion of features of different scales; at the same time, the remaining MM / G channels in each group are only processed by the 1×1 convolution kernel, which is responsible for linearly transforming the features of the channels and further adjusting the feature information; the above method is repeated G times, and finally a dual convolution integrated feature map with a depth of N is output; wherein M is the depth of the feature map input to the dual convolution module, and N is the depth of the output feature map.

[0051] Before using the walnut kernel quality detection model to detect real-time images, the model needs to be trained. The specific training and testing process is as follows:

[0052] A Hikvision MV-CS050-10GC camera was used to take 800 pictures of Yangbi walnut kernels of different qualities. The data were then expanded by combining random rotation, random brightness adjustment, random sharpening, random occlusion, random Gaussian blur, random addition of Poisson noise, and random contrast adjustment (e.g. Fig.10The images were expanded to 2380 images, a benchmark database for walnut kernel quality inspection was constructed, and the images were annotated using the image annotation software Labelimg (the specific annotated categories were Good walnut meat, Bad walnut meat, Diaphragmajuglandis, and Walnut shell. Good walnut mea indicated good walnut kernels, Bad walnut meat indicated bad walnut kernels, Diaphragmajuglandis indicated heartwood, and Walnut shell indicated shells. When the trained model was used for subsequent predictions, the predicted state categories of the target objects were Bad walnut meat, Diaphragmajuglandis, and Walnut shell as the second state; the predicted state category of the target object was Good walnut meat as the first state). The annotation file was saved in TXT format, and the dataset was redivided into training, validation, and test sets in a ratio of 8:1:1.

[0053] The images in the training set are input into the walnut kernel quality detection model. The images in the training set are enhanced by the online image enhancement method and then scaled to a size of 640×640 as the input of the backbone network of the walnut kernel quality detection model. After calculation by the walnut kernel quality detection model, the prediction box and category confidence are output, and the loss is calculated with the annotations in the training set and the parameters in the model are updated by back propagation. After each round of training, the evaluation index of the walnut kernel quality detection model is calculated using the validation set. After all training rounds are completed, the model weights of the generation with the best validation set evaluation index are saved as the final weights after the model training is completed. When testing, the optimal model weights are loaded and frozen, and forward reasoning is performed on the input image to output the prediction box position and category confidence predicted by the model.

[0054] The present invention is based on yolov8, and improves its backbone and neck network in a targeted manner, ultimately achieving improved model accuracy and lightweight. In order to further verify the effectiveness of the improvement strategy, precision (Precision, P), mean average precision (MAP), number of floating-point operations per second, and model memory usage are used as model performance evaluation parameters. The ablation test results are shown in Table 1. It can be seen from Table 1 that compared with the original yolov8 model and the model after the CSPPC model is introduced into the backbone network (the original yolov8 model is the first row of data in Table 1, the model after the CSPPC model is introduced into the backbone network is the second row of data in Table 1, and the model of the present invention is the third row of data in Table 1), the accuracy of the model of the present invention is improved by 1.9% and 3.2% respectively; the model memory usage and number of floating-point operations per second are reduced by 1.2MB and 1.6×10 compared with the original yolov8 model. 9 It can be seen from Table 1 that the method of the present invention has the highest accuracy and the model occupies the smallest space.

[0055] Table 1

[0056]

[0057] In order to further verify the advantages of the model of the present invention over the existing mainstream target detection algorithms, the method of the present invention is compared with three mainstream target detection methods. The comparison results are shown in Table 2. Combined with Table 2, it can be seen that the overall performance of the original yolov8 model is better than that of yolov9s and yolov10s; the accuracy and average precision of the method WKQD-yolov8 of the present invention are improved by 1.9% and 0.1% respectively compared with the traditional yolov8 network, which meets the quality detection requirements of walnut kernels; and the memory usage and floating-point operations per second of the model are 5.1MB and 6.5×10 9 , which are respectively reduced by 19.05% and 19.75% compared with the original yolov8 model, and can better complete the task of walnut kernel quality inspection.

[0058] Table 2

[0059]

[0060] It can be seen from the application of the above technical solutions that the walnut kernel quality detection model proposed in the present invention is based on an advanced algorithm, and its training data is collected from walnut kernels in Yangbi, Yunnan, covering various quality conditions, appearance characteristics and size specifications. The walnut kernel quality detection model of the present invention introduced based on the training set under real samples can accurately distinguish between qualified and substandard walnut kernels, providing strong technical support for the industry. Compared with traditional visual systems, the walnut kernel quality detection model based on deep learning can learn more details from complex data, effectively reducing misjudgments caused by manual operations or limitations of traditional machine vision settings; the repeatability and reliability of the production line have been greatly improved, ensuring stability and consistency in long-term operation.

[0061] The working principle of the walnut kernel visual inspection and sorting automated conveying production line equipment provided by the present invention is as follows:

[0062] After the equipment is turned on, the equipment is adjusted to the automatic operation mode through the HMI touch screen 53 (when the operation mode is in working state, the visual detection light source 55 is turned on to achieve the purpose of detecting that the ambient light source is sufficient). After reaching the automatic mode, the primary vibration feeder 4 and the secondary vibration feeder 8 start to work, and the walnut kernels enter the equipment through the trapezoidal lower hopper 6 to reach the primary receiving tray 3. The primary vibration feeder 4 vibrates to make the walnut kernels move linearly in the primary receiving tray 3 to reach the primary feeding slide 5, and the walnut kernels enter the secondary receiving tray 9 through inertia and vibration. Among them, the feeding baffle 13 prevents the walnut kernels from flying out of the equipment when entering the secondary receiving tray 9 through the primary feeding slide 5; the flexible baffle 69 slows down the time for the walnut kernels to enter the secondary feeding slide 46 from the secondary receiving tray 9, so that the walnut kernels are evenly distributed and enter the secondary receiving tray 9.

[0063] When the walnut kernel enters the secondary receiving tray 9, the secondary vibrating feeder 8 vibrates to make the walnut kernel move linearly in the secondary receiving tray 9, and enters the secondary feeding slide 46 by inertia and vibration. The walnut kernel enters the visual inspection transmission component 12 from the secondary feeding slide 46 by inertia, and triggers the feeding photoelectric sensor 47. The feeding photoelectric sensor 47 detects the signal, feeds the signal back to the S7-1200PLC, and transmits the signal to the first drive device 39. The first drive device 39 starts working, and the first drive device 39 drives the visual inspection transmission component 12 to start working. When the walnut kernel enters the detection area of ​​the visual inspection camera 21, the trained walnut kernel quality detection model is called to detect the pictures taken by the camera:

[0064] If the walnut kernel quality inspection model does not detect walnut kernels that do not meet the standards, the first drive device 39 will continue to drive the visual inspection transmission component 12 to work, and the qualified walnut kernels will be sent to the first state walnut kernel recovery hopper 59 through the visual inspection transmission component 12, and the qualified walnut kernels will be sent to the first state walnut kernel recovery mechanism 30 through the first state walnut kernel recovery hopper 59 to achieve the purpose of collecting qualified walnut kernels.

[0065] If the walnut kernel quality detection model detects walnut kernels that do not meet the standards, the signal of the walnut kernels that do not meet the standards will be transmitted to the PLC. After receiving the corresponding signal, the PLC controls the first-level vibration feeder 4, the second-level vibration feeder 8, and the first drive device 39 to stop. When the first-level vibration feeder 4, the second-level vibration feeder 8, and the first drive device 39 stop working, the first conveyor belt driven by the first drive device 39 also stops working. At the same time, the host computer reads the corresponding signal value in the PLC. When the three signal values ​​corresponding to the first-level vibration feeder 4, the second-level vibration feeder 8, and the first drive device 39 all reach the stop condition, the host computer will transmit the X-axis and Y-axis position information corresponding to the walnut kernels that do not meet the standards to the PLC. After receiving the position signal, the PLC will control The second preset direction servo motor 27 and the first preset direction stepper motor 25 start working, the second preset direction servo motor 27 and the first preset direction stepper motor 25 drive the second preset direction synchronous belt 10 and the first preset direction synchronous belt 36 to start working, the first preset direction synchronous belt 36 and the second preset direction synchronous belt 10 receive the position signal of the substandard walnut kernels transmitted by the PLC, and will run the corresponding distance according to the position information; when the first preset direction synchronous belt 36 and the second preset direction synchronous belt 10 are in place, the signal is transmitted to the dual-axis cylinder 2, and the dual-axis cylinder 2 receives the signal to start working; since the initial position of the dual-axis cylinder 2 is at the upper limit, the dual-axis cylinder 2 receives the signal to start working and return to the lower limit, and the lower limit reed sensor 24 receives the signal , the signal is transmitted to the flexible clamp 51, and the flexible clamp 51 starts working after receiving the signal, and grabs the substandard walnut kernels at the current position, and the grabbing delay is 3s. After the delay is reached, the signal is transmitted to the double-axis cylinder 2, and the double-axis cylinder 2 starts working after receiving the signal and returns to the upper limit position, and the upper limit reed sensor 23 receives the signal and transmits the signal to the second preset direction servo motor 27 and the first preset direction stepper motor 25; then the second preset direction servo motor 27 and the first preset direction stepper motor 25 drive the second preset direction synchronous belt coupling 68 and the first preset direction synchronous belt coupling 67, so that the second preset direction synchronous belt 10 and the first preset direction synchronous belt 36 start working; when the first preset direction synchronous belt 36 and the second preset direction synchronous belt 36 When the step belt 10 moves to the second conveyor belt position in the bad material transmission component 43 specified by the PLC, the signal is transmitted to the flexible clamp 51. After receiving the signal, the flexible clamp 51 begins to loosen the grasped walnut kernels until all substandard walnut kernels in the current detection area are grasped to the specified position of the second conveyor belt (if there are still substandard walnut kernels in the detection area, the above control operation will be repeated). If there are no substandard walnut kernels in the current detection area, the signal will be transmitted to the first-level vibration feeder 4, the second-level vibration feeder 8, and the first drive device 39. When the first-level vibration feeder 4, the second-level vibration feeder 8, and the first drive device 39 receive the signal, they all start working. The first drive device 39 drives the visual inspection transmission component 12 to start working and continue to detect walnut kernels.Based on this walnut kernel visual inspection and sorting automated conveying production line equipment, automated inspection and sorting of walnut kernels can be achieved.

[0066] The circuits involved in the present invention are all prior art and will not be described in detail here.

[0067] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A method for establishing a walnut kernel quality detection model, characterized in that: The walnut kernel quality detection model uses yolov8 as a framework, includes a backbone network, a neck network, and a head network, replaces the original C2f module of the backbone network with a CSPPC module, and replaces the original C2f module in the neck network with a Dual convolution module; takes the picture taken according to the visual detection component as the input of the backbone network to propose each layer of features, and at the same time, the backbone network uses the CSPPC module to enhance the extraction of detail features in the walnut kernel feature map, and obtains the walnut kernel surface image feature map through the SPPF operation after the last layer of the CSPPC module; takes the features of each layer and the walnut kernel surface image feature map as the input of the neck network, performs secondary processing through the new Dual convolution module in the Neck network, and obtains a multi-scale feature preprocessing map; takes the multi-scale feature preprocessing map as the input of the head network, and predicts the state category and confidence of each walnut kernel in the map.

2. The method for establishing a walnut kernel quality detection model according to claim 1, wherein: The CSPPS module includes a first convolution layer, a feature segmentation layer, a dual-part convolution layer, a feature splicing layer and a second convolution layer, wherein the dual-part convolution layer is composed of two partial convolutions; wherein the first convolution layer performs batch normalization on the height, width and number of channels of the input image as the input of the feature segmentation layer, and the output is divided into two parts through the feature segmentation layer: the number of channels of each part is reduced to half of the original number, the first part of the output enters the dual-part convolution layer, the partial convolution of the dual-part convolution layer is repeated and the feature map is spliced ​​with all dimensions unchanged, the spliced ​​feature map and the second part of the output enter the feature splicing layer together for secondary splicing, and finally the number of channels of the feature map output by the feature splicing layer is restored through the second convolution layer.

3. A walnut kernel sorting and automated conveying production line based on visual inspection, characterized in that: It comprises a feeding and conveying mechanism, a visual inspection mechanism, a walnut kernel sorting mechanism, a walnut kernel transmission and recovery mechanism, a first walnut kernel recovery mechanism (30), and a host computer, wherein the visual inspection mechanism comprises a visual inspection transmission component (12) and a visual inspection component; The discharge port of the feeding conveying mechanism is arranged in communication with the feeding port of the visual detection transmission component (12), and is used to convey the walnut kernels to the visual detection transmission component (12); the visual detection component and the walnut kernel sorting mechanism are installed on the side of the visual detection transmission component (12) away from the feeding conveying mechanism; The visual detection transmission component (12) is used to drive the walnut kernel to move along a first preset direction, the visual detection component is used to take a picture of the walnut kernel moving in the first preset direction, and the host computer calls the walnut kernel quality detection model to detect the picture taken by the visual detection component to determine whether the state of the walnut kernel is the first state or the second state; When the walnut kernel is in the first state, the visual detection transmission component (12) is used to guide the walnut kernel in the first state to the first state walnut kernel recovery mechanism (30) through the movement of the visual detection transmission component (12) along a first preset direction; When the walnut kernels are in the second state, the feeding conveying mechanism and the visual detection transmission component (12) stop conveying, and the walnut kernel sorting mechanism moves along the first and second preset directions to sort the walnut kernels in the second state to the bad material transmission and recovery mechanism; the walnut kernel transmission and recovery mechanism is used to move the walnut kernels in the second state along the second preset direction to guide them to the second state walnut kernel recovery mechanism (29).

4. The walnut kernel sorting automatic conveying production line based on visual detection according to claim 3 is characterized in that: The loading and conveying mechanism comprises a trapezoidal lower hopper (6) connected to the top of a primary feeding support frame (1), the outlet of the trapezoidal lower hopper (6) is communicated with the inlet of a primary receiving tray (3), a primary vibrating feeder (4) is installed above a primary vibrating feeding bottom plate (32) below the primary receiving tray (3), the outlet of the primary receiving tray (3) is communicated with the inlet of a primary feeding slide (5) installed on a primary slide support frame (31), the outlet of the primary feeding slide (5) is communicated with the inlet of a secondary receiving tray (9), a secondary vibrating feeder (8) is installed above a secondary vibrating feeding bottom plate (33) below the secondary receiving tray (9), The outlet of the secondary material receiving tray (9) is communicated with the inlet of the secondary feeding slide (46) installed on the secondary slide support frame (34), and the outlet of the secondary feeding slide (46) is communicated with the inlet of the visual detection transmission component (12); wherein, the working surfaces of the primary material receiving tray (3) and the secondary material receiving tray (9) are parallel to the conveying direction of the visual detection transmission component (12), and the working surface of the primary material receiving tray (3), the working surface of the secondary material receiving tray (9), and the working plane of the visual detection transmission component (12) are arranged in sequence up and down, and the primary feeding slide (5) and the secondary feeding slide (46) are arranged obliquely downward from the inlet to the outlet.

5. The walnut kernel sorting automatic conveying production line based on visual detection according to claim 3 is characterized in that: The visual inspection component comprises a visual inspection conveyor belt support frame (11), a visual inspection light source mounting plate (56), a camera connecting frame (57), and a visual inspection camera (21); the visual inspection light source mounting plate (56) and the camera connecting frame (57) are connected above the visual inspection platform support frame (11); a visual inspection light source (55) is mounted on the visual inspection light source mounting plate (56); a visual inspection camera (21) is mounted on the camera connecting frame (57); and a camera head of the visual inspection camera (21) extends from the center position of the visual inspection light source (55) to take pictures of walnut kernels on a first conveyor belt in a detection area in the visual inspection transmission component (12).

6. The walnut kernel sorting automatic conveying production line based on visual detection according to claim 3 is characterized in that: The walnut kernel sorting mechanism comprises two sorting mechanism mounting base plates (22), a first preset direction stepping motor (25), a second preset direction synchronous belt module (26), a second preset direction servo motor (27), a first preset direction synchronous belt module (41), a double-axis cylinder (2), and a flexible clamp (51); the two sorting mechanism mounting base plates (22) are mounted at intervals on a side of a conveyor belt frame of the visual detection transmission component (12) away from a feeding conveying mechanism; the second preset direction synchronous belt modules (26) are respectively mounted above the two sorting mechanism mounting base plates (22), and power is transmitted to the second preset direction synchronous belt modules (26) through the second preset direction servo motor (27); the two second preset direction synchronous belt modules (26) drive the first preset direction synchronous belt module (41) to move along the second preset direction; power is transmitted to the first preset direction synchronous belt module (41) through the first preset direction stepping motor (25); A direction synchronous belt module (41) is provided to drive a Z-axis connecting plate (35) to move along a first preset direction; the first preset direction synchronous belt module (41) is equipped with a first preset direction upper limit photoelectric switch (15), a first preset direction origin photoelectric switch (16) and a first preset direction lower limit photoelectric switch (17); the second preset direction synchronous belt module (26) is equipped with a second preset direction upper limit photoelectric switch (18), a second preset direction origin photoelectric switch (19) and a second preset direction lower limit photoelectric switch (20); the Z-axis connecting plate (35) is located between the first preset direction upper limit photoelectric switch (15) and the first preset direction lower limit photoelectric switch (17); a double-axis cylinder (2) is installed on the Z-axis connecting plate (35); an upper limit magnetic reed sensor (23) and a lower limit magnetic reed sensor (24) are installed on the double-axis cylinder (2); and a flexible clamp (51) is connected to the end of the double-axis cylinder (2).

7. The walnut kernel sorting automatic conveying production line based on visual detection according to claim 3 is characterized in that: The walnut kernel transmission and recovery mechanism comprises a second-state walnut kernel recovery mechanism (29) and a bad material transmission component (43), wherein the bad material transmission component (43) is used to drive the second-state walnut kernels sorted by the walnut kernel sorting mechanism to move along a second preset direction, and the walnut kernels in the second state are guided to the second-state walnut kernel recovery mechanism (29) by the movement of the bad material transmission component (43) along the second preset direction.

Images