A durian peel automatic peeling and sampling device based on vision and deep learning

Abstract

The present application relates to a kind of based on vision and automatic dicing sampling device of durian peel of deep learning. Including: clamping rotating mechanism, for the clamping and driving horizontal rotation of durian;CCD industrial camera, for the image acquisition of durian in sampling box;Cutting knife execution mechanism, for the peel of durian when walking in transverse;Including: transverse walking module and longitudinal feed module;The driving end of transverse walking module is equipped with longitudinal feed module, the driving end of longitudinal feed module is equipped with main shaft motor, and the output end of main shaft motor is equipped with peel cutter;Master unit, for receiving the image acquisition data of CCD industrial camera, and execute image processing, three-dimensional reconstruction, coordinate conversion and peel path control function.The present application can solve the technical problems of low efficiency, poor safety, uncontrollable depth, irregular shape and large area constant depth dicing in prior art.

Description

Technical Field

[0001] This invention belongs to the field of intelligent processing and testing sampling technology of agricultural products, and specifically relates to an automatic peeling and sampling device for durian peel based on vision and deep learning. Background Technology

[0002] Durian, a high-value tropical fruit, has a peel consisting of a hard, cone-shaped outer layer of spines and a dense, hard outer layer about 1.0–2.5 cm thick, with the flesh encased within. In durian variety selection, quality grading, processing adaptability evaluation, and customs import / export inspection and quarantine, a certain amount of peel samples is often required for subsequent analysis. For example, pesticide residue detection, peel fiber composition determination, heavy metal content, and microbial contamination testing all require collecting peel samples containing the intact spine structure and hard outer layer, with the sampling area typically covering 70%–80% of the entire fruit surface area, corresponding to a peel weight of approximately 300–500 g (depending on durian size). Durian fruit sizes vary greatly: large fruits are 20–35 cm in diameter and 20–50 cm in length, while small fruits are 8–20 cm in diameter and 10–20 cm in length, with some fruits exhibiting irregular oval or pear shapes.

[0003] Currently, durian peel sampling is mainly done manually, requiring operators to use specialized tools to manually hold the durian and peel it. This method has the following prominent problems: First, manual operation is inefficient and cannot meet the needs of large-scale, continuous sampling; second, the sharp points on the durian surface can easily injure operators, posing a safety hazard; third, the force, depth, and path of manual peeling are difficult to control precisely, and the consistency and repeatability of sampling are poor.

[0004] In the field of automated processing, existing fruit peeling or cutting equipment, such as automated processing devices designed for fruits with relatively regular shapes and smooth surfaces like apples and oranges, typically employs mechanical contouring or simple visual positioning methods. These technologies rely on fruits having relatively uniform geometric shapes and smooth surfaces, making them difficult to directly apply to fruits like durians, which have extremely irregular shapes and significant three-dimensional protrusions (spiky structures) on their surfaces. Specifically, conventional two-dimensional visual positioning methods cannot accurately obtain the three-dimensional contour information of the durian surface, resulting in a lack of basis for cutting path planning; while traditional mechanical contouring or fixed trajectory cutting methods cannot adapt to the shape differences between individual durians, easily causing cuts that are too deep and damage the flesh or too shallow and fail to peel effectively.

[0005] Furthermore, while some existing automated cutting systems employ vision-guided technology, they are primarily designed for industrial parts made of metal or wood, whose surface characteristics differ significantly from the natural biological structure of durian. Industrial parts typically possess well-defined geometric parameters and reflective properties, whereas the durian surface is not only complex in shape but also exhibits strong shadows and localized occlusion due to its spikes, placing higher demands on the robustness of image segmentation and 3D reconstruction algorithms. Simultaneously, dynamically coordinating the durian's rotational motion with the tool's lateral and longitudinal feeds during the peeling process to ensure the tool remains in contact with the surface and cuts to a constant depth is a technical challenge that has yet to be effectively resolved.

[0006] In summary, the present invention urgently needs to propose a device and method for automatically peeling and sampling durian peels that can adapt to the diversity of durian size and shape and achieve large surface area and constant depth, so as to replace manual operation and improve efficiency, safety and sampling consistency. Summary of the Invention

[0007] The purpose of this invention is to provide an automatic durian peeling and sampling device and method based on vision and deep learning. This invention can solve the technical problems of low efficiency, poor safety, uncontrollable depth, inability to adapt to irregular shapes, and inability to achieve large-area constant depth peeling in the prior art.

[0008] To address the aforementioned technical problems, this invention provides an automatic durian peeling and sampling device based on vision and deep learning, comprising: A clamping and rotating mechanism is used to clamp the durian and drive it to rotate horizontally; A CCD industrial camera is used to acquire images of durians inside a sampling box; A cutting actuator is used to peel the peel of a durian while it is moving laterally; it includes: a lateral movement module and a longitudinal feeding module; the longitudinal feeding module is installed at the drive end of the lateral movement module, a spindle motor is installed at the drive end of the longitudinal feeding module, and a peeling blade is installed at the output end of the spindle motor; The main control unit is used to receive image acquisition data from CCD industrial cameras and perform image processing, 3D reconstruction, coordinate transformation and stripping path control functions.

[0009] Preferably, the clamping rotation mechanism includes: The clamping module includes: a first drive assembly, a positive and negative threaded screw, a clamping arm, and a first guide rail; the first drive assembly is mounted on a frame and drives the positive and negative threaded screw to rotate. The positive and negative threaded screw is rotatably mounted on the frame via bearings. The clamping arm is threadedly sleeved on both sides of the positive and negative threaded screw. The bottom of the clamping arm is slidably connected to the first guide rail, which is mounted on the frame. The rotating module includes: a second drive assembly, a rotating shaft, a guide sleeve, and a three-jaw fixing pin; the rotating shaft is rotatably mounted on the top of the clamping arms arranged on both sides via bearings, and one side of the rotating shaft is driven to rotate by the second drive assembly, which is mounted on the clamping arm. The end of the rotating shaft not driven by the second drive assembly is rotatably mounted with the three-jaw fixing pin via a bearing, and the end of the rotating shaft driven by the second drive assembly is also mounted with the three-jaw fixing pin. The guide sleeve is symmetrically inserted into the left and right side walls of the sampling box, and the rotating shafts on both sides are slidably inserted into the guide sleeve.

[0010] Preferably, both the lateral travel module and the longitudinal feed module are motor-driven lead screw linear modules.

[0011] Preferably, the cutting actuator further includes a receiving box, which is located below the peeling blade and is mounted on the main spindle motor, moving synchronously with the main spindle motor; and a replaceable sampling collection bag is fitted onto the receiving box.

[0012] Preferably, it also includes a camera transfer mechanism, which adopts a motor-driven lead screw linear module, and the industrial camera is installed at the drive end of the camera transfer mechanism. The outer side of the CCD industrial camera is provided with an annular aperture, which is fixed to a shield plate. The shield plate is fixed to the drive end of the camera transfer mechanism. By controlling the movement of the shield plate, the light source hole opened on the top of the sampling box is switched between being shielded and exposed.

[0013] Preferably, it also includes a reflective transplanting mechanism and a reflective background plate; the reflective transplanting mechanism adopts a motor-driven lead screw linear module; the reflective transplanting mechanism is symmetrically arranged on both sides of the frame, and the reflective background plate is installed at the drive end of the reflective transplanting mechanism. The other end of the reflective background plate passes through the strip grooves opened on the left and right side walls of the sampling box. When the reflective background plates on both sides are controlled to approach and abut against each other, a background area with consistent brightness can be provided for the shooting field of the CCD industrial camera directly above; when the reflective background plates on both sides are controlled to move away from each other and exit the sampling box, space can be provided for peeling and cutting durian peel.

[0014] Preferred options also include: The water tank is equipped with a heating rod, a temperature sensor and a level gauge, which are used to heat, control the temperature and monitor the water level in the tank, respectively. A high-pressure water pump and water pipe are used to pump water from the water tank to clean the peeling blade. A wastewater collector, equipped with a filtration function, is connected to the bottom of the sampling box via a collection pipe for collecting and treating wastewater after cleaning.

[0015] This invention also provides an automatic durian peel cutting and sampling method based on vision and deep learning, employing an automatic durian peel cutting and sampling device based on vision and deep learning as described above, comprising: Step 1: Configuration of optical imaging system and image acquisition; A CCD industrial camera is placed above a durian, with a ring aperture outside the lens of the CCD industrial camera and a highly reflective diffuse reflective background plate placed below the durian. By controlling the durian to rotate at a constant speed around a rotation axis, the CCD industrial camera is triggered to acquire multiple images containing the shape of the durian and the reflective background plate at fixed angular rotation intervals. The acquired images are then labeled to generate a labeled dataset. Step 2: Durian instance segmentation and contour extraction based on deep learning; A durian segmentation model is obtained by training an instance segmentation neural network using a labeled dataset. During the actual peeling process, the image captured in real time by a CCD industrial camera is input into the durian segmentation model to output a binary mask of the durian region. By extracting the boundary contour of the binary mask, the sub-pixel level edge coordinates of the durian in the image plane at the rotation angle are obtained. Step 3: 3D contour reconstruction based on monocular vision and rotational kinematics; The intrinsic parameter matrix and distortion coefficient parameters of the CCD industrial camera, as well as the position of the durian rotation center axis in the world coordinate system, are pre-calibrated using the Zhang Zhengyou calibration method. The sub-pixel edges at each rotation angle are inversely projected to obtain spatial rays. The geometric constraints and contour continuity between the rays and the surface of the rotating body are used to calculate the radius values ​​at different axial positions. Finally, all angle data are fused to generate a three-dimensional surface model. Step 4: Coordinate system calibration and cutting point coordinate transformation; The hand-eye transformation matrix between the CCD industrial camera and the cutting actuator is pre-calibrated using the Zhang Zhengyou calibration method. Based on the three-dimensional surface model obtained in step three, the three-dimensional coordinates of the desired cutting point in the world coordinate system are determined. The hand-eye transformation matrix is ​​then used to transform the coordinates of the cutting point to the base coordinate system of the cutting actuator to generate the peeling motion trajectory. Step 5: Dynamic control of peeling while rotating; Based on the peeling motion trajectory obtained in step four, the main control unit synchronously controls the durian rotation and the action of the cutting blade actuator. During the uniform rotation of the durian, the peeling blade adjusts its position relative to the durian surface in real time according to the preset cutting depth offset, so that the peeling blade always fits the durian surface and maintains a constant penetration depth. At the same time, the peeling blade moves horizontally continuously or stepwise to achieve peeling sampling. During the movement, the deviation between the actual position of the peeling blade and the commanded position is compared in real time. When the deviation exceeds the threshold, the action stops and an alarm is triggered. After peeling is completed, the required durian peel sample is obtained.

[0016] Preferably, the instance segmentation neural network is Mask R-CNN or YOLACT, and data augmentation techniques are used during the training phase, including random rotation, brightness adjustment, and adding simulated spiky shadows, to improve the segmentation robustness to the irregularities on the durian surface. Between steps three and four, the method further includes: using the Iterative Closest Point (ICP) algorithm to register the reconstructed local point clouds at different rotation angles to the same world coordinate system, in order to correct the cumulative error caused by the eccentricity of the durian clamping or the runout of the rotation axis.

[0017] Preferably, in step three, calculating the radius value R(θ, z) includes: for each contour edge point at a rotation angle θ, calculating the spatial ray direction corresponding to the contour edge point based on the camera projection model; by utilizing the geometric characteristics of the durian rotating around the rotation axis, assuming the durian surface is a body of revolution, intersecting the ray with a set of cylindrical or conical surfaces with the rotation axis as the axis, and through iterative optimization to make the contour self-consistent at different angles, thereby simultaneously estimating the radius value R(θ, z) and the axial position z.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention achieves stable clamping and uniform rotation of the durian by setting up a clamping and rotating mechanism. Combined with an optical imaging system consisting of a CCD industrial camera, a ring aperture, and a movable reflective background plate, it can acquire clear, low-interference images of the durian's shape under uniform lighting conditions, avoiding the influence of sharp shadows and ambient stray light on contour recognition, and providing a reliable data foundation for subsequent high-precision recognition.

[0019] 2. This invention utilizes a deep learning instance segmentation network to accurately extract the contours of durians, and combines monocular vision and rotational kinematics to reconstruct the three-dimensional surface of the durian. It can adapt to the irregular shape, individual size differences, and surface spikes of the durian, solving the problem that traditional fixed trajectories cannot adapt to complex fruit shapes. Through hand-eye calibration, a unified conversion between image coordinates and the coordinates of the cutting mechanism is achieved, generating an adaptive peeling path that conforms to the durian surface, ensuring a constant peeling depth and uniform sampling layer thickness, avoiding over-cutting that damages the flesh or under-cutting that results in insufficient sampling.

[0020] 3. The cutting mechanism of this invention employs a linkage between lateral travel and longitudinal feed, working in conjunction with a spindle motor to drive a carbide end mill or a disc-shaped toothed cutter for peeling, enabling efficient cutting of hard fruit peels. A simultaneously mounted follow-up collection box and sampling bag ensure immediate collection of debris, guaranteeing sample cleanliness and integrity. Furthermore, the cutting depth is dynamically adjusted via load current feedback, and ICP point cloud registration corrects errors caused by clamping eccentricity and rotation axis runout, further improving peeling accuracy and system stability. In addition, a heated water tank, high-pressure water pump, and wastewater collection mechanism allow for hot water cleaning of residual adhesive residue on the cutter, three-jaw fixing pin, and other components during peeling, and wastewater collection, ensuring reliable continuous operation and preventing contamination of subsequent sampling. Attached Figure Description

[0021] Figure 1 This is a front view of the structure of an automatic durian peeling and sampling device based on vision and deep learning provided by the present invention.

[0022] Figure 2 This is a structural diagram of the camera transfer mechanism provided by the present invention.

[0023] Figure 3 This is a rear view of the structure of an automatic durian peeling and sampling device based on vision and deep learning provided by the present invention.

[0024] Figure 4 This is a front view of the clamping and rotating mechanism provided by the present invention.

[0025] Figure 5 This is a front view of the cutting actuator provided by the present invention.

[0026] Figure 6 This is a magnified view of a portion of the structure of the three-jaw fixing pin and the peeling blade provided by the present invention.

[0027] Figure 7 This is an overall structural diagram of the automatic durian peeling and sampling device including a main control unit provided by the present invention.

[0028] Figure 8 This is a flowchart of an automatic durian peeling and sampling method based on vision and deep learning provided by the present invention.

[0029] In the diagram: 1-Frame, 2-Clamping and rotating mechanism, 21-Clamping module, 211-First drive assembly, 212-Positive and negative threaded screws, 213-Clamping arm, 214-First guide rail, 22-Rotating module, 221-Second drive assembly, 222-Rotating shaft, 223-Guide sleeve, 224-Three-jaw fixing pin, 23-Collection box, 3-CCD industrial camera, 31-Camera transfer mechanism, 32-Annular aperture, 33-Shielding plate, 4-Sampling box, 41-Light source hole, 5-Cutter actuator, 51-Horizontal travel module, 52-Vertical feed module, 53-Spindle motor, 54-Peeling blade, 6-Reflective transfer mechanism, 7-Reflective background plate, 8-Strip groove, 9-Water tank, 10-Sewage collector, 11-Main control unit. Detailed Implementation

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0031] like Figures 1 to 7 As shown, this embodiment of the invention specifically provides an automatic durian peeling and sampling device based on vision and deep learning, comprising: The clamping and rotating mechanism 2 is used to clamp the durian and drive it to rotate horizontally; The CCD industrial camera 3 is used to acquire images of durians inside the sampling box 4. The sampling box 4 is a closed box with a cabinet door with a viewing window on the front and an open structure on the rear. The open structure is equipped with a retractable accordion seal cover, which is fixed to the housing of the spindle motor 53 and moves with the spindle motor 53 to achieve dynamic sealing of the open structure at the rear of the sampling box 4 without affecting the movement of the cutting actuator 5. The cutting mechanism 5 is used to peel the durian peel while it moves laterally; it includes a lateral movement module 51 and a longitudinal feed module 52; the longitudinal feed module 52 is installed at the drive end of the lateral movement module 51, and a spindle motor 53 is installed at the drive end of the longitudinal feed module 52, with a peeling blade 54 installed at the output end of the spindle motor 53; the peeling blade 54 is a carbide spiral end mill or a disc toothed cutter; by controlling the movement of the lateral movement module 51, the durian is moved axially (i.e., along its length) while being held and fixed; by controlling the movement of the longitudinal feed module 52, the peeling depth between the peeling blade 54 and the durian peel can be adjusted in real time; by controlling the movement of the spindle motor 53, the peeling blade 54 can be driven to rotate at high speed to peel the durian peel; The main control unit 11 is an embedded industrial computer with a built-in NVIDIA GPU for deep learning inference. It is used to receive image acquisition data from the CCD industrial camera 3 and perform image processing, 3D reconstruction, coordinate transformation and stripping path control functions.

[0032] As one optimized embodiment of the present invention, the clamping rotation mechanism 2 includes: The clamping module 21 includes: a first drive assembly 211, a positive and negative threaded screw 212, clamping arms 213, and a first guide rail 214. The first drive assembly 211 is mounted on the frame 1 and drives the positive and negative threaded screw 212 to rotate. The positive and negative threaded screw 212 is rotatably mounted on the frame 1 through bearings. Clamping arms 213 are threadedly fitted on the threads on both sides of the positive and negative threaded screw 212. The bottom of the clamping arms 213 is slidably connected to the first guide rail 214, which is mounted on the frame 1. When the first drive assembly 211 drives the positive and negative threaded screw 212 to rotate, the two clamping arms 213 move synchronously towards or away from each other to achieve centering clamping or loosening of the durian. The first drive assembly 211 is driven by a servo motor with a driving pulley, a driven pulley, and a synchronous belt transmission mechanism. Furthermore, by monitoring changes in the motor load current, the actual clamping force on the durian can be deduced using the motor current-torque characteristics, achieving constant force clamping without sensors; at the same time, an upper limit on the current is set to prevent overpressure damage to the durian.

[0033] The rotating module 22 includes: a second drive assembly 221, a rotating shaft 222, a guide sleeve 223, and a three-jaw fixing pin 224. The rotating shaft 222 is rotatably mounted on the top of the clamping arms 213 arranged on both sides via bearings. One side of the rotating shaft 222 is driven to rotate by the second drive assembly 221, which is mounted on the clamping arm 213. The end of the rotating shaft 222 not driven by the second drive assembly 221 is rotatably mounted with a three-jaw fixing pin 224 via a bearing. The end of the rotating shaft 222 driven by the second drive assembly 221 is also mounted with a three-jaw fixing pin 224. The tip of the three-jaw fixing pin 224 is conical, which facilitates reliable puncture and fixation. The guide sleeve 223 is symmetrically inserted into the left and right side walls of the sampling box 4. The rotating shafts 222 on both sides are slidably inserted into the guide sleeve 223. The guide sleeve 223 adopts a linear bearing structure. By controlling the rotation module 22, i.e., controlling the second drive assembly 221, the rotating shaft 222 and its three-jaw fixing pins 224 are used to drive the durian, which is being held in place, to rotate. The aforementioned second drive assembly 221 is driven by a servo motor and a gearbox.

[0034] As one of the optimized embodiments of the present invention, both the lateral walking module 51 and the longitudinal feeding module 52 adopt motor-driven lead screw linear modules. Specifically, the lead screw linear modules are driven by a servo motor with a driving pulley, a driven pulley, and a synchronous belt transmission mechanism.

[0035] As one of the optimized embodiments of the present invention, the cutting actuator 5 also includes a receiving box 23, which is located below the peeling and cutting blade 54 and is mounted on the main shaft motor 53 and moves synchronously with the main shaft motor 53; and a replaceable sampling collection bag is fitted on the receiving box 23, and the fruit peels peeled off by the peeling and cutting blade 54 automatically fall into the sampling collection bag.

[0036] As one optimized embodiment of the present invention, it also includes a camera transfer mechanism 31. The camera transfer mechanism 31 adopts a motor-driven lead screw linear module, and an industrial camera is installed at the drive end of the camera transfer mechanism 31. An annular aperture 32 is arranged on the outer side of the CCD industrial camera 3 to provide uniform shadowless illumination. The annular aperture 32 is fixed on a baffle plate 33, which is fixed to the drive end of the camera transfer mechanism 31. By controlling the movement of the baffle plate 33, the light source hole 41 opened on the top of the sampling box 4 can be switched between being blocked and exposed. When image acquisition and shooting are performed, the camera transfer mechanism 31 can be controlled to move the CCD industrial camera 3 to the axis of the light source hole 41 directly above the sampling box 4 for subsequent image acquisition. When subsequent peeling and cutting are performed, the camera transfer mechanism 31 can be controlled to move the CCD industrial camera 3 to one side, and the baffle plate 33 can be moved to align and block the light source hole 41, so that the fruit peel debris generated during peeling and cutting will not cause contamination of the lens of the CCD industrial camera 3.

[0037] As one optimized embodiment of the present invention, it also includes a reflective transplanting mechanism 6 and a reflective background plate 7; the reflective transplanting mechanism 6 adopts a motor-driven lead screw linear module; the reflective transplanting mechanism 6 is symmetrically arranged on both sides of the frame 1, and the reflective background plate 7 is installed on the driving end of the reflective transplanting mechanism 6. The other end of the reflective background plate 7 passes through the strip grooves 8 opened on the left and right side walls of the sampling box 4. By controlling the operation of the reflective transplanting mechanism 6, when the reflective background plates 7 on both sides are brought close to each other, a background area with consistent brightness and no reflection interference can be provided for the shooting field of the CCD industrial camera 3 located directly above; when peeling is required, the reflective background plates 7 on both sides are moved away from each other and exit the sampling box 4, which can provide clearance space for peeling durian peel.

[0038] As one optimized embodiment of the present invention, it further includes: Water tank 9 is equipped with heating rods, temperature sensors and level gauges, which are used to heat, control the temperature and monitor the water level in water tank 9, respectively. High-pressure water pump and water pipe are used to pump water from water tank 9 to clean the peeling blade 54. Wastewater collector 10 is equipped with a filtration function (with a removable and replaceable filter screen inside) and is connected to the bottom of sampling box 4 via a collection pipe for collecting and treating wastewater after cleaning.

[0039] like Figure 8 As shown, the present invention also provides an automatic durian peeling and sampling method based on vision and deep learning, which specifically includes the following steps: Step 1: Configuration of optical imaging system and image acquisition; A CCD industrial camera 3 is positioned above a durian, with a ring aperture 32 positioned outside the lens of the CCD industrial camera 3, and a high-reflectivity diffuse reflective background plate 7 positioned below the durian. By controlling the durian to rotate uniformly around the rotation axis 222 for one revolution, the CCD industrial camera 3 is triggered to acquire multiple images containing the shape of the durian and the reflective background plate 7 at fixed angle rotation intervals. The acquired images are then labeled to generate a labeled dataset. Step 2: Durian instance segmentation and contour extraction based on deep learning; A durian segmentation model is obtained by training an instance segmentation neural network using a labeled dataset. During the actual peeling process, the image acquired in real time by the CCD industrial camera 3 is input into the durian segmentation model to output a binary mask of the durian region. By extracting the boundary contour of the binary mask, the sub-pixel level edge coordinates of the durian in the image plane at the rotation angle are obtained. Step 3: 3D contour reconstruction based on monocular vision and rotational kinematics; The intrinsic parameter matrix and distortion coefficient parameters of the CCD industrial camera 3, as well as the position of the durian rotation center axis in the world coordinate system, are pre-calibrated using the Zhang Zhengyou calibration method. The sub-pixel edges at each rotation angle are inversely projected to obtain spatial rays. The radius values ​​at different axial positions are calculated by utilizing the geometric constraints and contour continuity between the rays and the surface of the rotating body. Finally, all angle data are fused to generate a three-dimensional surface model. Step 4: Coordinate system calibration and cutting point coordinate transformation; The hand-eye transformation matrix between the CCD industrial camera 3 and the cutting actuator 5 is pre-calibrated using the Zhang Zhengyou calibration method. Based on the three-dimensional surface model obtained in step three, the three-dimensional coordinates of the desired cutting point in the world coordinate system are determined. The hand-eye transformation matrix is ​​then used to transform the coordinates of the cutting point to the base coordinate system of the cutting actuator 5 to generate the peeling motion trajectory. Step 5: Dynamic control of peeling while rotating; Based on the peeling motion trajectory obtained in step four, the main control unit synchronously controls the durian rotation and the action of the cutting actuator 5. During the uniform rotation of the durian, the peeling blade 54 adjusts its position relative to the durian surface in real time according to the preset cutting depth offset, so that the peeling blade 54 always fits the durian surface and maintains a constant penetration depth. At the same time, the peeling blade 54 moves horizontally continuously or stepwise to achieve peeling and sampling. During the movement, the deviation between the actual position and the commanded position of the peeling blade 54 is compared in real time. When the deviation exceeds the threshold, the action stops and an alarm is triggered. After peeling is completed, the required durian peel sample is obtained.

[0040] As one of the optimized embodiments of the present invention, the instance segmentation neural network is Mask R-CNN or YOLACT, and data augmentation techniques are used during the training phase, including random rotation, brightness adjustment and adding simulated spiky shadows, to improve the segmentation robustness of the irregularities on the surface of the durian. Between steps three and four, the following is also included: using the Iterative Closest Point (ICP) algorithm to register the reconstructed local point clouds at different rotation angles to the same world coordinate system in order to correct the cumulative error caused by the eccentricity of the durian clamping or the runout of the rotation axis 222.

[0041] As one of the optimized embodiments of the present invention, in step three, the calculation of the radius value R(θ, z) includes: for each contour edge point under the rotation angle θ, the spatial ray direction corresponding to the contour edge point is calculated according to the camera projection model; by utilizing the geometric characteristics of the durian rotating around the rotation axis 222, assuming that the durian surface is a rotating body, the ray is intersected with a set of cylindrical or conical surfaces with the rotation axis 222 as the axis, and the contour is made self-consistent under different angles through iterative optimization, thereby simultaneously estimating the radius value R(θ, z) and the axial position z.

[0042] In one optimized embodiment of the present invention, the cutting depth offset is dynamically adjusted during the peeling process based on the load current feedback of the peeling blade 54: the cutting depth offset decreases when the load current increases, and increases when the load current decreases. Before the actual peeling, an idle scanning rotation is performed first, the CCD industrial camera 3 acquires images and reconstructs the three-dimensional model, and the generated trajectory is compared with the theoretical model. If there are local protrusions or depressions exceeding 2mm, the trajectory is automatically corrected locally.

[0043] The present invention also includes the following workflow: (1) Clamping and Imaging Preparation: The durian is manually placed into the sampling box 4. The clamping and rotating mechanism 2 drives the clamping arm 213 to center and clamp the durian through the positive and negative threaded screws 212. The three-jaw fixing pin 224 is inserted into the stem and tip of the durian to ensure reliable rotation without slippage. The main control unit controls the operation of the reflective transplanting mechanism 6. The reflective background plates 7 on both sides move towards each other to the middle to form a continuous and uniform bright background. The ring aperture 32 is opened, and the camera transplanting mechanism 31 drives the CCD industrial camera 3 to move to the shooting position.

[0044] (2) Image acquisition and deep learning segmentation: The second driving component 221 drives the durian to rotate uniformly around the rotation axis 222 for one revolution, and the camera acquires multiple frames of images at fixed angle intervals (such as 10° to 15°). The images are input into the trained instance segmentation neural network (Mask R-CNN or YOLACT), and the model outputs a binary mask of the durian region. Subpixel-level edge coordinates are obtained through contour extraction to eliminate spike shadows and background interference, thereby achieving accurate contour recognition.

[0045] (3) 3D Reconstruction and Trajectory Generation: Using pre-calibrated camera intrinsic parameters, distortion coefficients, and the position of rotation axis 222, spatial rays are obtained by inverse projection of edge points at various angles. Combined with the geometric constraints of the durian rotating body, the surface radius R(θ,z) at different angles and axial positions is calculated to complete the 3D surface reconstruction. The multi-view point cloud is registered using the ICP iterative nearest point algorithm to eliminate errors caused by clamping eccentricity and rotation axis 222 runout. Then, using the hand-eye calibration matrix, the peeling target point in the world coordinate system is converted into the coordinates of the cutting actuator 5 to generate a continuous peeling path that fits the fruit surface.

[0046] (4) Adaptive peeling while rotating; by controlling the reflective background plate 7 to retract and avoid, the cutting actuator 5 enters the working position. The durian continues to rotate, and the lateral walking module 51 drives the peeling blade 54 to move slowly along the axial direction (i.e., the length direction of the fixed points at both ends of the durian). The longitudinal feeding module 52 adjusts the depth of the peeling blade 54 in real time according to the three-dimensional model, so that the peeling blade 54 cuts into the peel at a constant penetration depth to avoid damaging the flesh. The main control unit reads the load current of the spindle motor 53 in real time. When the current increases, it indicates that the cutting resistance is too large (encountering thick shells or hard thorns), so the cutting depth is automatically reduced; when the current decreases, the cutting depth is appropriately increased to achieve constant force and constant depth adaptive control. The peel fragments cut off fall directly into the sampling bag on the follow-up collection box 23 to achieve complete sample collection, which can be easily removed and tested manually after the end.

[0047] (5) Cleaning and Resetting: After peeling, the peeling blade 54 retracts to the safe position, the clamping module 21 is released, and the durian and sampling sample can be removed. Then, the high-pressure water pump is started, and a person holds a water hose to spray hot water from the water tank 9 onto the peeling blade 54, the three-claw fixing pin 224, and other parts to rinse away any remaining durian glue residue. The wastewater flows into the bottom wastewater collector 10 for filtration. All mechanisms are reset, and the device enters standby mode, waiting for the next sampling operation.

[0048] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. An automatic durian peel peeling and sampling device based on vision and deep learning, characterized in that, include: A clamping and rotating mechanism is used to clamp the durian and drive it to rotate horizontally; A CCD industrial camera is used to acquire images of durians inside a sampling box; A cutting actuator is used to peel the peel of a durian while it is moving laterally; it includes: a lateral movement module and a longitudinal feeding module; the longitudinal feeding module is installed at the drive end of the lateral movement module, a spindle motor is installed at the drive end of the longitudinal feeding module, and a peeling blade is installed at the output end of the spindle motor; The main control unit is used to receive image acquisition data from CCD industrial cameras and perform image processing, 3D reconstruction, coordinate transformation and stripping path control functions.

2. The automatic durian peeling and sampling device based on vision and deep learning as described in claim 1, characterized in that, The clamping and rotating mechanism includes: The clamping module includes: a first drive assembly, a positive and negative threaded screw, a clamping arm, and a first guide rail; the first drive assembly is mounted on a frame and drives the positive and negative threaded screw to rotate. The positive and negative threaded screw is rotatably mounted on the frame via bearings. The clamping arm is threadedly sleeved on both sides of the positive and negative threaded screw. The bottom of the clamping arm is slidably connected to the first guide rail, which is mounted on the frame. The rotating module includes: a second drive assembly, a rotating shaft, a guide sleeve, and a three-jaw fixing pin; the rotating shaft is rotatably mounted on the top of the clamping arms arranged on both sides via bearings, and one side of the rotating shaft is driven to rotate by the second drive assembly, which is mounted on the clamping arm. The end of the rotating shaft not driven by the second drive assembly is rotatably mounted with the three-jaw fixing pin via a bearing, and the end of the rotating shaft driven by the second drive assembly is also mounted with the three-jaw fixing pin. The guide sleeve is symmetrically inserted into the left and right side walls of the sampling box, and the rotating shafts on both sides are slidably inserted into the guide sleeve.

3. The automatic durian peeling and sampling device based on vision and deep learning as described in claim 1, characterized in that, Both the lateral travel module and the longitudinal feed module are motor-driven lead screw linear modules.

4. The automatic durian peel peeling and sampling device based on vision and deep learning as described in claim 1, characterized in that, The cutting actuator also includes a receiving box, which is located below the peeling blade and is mounted on the main spindle motor, moving synchronously with the main spindle motor; and a replaceable sampling collection bag is fitted on the receiving box.

5. The automatic durian peeling and sampling device based on vision and deep learning as described in claim 1, characterized in that, It also includes a camera transfer mechanism, which adopts a motor-driven lead screw linear module, and the industrial camera is installed at the drive end of the camera transfer mechanism. The outer side of the CCD industrial camera is provided with an annular aperture, which is fixed to a shield plate. The shield plate is fixed to the drive end of the camera transfer mechanism. By controlling the movement of the shield plate, the light source hole opened on the top of the sampling box is switched between being shielded and exposed.

6. The automatic durian peel peeling and sampling device based on vision and deep learning as described in claim 1, characterized in that, It also includes a reflective transplanting mechanism and a reflective background plate; the reflective transplanting mechanism adopts a motor-driven lead screw linear module; the reflective transplanting mechanism is symmetrically arranged on both sides of the frame, and the reflective background plate is installed at the drive end of the reflective transplanting mechanism. The other end of the reflective background plate passes through the strip grooves opened on the left and right side walls of the sampling box. When the reflective background plates on both sides are controlled to approach and abut against each other, they can provide a background area with consistent brightness for the shooting field of the CCD industrial camera directly above; when the reflective background plates on both sides are controlled to move away from each other and exit the sampling box, they can provide clearance for peeling durian peel.

7. The automatic durian peeling and sampling device based on vision and deep learning as described in claim 1, characterized in that, Also includes: The water tank is equipped with a heating rod, a temperature sensor and a level gauge, which are used to heat, control the temperature and monitor the water level in the tank, respectively. A high-pressure water pump and water pipe are used to pump water from the water tank to clean the peeling blade. A wastewater collector, equipped with a filtration function, is connected to the bottom of the sampling box via a collection pipe for collecting and treating wastewater after cleaning.

8. A method for automatically peeling and sampling durian peel based on vision and deep learning, employing the automatic durian peel peeling and sampling device based on vision and deep learning as described in any one of claims 1 to 7, characterized in that, include: Step 1: Configuration of optical imaging system and image acquisition; A CCD industrial camera is placed above a durian, with a ring aperture outside the lens of the CCD industrial camera and a highly reflective diffuse reflective background plate placed below the durian. By controlling the durian to rotate at a constant speed around a rotation axis, the CCD industrial camera is triggered to acquire multiple images containing the shape of the durian and the reflective background plate at fixed angular rotation intervals. The acquired images are then labeled to generate a labeled dataset. Step 2: Durian instance segmentation and contour extraction based on deep learning; A durian segmentation model is obtained by training an instance segmentation neural network using a labeled dataset. During the actual peeling process, the image captured in real time by a CCD industrial camera is input into the durian segmentation model to output a binary mask of the durian region. By extracting the boundary contour of the binary mask, the sub-pixel level edge coordinates of the durian in the image plane at the rotation angle are obtained. Step 3: 3D contour reconstruction based on monocular vision and rotational kinematics; The intrinsic parameter matrix and distortion coefficient parameters of the CCD industrial camera, as well as the position of the durian rotation center axis in the world coordinate system, are pre-calibrated using the Zhang Zhengyou calibration method. The sub-pixel edges at each rotation angle are inversely projected to obtain spatial rays. The geometric constraints and contour continuity between the rays and the surface of the rotating body are used to calculate the radius values ​​at different axial positions. Finally, all angle data are fused to generate a three-dimensional surface model. Step 4: Coordinate system calibration and cutting point coordinate transformation; The hand-eye transformation matrix between the CCD industrial camera and the cutting actuator is pre-calibrated using the Zhang Zhengyou calibration method. Based on the three-dimensional surface model obtained in step three, the three-dimensional coordinates of the desired cutting point in the world coordinate system are determined. The hand-eye transformation matrix is ​​then used to transform the coordinates of the cutting point to the base coordinate system of the cutting actuator to generate the peeling motion trajectory. Step 5: Dynamic control of peeling while rotating; Based on the peeling motion trajectory obtained in step four, the main control unit synchronously controls the durian rotation and the action of the cutting blade actuator. During the uniform rotation of the durian, the peeling blade adjusts its position relative to the durian surface in real time according to the preset cutting depth offset, so that the peeling blade always fits the durian surface and maintains a constant penetration depth. At the same time, the peeling blade moves horizontally continuously or stepwise to achieve peeling sampling. During the movement, the deviation between the actual position of the peeling blade and the commanded position is compared in real time. When the deviation exceeds the threshold, the action stops and an alarm is triggered. After peeling is completed, the required durian peel sample is obtained.

9. The automatic durian peel cutting and sampling method based on vision and deep learning as described in claim 8, characterized in that, The instance segmentation neural network is Mask R-CNN or YOLACT. During the training phase, data augmentation techniques are used, including random rotation, brightness adjustment, and the addition of simulated spiky shadows, to improve the segmentation robustness to the irregularities on the durian surface. Between steps three and four, the method further includes: using the Iterative Closest Point (ICP) algorithm to register the reconstructed local point clouds at different rotation angles to the same world coordinate system, in order to correct the cumulative error caused by the eccentricity of the durian clamping or the runout of the rotation axis.

10. The automatic peeling and sampling method for durian peel based on vision and deep learning as described in claim 8, characterized in that, In step three, calculating the radius value R(θ, z) includes: for each contour edge point at a rotation angle θ, calculating the spatial ray direction corresponding to the contour edge point based on the camera projection model; by utilizing the geometric properties of the durian rotating around the rotation axis, assuming the durian surface is a body of revolution, intersecting the ray with a set of cylindrical or conical surfaces with the rotation axis as the axis, and through iterative optimization to make the contour self-consistent at different angles, thereby simultaneously estimating the radius value R(θ, z) and the axial position z.

Images