Method and system for intelligent detection and sorting of fruit defects based on machine vision

Abstract

The application discloses a fruit defect intelligent detection and sorting method and system based on machine vision, and relates to the technical field of industrial visual intelligent detection. The method projects a multi-frequency phase shift stripe pattern on fruits and extracts surface point cloud coordinates, and peels off surface high-frequency reflection components and subcutaneous low-frequency diffuse components. A three-dimensional reference is constructed based on the surface point cloud coordinates, the light intensity attenuation distribution characteristics of the diffuse components are extracted along the normal direction, a perspective scattering gradient map is generated through surface fitting, and the mutation pixel contour is extracted according to the gradient map to demarcate the tissue deterioration boundary. The three-dimensional mechanical yield parameters are calculated by combining the local curvature change rate to construct the mechanical weakening feature vector. The coordinates are mapped to the operation space to construct a grasping repulsion area, the grasping anchor points meeting the geometric gravity center are distributed in the safety contour, the end contact negative pressure is calibrated by combining the yield parameters, and the sorting instruction is generated and sent to the execution end. The system is used for implementing the above method.

Description

Technical Field

[0001] This invention relates to the field of industrial vision-based intelligent inspection technology, specifically to a method and system for intelligent detection and sorting of fruit defects based on machine vision. Background Technology

[0002] Fruit quality directly impacts its final commercial value and shelf life in modern agricultural processing and distribution. During harvesting, transportation, and sorting, fruit inevitably suffers surface bruising, localized browning, or subcutaneous damage. Faced with the demands of large-scale, high-throughput fruit processing, replacing traditional manual labor with automated equipment for fruit quality inspection and grading has become an inevitable trend in the industry. Machine vision, due to its non-contact nature and high inspection efficiency, is widely used in automated sorting lines, using image information collected from the fruit surface for quality assessment and subsequent physical distribution.

[0003] Existing fruit sorting methods have significant limitations in practical operation. Conventional visual inspection mainly relies on the extraction of apparent pixel features from two-dimensional images, which is easily affected by the reflection of natural wax on the peel or surface dirt, causing apparent blemishes to be confused with actual subcutaneous tissue damage. During the sorting execution stage, existing mechanical gripping devices typically use fixed spatial trajectories and uniform contact torques, failing to adapt mechanically to the local structural fragility of the fruit. When the end effector grips fruit with existing subcutaneous lesions or tissue deterioration, the uniform pressure can easily exceed the structural pressure limit of the affected area, directly causing the fruit to crack and leak juice, leading to secondary damage and cross-contamination of subsequent fruit and equipment on the entire sorting line. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a machine vision-based intelligent detection and sorting method and system for fruit defects to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a machine vision-based intelligent detection and sorting method for fruit defects, comprising the following steps: S1, projecting a multi-frequency phase-shift stripe pattern onto the fruit in the detection space and acquiring a phase-shift image sequence; performing phase unpacking and modulation depth analysis on the phase-shift image sequence to extract surface point cloud coordinates, and separating the surface high-frequency reflection component carrying contour information and the subcutaneous low-frequency diffuse component carrying internal tissue features; S2, constructing a three-dimensional reference based on the surface point cloud coordinates, and extracting the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component along the normal of the surface point cloud coordinates; performing surface fitting operation on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients, and combining the internal photon scattering coefficients to generate a perspective scattering gradient map; S 3. Extract the outer topological contour of the abrupt pixel set in the perspective scattering gradient map to delineate the tissue degradation boundary; combine the local curvature change rate of the surface point cloud coordinates within the tissue degradation boundary to calculate the three-dimensional mechanical yield parameter of the corresponding region of the degradation boundary, and combine the tissue degradation boundary coordinates and the three-dimensional mechanical yield parameter to construct a mechanical weakening feature vector; S4. Map the surface point cloud coordinates to the workspace, construct the gripping repulsion zone along the outer normal of the tissue degradation boundary, and optimally allocate gripping anchor points that satisfy the geometric centroid within the contour after removing the gripping repulsion zone from the surface point cloud coordinates; extract the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrate the end contact negative pressure matching the gripping anchor points, and combine the gripping anchor points and the end contact negative pressure to generate a sorting instruction and send it to the mechanical execution end.

[0006] In a preferred embodiment, the specific process of projecting a multi-frequency phase-shift stripe pattern onto the fruit in the detection space and acquiring a phase-shift image sequence is as follows: setting the basic spatial frequency and phase shift step, generating a multi-frequency phase-shift stripe pattern containing high-frequency, mid-frequency, and low-frequency components; driving the structured light projection module to sequentially project the multi-frequency phase-shift stripe pattern onto the surface of the fruit in the detection space through a hardware synchronous trigger controller; synchronously driving a multi-band image sensor to capture image frames whose corresponding surface curvature undergoes phase modulation within a single phase shift period of the multi-frequency phase-shift stripe pattern, and combining image frames of various frequency bands and different phase shift phases to construct a phase-shift image sequence.

[0007] In a preferred embodiment, the specific process of performing phase unpacking and modulation depth analysis on the phase-shifted image sequence, extracting surface point cloud coordinates, and separating the surface high-frequency reflection component carrying contour information and the subcutaneous low-frequency diffuse component carrying internal tissue features is as follows: extract the absolute folded phase distribution of the phase-shifted image sequence in the spatial domain; apply a multi-frequency heterodyne unpacking algorithm to unfold the absolute folded phase distribution, and perform inverse perspective mapping in combination with a preset stereo calibration matrix to calculate the three-dimensional spatial geometric coordinates to form the surface point cloud coordinates; analyze the modulation depth of the phase-shifted image sequence, and extract the AC amplitude distribution and DC bias distribution of the phase-shifted image sequence in the time domain; extract the surface high-frequency reflection component by extracting the AC amplitude distribution, and extract the subcutaneous low-frequency diffuse component by removing the ambient backlight offset in combination with the DC bias distribution.

[0008] In a preferred embodiment, the specific process of constructing a three-dimensional reference using surface point cloud coordinates and extracting the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component along the normal of the surface point cloud coordinates is as follows: a spatial three-dimensional reference is constructed using surface point cloud coordinates, and a local tangent plane matrix matching the surface point cloud coordinates is constructed. The normal vector set associated with the local tangent plane matrix is ​​extracted through cross product operation. The surface point cloud coordinates are specified as the spatial search origin, and the penetration path defined by the normal vector set is mapped to the pixel array of the subcutaneous low-frequency diffuse component. The brightness response amplitude of discrete sampling points on the penetration path is extracted to construct the light intensity attenuation distribution characteristics characterizing the photon's decreasing response along the propagation depth direction.

[0009] In a preferred embodiment, the specific process of performing surface fitting operation on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients and combining the internal photon scattering coefficients to generate a perspective scattering gradient map is as follows: Substitute the light intensity attenuation distribution characteristics into the semi-infinite medium diffuse analytical equation, perform a nonlinear least squares convergent fitting operation, and solve for the optimal approximation parameters of the semi-infinite medium diffuse analytical equation; extract the internal photon scattering coefficients that characterize the photon attenuation intensity from the optimal approximation parameters; bind the surface point cloud coordinates with the corresponding internal photon scattering coefficients, reconstruct the pixel grid in the two-dimensional unfolded projection domain, and generate a perspective scattering gradient map that encompasses the spatial continuous mapping of the internal photon scattering coefficients.

[0010] In a preferred embodiment, the specific process of extracting the outer topological contour of the abrupt pixel set in the perspective scattering gradient map and delineating the tissue degradation boundary is as follows: traverse the pixel matrix of the perspective scattering gradient map and extract the abrupt pixel set where the second derivative of the internal photon scattering coefficients crosses zero; apply a morphological dilation kernel to perform a closing operation on the abrupt pixel set to close the connected domain; trace the edge pixel coordinate sequence of the connected domain and connect the edge pixel coordinate sequences to generate the outer topological contour of the closed polygon; project and map the outer topological contour of the closed polygon back into the three-dimensional reference, and extract the corresponding spatial surface segment to delineate the tissue degradation boundary in three-dimensional space.

[0011] In a preferred embodiment, the specific process of constructing a mechanical weakening feature vector by combining the local curvature change rate of surface point cloud coordinates within the tissue degradation boundary, calculating the three-dimensional mechanical yield parameter of the region corresponding to the degradation boundary, and combining the tissue degradation boundary coordinates and the three-dimensional mechanical yield parameter is as follows: Extract the surface point cloud coordinates enclosed within the tissue degradation boundary, solve for the principal curvature and Gaussian curvature of the surface point cloud coordinates in the tangent space, and extract the spatial gradient of the principal curvature and Gaussian curvature to form the local curvature change rate; retrieve the preset scattering-elasticity numerical transformation matrix, extract the tissue apparent elastic modulus corresponding to the internal photon scattering coefficient in the perspective scattering gradient map, determine the geometric stress concentration coefficient of the corresponding tissue node based on the local curvature change rate, couple the tissue apparent elastic modulus and the geometric stress concentration coefficient, and analyze the critical stress value for plastic deformation of the corresponding tissue node; extract the spatial minimum value of the critical stress value as the three-dimensional mechanical yield parameter; splice the coordinates of the tissue degradation boundary and the three-dimensional mechanical yield parameter, and encapsulate to generate a mechanical weakening feature vector pointing to the extreme point of structural fragility.

[0012] In a preferred embodiment, the process of mapping surface point cloud coordinates to the workspace and constructing a gripping rejection zone along the outer normal of the tissue deterioration boundary, and then optimally allocating gripping anchor points that satisfy the geometric centroid within the contour of the surface point cloud coordinates after removing the gripping rejection zone, is as follows: Extract the base coordinate system of the mechanical actuator and use a hand-eye calibration extrinsic matrix to transform the surface point cloud coordinates into the workspace contained within the base coordinate system; extend a preset safety distance envelope outward along the outer normal of the tissue deterioration boundary and generate the gripping rejection zone through a spatial Boolean union operation; extract the remaining surface point cloud coordinates after removing the gripping rejection zone and perform discrete voxelization integration on the remaining surface point cloud coordinates to solve for the geometric centroid; in the remaining surface point cloud coordinate cluster with the shortest distance to the geometric centroid surface, optimally allocate gripping anchor points that are symmetrically distributed in a circular pattern based on the spatial degrees of freedom of the actuator's grippers.

[0013] In a preferred embodiment, the specific process of extracting the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrating the end contact negative pressure of the gripping anchor point, and combining the gripping anchor point and the end contact negative pressure to generate a sorting command and send it to the mechanical execution end is as follows: Extracting the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, and combining it with the local geometric contact area at the gripping anchor point, analyzing the ultimate vacuum adsorption pressure under the condition that the fruit as a whole does not break; under the upper limit constraint of the ultimate vacuum adsorption pressure, matching the end contact negative pressure to meet the requirement of offsetting the fruit's own gravity lifting; encapsulating the spatial pose matrix of the gripping anchor point and the timing control signal of the end contact negative pressure into the communication protocol message, and combining the gripping anchor point and the end contact negative pressure to generate a sorting command and send it to the mechanical execution end.

[0014] The machine vision-based intelligent fruit defect detection and sorting system, used to execute the aforementioned machine vision-based intelligent fruit defect detection and sorting method, includes: a visual analysis module, used to project multi-frequency phase-shift stripe patterns onto the fruit in the detection space and acquire phase-shift image sequences; perform phase unpacking and modulation depth analysis on the phase-shift image sequences to extract surface point cloud coordinates and separate the surface high-frequency reflection components carrying contour information and the subcutaneous low-frequency diffuse components carrying internal tissue features; a scattering analysis module, used to construct a three-dimensional reference based on the surface point cloud coordinates, extract the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse components along the normal of the surface point cloud coordinates; perform surface fitting operations on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients, and combine the internal photon scattering coefficients to generate a perspective scattering gradient map. The mechanical assessment module extracts the outer topological contour of the abrupt pixel set in the perspective scattering gradient map and delineates the tissue degradation boundary. It combines the local curvature change rate of the surface point cloud coordinates within the tissue degradation boundary to calculate the three-dimensional mechanical yield parameter of the corresponding region. The module then combines the tissue degradation boundary coordinates with the three-dimensional mechanical yield parameter to construct a mechanical weakening feature vector. The sorting decision module maps the surface point cloud coordinates to the workspace, constructs a gripping repulsion zone along the outer normal of the tissue degradation boundary, and optimally allocates gripping anchor points that satisfy the geometric centroid within the contour after removing the gripping repulsion zone from the surface point cloud coordinates. It extracts the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrates the end contact negative pressure matching the gripping anchor points, and combines the gripping anchor points and end contact negative pressure to generate a sorting command sent to the mechanical execution end.

[0015] The technical effects and advantages of this invention are as follows: (1) A machine vision-based intelligent detection and sorting method for fruit defects projects a multi-frequency phase-shift stripe pattern onto the fruit in the detection space during the detection stage. The surface point cloud coordinates are extracted through phase unpacking and modulation depth analysis. The high-frequency reflection component on the surface and the low-frequency diffuse component under the skin are separated. A three-dimensional reference is constructed using the surface point cloud coordinates. The light intensity attenuation distribution characteristics of the low-frequency diffuse component under the skin are extracted along the normal direction. The internal photon scattering coefficient is obtained through surface fitting calculation to generate a perspective scattering gradient map. Thus, the reflection interference signal on the fruit peel surface can be physically isolated from the real diffuse characteristics of the subcutaneous tissue. This overcomes the defect of existing visual detection methods that easily misjudge surface stains or natural reflections as internal tissue damage, and realizes accurate quantitative characterization of the real subcutaneous damage and its spatial attenuation trend.

[0016] (2) A machine vision-based intelligent detection and sorting method for fruit defects extracts abrupt pixel sets from the perspective scattering gradient map during the sorting stage to delineate the boundaries of tissue deterioration. It then calculates the three-dimensional mechanical yield parameter based on the local curvature change rate to construct a mechanically weakening feature vector. Simultaneously, it constructs a gripping rejection zone along the outer normal of the tissue deterioration boundary in the work space. Within the safe contour after removing this gripping rejection zone, it optimally allocates gripping anchor points that satisfy the geometric centroid. Finally, it generates sorting instructions based on the safe end-contact negative pressure matched with the three-dimensional mechanical yield parameter. This establishes a dynamic constraint response between the degree of internal tissue deterioration and the external mechanical force limit, overcoming the problem of existing mechanical devices using a fixed gripping mode to forcibly apply force, leading to the cracking and juice leakage of defective fruit. This achieves flexible sorting based on defect topology perception to prevent secondary damage.

[0017] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0018] Figure 1 This is a flowchart of the intelligent detection and sorting method for fruit defects based on machine vision according to the present invention. Figure 2 This is a schematic diagram of the attenuation distribution characteristics of the subcutaneous low-frequency diffuse component light intensity in an embodiment of the present invention; Figure 3 This is a schematic diagram of the surface point cloud mechanical yield characteristics and the topological distribution space of the gripping anchor points in an embodiment of the present invention; Figure 4 This is a flowchart of the intelligent fruit defect detection and sorting system based on machine vision according to the present invention. Detailed Implementation

[0019] This application's embodiments solve the problems of existing technologies being unable to accurately distinguish between apparent disturbances and actual subcutaneous damage, and the fixed grasping mode easily causing secondary damage to fragile fruits, through a machine vision-based intelligent detection and sorting method and system for fruit defects.

[0020] Example 1; please refer to Figure 1This invention provides a technical solution: a machine vision-based intelligent detection and sorting method for fruit defects, comprising the following steps: S1, projecting a multi-frequency phase-shift stripe pattern onto the fruit in the detection space and acquiring a phase-shift image sequence; performing phase unpacking and modulation depth analysis on the phase-shift image sequence to extract surface point cloud coordinates, and separating the surface high-frequency reflection component carrying contour information and the subcutaneous low-frequency diffuse component carrying internal tissue features; S2, constructing a three-dimensional reference based on the surface point cloud coordinates, and extracting the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component along the normal of the surface point cloud coordinates; performing surface fitting operation on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients, and combining the internal photon scattering coefficients to generate a perspective scattering gradient map; S3, ... Extract the outer topological contour of the abrupt pixel set in the perspective scattering gradient map to delineate the tissue degradation boundary; combine the local curvature change rate of the surface point cloud coordinates within the tissue degradation boundary to calculate the three-dimensional mechanical yield parameter of the corresponding region of the degradation boundary, and construct a mechanical weakening feature vector by combining the tissue degradation boundary coordinates and the three-dimensional mechanical yield parameter; S4, map the surface point cloud coordinates to the workspace, construct a gripping repulsion zone along the outer normal of the tissue degradation boundary, and optimally allocate gripping anchor points that satisfy the geometric centroid within the contour after removing the gripping repulsion zone from the surface point cloud coordinates; extract the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrate the end contact negative pressure matching the gripping anchor points, and combine the gripping anchor points and the end contact negative pressure to generate a sorting instruction and send it to the mechanical execution end.

[0021] In this implementation scheme, step S1 is used to acquire the external spatial geometry and internal tissue optical signals of the fruit to be inspected, separate interfering light rays, and analyze the underlying data. The multi-frequency phase-shifted stripe pattern refers to a grating projection image containing sinusoidal wave characteristics of different spatial frequencies. After being projected onto the fruit, it is affected by surface geometric undulations, resulting in stripe distortion and forming a phase-shifted image sequence. The phase unpacking process restores the truncated folded phase signals in the phase-shifted image sequence to spatially continuous absolute phases, thereby calculating the physical three-dimensional geometric coordinates to form surface point cloud coordinates. Modulation depth analysis quantifies the amplitude fluctuations of the image sequence in the time domain. The specular reflection caused by the outer wax layer of the peel manifests as a surface high-frequency reflection component with a large AC amplitude. The light rays transmitted into the pulp tissue and depolarized after multiple scatterings in the intercellular spaces manifest as a subcutaneous low-frequency diffuse component with DC bias characteristics. This step extracts the modulation depth difference to physically separate these two optical components.

[0022] Step S2 is used to quantify the spatial distribution of optical properties within the fruit pulp and establish a three-dimensional mapping relationship with the surface morphology. The three-dimensional reference is the spatial geometric skeleton of the fruit constructed from surface point cloud coordinates. The light intensity attenuation distribution characteristics characterize the decreasing law of photon energy with increasing penetration depth when the low-frequency diffuse component under the peel penetrates into the pulp along the normal direction of the peel surface. The surface fitting operation substitutes the extracted discrete light intensity attenuation data into the media diffuse analytical equation for nonlinear solution. The derived internal photon scattering coefficient is a physical parameter reflecting the density of the pulp cell structure and the state of free water. This coefficient becomes abnormal when the pulp suffers dark damage, water seepage, or browning. The perspective scattering gradient map is a continuous parameter mapping grid reconstructed in the projection domain after binding all the solved internal photon scattering coefficients with their corresponding surface point cloud coordinates, presenting the spatial gradient trend of the internal scattering characteristics of the fruit.

[0023] Step S3 is used to pinpoint the specific spatial boundaries of hidden deterioration within the pulp and calculate the mechanical compressive strength of the damaged area. The abrupt pixel set in the perspective scattering gradient map corresponds to the region where the internal photon scattering coefficient undergoes an abnormal step change. The outer topological contour is the outermost continuous closed pixel line surrounding this abnormal region. Based on this, the corresponding spatial surface segment can be extracted to delineate the actual physical extent of the damaged pulp, i.e., the tissue deterioration boundary. The local curvature change rate is a parameter reflecting the abruptness of the geometric unevenness of the peel surface within the tissue deterioration boundary, determining the stress concentration coefficient generated during external mechanical contact. The three-dimensional mechanical yield parameter refers to the critical stress extreme value at which the corresponding tissue node undergoes irreversible plastic deformation or rupture and juicing when subjected to external mechanical compression. The mechanical weakening feature vector is a data package encapsulated by splicing the three-dimensional coordinate matrix of the tissue deterioration boundary with the corresponding three-dimensional mechanical yield parameter, indicating the specific coordinates and compressive strength threshold of the physically vulnerable parts.

[0024] Step S4 is used to plan the spatial obstacle avoidance posture of the mechanical actuator and dynamically configure the matching safe gripping force. The working space refers to the three-dimensional coordinate reference system that the mechanical grippers or suction cups on the sorting production line can cover and physically interact with. The gripping repulsion zone is a virtual safe isolation volume space extended outwards from the tissue deterioration boundary; the system strictly prohibits physical contact between the mechanical end and the fruit within this volume space. The gripping anchor points are contact force coordinates calculated and allocated within the undamaged healthy skin area of ​​the fruit; their distribution must satisfy the geometric centroid formed by the center of the resultant force and the center of gravity of the fruit. The end-contact negative pressure is the ultimate vacuum adsorption pressure parameter calculated based on the mechanical weakening eigenvector. The sorting instruction encapsulates and combines the spatial posture matrix of the gripping anchor points with the electrical control signal of the end-contact negative pressure, driving the mechanical actuator to complete the physical diversion and guidance with an attitude that avoids physical vulnerabilities and a force within the safe pressure range.

[0025] Specifically, the process of projecting multi-frequency phase-shift stripe patterns onto fruits in the detection space and acquiring phase-shift image sequences is as follows: A base spatial frequency and phase-shift step are set to generate a multi-frequency phase-shift stripe pattern containing high-frequency, mid-frequency, and low-frequency components; a hardware synchronous trigger controller drives the structured light projection module to sequentially project the multi-frequency phase-shift stripe pattern onto the surface of the fruits in the detection space; a multi-band image sensor is synchronously driven to capture image frames whose surface curvature undergoes phase modulation within a single phase-shift period of the multi-frequency phase-shift stripe pattern; and image frames from various frequency bands and different phase-shift phases are combined to construct a phase-shift image sequence.

[0026] In this implementation scheme, the controller generates a discretized spatial light intensity matrix based on a preset digital micromirror deflection matrix. The distribution of the projected light rays emitted by the projection module in the spatial domain satisfies a specific physical equation, and the following multi-frequency phase-shifted light intensity distribution model is established: In this calculation formula, E represents the projection light intensity distribution matrix on the projection physical plane; p and q represent the horizontal and vertical pixel coordinates of the projection plane, respectively; A represents the ambient offset illumination reference quantity of the projection light source; and B represents the optical modulation amplitude of the projection system. This represents the spatial frequency of the k-th level, where k represents the frequency discrete index corresponding to the high-frequency band, mid-frequency band, and low-frequency band. Indicates the first Each phase shift step angle, and ; This indicates the total number of phase shift steps contained within a single phase shift cycle; This indicates the current phase shift step sequence number. The structured light projection module sequentially projects the aforementioned stripe pattern containing periodic spatial undulations into the detection space. When the grating stripes illuminate the fruit's outer skin, they undergo physical deformation due to the modulation of the surface's three-dimensional geometry. The multi-band image sensor synchronously exposes the fruit surface within the same phase shift period, capturing the reflected radiation flux modulated by both the fruit's geometric curvature and its tissue optical properties. The phase-shifted image sequence formed on the sensor target surface is recorded as follows: In this response equation, This represents the grayscale values ​​of an image sequence captured by the target surface of a multi-band image sensor; and These represent the horizontal and vertical pixel coordinates of the image sensor's imaging plane, respectively. The matrix representing the absolute optical reflectance of the outer surface of the fruit; This represents the background ambient light intensity response component reflected from the surface of the fruit; The fringe contrast amplitude response component represents the reflection from the surface of the fruit. This represents the folded phase matrix that undergoes geometric distortion after being modulated by the three-dimensional curvature of the fruit surface.

[0027] Specifically, the process of performing phase unpacking and modulation depth analysis on the phase-shifted image sequence to extract surface point cloud coordinates and separate the surface high-frequency reflection component carrying contour information and the subcutaneous low-frequency diffuse component carrying internal tissue features is as follows: Extract the absolute folded phase distribution of the phase-shifted image sequence in the spatial domain; apply a multi-frequency heterodyne unpacking algorithm to unfold the absolute folded phase distribution, and perform inverse perspective mapping in combination with a preset stereo calibration matrix to calculate the three-dimensional spatial geometric coordinates to form the surface point cloud coordinates; analyze the modulation depth of the phase-shifted image sequence to extract the AC amplitude distribution and DC bias distribution of the phase-shifted image sequence in the time domain; extract the surface high-frequency reflection component by extracting the AC amplitude distribution, and extract the subcutaneous low-frequency diffuse component by removing the ambient backlight offset in combination with the DC bias distribution.

[0028] In this implementation scheme, the system performs multi-dimensional signal analysis and physical attribute stripping on the acquired phase-shifted image sequence in both the time and spatial domains. For geometric contour extraction in the spatial domain, the system utilizes the grayscale distribution on the orthogonal time series to solve for the absolute folded phase distribution at the current frequency level. In this phase analysis formula, This represents the absolute folded phase distribution matrix extracted at the k-th frequency level. Subsequently, the folded phases at different spatial frequencies are cascaded and frequency-reduced using the multi-frequency heterodyne principle to obtain continuous absolute unfolded phases. Finally, based on the spatial triangulation principle, inverse perspective mapping is implemented to establish the conversion relationship between pixels and physical space. In the above spatial transformation formula, , and These represent the physical geometric coordinates mapped to the three-dimensional spatial coordinate system, which together constitute the basic nodes of the surface point cloud coordinates; This represents the camera intrinsic parameter matrix generated by the internal optical path calibration of the multi-band image sensor. The external calibration transformation matrix represents the relative pose relationship between the structured light projection module and the image sensor. After geometric extraction, the system performs optical component stripping on the phase-shifted image sequence in the time domain, and calculates the AC amplitude distribution and DC bias distribution in the time domain. ; In these two time-domain analytical formulas, U represents the AC amplitude distribution matrix in the time domain, corresponding to the surface high-frequency reflection component caused by the stratum corneum; D represents the DC bias distribution matrix in the time domain, corresponding to the basic diffuse signal that penetrates the stratum corneum and overflows after scattering within the tissue. To address the signal shift caused by ambient backlighting, the system extracts the subcutaneous low-frequency diffuse component through computation: In the above diffuse stripping formula, This represents the subcutaneous low-frequency diffuse component matrix after removing environmental interference; This represents the discrete frequency index corresponding to the low-frequency bandpass filter attribute; This represents the ambient backlight static response matrix pre-collected during the sensor offline calibration process; This represents the backlight suppression weighting coefficient. The method for determining the backlight suppression weighting coefficient is to establish a dynamic environmental compensation benchmark, and the specific calculation logic is as follows: ,and and These represent the maximum horizontal and vertical resolutions of the sensor target surface, respectively. This represents the unloaded DC bias distribution matrix extracted when the detection space is in an unloaded state.

[0029] Specifically, the process of constructing a three-dimensional reference using surface point cloud coordinates and extracting the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component along the normal of the surface point cloud coordinates is as follows: a spatial three-dimensional reference is constructed using surface point cloud coordinates, and a local tangent plane matrix matching the surface point cloud coordinates is constructed. The normal vector set associated with the local tangent plane matrix is ​​extracted through cross product operation. The surface point cloud coordinates are specified as the spatial search origin, and the transmission path defined by the normal vector set is mapped to the pixel array of the subcutaneous low-frequency diffuse component. The brightness response amplitude of discrete sampling points on the transmission path is extracted to construct the light intensity attenuation distribution characteristics characterizing the photon's decreasing response along the propagation depth direction.

[0030] In this implementation scheme, the system first extracts local geometric features based on surface point cloud coordinates to determine the physically orthogonal direction of light penetration through the fruit peel. For any point cloud target node in three-dimensional space, all spatial discrete points within its preset search radius are extracted to construct a local geometric covariance matrix. By performing singular value decomposition on the local geometric covariance matrix, the eigenvectors corresponding to the smallest singular values ​​are extracted as the initial normals of the target node. Combined with the fruit centroid coordinates, viewpoint consistency redirection is performed, and a set of normal vectors associated with the local tangent plane matrix is ​​calculated and generated. In this formula for solving the normal direction, Represents the topological index located in the horizontal and vertical directions of the surface point cloud coordinate matrix. The normal vector at that location; and These represent the horizontal and vertical topological indices on the two-dimensional unfolded plane, respectively. This represents the algebraic decomposition operator for extracting the eigenvectors corresponding to the minimum singular values ​​of the covariance matrix; This represents the total number of valid three-dimensional nearest neighbors contained within the local spherical neighborhood of the target node. Indicates the traversal number of the nearest neighbor sequence; This represents a column vector of spatial coordinates of a single three-dimensional nearest neighbor point within a local spherical search domain. This represents the column vector of three-dimensional spatial centroid coordinates of all nearest neighbor points within the local spherical search domain. After obtaining the normal vector set, using the surface point cloud coordinates as the spatial search origin, a series of equally spaced discrete depth penetration nodes are set along the corresponding normal vectors into the pulp tissue. These three-dimensional spatial nodes are projected onto the two-dimensional image plane of the subcutaneous low-frequency diffuse component using perspective transformation. The brightness response amplitude of the discrete sampling points is extracted to construct the light intensity attenuation distribution characteristics. In this formula, The characteristic matrix representing the one-dimensional light intensity attenuation distribution discretized along the photon propagation depth direction; This represents a discrete depth sampling level index that penetrates deep into the fruit pulp tissue; This indicates the physical penetration spacing step size between adjacent depth sampling levels; This represents a three-dimensional to two-dimensional bilinear interpolation brightness extraction function based on a spatial extrinsic matrix. The three-dimensional absolute physical coordinate matrix representing the target nodes on the fruit peel surface; This represents the subcutaneous low-frequency diffuse component matrix of the stripping output from the preceding process.

[0031] Specifically, the process of performing surface fitting operations on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients, and combining the internal photon scattering coefficients to generate a perspective scattering gradient map is as follows: Substitute the light intensity attenuation distribution characteristics into the diffuse analytical equation of a semi-infinite medium, perform a nonlinear least squares convergent fitting operation, and solve for the optimal approximation parameters of the diffuse analytical equation of the semi-infinite medium; extract the internal photon scattering coefficients that characterize the photon attenuation intensity from the optimal approximation parameters; bind the surface point cloud coordinates with the corresponding internal photon scattering coefficients, reconstruct the pixel grid in the two-dimensional unfolded projection domain, and generate a perspective scattering gradient map that encompasses the spatially continuous mapping of the internal photon scattering coefficients.

[0032] In this implementation scheme, the system inputs the acquired discrete light intensity attenuation distribution characteristics into an analytical physical equation based on radiative transfer theory to quantify the abrupt changes in microscopic optical transport caused by intercellular liquefaction or tissue browning in the fruit pulp. For each surface sampling node, a semi-infinite medium diffuse analytical equation is constructed to describe the energy dissipation law of photons in the fruit pulp medium: In this analytical equation, This represents the approximate predicted value of diffuse light intensity output by the analytical equation at the h-th depth sampling level; This represents the initial effective photon flux parameter incident on the fruit peel surface; Indicates the basic optical diffusion coefficient of fruit pulp tissue; The internal photon scattering coefficient to be solved represents the path attenuation rate of photons during random walks within the tissue. This represents a minimal constant to prevent division overflow caused by the denominator approaching zero. The system constructs a loss function based on the sum of squared residuals and uses the Gauss-Newton iterative method to perform nonlinear least squares convergent fitting operations. In this convergent fitting formula, This represents the internal photon scattering coefficient matrix in the optimal approximation parameters extracted after iterative calculation; This indicates the maximum number of sampling layers for the set penetration depth; This represents the signal-to-noise ratio (SNR) weighting coefficient that varies with penetration depth. The SNR weighting coefficient is determined by establishing a confidence penalty term based on the quantum efficiency decay law of the image sensor. The specific calculation logic is as follows: ,and This represents the dark current noise damping exponential constant of the hardware system obtained through offline calibration. The system extracts the values ​​from all nodes. The coordinates are then re-bound to the corresponding surface point cloud coordinates. A bicubic spline interpolation algorithm is used to perform pixel resampling operations on the unfolded two-dimensional parameter plane, generating a perspective scattering gradient map that encompasses the spatially continuous distribution of photon scattering coefficients within all regions. This allows the microscopic density changes in subcutaneous tissue caused by mechanical damage to be presented in the form of grayscale gradients within the two-dimensional mesh. Figure 2 As shown, the nonlinear least-squares convergence fitting results after substituting discrete sampling points into the analytical equation for diffuse diffusion in a semi-infinite medium are intuitively displayed. The horizontal axis represents the propagation depth, indicating the equidistant physical penetration distance of photons along the normal direction of the peel surface into the pulp; the vertical axis represents the light intensity response amplitude, used to characterize the relative brightness response intensity captured by the multi-band image sensor after removing environmental backlight interference. The legend includes two contrasting curves: the solid line marked with dots represents healthy pulp tissue, which, due to its intact and dense cell structure, stable basic optical diffusion coefficient, and long photon scattering path, exhibits a relatively gentle exponential decrease in light intensity response amplitude with increasing propagation depth; the dashed line marked with squares represents subcutaneous deteriorated tissue (such as areas with dark wounds and oozing water or internal browning), where abnormal local free water due to cell rupture causes a significant abrupt change in the internal photon scattering coefficient, resulting in a steeper light intensity attenuation gradient. Through the physical difference in the attenuation slope between the two curves, the system can accurately decouple and quantify the true subcutaneous damage without damaging the peel.

[0033] Specifically, the process of extracting the outer topological contour of the abrupt pixel set in the perspective scattering gradient map and delineating the tissue degradation boundary is as follows: traverse the pixel matrix of the perspective scattering gradient map and extract the abrupt pixel set where the second derivative of the internal photon scattering coefficients crosses zero; apply a morphological dilation kernel to perform a closing operation on the abrupt pixel set to close the connected domain; trace the edge pixel coordinate sequence of the connected domain and connect the edge pixel coordinate sequences to generate the outer topological contour of the closed polygon; project and map the outer topological contour of the closed polygon back into the three-dimensional reference, and extract the corresponding spatial surface fragment to delineate the tissue degradation boundary in three-dimensional space.

[0034] In this implementation scheme, the system applies the discrete Laplacian operator to calculate the spatial second-order partial derivative of the perspective scattering gradient map, locates the physical mapping region where the internal photon scattering coefficients exhibit abnormal step changes, and extracts the set of abrupt pixel changes through zero-crossing detection. In this feature extraction formula, This represents the set of aberration pixels that satisfy the zero-crossing condition of the second derivative; and These represent the lateral and longitudinal projection coordinates of the perspective scattering gradient map on the unfolded plane, respectively. Represents the perspective scattering gradient map matrix; This represents the discrete Laplace second-order difference operator; This represents the first-order spatial gradient operator; This represents the edge transition intensity threshold. The edge transition intensity threshold is determined by extracting the average first-order gradient magnitude of the undamaged healthy region of the entire image and multiplying it by a preset background variance fluctuation coefficient. After extracting the aberration pixel set, the system constructs morphological structural elements and performs a dilation-erosion closing operation on them to eliminate pixel discrete holes and bridge broken internal connected regions. The system uses an eight-neighbor boundary tracing algorithm to extract the outermost pixel sequence of this closed connected region, generating a closed polygonal outer topological contour. Subsequently, using an inverse perspective transformation model, this two-dimensional outer topological contour is mapped back to the previously established three-dimensional spatial reference. Spatial clipping operations are performed on the skin point cloud mesh along the viewpoint ray direction to extract the actual spatial surface fragments of the deteriorated area, thereby organizing the deteriorated boundary in three-dimensional physical space.

[0035] Specifically, the process of constructing a mechanical weakening feature vector by combining the local curvature change rate of surface point cloud coordinates within the tissue degradation boundary, solving the three-dimensional mechanical yield parameter of the corresponding region of the degradation boundary, and combining the tissue degradation boundary coordinates and the three-dimensional mechanical yield parameter is as follows: Extract the surface point cloud coordinates enclosed within the tissue degradation boundary, solve the principal curvature and Gaussian curvature of the surface point cloud coordinates in the tangent space, and extract the spatial gradient of the principal curvature and Gaussian curvature to form the local curvature change rate; retrieve the preset scattering-elasticity numerical transformation matrix, extract the tissue apparent elastic modulus corresponding to the internal photon scattering coefficient in the perspective scattering gradient map, determine the geometric stress concentration factor of the corresponding tissue node based on the local curvature change rate, couple the tissue apparent elastic modulus and the geometric stress concentration factor, and analyze the critical stress value of the corresponding tissue node undergoing plastic deformation; extract the spatial distribution minimum value of the critical stress value as the three-dimensional mechanical yield parameter; splice the coordinates of the tissue degradation boundary and the three-dimensional mechanical yield parameter, and encapsulate to generate a mechanical weakening feature vector pointing to the extreme point of structural fragility.

[0036] In this implementation scheme, the system traverses all physical nodes enclosed within the deterioration boundary of the tissue, constructs a local calculus surface model based on the coordinates of its adjacent spatial point clouds, solves for the maximum and minimum curvatures in the tangent plane direction, and extracts their spatial distribution characteristics to quantify the degree of surface geometric abrupt change: In this geometric parametric formula, This represents the rate of change of local curvature of the corresponding organizational node; This represents the traversal index of three-dimensional physical nodes within the deterioration boundary of the organization. and These represent the maximum and minimum principal curvatures of the corresponding node in the tangent space, respectively. The system synchronously retrieves the scattering and elasticity numerical transformation matrix stored at the bottom layer. This matrix is ​​generated by fitting the photon scattering attenuation response characteristics of non-destructive fruit samples collected synchronously offline with stress-strain feedback data from destructive mechanical probes. The system uses the internal photon scattering coefficients in the perspective scattering gradient map as the lookup index to extract the apparent elastic modulus of the tissue corresponding to the current node from the matrix. The system then determines the geometric stress concentration factor based on the rate of change of local curvature. in, This represents the empirical constant for curvature amplification based on the surface roughness of the fruit peel. Considering both the material's elastic modulus and geometric characteristics, the system analyzes the critical stress value for plastic fracture at this tissue node under external mechanical action using the continuum mechanics equations. In this mechanical analytical formula, This represents the critical stress value at which plastic deformation occurs at the corresponding tissue node; This represents the Poisson's ratio, an inherent physical constant of the pericarp tissue. The system performs a spatial minimum search operation on the critical stress values ​​of all nodes within the boundary; the extracted global minimum value is the three-dimensional mechanical yield parameter. This characterizes the ultimate physical pressure that the damaged area as a whole can withstand. The system will organize the three-dimensional coordinate matrix of the geometric centroid of the deteriorated boundary. With three-dimensional mechanical yield parameter Perform vector concatenation and encapsulate to generate mechanically weakened feature vectors. This feature vector directly indicates the location of the structurally vulnerable extreme point most prone to mechanical damage and the upper limit of the bearing capacity in the three-dimensional work space in the physical dimension.

[0037] Specifically, the process of mapping surface point cloud coordinates to the workspace and constructing a gripping rejection zone along the outer normal of the tissue deterioration boundary, and then optimally allocating gripping anchor points that satisfy the geometric centroid within the contour of the surface point cloud coordinates after removing the gripping rejection zone, is as follows: Extract the base coordinate system of the mechanical actuator and use the hand-eye calibration extrinsic parameter matrix to transform the surface point cloud coordinates into the workspace contained within the base coordinate system; extend a preset safety distance envelope outward along the outer normal of the tissue deterioration boundary and generate the gripping rejection zone through spatial Boolean union operation; extract the remaining surface point cloud coordinates after removing the gripping rejection zone and perform discrete voxelization integration on the remaining surface point cloud coordinates to solve for the geometric centroid; in the remaining surface point cloud coordinate cluster with the shortest distance to the geometric centroid surface, optimally allocate gripping anchor points that are symmetrically distributed in a circular pattern based on the spatial degrees of freedom of the actuator gripper.

[0038] In this implementation scheme, the system uses the hand-eye extrinsic parameter matrix calculated offline based on the calibration board to map the coordinate system of the vision sensor to the physical operating space of the robotic arm, and establishes the spatial transformation equation: In this spatial transformation equation, This represents the physical operation coordinate matrix mapped to the coordinate system of the mechanical actuator base; A topological index representing a discrete point cloud node; This represents the homogeneous matrix for the hand-eye calibration extrinsic transformation. This represents the surface point cloud coordinates obtained previously. Subsequently, using the physical coordinates corresponding to the tissue degradation boundary as a basis, the system expands along its three-dimensional extensional normal to generate a Boolean isolation volume, defining the grasping and repulsion zone formula as follows: In the formula for the exclusion region, This represents the three-dimensional envelope boundary matrix of the grasping rejection region; Represents the workspace coordinates within the boundary of organizational deterioration; Represents the unit normal vector of the workspace of the corresponding node; This indicates the preset safety extension distance. The method for determining this safety extension distance is to obtain the maximum motion envelope error of the mechanical actuator at its highest operating speed and add the maximum assembly tolerance of the end effector pneumatic gripper. The system removes points located in the full point cloud from the data. The internal coordinates are used to extract the remaining set of safe-to-access surface physical points, and discrete voxel calculus is used to solve for its equivalent geometric centroid. In this centroid equation, The geometric centroid space vector representing the remaining safe pulp; represents the equivalent closed integration domain enclosed by the remaining set of physical points on the surface; x', y', and z' represent three orthogonal coordinate system variables within the voxelized integration domain, respectively; This represents the homogeneous density constant function of the fruit. Finally, the system traverses the remaining set of surface physical points, and based on the geometric constraints of the suction cup hardware layout of the end gripper, searches for the set of surface nodes closest to the centroid projection normal to form the gripping anchor points: In this optimization formula, Indicates the first The three-dimensional coordinates of the gripping anchor point corresponding to each suction cup; Indicates the number of the individual suction cups contained in the end gripper; This represents the set of physical points on the remaining safe-to-contact surface after the rejection zone has been removed. This represents the circumferentially symmetrically distributed offset vector of each suction cup in space, based on the mechanical drawings of the end gripper. For example... Figure 3 As shown, this illustrates the flexible grasping planning process executed by the sorting decision module in a three-dimensional workspace to prevent secondary damage. The three Cartesian coordinate axes in the figure are X, Y, and Z, which together construct the three-dimensional physical workspace mapped to the mechanical actuator base using a hand-eye calibration extrinsic parameter matrix. The legend illustrates three types of key spatial topological data: First, the densely distributed, dark-colored grasping repulsion zone (deterioration boundary) is an absolutely isolated volume defined by abrupt pixel delineation based on the perspective scattering gradient map and extended outward along the outer normal of the tissue deterioration boundary. Within this region, the tissue's three-dimensional mechanical yield parameter is at a minimum, making it highly susceptible to plastic fracture and sap leakage under stress. Second, the light-colored, spherical envelope distribution of the safety surface point cloud represents the set of remaining safe contact points after removing the grasping repulsion zone from the full point cloud. Finally, the grasping anchor points, marked with diamond-shaped highlights, are three circumferentially symmetrically distributed end-force contact points optimally allocated by the system in the safety surface point cloud after solving for the fruit's geometric center of gravity through discrete voxelization integration, based on the spatial degrees of freedom of the gripper. This topological distribution ensures that the composite electrical sorting command satisfies both the torque balance required to counteract the fruit's weight increase and absolutely avoids structurally vulnerable extreme points in the physical interaction space.

[0039] Specifically, the process of extracting the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrating the end contact negative pressure of the gripping anchor point, and combining the gripping anchor point and the end contact negative pressure to generate a sorting command and send it to the mechanical execution end is as follows: Extract the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, combine it with the local geometric contact area at the gripping anchor point, and analyze the ultimate vacuum adsorption pressure under the condition that the fruit as a whole does not break; under the upper limit constraint of the ultimate vacuum adsorption pressure, match the end contact negative pressure that meets the requirement of offsetting the lifting of the fruit's own gravity; encapsulate the spatial pose matrix of the gripping anchor point and the timing control signal of the end contact negative pressure into the communication protocol message, and combine the gripping anchor point and the end contact negative pressure to generate a sorting command and send it to the mechanical execution end.

[0040] In this implementation scheme, to prevent the flexible gripping action from tearing the peel of the lesion area, the system uses a pre-generated mechanical weakening feature vector to apply an upper limit clamping limit to the pneumatic negative pressure. The system analyzes the ultimate vacuum adsorption pressure when irreversible microscopic tissue rupture occurs and establishes mechanical constraint equations: In this constraint equation, This represents the ultimate vacuum adsorption pressure threshold under conditions where the fruit peel remains intact. This represents the three-dimensional mechanical yield parameter extracted in the previous step; This indicates the effective physical contact area when a single flexible suction cup is fully compressed and adheres to the fruit peel. This represents the nonlinear shear stress concentration factor caused by the edge of the flexible suction cup to the fruit peel. This stress concentration factor is determined by extracting the absolute ratio of the peak value of the principal stress at the edge to the uniformly distributed stress at the center of the suction cup under rated compressive deformation through finite element simulation. Simultaneously, to ensure that the fruit can be stably picked up and removed from the production line without slipping, the system establishes a dynamic lifting torque balance equation to match the base negative pressure that counteracts its own weight. In this equilibrium equation, This indicates the negative pressure requirement for the end contact of the physical pickup action; The physical gravity scalar representing the force exerted on the fruit is obtained by multiplying the equivalent volume calculated from the previous voxel integral by the homogeneous density constant. The dynamic load amplification factor represents the dynamic load when the mechanical actuator is in the acceleration and lifting phase. Its value is determined by obtaining the peak composite acceleration vector of the mechanical actuator in the planned trajectory and calculating the magnitude of the vector sum of this vector and the gravitational acceleration. This indicates the total number of effective suction cups participating in synergistic adsorption; This represents the static coefficient of friction between the flexible suction cup material and the fruit peel surface at ambient temperature. The system executes hardware security verification logic to confirm... After its establishment, the target control negative pressure was locked at [value]. And calculate all the suction cup grab anchor points The six-degree-of-freedom pose parameters and the electrical duty cycle of the target control negative pressure are combined and converted into communication messages adapted to the industrial bottom layer, generating composite electrical sorting instructions which are sent to the programmable logic controller to drive the hardware execution end to complete the physical diversion task in a physical posture that avoids the lesion.

[0041] Example 2; please refer to Figure 4 A machine vision-based intelligent fruit defect detection and sorting system is used to execute the machine vision-based intelligent fruit defect detection and sorting method described in the embodiments. The system includes: a visual analysis module, used to project multi-frequency phase-shift stripe patterns onto the fruit in the detection space and acquire phase-shift image sequences; perform phase unpacking and modulation depth analysis on the phase-shift image sequences to extract surface point cloud coordinates and separate the surface high-frequency reflection components carrying contour information and the subcutaneous low-frequency diffuse components carrying internal tissue features; and a scattering analysis module, used to construct a three-dimensional reference based on the surface point cloud coordinates, extract the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse components along the normal of the surface point cloud coordinates; perform surface fitting operations on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients, and combine the internal photon scattering coefficients to generate a perspective scattering gradient. The system comprises several modules: a topological map and a mechanical assessment module. The mechanical assessment module extracts the outer topological contour of the abrupt pixel set in the perspective scattering gradient map to delineate the tissue degradation boundary. It combines the local curvature change rate of the surface point cloud coordinates within the tissue degradation boundary to calculate the three-dimensional mechanical yield parameter of the corresponding region of the degradation boundary. The mechanical weakening feature vector is constructed by combining the tissue degradation boundary coordinates and the three-dimensional mechanical yield parameter. The sorting decision module maps the surface point cloud coordinates to the workspace, constructs a gripping repulsion zone along the outer normal of the tissue degradation boundary, and optimally allocates gripping anchor points that satisfy the geometric centroid within the contour after removing the gripping repulsion zone from the surface point cloud coordinates. The three-dimensional mechanical yield parameter is extracted from the mechanical weakening feature vector, the end contact negative pressure matching the gripping anchor points is calibrated, and the gripping anchor points and end contact negative pressure are combined to generate a sorting instruction and send it to the mechanical execution end.

[0042] In this implementation scheme, the visual analysis module serves as the central hub for front-end data acquisition and preprocessing, directly coordinating the collaborative operation of the structured light projection hardware and the multi-band image sensor. At the physical level, this module drives the projector to output multi-frequency phase-shifted fringe patterns, synchronously controlling the image sensor to record a sequence of phase-shifted images geometrically modulated by the fruit surface. Subsequently, relying on digital signal processing algorithms, the module extracts the absolute folding phase and modulation depth of the image sequence, calculates the surface point cloud coordinates representing the three-dimensional spatial morphology of the fruit's exterior, and simultaneously separates the high-frequency surface reflection component carrying cuticle contour information from the low-frequency subcutaneous diffuse component penetrating the tissue, thus completing the decoupling and extraction of basic optical and geometric data.

[0043] The scattering analysis module receives the decoupled data from the underlying layers and is responsible for quantifying the optical transmission characteristics within the tissue under a three-dimensional spatial framework. This module uses the extracted surface point cloud coordinates as the framework to construct a three-dimensional spatial reference. It determines the normal direction of each coordinate node through vector operations, guiding the extraction of the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component radiating into the deeper layers of the pulp. The module's built-in computation unit inputs the discrete light intensity attenuation data into a nonlinear approximation operator, fitting and solving for the corresponding internal photon scattering coefficients. Finally, these coefficients are remapped and matched with the surface point cloud coordinates, rendering and outputting a perspective scattering gradient map reflecting the continuous spatial distribution of the pulp's internal state.

[0044] The mechanical assessment module is used to transform optical gradient features across domains into physical and mechanical stress indicators, enabling precise delineation of deteriorated regions. This module traverses the perspective scattering gradient map, detects abrupt pixel changes in the hash coefficients, extracts the surrounding topological contour, and maps it back to a 3D benchmark to delineate the physical boundary of tissue deterioration. The module further extracts the principal curvature and Gaussian curvature of the tangent space of the point cloud within the deterioration boundary to generate local curvature change rates. Combined with the built-in transformation benchmark, it calculates the critical stress values ​​for plastic deformation at each node, extracts the minimum value as the 3D mechanical yield parameter, and then concatenates the coordinate matrix with this yield parameter to encapsulate a mechanical weakening feature vector.

[0045] The sorting decision module is responsible for establishing the execution link from the vision algorithm to the mechanical control, and outputting safe physical action instructions to prevent secondary damage. This module transforms the surface point cloud coordinates to the robotic arm's working space based on the hand-eye calibration extrinsic parameter matrix, generates a solid isolation envelope surface along the outer normal of the tissue deterioration boundary, and constructs the grasping rejection zone through Boolean operations. The module then uses discrete integration to solve for the geometric centroid of the remaining safe point cloud region after removing the rejection zone, optimizes the allocation of grasping anchor points based on proximity, and directly retrieves the mechanical weakening eigenvector to limit the ultimate vacuum adsorption pressure, matching the target end contact negative pressure. Finally, the pose coordinates and negative pressure parameters are encapsulated into a communication message and sent to the mechanical actuator.

[0046] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for intelligent detection and sorting of fruit defects based on machine vision, characterized in that, Includes the following steps: S1. Project a multi-frequency phase-shifted stripe pattern onto the fruit in the detection space and acquire a phase-shifted image sequence; perform phase unpacking and modulation depth analysis on the phase-shifted image sequence, extract the surface point cloud coordinates, and peel off the surface high-frequency reflection component carrying contour information and the subcutaneous low-frequency diffuse component carrying internal tissue features. S2. Construct a three-dimensional reference using surface point cloud coordinates, and extract the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component along the normal of the surface point cloud coordinates; perform surface fitting operation on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients, and combine the internal photon scattering coefficients to generate a perspective scattering gradient map. S3. Extract the outer topological contour of the abrupt pixel set in the perspective scattering gradient map and delineate the tissue degradation boundary; combine the local curvature change rate of the surface point cloud coordinates within the tissue degradation boundary to calculate the three-dimensional mechanical yield parameter of the region corresponding to the degradation boundary, and combine the tissue degradation boundary coordinates and the three-dimensional mechanical yield parameter to construct the mechanical weakening feature vector. S4. Map the surface point cloud coordinates to the workspace, construct the gripping rejection zone along the outer normal of the tissue deterioration boundary, and optimally allocate gripping anchor points that satisfy the geometric centroid within the contour after removing the gripping rejection zone from the surface point cloud coordinates; extract the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrate the end contact negative pressure matching the gripping anchor points, and combine the gripping anchor points and end contact negative pressure to generate sorting instructions and send them to the mechanical execution end. 2.The machine vision-based fruit defect intelligent detection and sorting method according to claim 1, characterized in that: The specific process of projecting multi-frequency phase-shift stripe patterns onto the fruit in the detection space and acquiring phase-shift image sequences is as follows: By setting the base spatial frequency and phase shift step, a multi-frequency phase shift fringe pattern containing high-frequency, mid-frequency and low-frequency components is generated. The hardware synchronous trigger controller drives the structured light projection module to sequentially project multi-frequency phase-shift stripe patterns onto the surface of the fruit in the detection space. A synchronously driven multi-band image sensor captures image frames whose corresponding surface curvature undergoes phase modulation within a single phase shift cycle of a multi-frequency phase-shifted stripe pattern. The image frames of each frequency band and different phase shift phases are combined to construct a phase-shifted image sequence. 3.The machine vision-based fruit defect intelligent detection and sorting method according to claim 2, characterized in that: The specific process of performing phase unpacking and modulation depth analysis on the phase-shifted image sequence, extracting surface point cloud coordinates, and separating the surface high-frequency reflection component carrying contour information and the subcutaneous low-frequency diffuse component carrying internal tissue features is as follows: Extract the absolute folded phase distribution of the phase-shifted image sequence in the spatial domain; The absolute folded phase distribution is unfolded by applying a multi-frequency heterodyne unpacking algorithm, and the inverse perspective mapping is performed in combination with a preset stereo calibration matrix to solve the three-dimensional spatial geometric coordinates to form the surface point cloud coordinates. The modulation depth of the phase-shifted image sequence is analyzed, and the AC amplitude distribution and DC bias distribution of the phase-shifted image sequence in the time domain are extracted. The surface high-frequency reflection component is obtained by extracting the AC amplitude distribution, and the subcutaneous low-frequency diffuse component is extracted by combining the DC bias distribution to remove the environmental backlight offset. 4.The machine vision-based fruit defect intelligent detection and sorting method according to claim 1, characterized in that: The specific process of constructing a three-dimensional reference using surface point cloud coordinates and extracting the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component along the normal of the surface point cloud coordinates is as follows: A spatial three-dimensional reference is constructed using surface point cloud coordinates, and a local tangent plane matrix matching the surface point cloud coordinates is constructed. The normal vector set associated with the local tangent plane matrix is ​​extracted through cross product operation. The coordinates of the specified surface point cloud are used as the origin of the spatial search, and the data is mapped to the pixel array of the subcutaneous low-frequency diffuse component along the penetration path defined by the normal vector set. The brightness response amplitudes of discrete sampling points along the penetration path are extracted to construct a characteristic of the light intensity attenuation distribution that represents the decreasing response of photons along the propagation depth direction. 5.The machine vision-based fruit defect intelligent detection and sorting method according to claim 1, characterized in that: The specific process of performing surface fitting operation on the light intensity attenuation distribution characteristics to obtain the corresponding internal photon scattering coefficients, and combining the internal photon scattering coefficients to generate the perspective scattering gradient map is as follows: Substitute the light intensity attenuation distribution characteristics into the diffuse analytical equation of a semi-infinite medium, perform nonlinear least squares convergent fitting operation, and solve for the optimal approximation parameters of the diffuse analytical equation of a semi-infinite medium. Extract the internal photon scattering coefficients that characterize the photon attenuation intensity from the optimal approximation parameters; By binding the surface point cloud coordinates with the corresponding internal photon scattering coefficients, the pixel mesh is reconstructed in the two-dimensional unfolded projection domain to generate a perspective scattering gradient map that encompasses the spatially continuous mapping of the internal photon scattering coefficients. 6.The machine vision-based fruit defect intelligent detection and sorting method according to claim 1, characterized in that: The specific process of extracting the outer topological contour of the abrupt pixel set in the perspective scattering gradient map and delineating the boundary of tissue deterioration is as follows: Traverse the pixel matrix of the perspective scattering gradient map and extract the set of abrupt pixel changes where the second derivative of the internal photon scattering coefficients crosses zero. Applying a morphological dilation kernel to perform a closing operation on a set of abruptly changed pixels to close connected components; Tracing the edge pixel coordinate sequence of connected components, and connecting the edge pixel coordinate sequences to generate the outer topological contour of the closed polygon; The topological contour of the closed polygon is projected back to the three-dimensional datum, and the corresponding spatial surface fragments are extracted to delineate the tissue degradation boundary in three-dimensional space. 7.The machine vision-based fruit defect intelligent detection and sorting method according to claim 1, characterized in that: By combining the local rate of curvature change of surface point cloud coordinates within the deterioration boundary, the three-dimensional mechanical yield parameter of the region corresponding to the deterioration boundary is calculated. The specific process of constructing the mechanical weakening eigenvector by combining the deterioration boundary coordinates and the three-dimensional mechanical yield parameter is as follows: Extract the surface point cloud coordinates enclosed within the tissue deterioration boundary, solve for the principal curvature and Gaussian curvature of the surface point cloud coordinates in the tangent space, and extract the spatial gradients of the principal curvature and Gaussian curvature to form the local curvature change rate. The preset scattering-elasticity numerical transformation matrix is ​​retrieved, and the apparent elastic modulus of the tissue corresponding to the internal photon scattering coefficient in the perspective scattering gradient map is extracted. The geometric stress concentration factor of the corresponding tissue node is determined based on the local curvature change rate. The apparent elastic modulus of the tissue and the geometric stress concentration factor are coupled to analyze the critical stress value of the corresponding tissue node to undergo plastic deformation. The spatial minimum value of the critical stress is extracted as a three-dimensional mechanical yield parameter. By splicing the coordinates of the deterioration boundary of the structure with the three-dimensional mechanical yield parameter, a mechanical weakening feature vector pointing to the extreme point of structural fragility is generated.

8. The intelligent detection and sorting method for fruit defects based on machine vision according to claim 1, characterized in that: The specific process of mapping surface point cloud coordinates to the workspace, constructing a grasping rejection zone along the extensional normal of the tissue deterioration boundary, and optimizing the allocation of grasping anchor points that satisfy the geometric centroid within the contour after removing the grasping rejection zone from the surface point cloud coordinates is as follows: Extract the base coordinate system of the mechanical actuator, and use the hand-eye calibration extrinsic parameter matrix to transform the surface point cloud coordinates into the working space contained in the base coordinate system; A pre-defined safety distance envelope is extended outward along the extensional normal of the tissue deterioration boundary, and a grasping and repulsion zone is generated through spatial Boolean union operation; Extract the coordinates of the remaining surface point cloud after removing the grab rejection area, and perform discrete voxelization integration on the remaining surface point cloud coordinates to solve for the geometric centroid. In the remaining surface point cloud coordinate cluster that is closest to the geometric centroid surface, gripping anchor points that are distributed in a circular symmetry are optimally allocated based on the spatial degrees of freedom of the gripper at the execution end.

9. The intelligent detection and sorting method for fruit defects based on machine vision according to claim 1, characterized in that: The specific process of extracting the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrating the end contact negative pressure of the gripping anchor point, and combining the gripping anchor point and the end contact negative pressure to generate a sorting command and send it to the mechanical execution end is as follows: The three-dimensional mechanical yield parameter is extracted from the mechanical weakening feature vector. Combined with the local geometric contact area at the anchor point, the ultimate vacuum adsorption pressure under the condition that the fruit does not break as a whole is analyzed. Under the upper limit constraint of the ultimate vacuum adsorption pressure, the end contact negative pressure is matched to meet the need to counteract the lifting force of the fruit's own gravity; The spatial pose matrix of the gripping anchor point and the timing control signal of the end contact negative pressure are encapsulated into the communication protocol message. The gripping anchor point and the end contact negative pressure are combined to generate a sorting instruction and send it to the mechanical execution end.

10. A machine vision-based intelligent fruit defect detection and sorting system, used to execute the machine vision-based intelligent fruit defect detection and sorting method according to any one of claims 1-9, characterized in that, include: The visual analysis module is used to project multi-frequency phase-shift stripe patterns onto the fruits in the detection space and acquire phase-shift image sequences; Phase unpacking and modulation depth analysis are performed on the phase-shifted image sequence to extract the surface point cloud coordinates and separate the surface high-frequency reflection component carrying contour information and the subcutaneous low-frequency diffuse component carrying internal tissue features. The scattering analysis module is used to construct a three-dimensional reference based on surface point cloud coordinates, extract the light intensity attenuation distribution characteristics of the subcutaneous low-frequency diffuse component along the normal of the surface point cloud coordinates, perform surface fitting operation on the light intensity attenuation distribution characteristics, obtain the corresponding internal photon scattering coefficients, and combine the internal photon scattering coefficients to generate a perspective scattering gradient map. The mechanical evaluation module is used to extract the outer topological contour of the abrupt pixel set in the perspective scattering gradient map and delineate the tissue degradation boundary; combined with the local curvature change rate of the surface point cloud coordinates within the tissue degradation boundary, the three-dimensional mechanical yield parameter of the region corresponding to the degradation boundary is calculated, and the tissue degradation boundary coordinates and the three-dimensional mechanical yield parameter are combined to construct the mechanical weakening feature vector. The sorting decision module maps surface point cloud coordinates to the workspace, constructs a gripping rejection zone along the outer normal of the tissue deterioration boundary, and optimizes the allocation of gripping anchor points that satisfy the geometric centroid within the contour after removing the gripping rejection zone from the surface point cloud coordinates. It extracts the three-dimensional mechanical yield parameter from the mechanical weakening feature vector, calibrates the end contact negative pressure matching the gripping anchor points, and combines the gripping anchor points and end contact negative pressure to generate sorting instructions that are sent to the mechanical execution end.

Images