Wearable fruit multi-point quality detection method and device based on manipulator

Abstract

The invention belongs to the technical field of agricultural product nondestructive testing and automatic equipment, and discloses a wearable fruit multi-point quality detection method and device based on a manipulator, the wearable fruit multi-point quality detection method and device based on the manipulator are worn on an automatic fruit picking mechanical arm, and the integrated operation of picking, detecting and sorting of fruits can be achieved; and detection and grading of quality indexes of the internal maturity of the fruits are accurately completed. When the mechanical arm picks the fruits, the quality detection module synchronously collects data, the control module receives the data and outputs a quality result through model operation, then a sorting program drives the mechanical arm to transfer the fruits to the storage boxes of the corresponding grades, and picking-detection-sorting integrated operation is achieved. According to the invention, multi-modal quality data can be synchronously obtained, detection of multiple varieties of fruits is flexibly adapted, the detection precision and efficiency are improved, and the limitation of a traditional detection technology is solved.

Description

Technical Field

[0001] This invention belongs to the field of non-destructive testing and automated equipment technology for agricultural products, and particularly relates to a wearable multi-point quality testing method and device for fruits based on a robotic arm. Background Technology

[0002] Existing non-destructive testing equipment for apple quality includes portable devices and automated testing line equipment. Portable devices (such as a multifunctional integrated fruit sugar content and firmness testing device, publication number CN222913484U; portable fruit sugar content non-destructive testing instrument, announcement number CN223051182U) are single-point or small-area testing modes, which can only test local areas and cannot represent the overall quality of the apple. Moreover, manual testing is required after the fruit is picked. Automated testing line equipment (such as an automatic fruit sorting production line, publication number CN117600102A; apple quality non-destructive testing equipment, publication number: CN119915816A) requires the apples to be transported from the orchard to the testing station via a conveyor belt. This process is time-consuming, carries a high risk of fruit damage, and involves bulky equipment with poor adaptability and high costs. Both of these methods perform testing through a single modality, failing to take advantage of multimodal collaborative testing and thus unable to achieve synchronous acquisition and collaborative analysis of multimodal data.

[0003] One-sided sampling: Single-point sampling is difficult to reflect the distribution of surface defects and the overall differences in internal maturity of fruit. It is easy to misjudge quality due to local data deviation. In addition, it does not combine multi-component collaborative collection and cannot achieve data coverage of multi-point sampling locations.

[0004] The detection sensors are limited in number and accuracy: Single-spectrum detection often uses a single-chip design, which has a limited band coverage and makes it difficult to accurately capture the characteristic wavelengths related to the internal quality of fruits.

[0005] Inefficiency: The testing and harvesting equipment are separate, requiring additional transfer steps, which is cumbersome and time-consuming, affecting production efficiency.

[0006] Poor adaptability: Some devices are large in size and lack flexibility, making it difficult to integrate them with robotic arms. They are not suitable for testing multiple varieties of fruits, and the integration of hardware and software is low, making it difficult to meet the requirements of industrial-grade production in terms of overall quality judgment accuracy and stability.

[0007] Based on the above analysis, the problems and shortcomings of the existing technology are as follows: Sampling bias: Single-point sampling is difficult to reflect the distribution of surface defects and the overall differences in internal maturity of fruit. It is easy to misjudge quality due to local data deviations and cannot achieve data coverage of multiple sampling locations.

[0008] The detection sensors are limited in number and accuracy: Single-spectrum detection often uses a single-chip design, which has a limited band coverage and makes it difficult to accurately capture the characteristic wavelengths related to the internal quality of fruits.

[0009] Inefficiency: The testing and harvesting equipment are separate, requiring additional transfer steps, which is cumbersome and time-consuming, affecting production efficiency.

[0010] Poor adaptability: Some devices are large in size and lack flexibility, making it difficult to integrate them with robotic arms. They are not suitable for testing multiple varieties of fruits, and the integration of hardware and software is low, making it difficult to meet the requirements of industrial-grade production in terms of overall quality judgment accuracy and stability. Summary of the Invention

[0011] To address the problems existing in the prior art, this invention provides a wearable multi-point quality detection method and device for fruits based on a robotic arm.

[0012] This invention is implemented as follows: a wearable multi-point quality inspection method for fruit based on a robotic arm, the method comprising the following steps: (1) The robotic arm performs quality inspection simultaneously while performing fruit picking actions, and the picking and inspection are coordinated through the internal quality inspection module worn on the robotic arm; (2) When the robotic arm grasps the fruit, it acquires pressure data and controls the grasping pressure. After the fruit is firmly grasped, it acquires the raw data of the spectrum. (3) Transmit and verify the original data of the spectrum, remove abnormal data, then average the multiple original data to obtain the average spectrum, input the obtained average spectrum into the prediction model, output the prediction results of sugar content, acidity and hardness, and then obtain the final multi-level quality results of the fruit through the weight grading algorithm. (4) The control system automatically sorts the fruits according to the quality results and places them in the corresponding grade storage boxes to realize the integrated process of picking-testing-sorting.

[0013] Furthermore, the collaborative steps for harvesting and testing include: The detection system adopts a modular wearable design. The fingertip detection module is fixed to the mechanical fingertip with Velcro, and the suction cup detection module is integrated into the detachable silicone suction cup. The suction cup is installed in the palm of the robotic hand by threads. When the robotic hand performs fruit picking, it simultaneously triggers the multi-module detection system (including pressure detection module and internal quality detection module) worn on the robotic hand, realizing the coordinated linkage between picking action and quality detection. There is no need to set up an additional separate detection process, thus improving work efficiency.

[0014] Furthermore, the multi-source data acquisition step includes: 1. Pressure data acquisition and gripping control: The pressure detection module worn on the robotic arm collects pressure signals in real time during the gripping process and feeds the data back to the control system. The control system controls the robotic arm to stop gripping when the pressure reaches the preset "safe gripping threshold" (ensuring that the fruit is gripped firmly while avoiding squeezing damage), ensuring that the spectral detection module is in close contact with the fruit surface and ensuring the stability of spectral data acquisition.

[0015] 2. Multi-point spectral data acquisition: Through 13 spectral detection modules pre-installed on the robotic arm (specifically, 4 modules are deployed at each of the three fingertips and 1 module is deployed at the suction cup end), the raw spectral data of 13 detection points on the surface of the fruit are collected simultaneously; the spectral detection modules cover 18 characteristic bands in the 400-1100 nanometer wavelength range, covering the visible light and near-infrared spectral regions, to achieve comprehensive capture of information related to the internal quality of the fruit.

[0016] Furthermore, the data transmission, verification, anomaly removal, and averaging steps include: 1. Multimodal data transmission and verification: The acquired raw spectral data (including 13 channels of spectrum) is uniformly encapsulated with a verification field and transmitted to the control module; the receiving end verifies the data integrity through the verification field. If data loss or error is found, a retransmission mechanism is initiated to handle transmission anomalies and ensure data reliability.

[0017] 2. Outlier Removal: The Mahalanobis distance (MD) algorithm was used to clean the 13-channel raw spectral data and remove outliers. The Mahalanobis distance calculation formula is as follows:

[0018] ,

[0019] In the formula, x is the feature vector matrix of a single spectrum (dimension 1×m, where m is the number of spectral features, corresponding to the spectral values ​​of 18 bands), μ is the mean vector of the spectral data, and Σ is the covariance matrix of the spectral data; a Mahalanobis distance threshold is set, and when the Mahalanobis distance of a certain spectral data exceeds the threshold, it is judged as abnormal data and removed.

[0020] 3. Spectral data averaging: The average value of the valid multi-channel spectral data after removing outliers is calculated to obtain average spectral data, reducing single-point detection errors and improving data representativeness; the averaging calculation formula is:

[0021] ,

[0022] In the formula, is the average spectral value in the i-th band (i=1,2,...,18, covering the visible and near-infrared regions), and n is the number of effective spectral paths remaining after removing anomalies (n≤13). This represents the original spectral value of the effective spectrum of the k-th channel in the i-th band;

[0023] Furthermore, the model construction includes:

[0024] 1. Obtain the spectral data of the apple to be tested;

[0025] 2. MSC correction:

[0026] For each apple, a linear regression of the raw spectrum Araw with the reference spectrum Aref yields the spectral intensity scaling factor and the spectral baseline shift factor, described by the following formula:

[0027] ,

[0028] In the formula, Araw represents the original spectrum of the apple sample, Aref represents the reference spectrum, k represents the spectral intensity scaling factor of the sample, b represents the spectral baseline shift factor of the sample, and εi represents the regression residual.

[0029] The corrected spectral data of apples can be obtained using the MSC correction formula, which is:

[0030] ,

[0031] In the formula, Acorrected represents the spectral intensity of the sample after correction considering the transverse diameter information.

[0032] Each sample spectrum is corrected to be "aligned" with the reference spectrum, thereby removing intensity and baseline interference from scattering and retaining the characteristic absorption information corresponding to the chemical composition, resulting in the MSC-corrected spectral data.

[0033] 3. Feature Wavelength Extraction: Principal Component Analysis (PCA) is used for dimensionality reduction. The spectral data is decomposed by PCA, and the main principal components whose cumulative contribution rate meets the preset threshold are retained while redundant information is removed. Then, based on the loading coefficients of each main principal component, the wavelength positions corresponding to the local extrema of the loading coefficients are located, and the feature wavelength ranges that are sensitive to the internal quality of the fruit (sugar content, acidity, and firmness) are selected to form a simplified spectral feature set for subsequent model input.

[0034] 4. Construction of Internal Quality Prediction Model: Using the extracted spectral feature set as input, an internal quality prediction model based on partial least squares discriminant analysis (PLS-DA) is constructed. The model is trained and optimized using training samples (including spectral features of fruits of different quality grades and actual sugar content, acidity, and firmness values). The model outputs the predicted results of the sugar content, acidity, and firmness of the fruits.

[0035] 5. Multi-level quality fusion judgment: The internal quality prediction results are fused using a weighted grading algorithm: First, based on the characteristics of fruit varieties and market demand, corresponding weight coefficients are set for sugar content, acidity, and firmness; second, the prediction results of each indicator are mapped to standardized scores; finally, the comprehensive quality score is calculated by weighted summation, and the quality grades are divided according to the comprehensive score, ultimately outputting four quality classification results: excellent, good, qualified, and substandard.

[0036] Furthermore, after receiving the output quality grade results, the control system automatically generates sorting control instructions to drive the robotic arm to perform graded transfer operations: according to the preset storage box position coordinates, the robotic arm accurately places the fruit into the storage box corresponding to the quality grade, completing the fully automated operation from picking, testing to sorting.

[0037] The present invention also provides a wearable multi-point quality inspection device for fruit based on a robotic arm, the device comprising:

[0038] The system comprises a robotic arm module, a pressure detection module, a quality inspection module, a control module, and a quality sorting module. The robotic arm module includes a main robotic arm module, a robotic hand module, a picking and positioning camera module, and a control computer module. The pressure detection module is worn and fixed to the fingertips of the robotic hand module. The internal quality inspection module is worn and fixed to the inside of the hand's suction cup and the fingertips. The pressure detection unit detects pressure data on the fruit surface, and the internal quality inspection unit detects visible and near-infrared spectral data of the fruit passing through it. The quality sorting module includes multiple storage boxes corresponding to different quality grades. The control module includes a data acquisition unit, a transmission and anomaly removal unit, a model calculation unit, and a sorting control unit. The control module controls the acquisition, transmission, anomaly removal, averaging, predictive model calculation, and quality sorting of pressure and quality information.

[0039] Furthermore, the robotic arm module is a currently available multi-degree-of-freedom harvesting robotic arm. The robotic arm module includes a main robotic arm module, a robotic hand module, and a control computer module. The main robotic arm module is a multi-degree-of-freedom robotic arm, and the robotic hand module is a three-claw robotic hand with suction cups. The robotic hand module includes a fingertip unit and a detachable suction cup unit. The fingertip unit is used to wear an internal quality detection module and a pressure detection module. The detachable suction cup unit is used to integrate an internal detection module within the detachable suction cup. The control computer module stores and controls the program for the robotic arm to harvest and inspect fruits.

[0040] Furthermore, the pressure detection module includes a pressure detection unit, an anti-slip module, and a fixing module. The pressure detection unit uses a resistive thin-film pressure sensor. The fingertip anti-slip module uses anti-slip silicone material to cover the tactile sensor module, and hot-melt silicone is used to complete the planar coverage of the tactile sensor. The fixing module uses Velcro straps passing through the anti-slip silicone to fasten the pressure detection module to the fingertips of the three robotic arms. The pressure detection module is worn on the robotic arms and acquires pressure data when grasping fruit. It provides real-time feedback to control the robotic arms to stop grasping when a preset safe pressure threshold is reached, ensuring that the internal quality detection module is tightly attached to the fruit surface and completes the acquisition of visible and near-infrared spectral data of the fruit.

[0041] Furthermore, the internal quality inspection module includes a detachable suction cup integrated spectral detection module and a wearable finger sleeve integrated spectral detection module. The detachable suction cup integrated spectral detection module includes a detachable suction cup unit and a rigid suction cup end circuit board. The detachable suction cup unit includes a rigid outer shell of the suction cup, bolts at the bottom of the suction cup, and a black silicone suction cup at the top. Bolts connect the outer shell of the suction cup. Four nut pillars and four protruding strips are provided on the inner plane of the rigid outer shell of the suction cup. The four nut pillars are used to fix the near-infrared spectral detection module, and the four protruding strips are used to nest the inner black curved silicone light shield. There are four symmetrical holes at the four corners of the rigid suction cup end circuit board. The top black silicone suction cup is nested on the rigid outer shell to form the detachable suction cup integrated spectral detection module. The detachable suction cup integrated spectral detection unit is fixed to the inner thread of the palm of the robotic arm module by the bottom bolt. The wearable finger sleeve integrates a spectral detection module, including a flexible fingertip circuit board, surface anti-slip silicone, and a back fixing module. The flexible fingertip circuit board is made of polyimide (PI) material and measures 20mm × 50mm × 0.3mm. The surface anti-slip silicone covers the flexible circuit board with heat fusion, leaving four circular areas to fix the light shield. The back fixing module consists of Velcro straps passing through the anti-slip silicone. The Velcro straps of the fixing module secure the fingertip detection unit to the fingertips of the three robotic arms. The rigid suction cup end... An internal quality inspection unit is integrated on the circuit board and the flexible fingertip circuit board. This unit includes halogen lamps and visible / near-infrared spectral chips. The halogen lamps are arranged diagonally on the circuit board, with their illumination direction forming a 45° angle with the vertical direction. The visible / near-infrared spectral chips consist of two chips, AS7343 and AS7341. The AS7341 chip group covers nine wavelengths centered on visible light (410, 445, 480, 515, 555, 590, 630, 680 nm) and near-infrared light (910 nm). The AS7343 chip group covers wavelengths centered on visible light (405, 425, 450, 475, 515, 550, 555, 600, 640, 690, 745 nm) and near-infrared light (855 nm). The system comprises 12 spectral bands centered at nm, with overlapping bands at 515 and 555 nm. Data from these overlapping bands needs to be averaged. Other bands require complementary filling of gaps. This results in 18 spectral bands centered on visible light (405, 410, 425, 445, 450, 480, 515, 550, 555, 590, 600, 630, 640, 680, 690, 745 nm) and near-infrared light (855, 910 nm). Two spectral chips are mounted parallel to each other between four halogen lamps. The detachable suction cup integrated spectral detection module and the wearable finger sleeve integrated spectral detection module each have one palm and three fingertips, resulting in a total of 13 spectral detection modules. This enables multi-point spectral data acquisition of the fruit.

[0042] Furthermore, the control module includes a power supply module, a data acquisition unit, a transmission and anomaly removal unit, a model calculation unit, and a sorting control unit. The power supply module uses the existing lithium battery of the robotic arm, providing stable power to the internal quality detection module through voltage conversion (12V to 5V). The data acquisition unit is used to receive the raw spectral data output by the multimodal detection module. The transmission and anomaly removal unit performs unified encapsulation and transmission verification with verification fields on the raw spectral data and filters and removes abnormal data. The model calculation unit is used to make predictions using a preset prediction model and output multi-level quality results for the fruit. The sorting control unit is used to output robotic arm sorting action control commands based on the quality results, driving the robotic arm to complete fruit sorting and place the fruit in the corresponding grade fruit storage box.

[0043] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0044] This invention addresses the core pain points of existing fruit quality testing technologies, namely "partial sampling, poor equipment coordination, and low hardware and software integration." Through multi-dimensional technological innovation, it achieves significant advantages, with the following specific benefits compared to existing technologies:

[0045] (1) Multi-point location data acquisition solves the problem of one-sidedness of traditional single-point detection and improves the comprehensiveness of quality judgment.

[0046] Existing technologies often employ single-point sampling (such as collecting only a local spectrum of the fruit), which is insufficient to reflect the overall quality differences of the fruit and is prone to misjudgment due to local data deviations. This invention utilizes a wearable integrated design, installing a spectral detection module (AS7343 / AS7341 chip) on the suction cup and shaft end of a robotic arm to acquire visible and near-infrared spectra. Furthermore, the spectral module achieves multi-point sampling through 13 detection units (1 unit on the palm + 3 units on the fingertips × 4 units), fully covering the core dimensions of fruit quality, avoiding the one-sidedness of traditional single-point detection, and ensuring a more comprehensive and objective quality assessment.

[0047] (2) The integration of high-precision hardware design and intelligent algorithms solves the problem of insufficient traditional detection accuracy and improves the accuracy of quality judgment.

[0048] In existing technologies, single-spectral chips have limited band coverage (e.g., only covering the visible or near-infrared bands), and data processing often employs simple filtering or a single model, making it difficult to meet industrial requirements in terms of detection accuracy. This invention improves accuracy through dual innovations: In terms of hardware, the AS7343 (400-700nm) and AS7341 (650-1100nm) chips form a 50nm overlap response in the 650-700nm range, reducing the effective bandwidth of a single channel to 15-18nm, improving resolution by 20%-30% compared to traditional single-chip solutions. Furthermore, data stability is ensured through transparent optical channel calibration.

[0049] (3) Wearable integration and integrated process solve the problems of poor coordination and low efficiency of traditional equipment, and improve testing and production efficiency.

[0050] In existing technologies, the detection device and the harvesting equipment are independent of each other, requiring the transfer of fruit via conveyor belts. This process is cumbersome and time-consuming for single-sample testing, and some devices are bulky and difficult to integrate. This invention adopts a "wearable" design, directly embedding the detection module into a robotic arm to achieve simultaneous "harvesting-detection." Furthermore, it eliminates the need for additional transfer steps; after detection, the control module directly drives the robotic arm to sort, forming an integrated "harvesting-detection-sorting" process. Simultaneously, the four-axis robotic arm, equipped with detachable suction cups, can adapt to various fruit varieties such as apples, pears, and peaches, eliminating the need for frequent equipment changes, further reducing production line adjustment costs and improving overall production efficiency.

[0051] (4) High integration of hardware and software and stable control solve the problems of poor flexibility and low applicability of traditional devices, and adapt to the needs of industrial automation. In the existing technology, it is difficult to integrate high-precision detection hardware (such as spectrometers and high-definition cameras) with robotic arms, and there is a lack of unified control modules, making it difficult to adapt to industrial automated production lines. This invention solves this problem through "hardware modularization + control integration": In terms of hardware, the internal quality detection module can be directly worn on the fingertips and suction cups of the robotic arm. The fingertips adopt flexible circuit integration, and the suction cups, sensors and other components are detachable, which is convenient for maintenance and replacement; the control module is shared with the robotic arm control computer, integrating four major units: power supply (12V to 5V stable power supply), data conversion, model program and sorting program, without the need for additional control equipment; the overall device can be directly adapted to existing automatic fruit picking robotic arms without large-scale modification, making it more applicable and meeting the needs of continuous industrial operation.

[0052] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0053] The technical solution of this invention can be directly applied to fruit orchards, agricultural product processing enterprises, and other scenarios. Its expected benefits and commercial value are mainly reflected in three aspects: First, through the integrated process of "harvesting-testing-sorting," the traditional testing process eliminates fruit transportation and manual operation, significantly reducing labor and time costs, while also reducing fruit damage during transportation, increasing the yield of finished fruit, and indirectly increasing the economic benefits of enterprises. Second, the application of multimodal collaborative detection and high-precision models can achieve accurate grading of fruit quality, helping enterprises distinguish between superior, good, qualified, and substandard fruits. Differentiated pricing strategies can be formulated for different grades, increasing the market premium of high-grade fruits and enhancing product market competitiveness. Third, the device adopts a wearable design, is compatible with the detection of multiple varieties of fruits, and does not require large-scale modification of existing harvesting equipment, reducing the equipment investment costs of enterprises. At the same time, it meets the needs of industrial-grade continuous operation, can be quickly scaled up, has broad market promotion prospects, and can bring continuous revenue growth to equipment manufacturing and technical service companies. Attached Figure Description

[0054] Figure 1 This is a flowchart of a wearable multi-point quality detection method for fruits based on a robotic arm, provided in an embodiment of the present invention.

[0055] Figure 2 This is a structural diagram of a wearable multi-point quality inspection device for fruits based on a robotic arm, provided in an embodiment of the present invention.

[0056] Figure 3 This is a front view of the robotic arm provided in an embodiment of the present invention.

[0057] Figure 4 This is a side view of the suction cup provided in an embodiment of the present invention.

[0058] Figure 5 This is a top view of the suction cup circuit board provided in an embodiment of the present invention.

[0059] Figure 6 This is a diagram of the fingertip and finger sleeve module provided in an embodiment of the present invention.

[0060] Figure 7 This is a diagram showing the combination of the fingertip and finger sleeve robotic arm provided in an embodiment of the present invention.

[0061] Figure 8 This is a diagram of the assembled device provided in an embodiment of the present invention.

[0062] Figure 9 This is a modeling and sorting flowchart provided in an embodiment of the present invention.

[0063] Figure 9 This is a modeling and sorting flowchart provided in an embodiment of the present invention.

[0064] In the diagram: 1. Robotic arm; 2. Robotic hand; 3. Detachable silicone suction cup; 4. Embedded computer; 5. Suction cup end circuit board; 6. Finger tip flexible circuit; 7. Quality inspection module; 8. Pressure detection module; 301. Suction cup rigid shell; 302. Bottom internal threaded bolt of shell; 303. Top nut post of shell; 304. Protruding strip of shell; 305. Nested light shield; 501. AS7343; 502. AS7341; 503. Light source halogen lamp; 504. Screw holes of suction cup circuit board; 601. Finger tip anti-slip silicone; 602. Velcro strap; 603. Anti-slip silicone sheet holes; 801. Resistive thin-film pressure sensor. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0066] This invention addresses the technical bottlenecks in the industrial application of existing fruit quality testing methods. Traditional fruit grading systems often rely on single-vision or single-point spectral detection, with the detection unit independently positioned from the harvesting mechanism. This prevents simultaneous identification of internal quality during harvesting, leading to a disconnect between detection and sorting, delays in production cycle, and damage to the fruit surface caused by repeated gripping by the robotic arm. Furthermore, existing spectral detection methods often use fixed-angle illumination, making it difficult to obtain comprehensive information about the fruit surface. This results in detection results being significantly affected by the angle of illumination, reflectivity, and skin condition, especially when the fruit has complex shapes or defects, leading to decreased accuracy. Therefore, the industry urgently needs a system that can perform quality detection and intelligent sorting simultaneously with the robotic arm's harvesting actions, thereby improving detection accuracy and operational continuity without altering the production rhythm.

[0067] The wearable structure proposed in this invention integrates the quality inspection module with a robotic arm, achieving deep integration of the inspection unit and the end effector. When the robotic arm performs grasping or rotating actions, it automatically adjusts the sensor's incident angle and sampling direction based on its spatial posture, ensuring synchronous acquisition of spectral signals within the same timeframe. This structure, through a flexible bonding layer and a pressure feedback unit, ensures the force applied when grasping fruit and maintains a tight fit between the quality inspection module and the fruit surface, while avoiding indentations or light spot drift caused by traditional rigid devices. This design makes the robotic arm both an actuator and a multi-source sensing platform, structurally achieving coordination between harvesting and inspection actions.

[0068] At the data processing level, the system verifies the spectral signals, initiates a retransmission mechanism to handle transmission anomalies in case of data loss, and removes abnormal data using Mahalanobis distance to ensure data reliability. The average spectrum is first corrected by multivariate scattering, and then principal component dimensionality reduction and loading feature extraction are used to extract the main bands reflecting the sugar-acid ratio, soluble solids, and water content, providing input for subsequent model training.

[0069] The model construction uses the extracted spectral feature set as input to build an internal quality prediction model based on partial least squares discriminant analysis (PLS-DA). The model is trained and optimized using training samples, and outputs accurate predictions of fruit sugar content, acidity, and firmness. A weighted hierarchical algorithm is used to fuse the internal quality prediction results: First, based on fruit variety characteristics and market demand, corresponding weight coefficients are assigned to sugar content, acidity, and firmness; second, the prediction results of each indicator are mapped to standardized scores; finally, a weighted summation is used to calculate the comprehensive quality score, and quality grades are divided based on the comprehensive score, ultimately outputting four quality classification results: excellent, good, acceptable, and substandard.

[0070] The system control layer achieves real-time sorting through an integrated control algorithm. The quality grade signal output by the model is parsed by the central control module, automatically generating robotic arm paths and sorting instructions. The robotic arm executes corresponding transfer actions based on the grade instructions, placing the fruit into the corresponding storage boxes, thus achieving closed-loop control from harvesting to grading. This process requires no manual intervention; the control module dynamically adjusts the operating force and angle based on sensor data, ensuring high-precision sorting while maintaining the continuity of the harvesting rhythm.

[0071] The overall system operates on a collaborative closed loop of structure, perception, and intelligent decision-making. During its movement, the robotic arm simultaneously acquires information and performs execution functions, modeling spectral signals, and the model output drives the mechanical system to achieve adaptive sorting. By coupling sensing, algorithms, and control onto the same mechanical platform, this invention breaks away from the traditional model where detection and harvesting are independent, forming a fully automated quality inspection and grading solution for the post-harvest processing of intelligent agricultural products. It can be directly applied to fruit and vegetable harvesting production lines, achieving seamless integration from on-site harvesting to quality classification, and has significant industrial promotion value.

[0072] Example 1

[0073] like Figure 1 As shown in the figure, the wearable multi-point quality detection method for fruit based on a robotic arm provided by this invention includes the following steps:

[0074] S101, Harvesting and Inspection Collaboration: When the robotic arm performs fruit harvesting actions, it simultaneously triggers the multi-module inspection system (including pressure detection module and internal quality inspection module) worn on the robotic arm.

[0075] S102, Multi-source data acquisition; Pressure data acquisition and gripping control: A pressure detection module worn on the robotic arm collects pressure signals in real time during the gripping process and feeds the data back to the control system. Based on a preset "safe gripping threshold" (ensuring a firm grip on the fruit while preventing crushing damage), the control system stops the robotic arm from gripping when the pressure reaches the set value.

[0076] Multi-point spectral data acquisition: Through 13 spectral detection modules pre-installed on the robotic arm (specifically, 4 modules are deployed at each of the three fingertips and 1 module is deployed at the suction cup end), the raw spectral data of 13 detection points on the surface of the fruit are collected simultaneously.

[0077] S103, Data transmission, verification, anomaly removal and averaging; 1. Multimodal data transmission and verification: The acquired raw spectral data (including 13 spectra) is uniformly encapsulated with a verification field and transmitted to the control module; The receiving end verifies the data integrity through the verification field. If data loss or error is found, a retransmission mechanism is initiated to handle the transmission anomaly and ensure data reliability.

[0078] Outlier removal: The Mahalanobis distance (MD) algorithm was used to clean the 13-channel raw spectral data and remove outliers; the Mahalanobis distance calculation formula is as follows:

[0079] DM(x)=(x−μ)TΣ−1(x−μ)

[0080] Where x is the feature vector of a single spectrum, μ is the mean vector of 13 spectrum data, and Σ is the covariance matrix of 13 spectrum data; a Mahalanobis distance threshold is set (calibrated experimentally), and when the Mahalanobis distance of a certain spectrum data exceeds the threshold, it is judged as abnormal data and removed.

[0081] Spectral data averaging: The average value of the effective multi-channel spectral data after removing outliers is calculated to obtain average spectral data, which reduces single-point detection error and improves the representativeness of the data.

[0082] S104, Model Building and Sorting.

[0083] Obtain the spectral data of the apple to be tested;

[0084] Spectral data preprocessing; multivariate scattering correction of the spectral data of the apples to be tested; linear regression of the original spectrum A_raw of each apple with the reference spectrum A_ref: A_raw = k × A_ref + b + ε (ε is the residual) to obtain the parameters k and b, and through linear transformation A_corrected = kA_raw − b, the spectrum of each sample is corrected to a form "aligned" with the reference spectrum to obtain the corrected spectral data.

[0085] Feature wavelength extraction: Principal component analysis (PCA) is used for dimensionality reduction: the spectral data is decomposed by PCA, the main principal components whose cumulative contribution rate meets the preset threshold are retained, and redundant information is removed; then, based on the loading coefficient of each main principal component, the wavelength position corresponding to the local extreme value of the loading coefficient is located, and the feature wavelength range sensitive to the internal quality of the fruit (sugar content, acidity, firmness) is selected to form a simplified spectral feature set.

[0086] Internal quality prediction model construction: Using the extracted spectral feature set as input, an internal quality prediction model based on partial least squares discriminant analysis (PLS-DA) is constructed; the model is trained and optimized through training samples, and the model outputs accurate prediction results of the sugar content, acidity and firmness of the fruit.

[0087] Multi-level quality fusion judgment: The internal quality prediction results are fused using a weighted grading algorithm: First, based on the characteristics of fruit varieties and market demand, corresponding weight coefficients are set for sugar content, acidity, and firmness; second, the prediction results of each indicator are mapped to standardized scores; finally, the comprehensive quality score is calculated by weighted summation, and the quality grades are divided according to the comprehensive score, ultimately outputting four quality classification results: excellent, good, qualified, and substandard.

[0088] Sorting: After receiving the quality grade results from the control system, sorting control commands are automatically generated to drive the robotic arm to perform graded transfer operations. Based on the preset storage box location coordinates, the robotic arm accurately places the fruit into the storage box corresponding to the quality grade, completing the fully automated operation from picking and testing to sorting.

[0089] Example 2

[0090] This embodiment is used to illustrate the construction process and detection effect of the quality inspection model provided in Embodiment 1 above.

[0091] Due to the influence of the detection instrument itself and various external factors, the near-infrared spectra of collected fruits often contain irrelevant interference information such as dark current, stray light, and baseline drift. Therefore, preprocessing is required before modeling to eliminate or reduce irrelevant signal interference (Cao Songtao 2017). Wang et al. (2025) pointed out that near-infrared light produces a significant multivariate scattering effect when propagating in apple pulp. This scattering is not only affected by the density of pulp cells and tissue structure, but also by the larger the fruit diameter, the longer the propagation path of light in the pulp. The superposition of scattering results in the masking of characteristic information related to internal quality indicators (such as sugar content and soluble solids content) in the spectrum. To eliminate the scattering interference caused by differences in fruit diameter, multivariate scattering correction (MSC) is needed to preprocess the original spectra of apples of different diameters.

[0092] For each apple, a linear regression of the raw spectrum Araw with the reference spectrum Aref yields the spectral intensity scaling factor and the spectral baseline shift factor, described by the following formula:

[0093] A raw = k × A ref + b + ε

[0094] In the formula, Araw represents the original spectrum of the apple sample, Aref represents the reference spectrum, k represents the spectral intensity scaling factor of the sample, b represents the spectral baseline shift factor of the sample, and ε represents the regression residual.

[0095] The corrected spectral data of apples can be obtained using the MSC correction formula, which is:

[0096] Acorrected=kAraw−b,

[0097] In the formula, Acorrected represents the spectral intensity of the sample after correction considering the transverse diameter information.

[0098] Each sample spectrum is corrected to be "aligned" with the reference spectrum, thereby removing intensity and baseline interference from scattering and retaining the characteristic absorption information corresponding to the chemical composition, resulting in the MSC-corrected spectral data.

[0099] The continuous projection algorithm projects spectral data into a vector space, selecting a set of variables with the lowest redundancy, reducing the dimensionality of the data, and minimizing the collinearity among the selected spectral variables (Liu Jie 2011). Using the PCA class from the scikit-learn library, cluster analysis is performed on the corrected spectral data, retaining the first principal component (PC1, variance contribution rate ≥ 85%) and the second principal component (PC2, variance contribution rate ≥ 10%), replacing the original high-dimensional data with low-dimensional data. The X-loading coefficient method is used to select characteristic wavelengths: wavelengths with an absolute loading coefficient > 0.8 are selected as characteristic wavelengths for the internal quality (sugar content, acidity, and firmness) of apples.

[0100] After extracting the characteristic wavelengths, two machine learning algorithms, PLS-DA and SVM-GA, were used to construct internal quality prediction models for apples for single-point sampling and multi-point sampling, respectively, to compare the discrimination accuracy and verify the effectiveness of the method provided in Example 1.

[0101] Example 3

[0102] like Figure 2 As shown, the present invention provides a wearable multi-point quality inspection device for fruits based on a robotic arm, which is based on the wearable multi-point quality inspection method for fruits based on a robotic arm in Embodiment 1, and includes a robotic arm module, a quality inspection module, a control module and a quality sorting module.

[0103] The robotic arm module is an industrial-grade four-axis robotic arm, equipped with a three-jaw suction cup manipulator, and the suction cup unit is detachable;

[0104] The pressure detection module adopts a wearable design, with the pressure detection module (SFR402 sensor) attached to the fingertip;

[0105] The quality inspection module adopts a wearable design, with the internal quality inspection module (including AS7343 / AS7341 chips and halogen lamps) embedded in the robotic arm's suction cup and worn on the fingertips.

[0106] The control module is shared with the robotic arm control computer and includes power supply, data acquisition, transmission and anomaly rejection, model calculation and sorting control unit;

[0107] The quality sorting module contains four storage boxes corresponding to different quality grades, with soft foam lining the inner walls to prevent damage.

[0108] The device's collaborative working logic is as follows: When the robotic arm picks fruit, the pressure detection module collects pressure data and provides real-time feedback to control the gripping force of the robotic arm, ensuring that the quality detection module is in close contact with the apple surface. The quality detection module collects data synchronously, and the control module receives the data, removes anomalies, and outputs the quality result after model calculation. Subsequently, the sorting program drives the robotic arm to transfer the fruit to the corresponding grade storage box, realizing the integrated operation of "picking-detection-sorting".

[0109] like Figures 3 to 5 As shown, the overall structure of this invention consists of a robotic arm 1, a robotic hand 2, a detachable silicone suction cup 3, an embedded computer 4, and a multi-sensor detection assembly. The robotic arm 1 is rigidly connected to the robotic hand 2 via an end flange, forming an actuator capable of multi-degree-of-freedom rotation. The robotic hand 2 has a detachable silicone suction cup 3 at its front end, which integrates an adsorption mechanism and a spectral detection system. The suction cup 3 is electrically connected to the robotic fingertip via a suction cup end circuit board 5. The suction cup end circuit board 5 is connected to a flexible fingertip circuit 6 via a flexible flat cable. The flexible fingertip circuit 6 is further connected to the signal interface of the embedded computer 4, enabling synchronous data and control transmission. The robotic hand is covered with an anti-slip material to ensure gripping stability. The entire detection system is installed at the end of the robotic arm 1 and can perform synchronous detection tasks during the harvesting process.

[0110] The suction cup consists of a rigid outer shell 301 forming the main support frame. The bottom of the shell is fixed to the robotic arm end by internal threaded bolts 302, and the top is locked to the soft adsorption structure by a nut post 303. The outer wall of the shell has a raised strip 304 for sliding engagement with a nested light shield 305, thus creating a stable local light-shielding environment during spectral detection. A halogen lamp 503 is installed inside the suction cup to provide constant reflected illumination. AS7343 and AS7341 spectral detection chips 501 and 502 are embedded together on the suction cup end circuit board 5 and fixed in position by screw holes 504, forming a dual-band visible and near-infrared light acquisition unit. The inner wall of the suction cup uses a flexible buffer layer to maintain a constant spectral measurement angle when adsorbing onto the fruit surface, ensuring the stability and repeatability of the reflected signal.

[0111] The robotic fingertip is equipped with a flexible circuit 6 for connecting the quality inspection module 7 and the pressure detection module 8. The pressure detection module 8 consists of a resistive thin-film pressure sensor 801, embedded beneath the anti-slip silicone pad 601. When the robotic arm contacts the fruit surface, the sensor senses the contact force and converts it into an electrical signal. The flexible circuit 6 features Velcro straps 602 and anti-slip silicone pad holes 603 to stabilize and constrain the circuit and sensing unit, preventing slippage or signal interference during high-speed robotic arm movement. This structural layout allows for synchronous signal acquisition throughout the entire robotic arm movement process.

[0112] Upon receiving a control command, robotic arm 1 drives robotic hand 2 to move to the target fruit location and activates the suction mechanism. When suction cup 3 contacts the fruit surface, the flexible silicone layer achieves compliant adhesion. Pressure detection module 8 provides real-time feedback of pressure data, stopping the grip when a threshold is reached. Halogen lamp 503 illuminates the fruit surface, generating reflected light. AS7343 chip 501 collects 400-700 nm visible light signals, and AS7341 chip 502 collects 650-1100 nm near-infrared signals. All signals are transmitted to embedded computer 4 via flexible fingertip circuit 6. After data verification, abnormal data removal, and averaging, embedded computer 4 performs quality judgment based on a preset algorithm and feeds the results back to the control system. Robotic arm 1 then executes the graded transfer action, achieving a collaborative closed loop between detection and harvesting.

[0113] The purpose of this invention is to provide a wearable multi-point quality inspection method and device for fruits based on a robotic arm. This invention can simultaneously acquire quality data, flexibly adapt to the inspection of multiple varieties of fruits, improve the accuracy and efficiency of inspection, and overcome the limitations of traditional inspection technologies.

[0114] This embodiment takes apple quality inspection as the application scenario. Based on the integrated needs of "harvesting-inspection-sorting", it adopts an industrial-grade four-axis robotic arm equipped with a wearable internal quality inspection module. Combined with the PLS-DA model and weighted grading algorithm, it realizes the simultaneous detection and grading of the internal maturity (sugar content, acidity and firmness) of apples.

[0115] like Figure 3 , Figure 4 The main body of the robotic arm is a four-axis industrial robotic arm 1 with a load capacity of 3kg, a repeatability of ±0.03mm, and a working radius of 600mm, capable of multi-degree-of-freedom picking, sorting, and transfer actions. The robotic arm module is a customized three-claw suction cup type robotic arm 2 with the following parameters: Number of claws: 3, symmetrically distributed at 120°; Suction cup unit: detachable silicone suction cup 3, made of black food-grade silicone, with a rigid suction cup shell 301, M5 internal threaded bolts 302 at the bottom, and four M3 nut posts 303 (for fixing the spectral detection module) and four protruding strips 304 for nesting a light shield 305 at the top. The control computer module is an industrial-grade embedded computer 4.

[0116] The quality inspection module is equipped with an internal quality inspection unit (spectral detection module).

[0117] like Figure 5 The core chip combination uses the AS7343 visible spectrum chip 501 and the AS7341 near-infrared spectrum chip 502. Specific parameters are as follows: The AS7343 and AS7341 chips have different focuses in optical detection parameters, but they are also closely related. The AS7343 covers a detection band of 400-700nm and is equipped with 6 detection channels, each with a bandwidth of 20-25nm; while the AS7341 extends its detection band to 650-1100nm, has 8 more detection channels, and its channel bandwidth is slightly wider, at 25-30nm.

[0118] Light source 503: 4 halogen lamps, illuminating at a 45° angle to the vertical direction, diagonally distributed on both sides of the chip. Light-shielding component 305: A black curved silicone light-shielding cover, nestled within the raised strip of the suction cup housing, to prevent ambient light interference.

[0119] Suction cup end circuit board 5: Custom FR4 material circuit board, integrating chip driver circuit, light source control circuit and signal amplification circuit. There are 4 Φ3mm holes 504 at the four corners of the circuit board, which are fixed to the nut post of the suction cup shell by M3 screws.

[0120] Finger tip circuit board: It adopts a custom flexible circuit 6, which integrates chip driving circuit, light source control circuit and signal amplification circuit. The flexible circuit fits the entire fingertip and is covered with anti-slip silicone.

[0121] like Figure 7 The fingertip anti-slip module uses anti-slip silicone 601, which is applied to the surface of the flexible circuit using hot melt adhesive. The fixing module uses Velcro straps 602, which pass through the holes 603 on both sides of the anti-slip silicone sheet to secure the sensor to the mechanical fingertip.

[0122] Installation location: 1 detachable suction cup integrated spectral module (fixed to the center of the robotic hand palm) + 3 wearable finger sleeves integrated spectral modules (fixed to the fingertips of three fingers respectively), a total of 13 spectral detection units (1 group in the palm + 4 groups in each fingertip, each group containing 1 set of AS7343 / AS7341 chip and 4 halogen lamps), to achieve multi-point spectral acquisition on the surface of the apple.

[0123] Pressure detection module 8: SFR402 resistive thin-film pressure sensor 801, detection range 0-10N, resolution 0.01N, response time ≤10ms, operating temperature -20-60℃, dimensions 20mm×20mm×0.2mm. Located between the two sensors at the upper and lower fingertips of the fingertip spectral detection module, sharing the anti-slip module and fixing module of the wearable finger sleeve.

[0124] Number of sensors installed: Two sensors are fixed to each fingertip, for a total of six, to ensure synchronous contact with the apple surface during picking.

[0125] like Figure 8 , Figure 9 The control module is shared with the robotic arm's control computer, and its core components and functions are as follows:

[0126] Power Supply Unit: Employs a dedicated lithium battery for robotic arms, paired with a DC-DC voltage converter, to provide stable power to the spectral and pressure modules, ensuring voltage fluctuations ≤ ±0.1V. Spectral Signal: Converts analog signals to digital signals using an ADS1256 analog-to-digital converter; Pressure Signal: Converts resistance change signals to digital signals using an HX711 load cell amplifier; Data Standardization: A LabVIEW program uniformly converts the three types of data into JSON format, with a sampling frequency set to 1Hz.

[0127] Model program unit: developed based on Python 3.8, integrating the internal quality model (PLS-DA), and weighting the predicted results (sweetness, acidity and hardness) into four levels: excellent, good, qualified and substandard.

[0128] Sorting program unit: Based on the quality level output by the model, generate robotic arm action instructions: Excellent: The robotic arm rotates 90°, the claw opening degree is 80%, and the apple is placed into storage box 1; Good: The robotic arm rotates 180°, the claw opening degree is 80%, and the apple is placed into storage box 2; Acceptable: The robotic arm rotates 270°, the claw opening degree is 80%, and the apple is placed into storage box 3; Defective: The robotic arm rotates 0° (in place), the claw opening degree is 100%, and the apple is placed into storage box 4.

[0129] The quality sorting module includes: storage boxes: 4 rectangular plastic boxes, each corresponding to 4 quality grades. The inner walls of the boxes are lined with 5mm thick soft foam to prevent apples from being damaged by collision; layout: distributed in a ring at 90° intervals around the robotic arm.

[0130] Figure 10. III. Implementation Steps of the Detection Method

[0131] (I) Step 1: Equipment initialization and calibration

[0132] Robotic arm calibration: By setting the zero point position of the robotic arm, the picking point (50mm from the fruit surface), the detection point (20mm from the center of the apple to the spectral module), and the sorting point (each storage box) are calibrated, with a repeatability accuracy error of ≤0.1mm.

[0133] Spectral Module Calibration: Standard Baseline Acquisition: With the fruit placed close to the light source in the absence of light, below the spectral detection module, the raw response values ​​of AS7343 (400-700nm, 10nm interval) and AS7341 (650-1100nm, 15nm interval) are collected as the "standard baseline value"; Real-time Calibration: During the detection process, the raw response value is re-acquired after every 100 samples. If the deviation from the standard baseline value is >4%, the halogen lamp drive voltage is finely adjusted through the control circuit (adjustment range 3-5V) until the deviation is ≤2%.

[0134] (II) Step 2: Multi-source data acquisition

[0135] 1. Picking and positioning: The robotic arm moves to the apple picking point according to the preset program, grabs the apple with three claws, and performs inspection at the same time (keeping the apple in close contact with the quality inspection module).

[0136] 2. Spectral data acquisition: The halogen lamps of 13 spectral modules are turned on, and the AS7343 / AS7341 chip synchronously acquires diffuse reflectance spectral data of 13 points on the surface of the apple. The acquisition time is 0.5s, and the data is transmitted to the control computer through the circuit board.

[0137] 3. Pressure data acquisition: Three pressure sensors collect pressure signals on the apple surface in real time, with a sampling frequency of 10Hz and a total of 10 data points. The data is transmitted to the control computer through the HX711 amplifier to provide real-time feedback and control the gripping force of the mechanical fingers.

[0138] 4. Data storage: The control computer stores the three types of data on the hard drive in the format of "sample ID-collection time-spectral data".

[0139] (III) Step 3: Spectral data anomaly removal and averaging

[0140] 1. Spectral data anomaly removal

[0141] The Mahalanobis distance (MD) algorithm was used to clean the raw data of 13 spectra and remove abnormal data. The Mahalanobis distance calculation formula is: DM(x)=(x−μ)TΣ−1(x−μ). A Mahalanobis distance threshold was set (calibrated experimentally). When the Mahalanobis distance of a certain spectra exceeds the threshold, it is judged as abnormal data and removed.

[0142] 2. Averaging of Spectral Data

[0143] The average value is calculated for the effective multi-channel spectral data after removing outliers to obtain average spectral data, thereby reducing single-point detection error and improving the representativeness of the data.

[0144] (iv) Step 4: Quality Inspection Model Training and Prediction

[0145] 1. Spectral data preprocessing and feature extraction

[0146] MSC correction: The spectral data of the apples to be tested are corrected by multivariate scattering. For the original spectrum Araw of each apple, a linear regression is performed with the reference spectrum Aref: Araw = k × Aref + b + ε (ε is the residual) to obtain the parameters k and b. Through the linear transformation Acorrected = kAraw − b, the spectrum of each sample is corrected to a form that is "aligned" with the reference spectrum to obtain the corrected spectral data.

[0147] PCA dimensionality reduction: The PCA class from the scikit-learn library was used to reduce the dimensionality of the corrected spectral data, retaining the first principal component (PC1, variance contribution rate ≥ 85%) and the second principal component (PC2, variance contribution rate ≥ 10%), replacing the original high-dimensional data with the low-dimensional data. X-loading coefficient method for selecting characteristic wavelengths: Wavelength points with an absolute loading coefficient > 0.8 were selected as characteristic wavelengths for the internal quality (sugar content, acidity, and firmness) of apples.

[0148] 2. Internal Quality Model (PLS-DA) Training and Prediction

[0149] Sample Splitting: 1000 apple samples were selected (covering four grades: excellent, good, qualified, and defective, 250 of each), and divided into a training set (700 samples) and a prediction set (300 samples) in a 7:3 ratio. The input features were the selected spectral wavelength data. Model Training: The PLS-DA model was trained using the training set to fit the mapping relationship between the apple spectral features and internal maturity (sugar content, acidity, and firmness), and the predicted internal quality results were output.

[0150] (V) Step 5: Quality Judgment and Sorting

[0151] Multi-level quality fusion judgment: The internal quality prediction results are fused using a weighted grading algorithm: First, based on the characteristics of fruit varieties and market demand, corresponding weight coefficients are set for sugar content, acidity, and firmness; second, the prediction results of each indicator are mapped to standardized scores; finally, the comprehensive quality score is calculated by weighted summation, and the quality grades are divided according to the comprehensive score, ultimately outputting four quality classification results: excellent, good, qualified, and substandard.

[0152] Sorting: The computer-controlled sorting program generates robotic arm movement instructions (rotation angle, gripper opening and closing degree, movement speed) based on the overall quality grade. The robotic arm moves to the corresponding storage box according to the instructions, the gripper opens (opening degree 80%-100%), and the apples are placed into the storage box, completing the sorting; after sorting, the robotic arm returns to the picking point to prepare for the next sample test.

[0153] Specific application areas or related products of this invention:

[0154] (a) Specific application areas

[0155] This invention is primarily applied in the field of non-destructive testing and automated equipment technology for agricultural products, focusing on key links in the fruit planting and processing industry chain. Specifically, it can be divided into the following scenarios: First, for large-scale fruit orchards, it enables immediate quality testing and grading of spherical / near-spherical fruits such as apples and pears after harvesting, solving the problems of low efficiency, high misjudgment rate, and high damage rate caused by traditional manual methods and transportation. It achieves an integrated "harvesting-testing-sorting" process, meeting the needs of high-efficiency processing. Second, it is used in the pre-processing stage of fruit processing enterprises, providing raw material screening services for processing into juice, dried fruit, etc., by accurately detecting the internal maturity, external defects, and texture hardness of fruits, ensuring product quality and reducing labor costs and raw material waste. Third, it assists in the research and development and production of agricultural product quality testing equipment, providing manufacturers with a core technology solution of "wearable multi-point testing," promoting the upgrade of testing equipment from "single-point" to "multi-point integration."

[0156] (ii) Related Products

[0157] The specific related products that can be developed or adapted based on the technical solution of this invention are as follows:

[0158] Wearable fruit picking and inspection integrated robotic arm: Based on an industrial-grade four-axis robotic arm, it integrates an AS7343 / AS7341 dual-chip spectral detection module and an SFR402 pressure detection module, and is equipped with a detachable suction cup (adapted to different fruit sizes) and a grading storage box, which can be directly applied to automated operations in orchards or sorting centers.

[0159] Fruit quality testing module (modular product): The spectral detection unit in this invention is designed as a modular component, which can be adapted to existing fruit picking robotic arms or sorting lines. It can upgrade and transform traditional equipment without replacing the entire set of equipment, reduce the equipment replacement cost of enterprises, and meet the technical upgrade needs of small and medium-sized plantations or processing enterprises.

[0160] Evidence related to the technical effects obtained by the embodiments of the present invention:

[0161] Based on the wearable multi-point quality inspection method, system, and computer program for fruits based on a robotic arm in this embodiment, a quality inspection device was constructed. The inspection method and system of this embodiment were verified using the constructed non-destructive testing device. Internal quality inspection (GA-SVM): the multi-point detection accuracy (0.815) was improved by 2.9% compared to single-point detection (0.786). Internal quality inspection (PLS-DA): the multi-point detection accuracy (0.866) was improved by 4.0% compared to single-point detection (0.826). This confirms the rationality of the design of this invention, which covers multiple sampling points on the fruit surface using multiple sets of spectral units, and solves the problem of "local data deviation leading to misjudgment of quality" in traditional single-point sampling. The problems are as follows: the internal quality inspection (PLS-DA) improves the single-point and multi-point sampling by 6% and 5% respectively compared with the internal quality inspection (GA-SVM), which confirms the advantages of the model constructed by PLS-DA in this invention and shows that the wearable fruit multi-point quality inspection method and system based on the robotic arm is feasible; however, some errors in the prediction of this system cannot be ignored. The reason for the error may be that the external temperature interferes with the detection process, so the temperature correction of the system still needs further research.

[0162] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as an embedded microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented using hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or can be implemented using software executed by various types of processors, or can be implemented using a combination of the above-described hardware circuitry and software, such as firmware.

[0163] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A wearable multi-point quality inspection method for fruits based on a robotic arm, characterized in that, include: While the robotic arm is performing fruit picking, the internal quality detection module worn on the surface of the robotic arm acquires multi-point spectral data of the contact points on the fruit surface. During the process of the robotic arm grasping the fruit, the contact pressure is obtained by the pressure detection module, and the robotic arm is controlled to stop grasping based on the pressure feedback so that the internal quality detection module can fit the fruit surface. The raw spectral data collected from multiple points are subjected to transmission verification, anomaly removal, and averaging to obtain the average spectrum; The average spectrum is input into the prediction model to obtain the predicted results of fruit sugar content, acidity and firmness; Based on the quality results, the robotic arm is controlled to sort the fruits into the corresponding grade storage boxes, realizing a continuous process of picking, testing and sorting.

2. The method according to claim 1, characterized in that, The method for anomaly removal of spectral data is as follows: using a single spectrum as a feature vector, the distance metric formed by the single spectrum and the mean feature vector and corresponding covariance matrix of multiple spectra is used as an anomaly evaluation index. When the statistical distance between a single spectrum and the mean vector is greater than a preset threshold, the spectral data is judged as an anomaly and removed.

3. The method according to claim 1, characterized in that, The input spectrum of the prediction model is processed as follows before being input: A linear relationship is fitted between the original spectrum and the reference spectrum to obtain the spectral intensity scaling factor and the spectral baseline offset. These two factors are then used to correct the intensity and baseline of the original spectrum to eliminate spectral deviations caused by scattering. The corrected spectrum is subjected to dimensionality reduction analysis. Principal components with cumulative contribution rates reaching a preset ratio are selected, and characteristic wavelength ranges are determined based on the extreme loading positions of the principal components to form a simplified spectral feature set.

4. The method according to claim 1, characterized in that, Fruit quality grades are determined by weighted summation of predicted sugar content, acidity, and firmness to obtain a comprehensive score, which is then divided into multiple quality grades according to a preset range.

5. A wearable multi-point quality inspection device for fruits based on a robotic arm, characterized in that, It includes a robotic arm module, a pressure detection module, an internal quality inspection module, and a control module; The pressure detection module is installed on the surface of the mechanical finger tip and is used to detect the contact pressure on the fruit surface; The internal quality inspection module is located at the tip of the mechanical finger and the palm and is used to collect multiple spectral data in the visible and near-infrared bands. The control module is used to perform spectral data acquisition, data transmission, anomaly removal, spectral processing, prediction calculation, and sorting control.

6. The apparatus according to claim 5, characterized in that, The pressure detection module includes a pressure detection unit, an anti-slip cover layer, and a fixing structure for fixing the detection module to the tip of the robotic finger; the anti-slip cover layer covers the surface of the pressure detection unit, and the fixing structure is fastened to the tip of the robotic finger to ensure that the pressure detection module is stably attached to the surface of the robotic hand.

7. The apparatus according to claim 5, characterized in that, The internal quality inspection module includes a wearable finger-type spectral detection structure, which includes a flexible circuit board, an anti-slip layer for covering the surface of the circuit board, and a fixing structure set on the back of the anti-slip layer. A light source and a spectral detection unit are set on the flexible circuit board, and the light sources are arranged diagonally and illuminate the surface of the fruit at an angle.

8. The apparatus according to claim 5, characterized in that, The internal quality inspection module also includes a detachable suction cup-type spectral detection structure, which includes an adsorption unit, a rigid housing, a circuit board, and a light-shielding component. The adsorption unit is used to fix the structure to the palm of the robotic arm, and the rigid housing has mounting posts for fixing the spectral detection unit and a limiting structure for embedding the light-shielding component.

9. The apparatus according to claim 5, characterized in that, The control module includes a power conversion unit, a data acquisition unit, a data transmission and verification unit, a spectral preprocessing unit, a prediction model calculation unit, and a sorting execution unit; The power conversion unit converts the power supply to the robotic arm into a stable voltage required by the internal quality inspection module. The data transmission and verification unit encapsulates and verifies the spectral data and removes incomplete data using anomaly rules; The prediction model calculation unit outputs the fruit quality grade result, and the sorting execution unit controls the robotic arm to complete the sorting based on the result.

10. An integrated system for fruit harvesting, quality inspection, and quality sorting, characterized in that, This includes a robotic arm harvesting mechanism, a wearable multi-point spectral data acquisition mechanism, a pressure feedback control mechanism, and an automatic sorting mechanism; The wearable multi-point spectral data acquisition mechanism acquires multiple spectral data when the fruit is grasped and attached; The pressure feedback control mechanism is used to keep the spectral acquisition mechanism in contact with the fruit at a pressure that does not damage it. The automatic sorting mechanism places the fruit into the corresponding storage location based on the quality prediction results, realizing the linkage between harvesting, internal quality inspection and quality sorting.

Images