Integrated screening and sorting method for litchi based on embodied robot

By combining embodied robots with multimodal detection models, the appearance and internal quality parameters of lychees can be acquired simultaneously, solving the problems of low grading accuracy and poor detection efficiency in lychee sorting. This enables fully automated sorting throughout the entire process, adapting to flexible sorting of various fruits and vegetables and food-grade clean production.

CN123102633APending Publication Date: 2026-11-17广东海川机器人有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611084401.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In the lychee production line sorting operation, the appearance and internal quality inspection are separated, resulting in low grading accuracy and poor inspection efficiency. The parameters of the initial screening and secondary fine sorting stations are disconnected, making it impossible to achieve continuous sorting throughout the entire process. It is also difficult to balance sorting efficiency with food-grade clean production requirements.

Method used

The embodied robot is used for the integrated sorting of lychees, including initial screening and fine sorting. The multimodal fusion detection model simultaneously acquires appearance and internal quality parameters, and combines these parameters to perform coarse screening and fine grading actions, achieving fully automated sorting throughout the entire process and meeting food-grade clean production standards.

Benefits of technology

It improves grading accuracy and detection efficiency, realizes full-process automation of primary screening and fine sorting, adapts to the flexible sorting needs of multiple types of fruits and vegetables, and meets food production compliance requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123102633A_ABST
    Figure CN123102633A_ABST
Patent Text Reader

Abstract

This application relates to the field of intelligent sorting technology for agricultural products, and provides an integrated sorting method for lychees based on an embodied robot, combining initial screening and fine sorting. The method collects visual and near-infrared spectral data of lychees entering the sorting line to obtain corresponding appearance and internal quality parameters. Based on the appearance quality parameters, a coarse screening process is performed at the initial screening station to obtain basic variety information of the lychees. Lychees that do not meet the preset basic quality requirements are then removed, completing the coarse screening process. The lychees that have passed the coarse screening are then transported to a preset fine sorting station, where a fine grading process is performed based on the appearance and internal quality parameters, completing the integrated initial screening and fine sorting of the lychees. By using multimodal visual and spectral data fusion detection, the problem of asynchronous internal and external quality detection is solved, effectively improving grading accuracy and detection efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of intelligent sorting technology for agricultural products, and in particular to an integrated sorting method for lychees based on embodied robots, which combines primary screening and fine sorting. Background Technology

[0002] Currently, the automated sorting process for lychees still has the following shortcomings: First, the appearance and internal quality inspection processes are separated, making it impossible to simultaneously obtain appearance indicators such as fruit shape and defects, as well as internal indicators such as sugar content and ripeness, resulting in low grading accuracy and poor inspection efficiency. Second, the sorting parameters at the preliminary screening and secondary fine sorting stations are disconnected, making it impossible to complete the entire process of sorting based on unified inspection results, resulting in insufficient overall automation. Third, the sorting execution process struggles to balance sorting efficiency with food-grade clean production requirements, making it unable to quickly adapt to the flexible sorting needs of multiple types of fruits and vegetables. Summary of the Invention

[0003] This application provides an integrated sorting method for lychee primary screening and fine sorting based on embodied robots, which aims to solve the shortcomings of current lychee assembly line sorting operations.

[0004] In a first aspect, embodiments of this application provide an integrated sorting method for lychee primary screening and fine sorting based on an embodied robot, the method comprising: Collect visual and near-infrared spectral data of lychees entering the sorting line; input the visual and near-infrared spectral data into a pre-trained multimodal fusion detection model to obtain the corresponding lychee appearance quality parameters and internal quality parameters. Based on the appearance quality parameters, a coarse screening action is performed at the primary screening station to obtain the basic variety information of lychees. Lychees that do not meet the preset basic quality are removed based on the basic variety information to complete the coarse screening action. The lychees that have passed the coarse screening process are transported to a preset fine sorting station. A fine grading process is performed by combining appearance quality parameters and internal quality parameters. The quality score information corresponding to the lychees is obtained based on the appearance quality parameters and internal quality parameters. The lychees are then divided into the corresponding quality grade range based on the quality score information, thus completing the integrated sorting of the lychees from initial screening to fine sorting.

[0005] In some embodiments, the acquisition of visual data and near-infrared spectral data of lychees to be sorted entering the sorting line includes: identifying the position information of the lychees to be sorted on the conveyor line; and when the lychees to be sorted reach the detection station, simultaneously triggering the high-definition image acquisition unit and the near-infrared spectral acquisition unit to acquire visual data and near-infrared spectral data of a single lychee to be sorted, respectively.

[0006] In some embodiments, the step of inputting appearance visual data and near-infrared spectral data into a pre-trained multimodal fusion detection model to obtain the appearance quality parameters and intrinsic quality parameters of the corresponding lychee includes: extracting features from the appearance visual data to obtain appearance dimension feature data; extracting features from the near-infrared spectral data to obtain intrinsic dimension feature data; performing spatiotemporal alignment and feature fusion of the appearance dimension feature data and intrinsic dimension feature data; inputting the data into the multimodal fusion detection model for inference; and outputting the appearance quality parameters and intrinsic quality parameters of the corresponding lychee respectively.

[0007] In some embodiments, the step of performing coarse screening at the primary screening station based on appearance quality parameters to obtain basic variety information of lychees includes: identifying and comparing the fruit shape, skin texture and shell color features in the appearance visual data, matching the benchmark data in the preset variety feature library, and obtaining the basic variety information of the corresponding lychee.

[0008] In some embodiments, the step of removing lychees that do not meet the preset basic quality based on the basic variety information to complete the coarse screening and sorting action includes: calling the basic quality threshold of the corresponding variety based on the basic variety information, comparing the appearance quality parameters with the corresponding basic quality threshold, determining that lychees with appearance quality parameters lower than the basic quality threshold are unqualified lychees, and controlling the actuator of the primary screening station to push the unqualified lychees to the defective material channel to complete the coarse screening and sorting action.

[0009] In some embodiments, the step of conveying the lychees that have passed the coarse screening to a preset fine sorting station, performing a fine grading action by combining appearance quality parameters and internal quality parameters, and obtaining the quality score information corresponding to the lychees based on the appearance quality parameters and internal quality parameters includes: matching the grading weight configuration according to the basic variety information of the corresponding lychees, assigning corresponding weight coefficients to the appearance quality parameters and internal quality parameters respectively, and performing a weighted calculation on the appearance quality parameters and internal quality parameters according to the weight coefficients to obtain the quality score information of the corresponding lychees.

[0010] In some embodiments, the step of classifying lychees into corresponding quality grade intervals based on the quality scoring information to complete the integrated sorting of lychees for initial screening and fine sorting includes: comparing the acquired quality scoring information with multiple preset quality grade interval thresholds one by one to determine the target quality grade interval into which the quality scoring information falls; controlling the actuator of the fine sorting station to transport the corresponding lychees to the discharge channel corresponding to the target quality grade interval to complete the integrated sorting of lychees for initial screening and fine sorting.

[0011] In some embodiments, the method further includes: periodically statistically analyzing the distribution data of quality parameters of sorted lychees, combining the incoming material fluctuation characteristics of the current batch of lychees, dynamically adjusting the threshold range of each quality grade interval, synchronously updating the basic quality threshold of the primary screening station, and completing the adaptive calibration of the sorting standard.

[0012] In some embodiments, the method further includes: real-time acquisition of the conveying speed and incoming material density data of the sorting line; dynamic adjustment of the execution triggering sequence of the primary screening station and the fine sorting station based on the detection time and sorting execution time of a single lychee; matching the conveying rhythm of the line to ensure that the sorting action of a single lychee is synchronized with the corresponding conveying progress.

[0013] In some embodiments, the method further includes: collecting quality parameters and grading results data generated during the entire sorting process, combining them with verification results data generated by manual sampling, incrementally training the multimodal fusion detection model, optimizing the parameters of the multimodal fusion detection model, and improving the accuracy of the detection results.

[0014] This application has the following beneficial effects: 1. By fusing multimodal visual and spectral data, the appearance and internal quality parameters of lychees are output simultaneously, solving the problem of asynchronous internal and external quality detection and effectively improving grading accuracy and detection efficiency.

[0015] 2. Based on the unified test results, the initial screening and coarse sorting and fine sorting are performed sequentially to achieve a seamless and automated sorting process from initial screening to fine sorting, solving the problem of parameter disconnect between the two processes and improving the efficiency of collaborative operation of the production line.

[0016] 3. The entire process uses actuators that meet food-grade cleanliness standards to complete the conveying and sorting operations, meeting the compliance requirements of food production and adaptable to flexible sorting and switching of multiple types of fruits and vegetables.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart illustrating the steps of an integrated sorting method for lychee primary screening and fine sorting based on an embodied robot, provided in one embodiment of this application. Figure 2This is a schematic diagram illustrating the principle of an integrated sorting method for lychee primary screening and fine sorting based on an embodied robot, provided in one embodiment of this application. Figure 3 This is a schematic diagram of the embodied intelligent agent that executes the integrated sorting method for lychee primary screening and fine sorting based on embodied robots provided in an embodiment of this application. Figure 4 This is a schematic block diagram of a lychee primary screening and fine sorting integrated sorting system based on an embodied robot provided in one embodiment of this application; Figure 5 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0023] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0024] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] Currently, the automated sorting process for lychees still has the following shortcomings: First, the appearance and internal quality inspection processes are separated, making it impossible to simultaneously obtain appearance indicators such as fruit shape and defects, as well as internal indicators such as sugar content and ripeness, resulting in low grading accuracy and poor inspection efficiency. Second, the sorting parameters at the preliminary screening and secondary fine sorting stations are disconnected, making it impossible to complete the entire process of sorting based on unified inspection results, resulting in insufficient overall automation. Third, the sorting execution process struggles to balance sorting efficiency with food-grade clean production requirements, making it unable to quickly adapt to the flexible sorting needs of multiple types of fruits and vegetables.

[0027] Please refer to Figure 1 and Figure 2 This application provides an integrated sorting method for lychees based on embodied robots, encompassing initial screening and fine sorting, which can be run on computer-controlled equipment. The computer control equipment can be deployed in a local industrial control cabinet, a single server, or a server cluster, or embedded in a handheld terminal, industrial tablet, wearable device, or embodied intelligent execution device. All data acquisition, transmission, storage, and processing in this method are performed with the authorization of the relevant parties and in compliance with data security and food production standards, and do not involve any privacy violations.

[0028] The sorting system corresponding to this application comprises six components: a conveying unit, a multimodal detection unit, a primary screening unit, a fine sorting unit, a grading and discharge unit, and a control unit. All components in direct contact with lychees are made of materials that meet food contact safety standards, with smooth surfaces and no dead corners, allowing for direct cleaning and disinfection, meeting food-grade clean production requirements. The system can be adapted to the sorting needs of various spherical fruits and vegetables, such as lychees and longans, through parameter configuration without requiring modifications to the hardware structure. The conveying unit uses a horizontal conveyor belt structure, which transports the fruits and vegetables to be sorted sequentially to each workstation at individual intervals. The multimodal detection unit is located behind the feed section of the conveyor belt, used to simultaneously collect data on the appearance and internal quality of the fruits and vegetables. The primary screening unit is located behind the multimodal detection unit, used to remove obviously unqualified fruits and vegetables. The fine sorting unit is located behind the primary screening unit, used to complete multi-level fine sorting. The grading and discharge unit is located at the end of the conveyor belt, corresponding to discharge channels for multiple quality grades. The control unit is electrically connected to all the above units for data processing, logical decision-making, and motion control.

[0029] The provided integrated sorting method for lychees based on embodied robots, encompassing both initial screening and final sorting, includes steps S101 to S103. Details are as follows: Step S101. Collect visual data and near-infrared spectral data of the lychees to be sorted into the sorting line; input the visual data and near-infrared spectral data into the pre-trained multimodal fusion detection model to obtain the appearance quality parameters and internal quality parameters of the corresponding lychees.

[0030] Specifically, this step forms the basis of the entire inspection process. Its core is to collect two types of data—appearance and internal structure—of a single lychee at the same workstation and at the same time, and to output the two types of quality parameters synchronously through a fusion model, thus solving the problem of asynchronous internal and external inspection in existing technologies.

[0031] After being loaded and aligned, the lychees to be sorted are conveyed to the multimodal inspection station at individual intervals. A position detection component is installed on the feed side of the inspection station to identify the real-time position of the lychees on the conveyor line. When a lychee moves to the center of the inspection area, the position detection component sends a trigger signal to the control unit, which then simultaneously activates the appearance acquisition component and the spectral acquisition component.

[0032] The appearance acquisition unit comprises three sets of high-definition image acquisition devices, positioned directly above, to the left, and to the right of the conveyor belt. These devices simultaneously capture images of the lychee from three angles, eliminating blind spots inherent in single-sided acquisition. A diffuse white light supplementary lighting device is installed in the acquisition area to provide a uniform and stable illumination environment, preventing glare from the fruit shell from affecting color recognition accuracy. The acquired visual data completely covers the entire outer surface of the lychee, including information such as fruit shape, size, color, skin texture, blemish distribution, stem condition, and shell damage.

[0033] The spectral acquisition unit comprises a near-infrared light source and a spectral receiver, employing a diffuse reflection acquisition method and positioned directly in front of the central region of the litchi fruit. The near-infrared light source emits near-infrared light of a specific wavelength to illuminate the litchi peel. After reflection from the inside of the pulp, the light is collected by the spectral receiver, forming near-infrared spectral data. This near-infrared spectral data reflects the litchi's internal quality information, including sugar content, moisture content, ripeness, internal browning, pulp damage, and pest and disease infestation.

[0034] The control unit inputs the acquired visual appearance data and near-infrared spectral data into a pre-trained multimodal fusion detection model. The multimodal fusion detection model is pre-trained using a large number of labeled samples, covering litchi samples of different varieties, maturity levels, appearance conditions, and internal qualities. Each sample simultaneously includes an appearance image, spectral data, and corresponding manually labeled quality results.

[0035] The model first processes the two types of input data separately: for visual appearance data, it extracts fruit region, normalizes illumination, and calibrates size to extract feature data in the appearance dimension; for near-infrared spectral data, it performs smoothing, noise reduction, baseline correction, and numerical standardization to extract feature data in the intrinsic dimension. The model then performs spatiotemporal alignment of the two types of feature data to ensure that both types of features correspond to the same detection area of ​​the same lychee. The aligned features are then fused and stitched together to form a unified multimodal feature vector. Finally, the model performs inference calculations based on the multimodal feature vector, simultaneously outputting the corresponding appearance quality parameters and intrinsic quality parameters of the lychee.

[0036] The appearance quality parameters include seven quantitative indicators: fruit shape regularity, fruit transverse diameter, fruit longitudinal diameter, color uniformity, skin integrity, percentage of blemishes, and fruit stem condition. The internal quality parameters include four quantitative indicators: soluble solids content, maturity grade, degree of internal browning, and flesh integrity.

[0037] Step S102. Perform coarse screening at the primary screening station according to the appearance quality parameters to obtain the basic variety information of lychees. Remove lychees that do not meet the preset basic quality according to the basic variety information to complete the coarse screening.

[0038] Specifically, this step is the first sorting stage in the entire process. Its core is to identify the product type and quickly remove unqualified products based on appearance quality parameters, thereby reducing the processing volume of subsequent fine sorting stages, improving overall sorting efficiency, and ensuring that the initial screening standard and subsequent fine sorting standards are based on the same test results to avoid disconnection of process parameters.

[0039] The control unit extracts fruit shape, size, color, and texture-related indicators from appearance quality parameters, and combines them with four core features extracted from visual appearance data, including fruit top morphology and epidermal protrusion density. These features are then compared one by one with benchmark data in a pre-set variety feature library. The pre-set variety feature library stores feature benchmark intervals for different lychee varieties, with each variety corresponding to a set of independent feature range parameters.

[0040] During the comparison process, a score is awarded for each successful match of a feature category. The scores of the four feature categories are added together to obtain the total matching degree. The control unit selects the variety with the highest total matching degree that exceeds the preset matching threshold as the basic variety information for the corresponding lychee. If the total matching degree of all varieties is lower than the threshold, it is determined to be a generic variety, and the system's preset generic quality threshold is used. The preset variety feature library supports the input of new variety parameters, allowing for adaptation to more categories without hardware modifications.

[0041] Based on the identified basic variety information, the control unit calls up the corresponding basic quality thresholds for the variety. The basic quality thresholds include five independent thresholds: upper limit for fruit shape deviation, upper limit for skin damage area, upper limit for blemish percentage, upper limit for rotten area, and fruit stem missing judgment criteria. Different threshold values ​​can be set for different varieties.

[0042] The control unit compares the appearance quality parameters of the corresponding lychee with five thresholds one by one. If any parameter exceeds the corresponding threshold range, the lychee is determined to be unqualified. Unqualified lychees include those that are severely deformed, have large areas of damaged skin, are rotten or moldy, are severely affected by pests or diseases, or have missing stems that result in loss of marketability.

[0043] The control unit calculates the time it takes for a lychee to move from the inspection station to the primary screening station based on the conveyor belt speed and the distance between the inspection station and the primary screening station, and sends an action command to the primary screening unit in advance. When the corresponding lychee moves to the primary screening station, the food-grade flexible pusher of the primary screening unit extends and smoothly pushes the unqualified lychee into the defective material channel. After the action is completed, the pusher immediately resets, without affecting the normal conveying of the next lychee. Lychees that meet the basic quality requirements continue to be conveyed to the refining station by the conveyor belt.

[0044] Step S103. The lychees that have passed the coarse screening and sorting action are transported to the preset fine sorting station. The fine grading action is performed by combining the appearance quality parameters and the internal quality parameters. The quality score information corresponding to the lychee is obtained according to the appearance quality parameters and the internal quality parameters. The lychees are divided into the corresponding quality grade range according to the quality score information, thus completing the integrated sorting of the lychees by initial screening and fine sorting.

[0045] Specifically, this step is the core grading link in the entire process. The core is to complete the quantitative scoring and grading by comprehensively considering two types of quality parameters: appearance and internal quality. This enables a seamless sorting process from initial screening to fine sorting, improving grading accuracy and consistency.

[0046] The control unit calls up the matching grading weight configuration based on the basic variety information of the corresponding lychee. The grading weight configuration is preset by variety, and multiple weight schemes can be configured for each variety, corresponding to different application scenarios such as fresh fruit sales, gift packaging, and deep processing. It can be switched directly according to production needs.

[0047] The weighting configuration includes two primary weighting categories: overall appearance weighting and overall internal weighting. Under the overall appearance weighting, three secondary sub-weights are set: fruit shape, skin integrity, and color uniformity. Under the overall internal weighting, three secondary sub-weights are set: soluble solids content, maturity, and flesh integrity. In the calculation process, each quality parameter is first normalized to a uniform numerical range. Then, the overall appearance score and overall internal score are calculated according to the secondary sub-weights. Finally, the two overall scores are added together according to the primary weighting to obtain the final quality score for the corresponding lychee. The quality score is a single-value quantified result; a higher value indicates better overall quality.

[0048] The control unit is pre-set with four consecutive quality grade ranges, from high to low, corresponding to Grade 1 high-quality fruit, Grade 2 commercial fruit, Grade 3 processed fruit, and fruit awaiting re-inspection. Each grade has a fixed upper and lower limit for scoring.

[0049] The control unit compares the calculated quality score information one by one, starting from the highest level, to determine the target quality level range into which the score falls. Then, based on the conveyor belt speed and the position of the precision execution unit, the control unit calculates the time it takes for the lychee to reach the corresponding execution position and sends the corresponding level of action command to the precision execution unit in advance.

[0050] The precision grading unit is equipped with four independent flexible paddles and discharge channels corresponding to the four grades. When a lychee moves to the corresponding grade's execution position, the corresponding paddle extends, guiding the lychee into the matching discharge channel, completing the precision grading. All discharge channels adopt a smooth transition structure to avoid damage to the lychees during their fall, ensuring the integrity of the fruit.

[0051] In some embodiments, the acquisition of visual data and near-infrared spectral data of lychees to be sorted entering the sorting line includes: identifying the position information of the lychees to be sorted on the conveyor line; and when the lychees to be sorted reach the detection station, simultaneously triggering the high-definition image acquisition unit and the near-infrared spectral acquisition unit to acquire visual data and near-infrared spectral data of a single lychee to be sorted, respectively.

[0052] This embodiment specifically illustrates the method for synchronously acquiring appearance visual data and near-infrared spectral data.

[0053] In this embodiment, the position detection component uses a through-beam photoelectric sensor, which is installed on the feed side edge of the multimodal detection station. The sensor's optical path spans the surface of the conveyor belt. When the lychee moves to the sensor position with the conveyor belt, the lychee fruit blocks the optical path, and the sensor outputs a level change signal, which is sent to the control unit as a trigger reference for synchronous acquisition.

[0054] Upon receiving the trigger signal, the control unit simultaneously sends acquisition commands to the three sets of high-definition image acquisition devices and the near-infrared spectral acquisition device, ensuring that the acquisition time difference between the two types of data does not exceed ten milliseconds, thus achieving synchronization in the time dimension. The optical axes of the three sets of high-definition image acquisition devices are all aligned with the center of the detection area, and the acquisition range completely covers the maximum external dimensions of the lychee, ensuring that the entire outer surface of a single lychee is within the image acquisition range, thus achieving synchronization in the spatial dimension.

[0055] The supplemental lighting device uses a high color rendering index diffuse LED light source, with a diffuser plate installed on the surface of the light source to avoid specular reflection on the surface of the lychee peel. The light source and receiving probe of the spectral acquisition device are arranged at a fixed angle, directly facing the center height of the conveyor belt, to ensure that the acquired spectral data corresponds to the equatorial position of the lychee fruit, reducing the detection position deviation of lychees of different sizes.

[0056] After data acquisition, the control unit stitches the three sets of appearance images together, extracts the complete lychee fruit area, removes the conveyor belt background, and generates panoramic visual data of a single lychee. At the same time, it removes the first and last noise bands from the acquired raw spectral data, retains the spectral data of the effective detection bands, and uses them as the basic data for subsequent model input.

[0057] In some embodiments, the step of inputting appearance visual data and near-infrared spectral data into a pre-trained multimodal fusion detection model to obtain the appearance quality parameters and intrinsic quality parameters of the corresponding lychee includes: extracting features from the appearance visual data to obtain appearance dimension feature data; extracting features from the near-infrared spectral data to obtain intrinsic dimension feature data; performing spatiotemporal alignment and feature fusion of the appearance dimension feature data and intrinsic dimension feature data; inputting the data into the multimodal fusion detection model for inference; and outputting the appearance quality parameters and intrinsic quality parameters of the corresponding lychee respectively.

[0058] This embodiment specifically illustrates the feature processing and parameter output implementation of the multimodal fusion detection model.

[0059] In this embodiment, the preprocessing of the appearance visual data is divided into three steps: the first step is to perform background segmentation, which removes the background pixels of the conveyor belt by pixel threshold segmentation and extracts the independent litchi fruit body area; the second step is to perform illumination normalization, which adjusts the brightness and contrast of the image and eliminates the influence of illumination fluctuations in different batches on color recognition; the third step is to perform size calibration, which converts the pixel size in the image into actual physical size parameters according to the pre-calibrated conversion ratio between pixels and actual size.

[0060] After preprocessing, the model extracts features from the visual appearance data, extracting the shape features of the fruit outline, the color distribution features of the fruit shell, the texture features of the skin, and the area and location features of the blemish areas, and combining them to form an appearance dimension feature vector.

[0061] The preprocessing of near-infrared spectral data consists of three steps: the first step is to perform smoothing filtering to remove random noise during the spectral acquisition process; the second step is to perform baseline correction to eliminate the overall numerical deviation caused by the equipment baseline drift; and the third step is to perform numerical standardization to map the spectral absorbance values ​​to a uniform numerical range and eliminate the influence of the overall light intensity difference between different samples.

[0062] After preprocessing, the model extracts features from the near-infrared spectral data, filters feature bands related to sugar content, maturity, and browning, extracts the absorbance values ​​and curve variation features of the corresponding bands, and combines them to form an intrinsic dimension feature vector.

[0063] The model then performs spatiotemporal alignment verification on the two types of feature vectors. After confirming that the two types of features correspond to the same trigger time and the same fruit area, the two feature vectors are concatenated into a unified multimodal feature vector in dimensional order. The multimodal feature vector is then input into a fully connected inference network, which outputs the values ​​of seven appearance quality parameters and four intrinsic quality parameters, respectively, thus completing the multimodal fusion detection process.

[0064] In some embodiments, the step of performing coarse screening at the primary screening station based on appearance quality parameters to obtain basic variety information of lychees includes: identifying and comparing the fruit shape, skin texture and shell color features in the appearance visual data, matching the benchmark data in the preset variety feature library, and obtaining the basic variety information of the corresponding lychee.

[0065] This embodiment specifically illustrates the method for identifying basic lychee variety information.

[0066] In this embodiment, a preset variety feature library stores the baseline feature parameters of multiple mainstream lychee varieties. Each variety corresponds to the baseline range of four core features, namely the length-to-width ratio range of the fruit shape, the phase range of the main color of the fruit shell, the density range of the epidermal protrusion, and the range of the degree of depression at the top of the fruit.

[0067] During variety identification, the control unit first extracts the actual values ​​of the four features mentioned above from the appearance quality parameters and appearance visual data. Then, it compares the actual value of each feature with the benchmark range of the corresponding variety one by one. If the actual value falls within the benchmark range of a certain variety, the feature is considered a successful match and receives the corresponding score; otherwise, it is considered a failed match and receives zero points.

[0068] The scores of the four features are added together to obtain the total matching degree of the variety. The control unit traverses the total matching degree values ​​of all stored varieties and selects the variety with the highest value for secondary verification. If the highest total matching degree exceeds the preset matching threshold, the corresponding variety is determined as the basic variety information of the lychee; if the highest total matching degree does not reach the matching threshold, it is determined as a general variety, and the system's preset general basic quality threshold and general grading weight are called.

[0069] The preset variety feature library supports adding and modifying variety parameters. When it is necessary to adapt to new varieties or other types of fruits and vegetables, you only need to enter the corresponding variety's feature benchmark range and matching threshold. There is no need to adjust the hardware structure and core algorithm to complete the flexible switching.

[0070] In some embodiments, the step of removing lychees that do not meet the preset basic quality based on the basic variety information to complete the coarse screening and sorting action includes: calling the basic quality threshold of the corresponding variety based on the basic variety information, comparing the appearance quality parameters with the corresponding basic quality threshold, determining that lychees with appearance quality parameters lower than the basic quality threshold are unqualified lychees, and controlling the actuator of the primary screening station to push the unqualified lychees to the defective material channel to complete the coarse screening and sorting action.

[0071] This embodiment specifically illustrates the determination and execution method of the coarse screening and sorting action at the primary screening station.

[0072] In this embodiment, the basic quality thresholds are set independently for each variety, including five independent judgment indicators: the upper limit of fruit shape deviation rate, the upper limit of the percentage of skin damage area, the upper limit of the percentage of total surface blemish area, the upper limit of the rotten area, and the judgment standard for missing fruit stems. These five indicators are parallel in their judgment relationship; if any one of them exceeds the threshold, the corresponding lychee is judged as unqualified.

[0073] The fruit shape deviation rate is calculated by the ratio of the deviation between the actual fruit shape outline and the standard circular outline; the percentage of the skin damage area is calculated by the ratio of the pixel area of ​​the damaged area to the total pixel area of ​​the fruit; the percentage of the total surface blemish area is calculated by the ratio of the sum of the pixel areas of all blemish areas to the total pixel area of ​​the fruit; the area of ​​the rotten area is obtained by identifying the area of ​​the dark brown rotten area; the missing fruit stem is determined by the morphological characteristics of the fruit stem area.

[0074] The actuator employs a food-grade flexible paddle structure. The surface of the paddle that contacts the lychee is made of soft silicone with rounded edges to prevent scratching the fruit shell. The control unit calculates the conveying time required for the lychee to move from the detection station to the paddle position based on the real-time operating speed of the conveyor belt and the horizontal distance between the detection station and the initial screening paddle position. This calculation is then subtracted from the response time of the paddle action to obtain the advance instruction sending time.

[0075] The control unit sends the action command in advance according to the calculated time, ensuring that the lychee is just in front of the deflector when the deflector extends, smoothly pushing the defective lychee into the side defective material channel. The deflector immediately resets after being extended into place. The entire action time is less than the interval between two lychees, so it will not interfere with the normal conveying of the next lychee.

[0076] In some embodiments, the step of conveying the lychees that have passed the coarse screening to a preset fine sorting station, performing a fine grading action by combining appearance quality parameters and internal quality parameters, and obtaining the quality score information corresponding to the lychees based on the appearance quality parameters and internal quality parameters includes: matching the grading weight configuration according to the basic variety information of the corresponding lychees, assigning corresponding weight coefficients to the appearance quality parameters and internal quality parameters respectively, and performing a weighted calculation on the appearance quality parameters and internal quality parameters according to the weight coefficients to obtain the quality score information of the corresponding lychees.

[0077] This embodiment specifically illustrates how the lychee quality scoring information is calculated.

[0078] In this embodiment, the grading weight configuration is stored by variety, with multiple switchable weight schemes for each variety. Each scheme includes two levels: primary weight and secondary weight. The primary weight includes the total weight of appearance quality and the total weight of internal quality, with the sum of the two weight values ​​being a fixed value. The secondary weight consists of sub-item weights under the primary weight. Under appearance quality, three sub-weights are set: fruit shape regularity, skin integrity, and color uniformity. Under internal quality, three sub-weights are set: soluble solids content, maturity, and flesh integrity. The sum of all sub-weight values ​​under the same primary weight is a fixed value.

[0079] The quality rating calculation is divided into three levels: The first level is the sub-item score calculation, which normalizes the actual value of each quality parameter to a value range of zero to one hundred, multiplies it by the weight ratio of the corresponding sub-item, and obtains the sub-item score for that item.

[0080] The second level is the calculation of scores by category. The scores of the three sub-items under appearance quality are added together to obtain the overall appearance score; the scores of the three sub-items under internal quality are added together to obtain the overall internal score.

[0081] The third level is the calculation of the overall score. The overall appearance score is multiplied by the overall appearance weight ratio, and the overall internal score is multiplied by the overall internal weight ratio. The two results are added together to obtain the final quality score information for the corresponding lychee.

[0082] Different weighting schemes can be switched for different application scenarios. For example, in the case of gift fruit, the total weight of appearance quality can be increased, while in the case of deep processing, the total weight of internal quality can be increased to adapt to different production needs.

[0083] In some embodiments, the step of classifying lychees into corresponding quality grade intervals based on the quality scoring information to complete the integrated sorting of lychees for initial screening and fine sorting includes: comparing the acquired quality scoring information with multiple preset quality grade interval thresholds one by one to determine the target quality grade interval into which the quality scoring information falls; controlling the actuator of the fine sorting station to transport the corresponding lychees to the discharge channel corresponding to the target quality grade interval to complete the integrated sorting of lychees for initial screening and fine sorting.

[0084] This embodiment specifically illustrates the implementation method of refined grading action and material discharge.

[0085] In this embodiment, the quality grade range is pre-set as four consecutive numerical ranges, corresponding to Grade 1 high-quality fruit, Grade 2 commercial fruit, Grade 3 processed fruit, and fruit awaiting re-inspection in descending order. Each grade has a clear upper and lower limit for scoring, and the ranges have no overlap or breaks.

[0086] The grading process uses a descending order of matching logic. First, it checks whether the quality score meets the requirements for Grade 1 premium fruit. If it does, it is graded as Grade 1 premium fruit. If not, it checks whether it meets the requirements for Grade 2 commercial fruit, and so on, until the corresponding grade range is matched.

[0087] The precision execution unit is arranged with four sets of independent flexible paddles along the conveyor belt conveying direction. Each set of paddles corresponds to a discharge channel of a certain grade, and the distance between the four sets of paddles and the detection station increases sequentially. After the control unit determines the grade of the corresponding lychee, it calculates the time it takes for the lychee to reach the paddle position based on the conveyor belt running speed and the distance between the paddles of the corresponding grade, and sends the action command in advance.

[0088] When a lychee moves to the corresponding dispensing plate, the plate extends and guides the lychee into the corresponding discharge channel. The plate immediately resets after the action is complete. Lychees not separated by the first three dispensing plates fall into the final inspection channel for manual re-checking to prevent mis-sorting. All discharge channels are equipped with buffer ramps and flexible support pads to reduce the impact of falling lychees and minimize damage.

[0089] In some embodiments, the method further includes: periodically statistically analyzing the distribution data of quality parameters of sorted lychees, combining the incoming material fluctuation characteristics of the current batch of lychees, dynamically adjusting the threshold range of each quality grade interval, synchronously updating the basic quality threshold of the primary screening station, and completing the adaptive calibration of the sorting standard.

[0090] This embodiment specifically illustrates the adaptive calibration implementation method of sorting standards.

[0091] In this embodiment, the system is set with a fixed statistical cycle. A statistical analysis is triggered once every fixed number of lychees are processed. The statistical content includes six data items: the average value and distribution range of appearance quality parameters, the average value and distribution range of internal quality parameters, the distribution of quality scores, the proportion of each grade, and the initial screening failure rate.

[0092] The system compares the statistical data of the current period with the data of historical benchmark batches, calculates the deviation of various parameters, and obtains the incoming material fluctuation characteristics of the current batch. When the deviation of any core parameter exceeds the preset fluctuation threshold, the system triggers an adaptive adjustment of the sorting standards.

[0093] The adjustment rules are as follows: Based on the overall offset direction of the current batch parameters, adjust the threshold range of the quality grade interval accordingly. The adjustment range should match the deviation range, and the single adjustment range should not exceed the preset maximum adjustment ratio to avoid significant fluctuations in the standard. Simultaneously, adjust the basic quality threshold of the initial screening station by the same ratio to ensure that the initial screening standard is consistent with the precision grading standard, avoiding a disconnect between the judgment standards of the two processes.

[0094] After the adjustment is completed, the system automatically verifies whether the adjusted level proportions are within a reasonable range. If the verification is successful, the new threshold standard is officially implemented; if the verification fails, the system reverts to the standard before the adjustment. All adjustment processes are automatically recorded and support manual review and correction.

[0095] In some embodiments, the method further includes: real-time acquisition of the conveying speed and incoming material density data of the sorting line; dynamic adjustment of the execution triggering sequence of the primary screening station and the fine sorting station based on the detection time and sorting execution time of a single lychee; matching the conveying rhythm of the line to ensure that the sorting action of a single lychee is synchronized with the corresponding conveying progress.

[0096] This embodiment specifically illustrates the collaborative control method for the conveyor cycle time of the production line.

[0097] In this embodiment, the system collects the running speed of the conveyor belt in real time through the encoder of the conveyor belt drive motor, and calculates the incoming material density data by counting the number of lychees passing through per unit time through the photoelectric sensor at the detection station.

[0098] The system pre-stores the baseline values ​​for the detection time and sorting execution time of a single lychee. Combined with the real-time conveying speed, it calculates the conveying time for the lychee to move from the detection station to the initial screening execution station and the fine sorting execution stations, and then subtracts the processing time of the corresponding links to obtain the advance sending time of each execution instruction.

[0099] When the conveying speed changes, the system updates the advance sending time of instructions for each station in real time, and dynamically adjusts the execution triggering sequence of the primary screening station and the fine sorting station to ensure that the actuator completes its action just as the lychee arrives at the execution position, avoiding missed sorting due to premature action or incorrect sorting due to delayed action.

[0100] When the incoming material density exceeds the preset upper limit threshold, the system automatically reduces the running speed of the conveyor belt and extends the interval between individual lychees to ensure sufficient processing time for the detection and execution stages, and to avoid sorting failures caused by the overlap of multiple lychees; when the incoming material density returns to the normal range, the system automatically restores the set running speed of the conveyor belt.

[0101] In some embodiments, the method further includes: collecting quality parameters and grading results data generated during the entire sorting process, combining them with verification results data generated by manual sampling, incrementally training the multimodal fusion detection model, optimizing the parameters of the multimodal fusion detection model, and improving the accuracy of the detection results.

[0102] This embodiment specifically illustrates the incremental training and accuracy optimization implementation method of the multimodal fusion detection model.

[0103] In this embodiment, the system extracts re-inspection samples from the sorted lychees at a fixed ratio. Quality inspectors manually inspect the appearance and internal quality of the samples and record the results as verification data.

[0104] The system compares the automatically detected quality parameters with the results of manual verification item by item. When the error of any parameter exceeds the preset allowable error range, the original collected data of the sample and the manually labeled results are combined into incremental training samples.

[0105] The system is based on the currently running multimodal fusion detection model. It uses accumulated incremental training samples to fine-tune the output layer parameters of the model. The training process does not modify the underlying feature extraction network, ensuring the basic stability of the model. After each round of incremental training, the system uses a validation sample set to test the detection accuracy of the model. If the accuracy improves and meets the preset accuracy requirements, the new model is replaced with the online running model during the production line's idle periods, completing the closed-loop optimization of model accuracy.

[0106] The entire training and replacement process does not affect normal sorting production. Parameter changes before and after model updates are automatically archived, supporting version rollback and effect comparison.

[0107] In some embodiments, such as Figure 2 and Figure 3 As shown, this embodiment, combined with an embodied intelligent executor, illustrates the complete implementation method of integrated sorting throughout the entire process.

[0108] In this embodiment, the sorting process is completed independently by an embodied intelligent agent. The embodied intelligent agent consists of three components: a perception module, a decision-making module, and an execution module. It can autonomously complete the entire process of environmental perception, decision-making, and action execution.

[0109] The sensing module includes a visual sensing unit, a spectral sensing unit, a position sensing unit, and a force sensing unit. The visual sensing unit corresponds to high-definition image acquisition, the spectral sensing unit corresponds to near-infrared spectral acquisition, the position sensing unit corresponds to the detection of lychee position and conveying status, and the force sensing unit is integrated into the end effector to sense the contact force of the sorting action.

[0110] The decision-making module is embedded in the edge computing unit and runs multimodal fusion detection algorithms, variety recognition algorithms, quality scoring algorithms, beat control algorithms, and adaptive calibration algorithms. It can process all the data collected by the perception module in real time and generate corresponding execution decisions.

[0111] The execution module includes a conveyor drive unit, a primary screening execution end, a fine sorting execution end, and an autonomous inspection unit. The conveyor drive unit controls the running speed of the conveyor belt, the primary screening and fine sorting execution ends are used to complete the sorting and conveying actions, and the autonomous inspection unit can move along the production line to detect the equipment operating status and fruit sorting effect.

[0112] The complete sorting process is as follows: The embodied intelligent agent detects lychees entering the detection area through the position sensing unit, and simultaneously triggers the visual sensing unit and the spectral sensing unit to collect data; the decision module processes the collected data to obtain appearance quality parameters, internal quality parameters and basic variety information, and completes the initial screening and fine sorting grade determination; according to the decision results, the execution module triggers the initial screening and fine sorting execution ends in sequence according to the conveying rhythm to complete the sorting action; the force sensing unit provides real-time feedback on the contact force of the execution end and automatically adjusts the extension range of the paddle to avoid excessive force that could damage the lychees.

[0113] During operation, the embodied intelligent agent can autonomously collect sorting data, perform adaptive calibration of sorting standards and incremental optimization of the model, and also check for abnormal conditions on the production line through autonomous inspection units, without the need for continuous manual monitoring. The system can quickly adapt to the sorting needs of different types of fruits and vegetables through parameter configuration, achieving flexible switching between multiple categories, and the entire process complies with food-grade clean production standards.

[0114] Please see Figure 4 As shown, Figure 4This is a schematic diagram of the structure of the lychee primary screening and fine sorting integrated system 200 based on embodied robots provided in this application embodiment. This lychee primary screening and fine sorting integrated system 200 based on embodied robots is used to execute the steps of the lychee primary screening and fine sorting integrated method based on embodied robots shown in the above embodiments. The lychee primary screening and fine sorting integrated system 200 based on embodied robots can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, a laptop computer, a wearable device, or a robot.

[0115] like Figure 4 As shown, the integrated sorting system 200 for lychee primary screening and fine sorting based on embodied robots includes: The data acquisition unit 201 is used to collect the appearance visual data and near-infrared spectral data of the lychees to be sorted into the sorting line; and input the appearance visual data and near-infrared spectral data into the pre-trained multimodal fusion detection model to obtain the appearance quality parameters and internal quality parameters of the corresponding lychees. The coarse screening and sorting unit 202 is used to perform coarse screening and sorting at the primary screening station according to the appearance quality parameters, obtain the basic variety information of lychee, and remove lychees that do not meet the preset basic quality according to the basic variety information to complete the coarse screening and sorting action. The sorting completion unit 203 is used to transport the lychees that have passed the coarse screening to the preset fine sorting station, and perform fine grading by combining appearance quality parameters and internal quality parameters. Based on the appearance quality parameters and internal quality parameters, the unit obtains the quality score information corresponding to the lychees, and divides the lychees into the corresponding quality grade range based on the quality score information, thus completing the integrated sorting of the lychees by initial screening and fine sorting.

[0116] In some embodiments, the acquisition of visual data and near-infrared spectral data of lychees to be sorted entering the sorting line includes: identifying the position information of the lychees to be sorted on the conveyor line; and when the lychees to be sorted reach the detection station, simultaneously triggering the high-definition image acquisition unit and the near-infrared spectral acquisition unit to acquire visual data and near-infrared spectral data of a single lychee to be sorted, respectively.

[0117] In some embodiments, the step of inputting appearance visual data and near-infrared spectral data into a pre-trained multimodal fusion detection model to obtain the appearance quality parameters and intrinsic quality parameters of the corresponding lychee includes: extracting features from the appearance visual data to obtain appearance dimension feature data; extracting features from the near-infrared spectral data to obtain intrinsic dimension feature data; performing spatiotemporal alignment and feature fusion of the appearance dimension feature data and intrinsic dimension feature data; inputting the data into the multimodal fusion detection model for inference; and outputting the appearance quality parameters and intrinsic quality parameters of the corresponding lychee respectively.

[0118] In some embodiments, the step of performing coarse screening at the primary screening station based on appearance quality parameters to obtain basic variety information of lychees includes: identifying and comparing the fruit shape, skin texture and shell color features in the appearance visual data, matching the benchmark data in the preset variety feature library, and obtaining the basic variety information of the corresponding lychee.

[0119] In some embodiments, the step of removing lychees that do not meet the preset basic quality based on the basic variety information to complete the coarse screening and sorting action includes: calling the basic quality threshold of the corresponding variety based on the basic variety information, comparing the appearance quality parameters with the corresponding basic quality threshold, determining that lychees with appearance quality parameters lower than the basic quality threshold are unqualified lychees, and controlling the actuator of the primary screening station to push the unqualified lychees to the defective material channel to complete the coarse screening and sorting action.

[0120] In some embodiments, the step of conveying the lychees that have passed the coarse screening to a preset fine sorting station, performing a fine grading action by combining appearance quality parameters and internal quality parameters, and obtaining the quality score information corresponding to the lychees based on the appearance quality parameters and internal quality parameters includes: matching the grading weight configuration according to the basic variety information of the corresponding lychees, assigning corresponding weight coefficients to the appearance quality parameters and internal quality parameters respectively, and performing a weighted calculation on the appearance quality parameters and internal quality parameters according to the weight coefficients to obtain the quality score information of the corresponding lychees.

[0121] In some embodiments, the step of classifying lychees into corresponding quality grade intervals based on the quality scoring information to complete the integrated sorting of lychees for initial screening and fine sorting includes: comparing the acquired quality scoring information with multiple preset quality grade interval thresholds one by one to determine the target quality grade interval into which the quality scoring information falls; controlling the actuator of the fine sorting station to transport the corresponding lychees to the discharge channel corresponding to the target quality grade interval to complete the integrated sorting of lychees for initial screening and fine sorting.

[0122] In some embodiments, the method further includes: periodically statistically analyzing the distribution data of quality parameters of sorted lychees, combining the incoming material fluctuation characteristics of the current batch of lychees, dynamically adjusting the threshold range of each quality grade interval, synchronously updating the basic quality threshold of the primary screening station, and completing the adaptive calibration of the sorting standard.

[0123] In some embodiments, the method further includes: real-time acquisition of the conveying speed and incoming material density data of the sorting line; dynamic adjustment of the execution triggering sequence of the primary screening station and the fine sorting station based on the detection time and sorting execution time of a single lychee; matching the conveying rhythm of the line to ensure that the sorting action of a single lychee is synchronized with the corresponding conveying progress.

[0124] In some embodiments, the method further includes: collecting quality parameters and grading results data generated during the entire sorting process, combining them with verification results data generated by manual sampling, incrementally training the multimodal fusion detection model, optimizing the parameters of the multimodal fusion detection model, and improving the accuracy of the detection results.

[0125] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described integrated sorting system for lychee primary screening and fine sorting based on embodied robots and its various modules can be found in the corresponding embodiments of the above-described integrated sorting method for lychee primary screening and fine sorting based on embodied robots, and will not be repeated here.

[0126] The aforementioned integrated sorting method for lychee primary screening and fine sorting based on embodied robots can be implemented as a computer program, which can be used in various ways, such as... Figure 4 It runs on the system shown.

[0127] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. The computer device includes a processor, a memory, and a network interface connected via a device bus, wherein the memory may include a storage medium and internal memory.

[0128] The storage medium can store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any integrated sorting method for lychee primary screening and fine sorting based on an embodied robot.

[0129] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0130] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any integrated sorting method for lychee primary screening and fine sorting based on embodied robots.

[0131] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0133] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Collect visual and near-infrared spectral data of lychees entering the sorting line; input the visual and near-infrared spectral data into a pre-trained multimodal fusion detection model to obtain the corresponding lychee appearance quality parameters and internal quality parameters. Based on the appearance quality parameters, a coarse screening action is performed at the primary screening station to obtain the basic variety information of lychees. Lychees that do not meet the preset basic quality are removed based on the basic variety information to complete the coarse screening action. The lychees that have passed the coarse screening process are transported to a preset fine sorting station. A fine grading process is performed by combining appearance quality parameters and internal quality parameters. The quality score information corresponding to the lychees is obtained based on the appearance quality parameters and internal quality parameters. The lychees are then divided into the corresponding quality grade range based on the quality score information, thus completing the integrated sorting of the lychees from initial screening to fine sorting.

[0134] In some embodiments, the acquisition of visual data and near-infrared spectral data of lychees to be sorted entering the sorting line includes: identifying the position information of the lychees to be sorted on the conveyor line; and when the lychees to be sorted reach the detection station, simultaneously triggering the high-definition image acquisition unit and the near-infrared spectral acquisition unit to acquire visual data and near-infrared spectral data of a single lychee to be sorted, respectively.

[0135] In some embodiments, the step of inputting appearance visual data and near-infrared spectral data into a pre-trained multimodal fusion detection model to obtain the appearance quality parameters and intrinsic quality parameters of the corresponding lychee includes: extracting features from the appearance visual data to obtain appearance dimension feature data; extracting features from the near-infrared spectral data to obtain intrinsic dimension feature data; performing spatiotemporal alignment and feature fusion of the appearance dimension feature data and intrinsic dimension feature data; inputting the data into the multimodal fusion detection model for inference; and outputting the appearance quality parameters and intrinsic quality parameters of the corresponding lychee respectively.

[0136] In some embodiments, the step of performing coarse screening at the primary screening station based on appearance quality parameters to obtain basic variety information of lychees includes: identifying and comparing the fruit shape, skin texture and shell color features in the appearance visual data, matching the benchmark data in the preset variety feature library, and obtaining the basic variety information of the corresponding lychee.

[0137] In some embodiments, the step of removing lychees that do not meet the preset basic quality based on the basic variety information to complete the coarse screening and sorting action includes: calling the basic quality threshold of the corresponding variety based on the basic variety information, comparing the appearance quality parameters with the corresponding basic quality threshold, determining that lychees with appearance quality parameters lower than the basic quality threshold are unqualified lychees, and controlling the actuator of the primary screening station to push the unqualified lychees to the defective material channel to complete the coarse screening and sorting action.

[0138] In some embodiments, the step of conveying the lychees that have passed the coarse screening to a preset fine sorting station, performing a fine grading action by combining appearance quality parameters and internal quality parameters, and obtaining the quality score information corresponding to the lychees based on the appearance quality parameters and internal quality parameters includes: matching the grading weight configuration according to the basic variety information of the corresponding lychees, assigning corresponding weight coefficients to the appearance quality parameters and internal quality parameters respectively, and performing a weighted calculation on the appearance quality parameters and internal quality parameters according to the weight coefficients to obtain the quality score information of the corresponding lychees.

[0139] In some embodiments, the step of classifying lychees into corresponding quality grade intervals based on the quality scoring information to complete the integrated sorting of lychees for initial screening and fine sorting includes: comparing the acquired quality scoring information with multiple preset quality grade interval thresholds one by one to determine the target quality grade interval into which the quality scoring information falls; controlling the actuator of the fine sorting station to transport the corresponding lychees to the discharge channel corresponding to the target quality grade interval to complete the integrated sorting of lychees for initial screening and fine sorting.

[0140] In some embodiments, the method further includes: periodically statistically analyzing the distribution data of quality parameters of sorted lychees, combining the incoming material fluctuation characteristics of the current batch of lychees, dynamically adjusting the threshold range of each quality grade interval, synchronously updating the basic quality threshold of the primary screening station, and completing the adaptive calibration of the sorting standard.

[0141] In some embodiments, the method further includes: real-time acquisition of the conveying speed and incoming material density data of the sorting line; dynamic adjustment of the execution triggering sequence of the primary screening station and the fine sorting station based on the detection time and sorting execution time of a single lychee; matching the conveying rhythm of the line to ensure that the sorting action of a single lychee is synchronized with the corresponding conveying progress.

[0142] In some embodiments, the method further includes: collecting quality parameters and grading results data generated during the entire sorting process, combining them with verification results data generated by manual sampling, incrementally training the multimodal fusion detection model, optimizing the parameters of the multimodal fusion detection model, and improving the accuracy of the detection results.

[0143] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the integrated sorting method for lychee primary screening and fine sorting based on an embodied robot as provided in any embodiment of this application.

[0144] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0145] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for integrated primary screening and fine sorting of lychees based on embodied robots, characterized in that, include: Collect visual and near-infrared spectral data of lychees entering the sorting line; Visual data and near-infrared spectral data are input into a pre-trained multimodal fusion detection model to obtain the corresponding appearance quality parameters and internal quality parameters of lychees. Based on the appearance quality parameters, a coarse screening action is performed at the primary screening station to obtain the basic variety information of lychees. Lychees that do not meet the preset basic quality are removed based on the basic variety information to complete the coarse screening action. The lychees that have passed the coarse screening process are transported to a preset fine sorting station. A fine grading process is performed by combining appearance quality parameters and internal quality parameters. The quality score information corresponding to the lychees is obtained based on the appearance quality parameters and internal quality parameters. The lychees are then divided into the corresponding quality grade range based on the quality score information, thus completing the integrated sorting of the lychees from initial screening to fine sorting.

2. The method according to claim 1, characterized in that, The visual and near-infrared spectral data of the lychees entering the sorting line are collected, including: Identify the location information of the lychees to be sorted on the conveyor line; When the lychees to be sorted arrive at the inspection station, the high-definition image acquisition unit and the near-infrared spectral acquisition unit are triggered simultaneously to acquire the appearance visual data and near-infrared spectral data of a single lychee to be sorted.

3. The method according to claim 1, characterized in that, The step involves inputting visual appearance data and near-infrared spectral data into a pre-trained multimodal fusion detection model to obtain the corresponding appearance quality parameters and internal quality parameters of the lychee, including: Feature extraction is performed on the visual data of the appearance to obtain the feature data of the appearance dimension; Feature extraction is performed on near-infrared spectral data to obtain intrinsic dimensional feature data; The appearance dimension feature data and the internal dimension feature data are spatiotemporally aligned and feature fused, and then input into a multimodal fusion detection model for inference, outputting the corresponding appearance quality parameters and internal quality parameters of the lychee respectively.

4. The method according to claim 1, characterized in that, The process of performing coarse screening at the primary screening station based on appearance quality parameters to obtain basic lychee variety information includes: The fruit shape, skin texture and shell color features in the appearance visual data are identified and compared, and matched with the benchmark data in the preset variety feature library; Obtain basic variety information for the corresponding lychee.

5. The method according to claim 4, characterized in that, The step of removing lychees that do not meet the preset basic quality based on the basic variety information to complete the coarse screening and sorting action includes: The basic quality threshold of the corresponding variety is retrieved based on the basic variety information; The appearance quality parameters are compared with the corresponding basic quality thresholds, and lychees with appearance quality parameters lower than the basic quality thresholds are determined to be unqualified lychees. The actuator at the primary screening station pushes unqualified lychees to the defective material channel to complete the coarse screening and sorting action.

6. The method according to claim 1, characterized in that, The process involves conveying the lychees that have passed the coarse screening to a preset fine sorting station, where a fine grading process is performed based on appearance and internal quality parameters. The quality score information corresponding to the lychees is obtained based on the appearance and internal quality parameters, including: Based on the basic variety information of the corresponding lychee, the grading weight configuration is matched, and corresponding weight coefficients are assigned to the appearance quality parameters and the internal quality parameters, respectively. The appearance quality parameters and internal quality parameters are weighted according to the weighting coefficients to obtain the corresponding quality score information of lychees.

7. The method according to claim 6, characterized in that, The step of classifying lychees into corresponding quality grade ranges based on the quality scoring information, thus completing the integrated sorting and final selection of lychees, includes: The acquired quality score information is compared one by one with multiple preset quality level range thresholds to determine the target quality level range into which the quality score information falls. The actuator at the fine sorting station will transport the corresponding lychees to the discharge channel corresponding to the target quality grade range, completing the integrated sorting of lychees from initial screening to fine sorting.

8. The method according to claim 1, characterized in that, The method further includes: Periodically analyze the distribution data of quality parameters of sorted lychees; Based on the fluctuation characteristics of the current batch of lychees, the threshold range of each quality grade is dynamically adjusted, and the basic quality threshold of the primary screening station is updated simultaneously to complete the adaptive calibration of the sorting standard.

9. The method according to claim 1, characterized in that, The method further includes: Real-time acquisition of conveyor speed and material density data for the sorting line; Based on the detection time and sorting time of a single lychee, the execution triggering sequence of the primary screening station and the fine sorting station is dynamically adjusted to match the conveyor belt rhythm of the production line, so as to ensure that the sorting action of a single lychee is synchronized with the corresponding conveying progress.

10. The method according to claim 1, characterized in that, The method further includes: The quality parameters and grading results generated throughout the sorting process are collected and combined with the verification results generated by manual sampling to incrementally train the multimodal fusion detection model and optimize its parameters.