Method and system for hierarchical correlation tracking and defect quantification of rotating spherical fruit target

Abstract

The invention discloses a hierarchical association tracking and defect quantification method and system for a rotating spherical fruit target. A conveying belt video is collected, a fruit bounding box and a defect mask are obtained through detection and segmentation, and TID and QTID hierarchies are established according to the inclusion relation; mahalanobis distance motion gating and EMA are adopted to update ReID cosine gating, multi-frame confirmation is carried out, existence degree threshold values are weighted by confidence coefficients, false detection is eliminated, TIDs are combined, and a unique logic ID is allocated; mapping a defect mask to a fruit local coordinate and rasterizing, de-weighting the same physical defect based on a signature IoU threshold, determining a representative QTID according to a historical maximum area, establishing alias mapping, and de-weighting a defect union set during trajectory combination; and determining and grading the defect area of the representative QTID under the statistical logic ID, and outputting to a PLC (Programmable Logic Controller) or an executing mechanism. The system comprises an image acquisition unit, a detection segmentation unit, a hierarchical binding unit, a track association unit, a defect duplicate removal unit, a hierarchical decision unit and a control output unit, and track stability and defect statistics consistency are improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, pattern recognition, intelligent agricultural equipment, and industrial automation control technology, specifically to a hierarchical association tracking and defect quantification method and system for rotating spherical fruit targets. Background Technology

[0002] With the development of smart agriculture and smart manufacturing technologies, the demand for automated sorting and appearance quality inspection in the post-harvest processing of fruits and vegetables is increasing. In traditional sorting production lines, appearance defects of spherical fruits (such as surface scars, rot, punctures, etc.) often rely on manual visual screening, which limits efficiency and consistency.

[0003] Unlike rigid parts with stable orientation, spherical fruits on roller conveyor belts typically exhibit a complex motion involving translation, random spin, and micro-bouncing. This results in nonlinear changes in the texture of the image over time, and since defects are located on the spherical surface, their projected shape undergoes significant perspective distortion with rotation. Existing DeepSORT methods based on Kalman filtering and their IoU-based association strategies still suffer from the following problems when applied to this scenario:

[0004] (1) Tracking models are difficult to adapt to complex motion, resulting in trajectory fragmentation: Existing technologies mostly use linear constant velocity models, which are not adaptable to appearance changes caused by spin. They are prone to losing targets and misjudging them as new targets at texture changes or short-term occlusions, resulting in frequent trajectory ID switching and difficulty in continuously accumulating historical detection data of the same fruit.

[0005] (2) Rigid IoU matching is not tolerant to spherical perspective distortion, which leads to repeated counting of defects: When defects move from the center to the edge as the fruit rotates, they are prone to shape compression and stretching. Standard IoU is difficult to identify defects with large geometric differences but topological homology, which leads to the same physical defect being mistakenly recorded as multiple independent defect IDs, resulting in an inflated defect area count and thus misclassification.

[0006] (3) The weak anti-interference ability of simple counting logic leads to the jitter of the classification result: There are factors such as motion fuzziness, uneven lighting, and dust interference in the industrial field, which can easily produce false detections with low confidence. If noise and effective detection are given equal weight, the judgment jitter is likely to occur near the classification threshold, reducing the repeatability accuracy of the equipment.

[0007] Therefore, a technical solution is needed that can adapt to the spin motion of spherical fruits, achieve stable trajectory correlation, and deweight the spherical defects, so as to improve the consistency and robustness of defect statistics and graded output. Summary of the Invention

[0008] The purpose of this invention is to overcome the problems of trajectory fragmentation, repeated counting of defects, and jitter of grading results in the existing technology for the scenario of rotating spherical fruit on roller conveyor belts, and to provide a grading association tracking and defect quantification method and system for rotating spherical fruit targets, so as to achieve stable tracking of the same fruit instance, deduplication and measurement of the same physical defects, and stable output of grading results.

[0009] To achieve the above objectives, the present invention provides the following technical solution.

[0010] I. Methods

[0011] This invention provides a hierarchical correlation tracking and defect quantification method for rotating spherical fruit targets, including:

[0012] Step S1: Multi-task perception and hierarchical binding. Acquire the video sequence of the detection area of ​​the roller conveyor belt; perform target detection and instance segmentation on each frame of the video sequence to obtain the bounding box of the parent fruit target and the pixel mask of the child defect target, and obtain their respective confidence scores; based on the spatial inclusion relationship between the defect mask and the bounding box of the fruit target, bind the defect mask to its corresponding fruit target, and establish a parent-child hierarchical data structure of fruit trajectory identifier TID and defect trajectory identifier QTID.

[0013] Step S2: Fruit trajectory association and validity verification (VisionLock). A Kalman filter prediction model is established for each fruit trajectory. The squared Mahalanobis distance between the current observation and the prediction is calculated and compared with the motion consistency gating threshold. Comparison is used to filter candidate trajectories; fruit appearance ReID feature vectors are extracted and updated using exponential moving average; the cosine distance between appearance prototype vectors is calculated and gated with an appearance consistency threshold. Candidate trajectories are compared and screened; candidate trajectory pairs that pass both motion consistency gating and appearance consistency gating are confirmed for multiple frames. After confirmation, the confidence weighted existence sum S of multiple TIDs belonging to the same physical fruit instance is calculated, and when S exceeds a preset threshold, the fruit instance is confirmed as a valid target and assigned a unique logical ID to suppress the impact of short-term false detections on subsequent statistics.

[0014] Step S3: Defect topology deduplication and merging (DefectLock). Defect masks bound to the same fruit target are mapped to the local coordinate system of that fruit target and rasterized into a fixed-size N×N binary feature mesh; the signature intersection-union ratio (CIU) is calculated for candidate defects. A signature threshold θ is set to tolerate spherical perspective distortion, and the signature intersection ratio is not less than the signature threshold. When different perspectives are used, defect observations are associated as the same physical defect, thereby identifying and associating different perspective observations belonging to the same physical defect. For the defect observation set that is determined to be the same physical defect, the QTID with the largest historical defect area is selected as the representative QTID of the physical defect, and an alias mapping is established so that other QTIDs in the set point to the representative QTID. When fruit TID merging occurs, the defect list attached to each is subjected to a set union deduplication operation based on the representative QTID, and the representative QTID is assigned to the logical ID along with the TID merging to eliminate statistical redundancy caused by trajectory breakage or repeated detection.

[0015] Step S4: Grading Decision and Hardware Execution. The cumulative defect area of ​​all QTIDs under the statistical logic ID is compared with the preset grading threshold to obtain the grading result. The grading result is used to generate control instructions and sent to the PLC or actuator to complete the sorting execution.

[0016] II. System

[0017] The present invention also provides a hierarchical association tracking and defect quantification system for rotating spherical fruit targets, comprising: an image acquisition unit, a detection and segmentation unit, a hierarchical binding unit, a fruit trajectory association unit, a defect deduplication unit, a hierarchical decision unit, and a control output unit.

[0018] The system comprises the following components: an image acquisition unit for acquiring video sequences of the detection area of ​​the roller conveyor belt; a detection and segmentation unit for outputting the fruit target BBOX and defect target pixel masks, and corresponding confidence scores; a hierarchical binding unit for establishing the parent-child hierarchical relationship between fruit TID and defect QTID based on geometric inclusion relationships; a fruit trajectory association unit for performing motion consistency gating based on Kalman prediction, appearance consistency gating based on ReID and EMA, multi-frame confirmation, and TID merging to form a logical ID; and a defect deduplication unit for performing local mapping of defect masks, fixed meshing, and other functions. The system matches and represents QTID selection and alias mapping, and performs defect set union and deduplication when merging TIDs; the hierarchical decision unit is used to output hierarchical results based on the cumulative defect area of ​​the representative QTID; the control output unit is used to send the hierarchical results to the PLC or actuator to complete the sorting.

[0019] The present invention has at least the following beneficial effects:

[0020] (1) Improve the stability of fruit trajectory association: By combining motion consistency gating and appearance consistency gating, and introducing a multi-frame confirmation mechanism, the probability of trajectory breakage and ID switching caused by spin appearance changes and short-term occlusion can be reduced, so that the historical detection information of the same fruit instance can be continuously accumulated.

[0021] (2) Improve the consistency of defect area statistics: By mapping the local coordinates of the defect mask, fixing the mesh and matching the signature IoU, and combining the "maximum area represents QTID + alias mapping + set union deduplication" mechanism, the repeated counting of the same physical defect caused by spherical perspective distortion and repeated detection can be reduced, thereby improving the stability of quantitative defect statistics.

[0022] (3) Improve the robustness of the graded output: Quantify the effectiveness of the target by the total existence S weighted by confidence, and set an effective threshold to suppress the impact of short-term false detections with low confidence on statistics and grading, thereby reducing the jitter of the grading results near the critical threshold and improving the repeatability and adaptability of the system under industrial field conditions. Attached Figure Description

[0023] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0024] Figure 1 This is a schematic diagram of the data flow of a rotating spherical fruit in a conveyor belt scenario according to an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of the process by which the VisionLock module associates and merges multiple TIDs of the same fruit instance in an embodiment of the present invention.

[0026] Figure 3 This is a flowchart illustrating the process of the DefectLock module performing topological deduplication, representative selection, and merging with TIDs to belong to logical IDs for multiple QTIDs of the same physical defect in an embodiment of the present invention.

[0027] Figure 4 This is a schematic diagram of the sorting mechanism and its application object in an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solution of this invention, the implementation methods of this invention will be described in detail below with reference to specific algorithm flows, parameter configurations, and hardware interaction processes. It should be noted that this invention is not limited to the following embodiments, and those skilled in the art can make various modifications or equivalent substitutions to the implementation methods without departing from the concept of this invention.

[0029] I. Explanation of terms and symbols.

[0030] For ease of understanding, the terms used in this article are defined as follows:

[0031] (1) BBOX: The bounding box of the fruit target;

[0032] (2) TID: Fruit trajectory identifier, used to represent the fruit trajectory obtained by continuous tracking of the same segment;

[0033] (3) Logical ID: A unified identifier obtained by confirming and merging multiple TIDs belonging to the same physical fruit instance;

[0034] (4) QTID: Defect trajectory identifier, used to characterize the identifier of defect observation; in this embodiment, defect observations can be merged into the same physical defect set in time sequence;

[0035] (5) Represents QTID: A unique output identifier selected from the set of QTIDs corresponding to the same physical defect according to the "historical maximum defect area";

[0036] (6) The cross-union ratio (CUC) is calculated between the binary meshes obtained by rasterizing the defect mask in the local coordinate system of the fruit.

[0037] (7) The confidence-weighted sum of existence is used to measure the effectiveness of a target over time.

[0038] (8) ROI: Region of Interest;

[0039] (9) Squared Mahalanobis distance; Cosine distance; EMA smoothing factor; N: grid size; : Signature IoU threshold; Multi-frame confirmation threshold.

[0040] (10) Figure 4 In the diagram, 1 represents the base, 2 represents the rollers, 3 represents the fruit, 4 represents the industrial camera, and 5 represents the stand.

[0041] II. System Deployment and Hardware Configuration (Example).

[0042] This embodiment uses a hawthorn sorting production line as an example, but the invention is also applicable to other spherical fruits (such as apples, oranges, tomatoes, etc.). The conveyor belt speed is, for example, 50 mm / s.

[0043] To support the algorithm's operation, the hardware environment configuration is as follows:

[0044] (1) Image acquisition: An industrial camera and a strobe light source are used to reduce motion blur. The lens focal length is, for example, a 12mm fixed-focus lens, and the installation height from the conveyor belt plane is preferably 0.2m to 0.3m. The camera resolution is preferably 1920×1080, the acquisition frame rate is preferably 30fps, and the exposure time is preferably 0.2ms to 2ms. The light source is a ring-shaped high-brightness LED strobe light, which is hard-triggered and synchronized with the camera exposure signal to reduce the influence of ambient light fluctuations on the detection results.

[0045] (2) Processing unit: Jetsonorin embedded edge computing unit is adopted. The detection and tracking model adopts Tensort inference acceleration to achieve real-time processing, and the end-to-end latency of a single frame is controlled within 25ms.

[0046] (3) Control unit: PLCs7-1200 is used to record the classification data and execute subsequent actions.

[0047] (4) Communication: The edge computing unit and the PLCs7-1200 communicate via network cable to complete the synchronous writing and execution of the hierarchical results register.

[0048] III. Overall Process and Data Structure.

[0049] See Figure 1 The method of this invention can be generally divided into four stages: perception and hierarchical binding, fruit trajectory association with TID level, defect deduplication and merging with QTID level, and hierarchical decision-making and control output. The system first performs fruit detection and defect segmentation on each frame of image, binding the defects to their respective fruits; then, it performs joint motion and appearance gating on the fruit trajectory in the time dimension, merging them to form logical IDs; and finally, it performs topological consistency deduplication on the defects within the fruit's local coordinate system, outputting the defect quantification result and hierarchical instructions in units of logical IDs.

[0050] IV. Step S1: Multi-task awareness and hierarchical binding.

[0051] After each frame of image is input into the detection and instance segmentation network, it outputs two types of target information: the parent target fruit, the box, and the confidence score. Sub-target defects: pixel mask and confidence level The system performs geometric binding: it calculates the geometric center point (or centroid) of each defect mask. If this center point is located inside a fruit box, the defect observation is added to the defect list of that fruit. A temporary defect identifier is assigned to each defect observation as a QTID candidate, and its mask, area, and confidence level are recorded. Simultaneously, appearance ReID feature vectors are extracted from the fruit's ROI. It is then subjected to L2 normalization for subsequent appearance gating.

[0052] 5. Step S2: Construct robust tracking and merging of fruit trajectories (TID) (VisionLock).

[0053] See Figure 2 To address the trajectory fragmentation issue caused by the rolling of spherical fruits, this embodiment employs a dual gating and multi-frame confirmation mechanism for trajectory repair and merging. Motion consistency gating is achieved through Kalman prediction combined with Mahalanobis distance measurement, while appearance consistency gating is achieved through ReID embedding with cosine distance. Multi-frame confirmation prevents jitter, and confidence-weighted accumulation of presence frames suppresses short-term false detections.

[0054] S2.1 Kalman prediction and motion consistency gating.

[0055] The system maintains a Kalman filter for each fruit instance identified as a parent trajectory. The system state vector is defined as an 8-dimensional vector.

[0056] ;

[0057] in,( () represents the center coordinates of the fruit BBOX. Indicates the aspect ratio of BBOX. Indicates the height of the BBOX. ) represent the rate of change of the above variables on the image plane.

[0058] For newly generated trajectories or trajectories that regenerate after a track break. Candidate trajectories are selected from the historical trajectory cache pool. And calculate the frame interval between the two. ,in The frame number is the newly generated trajectory or the trajectory regenerated after a track break. This is the frame number of the last valid update of the candidate trajectory. Set the time-gating threshold. At a frame rate of 30fps, the optimal limit window for physical association is 0.2s, i.e. Frames are used to tolerate short-term occlusion or detect flicker. When If the target has been lost for too long, the candidate is excluded; At that time, the candidate trajectory state is predicted forward. Step, to obtain the prior predicted state at the current time. and predicted covariance .

[0059] Let the observation vector of the trajectory to be merged in the current frame be denoted as ;

[0060] ;

[0061] Predicted observations , where H is the observation matrix that maps the 8-dimensional state to the 4-dimensional observation space. Define the innovation covariance matrix:

[0062] ;

[0063] Where R is the covariance matrix of the observation noise inside the tracker. Calculate the squared Mahalanobis distance:

[0064] ;

[0065] Set motion consistency gating threshold If and only if ≤ At that time, the candidate trajectory pair is determined to pass the motion consistency gating. Preferably, The value corresponding to the 0.99 quantile of the chi-square distribution with 4 degrees of freedom is 13.277.

[0066] S2.2ReID Appearance Consistency Gating and EMA Update

[0067] To adapt to the texture gradation caused by fruit spin, the system maintains an appearance prototype vector for each trajectory. It is updated using an exponential moving average (EMA).

[0068] When the trajectory is The first time a valid appearance feature vector is detected and obtained is at time 1. At that time, construct the initial appearance prototype:

[0069] ;

[0070] When the The trajectory was successfully matched with the detection result, and the appearance feature vector was obtained. Update the appearance prototype at that time:

[0071] ;

[0072] When the If the trajectory is missed or occluded and effective features cannot be extracted, the system forcibly freezes the appearance prototype of the previous frame.

[0073] ;

[0074] In this embodiment, the EMA smoothing coefficient The preferred value is 0.2, which provides a better trade-off between identity stability and rotational adaptability.

[0075] For candidate trajectory pairs that pass motion gating, extract their appearance prototype vector. and , This is the appearance prototype vector of the historical trajectory. Calculate the cosine distance for the appearance prototype vector of the new trajectory:

[0076] ;

[0077] Set appearance gating threshold Preferred .when ≤ If the appearance is consistent, the candidate is retained; otherwise, it is discarded.

[0078] S2.3 Multi-frame confirmation (with counter support).

[0079] To avoid false merging due to single-frame false detections or noise, a timing hysteresis confirmation mechanism is introduced. The system maintains a support counter for each candidate association pair. Its update rule is, for example, when both conditions are met. ≤ and ≤ hour, When none of the above conditions are met, .in To confirm the threshold; preferably, in Under the given conditions, if the confirmation timescale is 0.2s, then Only when Reaching the confirmation threshold Only then is it determined that the candidate trajectories belong to the same physical fruit instance.

[0080] S2.4 Confidence-weighted sum of existence .

[0081] After determining that multiple TIDs belong to the same fruit instance, in order to suppress the impact of short-term false detections on the statistics, instead of directly using simple frame counting, the sum of existence degrees is calculated. :

[0082] ;

[0083] in This represents the set of TIDs that have been confirmed to belong to the same fruit instance; This represents the set of frames associated with this TID; is the confidence score of the detection result corresponding to this TID in frame t.

[0084] VI. Step S3: Deduplication of defective instances with topological consistency (DefectLock).

[0085] See Figure 3To avoid the same physical defect being counted repeatedly from different perspectives, this embodiment performs topological consistency matching and merging of defect shapes within the local coordinate system of the fruit. A set of QTIDs for the same defect instance is constructed, and a representative QTID is determined through a maximum area representation mechanism. The attribution of the QTID list is normalized, and the QTIDs are deduplicated and merged along with the TIDs of the fruit trajectory, eliminating duplicate attachment and counting of defects caused by trajectory breaks or repeated detection.

[0086] S3.1 Defect mask local mapping and signature IoU matching.

[0087] Using the top left corner of the fruit BBOX as Width and height are For any pixel in the mask Calculate normalized local coordinates: , and will Cut off to Interval. The defect mask in the local coordinate system is rasterized into a fixed-size binary mesh. The nearest neighbor method is preferred, and .

[0088] For any two defect observation grids and Define signature intersection and comparison for:

[0089] ;

[0090] when If the two instances belong to the same physical defect instance and are placed in the same set, then they are determined to be different defect instances. Preferably, To tolerate the decrease in overlap caused by spherical perspective distortion.

[0091] The maximum area in S3.2 represents the mapping between QTID and alias.

[0092] QTID set of the same physical defect instance Internally, maintain the historical maximum defect area for each QTID. :

[0093] ;

[0094] in This represents the pixel area of ​​the defect mask for the QTID in the current frame.

[0095] set The representative QTID is defined as:

[0096] ;

[0097] If it appears In the case of a tie, the QTID with the longer frame count is used as the deterministic secondary rule.

[0098] And establish alias mapping Make each QTID in the set point to the QTID representing it:

[0099] , ;

[0100] This achieves the goal of "only one representative QTID is retained for the same physical defect" at the statistical and output level, and that QTID satisfies the constraint of "coming from the one with the largest area in its QTID set".

[0101] S3.3 QTID list normalization and deduplication by merging with TID.

[0102] For any fruit trajectory TID with a list of defective QTIDs, normalization is first performed: each QTID in the list is replaced with its alias-mapped representative QTID, and the replaced list is deduplicated so that the same representative QTID is retained only once. When multiple fruit trajectory TIDs are merged and assigned the same logical ID, their respective representative QTID sets are deduplicated using a union:

[0103] ;

[0104] This eliminates the repeated loading and counting of defects caused by broken tracks or repeated detection, and assigns the QTID to the allocated logical ID. ).

[0105] 7. Step S4: Determine whether to merge TIDs and assign logical IDs.

[0106] Determine if the FPS of all TIDs for the same hawthorn instance is greater than 30 frames, weighted by confidence. This is based on the sum of the existence scores obtained in step S2.4. When, confirm the merging of all TIDs for this hawthorn instance and generate a unique logical ID; when At this time, the TID of this hawthorn instance will not be merged and will not proceed to subsequent processes. Threshold The calibration can be based on the actual false detection rate. Preferably, the threshold is... Take 30.

[0107] 8. Step S5: Grading determination and industrial control output.

[0108] S5.1 Calculation of cumulative defect area.

[0109] Once a fruit leaves the ROI area or is determined to have completed tracking, the system freezes the logical ID and calculates its cumulative defect area. The largest historical area representing QTID. As a defect quantification value, the following is calculated:

[0110] ;

[0111] S5.2 Classification Threshold Determination (Example).

[0112] processor according to Output a grading signal. The grading threshold, measured in pixels, can be determined based on the device's field of view and grading standards. For example: Grade 0 (Excellent): Grade 1 (processed fruit): Level 2 (Secondary Result): The thresholds mentioned above are for illustrative purposes only and can be adjusted according to the fruit variety, imaging resolution, and quality standards in practical applications.

[0113] S5.3 Control output (example).

[0114] The edge computing unit writes the grading results and the spatial or temporal positioning information of the corresponding logical ID into the PLC register. The PLC then drives actuators such as pneumatic nozzles and electromagnetic levers to complete the sorting.

[0115] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or equivalent substitutions can be made to the above embodiments without departing from the spirit of the present invention, and such modifications or substitutions should all fall within the scope of protection of the present invention.

Claims

1. A hierarchical correlation tracking and defect quantification method for rotating spherical fruit targets, characterized in that, include: Acquire video sequences of the detection area of ​​the roller conveyor belt; Target detection and instance segmentation are performed on each frame of the video sequence to obtain the bounding box of the parent fruit target and the pixel mask of the child defect target, and the confidence score of each is obtained. Based on the spatial inclusion relationship between the defect mask and the bounding box of the fruit target, the defect mask is bound to its corresponding fruit target, and a parent-child hierarchical data structure of fruit trajectory identifier TID and defect trajectory identifier QTID is established. For fruit target trajectory association: a Kalman filter prediction model is established for each fruit trajectory, the squared Mahalanobis distance between the current observation and the prediction is calculated, and the result is compared with the motion consistency gating threshold. Comparison to filter candidate trajectories; Extract the fruit appearance ReID feature vector and update the appearance prototype vector using exponential moving average. Calculate the cosine distance between appearance prototype vectors and use it as a gating threshold for appearance consistency. Comparison to filter candidate trajectories; Candidate trajectory pairs that pass both motion consistency gating and appearance consistency gating are confirmed for multiple frames. After confirmation, the confidence weighted existence sum S of multiple TIDs belonging to the same physical fruit instance is calculated. When S exceeds a preset threshold, the fruit instance is confirmed as a valid target and a unique logical ID is assigned. Perform topological deduplication on defective targets: map the defect masks bound to the same fruit target to the local coordinate system of the fruit target, and rasterize them into a fixed-size N×N binary feature mesh; Calculate the signature crossover ratio (CLORD) for candidate defects, and ensure that the CLORD is not less than the signature threshold. At the same time, defect observations from different perspectives are associated with the same physical defect; For a set of defect observations that are determined to be the same physical defect, the QTID with the largest historical defect area is selected as the representative QTID of the physical defect, and an alias mapping is established so that other QTIDs in the set point to the representative QTID. When a TID merging occurs, the defect list is deduplicated by performing a union based on the representative QTID and the representative QTID is assigned to the logical ID; The cumulative defect area of ​​all QTIDs under the logical ID is counted and compared with a preset grading threshold to obtain the grading result. The grading result is used to generate control instructions and sent to the PLC or actuator to complete the sorting execution.

2. The method according to claim 1, characterized in that, The step of binding the defect mask to its corresponding fruit target based on the spatial inclusion relationship between the defect mask and the fruit target bounding box includes: Calculate the geometric center point or centroid of the defect mask; when the geometric center point is located inside the bounding box of a fruit target, bind the defect mask to the fruit target.

3. The method according to claim 1, characterized in that, The association of the fruit target trajectory also includes time gating, and the method further includes: Candidate trajectories are selected from the historical trajectory cache pool, and the frame interval between the current trajectory and the candidate trajectories is calculated. Only when Only when the time threshold is not greater than a preset time control threshold will the candidate trajectory be subjected to Kalman prediction and enter the association screening.

4. The method according to claim 1, characterized in that, The state vector of the Kalman filter prediction model is an 8-dimensional vector containing the bounding box center coordinates, aspect ratio, height, and rate of change. The motion consistency gating threshold... Used for gating the squared Mahalanobis distance.

5. The method according to claim 1, characterized in that, The method of updating the appearance prototype vector using exponential moving average includes: when the trajectory obtains the appearance ReID feature vector in the current frame, according to... Update in a different way The smoothing coefficient is the exponential moving average; when the trajectory does not obtain a valid appearance ReID feature vector in the current frame, the appearance prototype vector of the previous frame is frozen and not updated.

6. The method according to claim 1, characterized in that, The multi-frame confirmation includes: Maintain support counters for candidate trajectory pairs; When the candidate trajectory pairs simultaneously satisfy the condition that the squared Mahalanobis distance is not greater than And the cosine distance is no greater than At that time, the support counter increments; When the condition is not met, the support counter is decremented; When the support counter reaches the preset confirmation threshold M, the candidate trajectory pair is confirmed to belong to the same physical fruit instance and TID merging is performed.

7. The method according to claim 1, characterized in that, The confidence-weighted existence sum S is: the cumulative sum of the detection confidence scores of each TID merged into the same fruit instance in each frame. When S exceeds the preset existence threshold, the fruit instance corresponding to the logical ID is confirmed as a valid target.

8. The method according to claim 1, characterized in that, The process of mapping defect masks bound to the same fruit target to the local coordinate system of the fruit target and rasterizing them into a fixed-size N×N binary feature mesh includes: Using the top left corner of the fruit target bounding box as the origin and the width and height of the bounding box as the scale, the defect mask pixels are normalized and truncated to the [0,1] interval; the normalized defect mask is rasterized into an N×N binary grid using the nearest neighbor method.

9. The method according to claim 1, characterized in that, The signature crossover ratio is the crossover ratio between two N×N binary grids, wherein the signature crossover ratio is not less than a signature threshold. At that time, defect observations from different perspectives are associated with the same physical defect, including; When the intersection-union ratio between the two N×N binary grids is not less than the signature threshold When the corresponding defect observations are determined to belong to the same physical defect instance.

10. The method according to claim 1, characterized in that, The selection of the representative QTID also includes deterministic rules, and the method further includes: When multiple QTIDs of the same physical defect instance have the same historical maximum defect area, the QTID with the longer frame count is selected as the representative QTID.

11. The method according to claim 1, characterized in that, The cumulative defect area is the sum of the historical maximum defect areas of each QTID represented under the logical ID. The control command includes the classification result and the spatial or temporal positioning information corresponding to the logical ID.

12. A hierarchical correlation tracking and defect quantification system for rotating spherical fruit targets, characterized in that, include: The image acquisition unit is used to acquire video sequences of the detection area of ​​the roller conveyor belt; The detection and segmentation unit is used to perform target detection and instance segmentation on each frame of the video sequence to obtain the bounding box of the parent fruit target and the pixel mask of the child defect target, and obtain their respective confidence scores. The hierarchical binding unit is used to bind the defect mask to its corresponding fruit target based on the spatial inclusion relationship between the defect mask and the fruit target bounding box, and to establish a parent-child hierarchical data structure between the fruit trajectory identifier TID and the defect trajectory identifier QTID. The fruit trajectory association unit is used to perform trajectory association on fruit targets: it establishes a Kalman filter prediction model for each fruit trajectory, calculates the squared Mahalanobis distance between the current observation and the prediction, and applies it to a motion consistency gating threshold. The comparison is used to filter candidate trajectories; it is also used to extract fruit appearance ReID feature vectors and update appearance prototype vectors using exponential moving average, calculate the cosine distance between appearance prototype vectors and use it as a gating threshold for appearance consistency. The comparison is used to filter candidate trajectories; it is also used to perform multi-frame confirmation on candidate trajectory pairs that pass both the motion consistency gating and the appearance consistency gating. After confirmation, the confidence weighted existence sum S of multiple TIDs belonging to the same physical fruit instance is calculated, and when S exceeds a preset threshold, the fruit instance is confirmed as a valid target and a unique logical ID is assigned. The defect deduplication unit is used to perform topological deduplication on defect targets: mapping defect masks bound to the same fruit target to the local coordinate system of the fruit target and rasterizing them into a fixed-size N×N binary feature mesh; it is also used to calculate the signature intersection-union ratio (CIU) for candidate defects, and ensure that the CIU is not less than a signature threshold. At the same time, it associates defect observations from different perspectives as the same physical defect; it is also used to select the QTID with the largest historical defect area as the representative QTID of the physical defect for the defect observation set that is determined to be the same physical defect, and to establish an alias mapping so that other QTIDs in the set point to the representative QTID. It is also used to perform union deduplication on the defect list based on the representative QTID and assign the representative QTID to the logical ID when TID merging occurs; The hierarchical decision unit is used to calculate the cumulative defect area of ​​all QTIDs under the logical ID and compare it with a preset hierarchical threshold to obtain the hierarchical result. The control output unit is used to generate control commands from the grading results and send them to the PLC or actuator to complete the sorting execution.

13. The system according to claim 12, characterized in that, The fruit trajectory association unit is also used to perform time-gated screening of candidate trajectory pairs before trajectory merging, and to perform motion consistency gating based on the squared Mahalanobis distance and the motion consistency threshold. and the cosine distance and appearance consistency gating threshold Implement joint gating.

14. The system according to claim 12, characterized in that, The defect deduplication unit is also used to maintain the historical maximum defect area of ​​each QTID for the same physical defect instance, and use the QTID with the largest historical maximum defect area as the representative QTID, and point other QTIDs in the same physical defect instance to the representative QTID through alias mapping.

Images