Pear picking robot binocular positioning method based on YOLO-CDS depth detection network and RAFT-Stereo cooperation

Abstract

The invention provides a pear picking robot binocular positioning method based on YOLO-CDS depth detection network and RAFT-Stereo cooperation, and aims to solve the problems that obstacles such as iron wires and ropes in a modern orchard affect pear recognition and a semi-global matching (SGM) algorithm in a binocular vision positioning system is insufficient in feature matching robustness in a complex scene. A YOLO-CDS model is introduced, and a C2f-EMBC module, a DySample and a Shape-IoU loss function are utilized, so that the feature extraction and bounding box regression precision is improved; rAFT-Stereo is adopted to replace an SGM algorithm, so that the robustness of parallax estimation is enhanced; and the depth value is optimized in combination with an IQR (quartile distance), so that the positioning precision is improved. Experiments show that the average accuracy of the method on various pear data sets is 97.5%, the average relative error of pear coordinate positioning is smaller than 3%, the positioning success rate in an orchard picking test is 100%, the picking success rate is 91.93%, powerful technical support is provided for pear picking robot application, and the method has important application value and popularization prospects.

Description

Technical Field

[0001] This invention belongs to the field of agricultural harvesting robot technology, specifically involving a binocular localization method for pear-harvesting robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration. Background Technology

[0002] The pear industry ranks third among my country's fruit industries, with Jiangsu Province, thanks to its unique natural conditions, becoming one of the major pear producing areas. As early as January 2014, Jiangsu Province was designated as a key region for specialty pear production. Through continuous optimization of its specialty pear structure, strengthened technological research and development, and standardized production, Jiangsu Province has built an integrated pear industry chain encompassing production, processing, and marketing, showcasing a broad development blueprint.

[0003] However, with the continuous expansion of pear planting area in Jiangsu Province, the pear industry also faces many challenges. On the one hand, traditional manual harvesting methods are labor-intensive, inefficient, and the fruit is easily damaged, making it difficult to meet the needs of the rapid development of the modern pear industry. On the other hand, the aging of the workforce and the shortage of labor are becoming increasingly serious problems, and the difficulty and high cost of hiring have become major bottlenecks restricting the development of Jiangsu Province's modern agricultural industrial system (pear). Against the backdrop of accelerating agricultural modernization and rapid advancements in intelligent technology, developing intelligent algorithms capable of automatically identifying and locating pears is of significant practical importance for improving agricultural production efficiency and reducing the labor burden. It has become a key step in promoting the development of Jiangsu Province's modern agricultural industrial system (pear) and improving production efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration, so as to solve the problem of pear identification and localization in complex backgrounds.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: This invention provides a binocular localization method for pear-picking robots based on the YOLO-CDS depth detection network and RAFT-Stereo collaboration, including: The binocular camera acquires color images from the left eye and inputs them into the YOLO-CDS depth detection network. Based on the different occlusion conditions of pears in the natural environment of the orchard, the pears are classified into multiple categories. The channel weights are dynamically adjusted through the C2f-EMBC module to optimize the sampling process. The RAFT-Stereo algorithm is used to replace the SGM algorithm of the stereo camera for disparity map calculation, which improves the robustness of stereo visual localization and the accuracy of depth estimation. The interquartile range optimization is performed on the depth values ​​obtained by YOLO-CDS and RAFT-Stereo matching to remove mismatched feature points in the background and foreground, and the optimal depth information of the target region is calculated. Based on the obtained optimal depth information, and combined with the transformation relationship between the camera coordinate system and three-dimensional space, the precise three-dimensional spatial coordinates of the pear fruit are calculated.

[0006] Further steps in constructing the YOLO-CDS depth detection network include: Based on the YOLOv8 model, the structure and feature extraction capabilities of the BottleNeck module of YOLOv8 were analyzed. The EMBC module was designed to replace the BottleNeck module, and the improved module was named C2f-EMBC to improve the ability to capture key features of pear fruit targets of different classifications. Replace all upsampling modules in YOLOv8s with the DySample module, which can dynamically adjust sampling offset and grouping strategy, to more accurately capture feature details of different pear categories; The Shape-IoU loss function, which takes into account the shape and scale of the real bounding box, is used to replace the CIoU loss function, making the regression of the target predicted box more accurate and reducing the number of mismatched feature points in the detected box. The improved network was named YOLO-CDS, where C, D, and S represent the first letters of C2f-EMBC, DySample, and Shape-IoU, respectively.

[0007] Furthermore, the usage process of the C2f-EMBC module includes: The EMBC module first performs feature fusion and enhancement on the input feature map using a 1×1 ordinary convolution, and then increases the dimensionality of the image. Then, Depthwise Conv is used to reduce the number of model parameters and computational cost. In order to further compensate for the loss of feature information that may be caused by the lack of channel fusion in Depthwise Conv, the Effective SE module is used to dynamically adjust the weights of each channel, optimize the feature expression ability of the neural network, and make the diverse features more informative while reducing information loss. Then, a 1×1 ordinary convolution is used again to fuse and reduce the dimensionality of the feature maps; Finally, through the Dropout layer, some activated neurons are randomly selected to stop working with a certain probability, thereby achieving regularization, model averaging, and feature independence enhancement, and improving the generalization of the model. After passing through the Shortcut structure, the feature map processed by the EMBC module is concatenated with the input feature map. In the entire structure, the SiLu activation function is used to improve the model's expressive efficiency and convergence speed.

[0008] Furthermore, the DySample module is a dynamic upsampling module. The DySample is designed from the perspective of point sampling. It dynamically generates an upsampling kernel by generating an offset, replacing the fixed kernel of traditional bilinear interpolation, and achieving the effect of dynamic upsampling.

[0009] Furthermore, the Shape-IoU loss function is formulated as follows: Where scale is the scaling factor, which is related to the scale of the target in the dataset; ww and hh are the weight coefficients in the horizontal and vertical directions, respectively, and their values ​​are related to the shape of the ground truth box; w, h: the width and height of the predicted box; x, y: the coordinates of the center point of the predicted box. , The width and height of the actual bounding box; , : Center coordinates of the ground truth bounding box; c: Normalized length; scale: Scale factor, which controls the influence of large and small targets; θ: Exponential hyperparameter, usually set to 4; (t∈{w,h}) reflects the relative difference between the predicted bounding box and the ground truth bounding box in the corresponding dimension (width or height); : Overall measure of shape difference; IoU: Intersection over Union (IoU) ratio between predicted bounding boxes and ground truth bounding boxes; Position error term; Overall regression loss.

[0010] Furthermore, the RAFT-Stereo stereo matching algorithm includes a feature extractor, a correlation pyramid, and an iterative update operator based on gated recurrent units (GRUs). The computational process includes: Key features are extracted from the corrected left and right images, and a correlation pyramid is constructed to capture multi-scale correlations. The context encoder is used to extract image context features and initialize the hidden state, and the disparity map is preset to zero. Subsequently, during the iterative optimization process, the gated loop unit intelligently samples relevant features from the relevant pyramid based on the current disparity estimate, and combines the initial image features, the current hidden state, and the relevant features to generate an updated hidden state and disparity estimate; Through multi-layer cyclic field transformation, RAFT-Stereo gradually refines the positioning of corresponding points in the left and right images, and finally outputs a smooth and accurate disparity map, which, together with the relevant parameters of the stereo camera, calculates the depth information of the target.

[0011] Furthermore, the depth information of the target is calculated using relevant parameters from the stereo camera, including: Depth information can be derived from the principle of similar triangles, and the specific calculation formula is as follows: Where B is the baseline distance, which is the horizontal distance between the optical centers of the left and right cameras; f is the focal length of the camera. , Let P be the horizontal pixel coordinates of point P in the left and right images; Z is the pixel disparity value between the left and right cameras after RAFT-Stereo optimization, representing the horizontal offset of point P's projection in the left and right pixel coordinate systems; Z is the depth coordinate of point P in the camera coordinate system. The horizontal and vertical coordinates (X, Y) of point P can be calculated as follows: Where (u, v) are the pixel coordinates of point P in the image; , P is the principal point of the image; X is the horizontal coordinate of point P in the camera coordinate system; Y is the vertical coordinate of point P in the camera coordinate system.

[0012] Furthermore, the depth values ​​obtained from YOLO-CDS and RAFT-Stereo matching are optimized using the interquartile range (IQR) to remove mismatched feature points in the background and foreground. In calculating the optimal depth information for the target region, the IQR method involves arranging all values ​​from smallest to largest and dividing them into four equal parts. The values ​​at the three dividing points are called quartiles. There are three quartiles: the first quartile, commonly referred to as the lower quartile, the second quartile, the median, and the third quartile, called the upper quartile, denoted by Q1, Q2, and Q3, respectively. Q1, Q2, and Q3 represent the 25th, 50th, and 75th percentile positions after arranging all values ​​in the sample from smallest to largest, respectively. IQR is the difference between the third quartile (Q3) and the first quartile (Q1) in the dataset. Based on IQR, Q1, and Q3, upper and lower thresholds can be set to filter and remove data points below the lower limit or above the upper limit. The specific calculation formulas for the upper and lower thresholds are as follows: In the formula This indicates the lower limit of the depth value threshold; This represents the upper limit of the depth value threshold; k is the threshold coefficient, usually taken as 1.5. The filtering formula is: Z e ={Z e1 Z e2 ,...,Z en} represents the set of valid depth points.

[0013] Beneficial effects: In summary, developing intelligent algorithms capable of automatically identifying and locating pears is of significant practical importance for improving agricultural production efficiency and reducing labor burden. Addressing the impact of obstacles such as wires and ropes on pear identification in modern orchards, and the insufficient robustness of semi-global matching (SGM) algorithms in complex scenarios within binocular vision localization systems, this invention proposes a binocular localization method for pear-picking robots based on the YOLO-CDS depth detection network and RAFT-Stereo collaboration. In the YOLO-CDS depth detection network, based on the YOLOv8s model, the structure and feature extraction capabilities of the BottleNeck module in YOLOv8s are analyzed. A redesigned C2f-EMBC module is used to replace it, improving the ability to capture key features of different pear categories. All upsampling modules in YOLOv8s are replaced with the DySample module, which can dynamically adjust sampling offset and grouping strategies, aiming to more accurately capture the feature details of different pear categories. The Shape-IoU loss function, which considers the true bounding box shape and scale, is used instead of the CIoU loss function. This makes the regression of the predicted bounding box more accurate, reduces the number of mismatched feature points in the detected bounding box, and improves the detection accuracy of the model in complex environments. In the RAFT-Stereo stereo matching technology collaborative optimization algorithm, RAFT-Stereo replaces the traditional SGM algorithm by using a deep learning model to capture the pixel-level correspondence between image pairs and generate an accurate disparity map, aiming to improve the robustness of binocular vision localization and the accuracy of depth estimation. Furthermore, the Interquartile Range (IQR) optimization algorithm is introduced to optimize depth value calculation, reduce depth calculation errors caused by mismatched points, and thus improve the accuracy of the entire localization system. Attached Figure Description

[0014] Figure 1 This is a diagram of the YOLO-CDS network structure of the present invention; Figure 2 This is a schematic diagram showing the design details of the C2f-EMBC module of the present invention; Figure 3 This is a structural diagram of the DySample module of the present invention; Figure 4This is a comparison diagram of the ground truth bounding box and the predicted bounding box of the Shape-IoU loss function of this invention; Figure 5 This is a flowchart of the RAFT-Stereo disparity estimation process of the present invention; Figure 6 This is a schematic diagram of the binocular ranging principle of the present invention; Figure 7 This is the principle of the interquartile range optimization method of the present invention; Figure 8 This is a sample image of the pear fruit dataset of the present invention; Figure 9 This is a diagram of the test platform and test scenario of the present invention; Figure 10 This is a diagram of the experimental platform and testing base of the present invention; Figure 11 This is a performance comparison chart of the improved model of this invention; Figure 12 This is a graph showing the detection results of seven types of pears using YOLOv8s and YOLO-CDS of the present invention; Figure 13 This is a comparison of the disparity maps of the SGBM algorithm and the RAFT-Stereo algorithm of this invention. Figure 14 This is a comparison of depth map imaging between the RAFT-Stereo of this invention and the IQR-optimized RAFT-Stereo; Figure 15 These are some scenarios of failed harvesting during the orchard deployment experiment of this invention; Figure 16 This is a flowchart illustrating the usage of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0016] Please see Figures 1-14 Current research generally treats object detection and 3D localization as independent tasks. However, in reality, there is an error coupling effect between the two in stereo vision systems. The depth calculation of binocular cameras relies on disparity matching algorithms, and its accuracy is directly affected by the quality of feature points within the object detection box. Light interference and obstacle occlusion in natural orchards can affect the detection performance of the detection model. When the detection model misses or falsely detects, it will lead to two types of problems with the feature points in the target area: Mismatch point proliferation: erroneous detection boxes contain a large number of non-target features (such as pseudo-contours caused by foliage occlusion, or mismatched pixels in the background), which introduces outliers into disparity calculations.

[0017] Missing valid information: Insufficient matching points within the correct detection bounding box may be caused by uneven lighting or missing textures, which reduces the confidence of the depth value.

[0018] To address the aforementioned issues and systematically resolve challenges such as lighting, obstacle occlusion, and background interference, thereby improving the localization accuracy of pear targets, a binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration is proposed. The improvement strategy mainly includes the following aspects: (1) To address the impact of varying light levels and various obstructions (such as branches, leaves, and wires) on the accuracy of pear detection in complex orchard environments, this invention introduces the C2f-EMBC module. This module, by combining Depthwise Conv and Effective SE, significantly enhances the ability to capture key features of the target while reducing the number of model parameters and computational load. Specifically, Depthwise Conv reduces computational complexity by decomposing traditional convolution operations; Effective SE dynamically adjusts channel weights to optimize feature representation.

[0019] (2) In order to further improve the model's ability to detect small targets and detailed features, this invention introduces the DySample dynamic upsampling module. Traditional upsampling methods (such as bilinear interpolation) are prone to losing detailed information when processing small targets, while DySample can better capture semantic information in the image by dynamically adjusting the sampling offset and grouping strategy.

[0020] (3) In order to optimize bounding box regression and reduce mismatches caused by scaling, this invention adopts the Shape-IoU loss function. Shape-IoU optimizes the bounding box matching evaluation by introducing shape constraints and dynamic weights, thereby reducing mismatches.

[0021] (4) To improve the robustness and accuracy of binocular visual localization, this invention introduces the RAFT-Stereo stereo matching algorithm to replace the traditional SGM algorithm. RAFT-Stereo, through iterative optimization of a deep learning model, can effectively address challenges in complex environments such as low-texture regions, occlusion, and lighting variations. Specifically, RAFT-Stereo refines disparity estimation step by step through a feature extractor, a correlation pyramid, and an iterative update operator using a GRU, generating an accurate disparity map.

[0022] (5) To further improve the accuracy of pear depth information, this invention introduces a depth value optimization strategy based on the interquartile range (IQR). In complex orchard environments, the YOLO-CDS detection frame may contain a large number of non-pear pixels (such as foregrounds obscured by leaves or noise in the background), and the depth values ​​of these pixels will interfere with the calculation of the true depth of the pear. By calculating the quartiles (Q1 and Q3) and IQR of the depth values, upper and lower thresholds are dynamically set to remove outliers.

[0023] Based on the above improvement strategies, this invention provides a binocular localization method for pear-picking robots based on the collaborative optimization of the YOLO-CDS depth detection network and RAFT-Stereo, including: S1. The binocular camera acquires a color image of the left eye and inputs it into the YOLO-CDS depth detection network. Based on the different occlusion conditions of pears in the natural environment of the orchard, the pears are classified and identified in multiple ways. The channel weights are dynamically adjusted through the C2f-EMBC module to optimize the sampling process. S2. Replace the SGM algorithm of the stereo camera with the RAFT-Stereo algorithm for disparity map calculation to improve the robustness of stereo visual localization and the accuracy of depth estimation. S3. Perform interquartile range optimization on the depth values ​​obtained by YOLO-CDS and RAFT-Stereo matching, remove mismatched feature points in the background and foreground, and calculate the optimal depth information of the target region. S4. Based on the obtained optimal depth information, and combined with the transformation relationship between the camera coordinate system and three-dimensional space, calculate the precise three-dimensional spatial coordinates of the pear.

[0024] This method addresses the impact of various obstructions on pear tree identification in modern orchards. Based on the characteristics of these obstructions, the YOLO-CDS model is used to achieve accurate detection of pears in multiple categories. Furthermore, an improved stereo matching framework combining "iterative optimization of deep learning models + dynamic removal of statistical outliers" is proposed. By constructing a novel visual localization method that integrates a RAFT-Stereo-based depth estimation algorithm with quartile statistical optimization, the localization accuracy of pears in complex orchards is improved.

[0025] The above methods will be analyzed in a comprehensive manner below.

[0026] In step S1, the YOLO-CDS deep detection network is based on the YOLOv8 model. The structure and feature extraction capabilities of the BottleNeck module of YOLOv8 are analyzed, and the EMBC module (Efficient Multi-scale Bottleneck Convolution) is designed to replace the BottleNeck module. The improved module is named C2f-EMBC to improve the ability to capture key features of different pear fruit targets. Replace all upsampling modules in YOLOv8s with the DySample module, which can dynamically adjust sampling offset and grouping strategy, to more accurately capture feature details of different pear categories; The Shape-IoU loss function, which takes into account the shape and scale of the real bounding box, is used to replace the CIoU loss function, making the regression of the target predicted box more accurate and reducing the number of mismatched feature points in the detected box. The improved network was named YOLO-CDS, where C, D, and S represent the first letters of C2f-EMBC, DySample, and Shape-IoU, respectively.

[0027] To better illustrate the YOLO-CDS depth detection network, the EMBC module in C2f-EMBC first uses a 1×1 ordinary convolution to perform feature fusion and enhancement on the input feature map, and then upscales the image. The principle is as follows: Figure 2 As shown in a; Then, Depthwise Conv is used to reduce the number of model parameters and computational cost, based on the following principle: Figure 2 As shown in b, in order to further compensate for the loss of feature information that may be caused by the lack of channel fusion in Depthwise Conv, the Effective SE module is used to dynamically adjust the weights of each channel, optimize the feature expression ability of the neural network, and make the diverse features more informative while reducing information loss. Then, a 1×1 ordinary convolution is used again to fuse and reduce the dimensionality of the feature maps, the principle of which is as follows: Figure 2 As shown in c; Finally, through the Dropout layer, some activated neurons are randomly selected and stopped working with a certain probability to achieve regularization, model averaging, and feature independence enhancement, thereby improving the model's generalization ability. After passing through the Shortcut structure, the feature map processed by the EMBC module is concatenated with the input feature map. In the entire structure, the SiLu activation function is used to improve the model's expression efficiency and convergence speed. The designed EMBC module replaces the BottleNeck module in C2f, and the generated C2f-EMBC module can extract target features more comprehensively and efficiently. At the same time, the weights between channels are dynamically adjusted to optimize the quality of feature representation, making it scale more evenly in the three dimensions of depth, width, and resolution, thereby achieving better detection performance.

[0028] In addition, the DySample module is a fast, efficient and universal dynamic upsampling module. Unlike common kernel-based upsampling methods, DySample is designed from the perspective of point sampling. It dynamically generates the upsampling kernel by generating offsets, replacing the fixed kernel of traditional bilinear interpolation, thereby achieving the effect of dynamic upsampling.

[0029] In the aforementioned dynamic upsampling process, DySample divides a point into s² upsampling points, thereby improving the clarity and effectiveness of the upsampling. The structure of DySample introduced in this invention is as follows: Figure 3 As shown, firstly, an upsampling scale factor s and a feature of size C×H×W are given. Figure X Considering the adjacent s 2 The positional relationships between the points are such that we divide the feature map into g groups along the channel dimension, using a single input and output channel with C and 2s respectively. 2 The linear layer generates g groups of different offsets, resulting in a relatively uniform initial sampling position distribution, which are eventually merged into a single array of size 2gs. 2 The offset O, representing ×H×W, is used by the output of the linear layer to generate a set of vectors to guide the displacement of each point during the feature map upsampling process. Then, through pixel recombination, the generated offset O is shaped into a 2g×sH×sW shape, so that each position corresponds to the upsampled position s. 2 The offset information of each point is used in this process, which is called the "linear + pixel shuffling" (LP) structure. The sampling set S is finally obtained by adding the offset O to the original sampling grid g. The positions in the sampling set S are used to resample the assumed bilinear interpolation, and finally an upsampled feature of size C×sH×sW is generated. Figure X ', where the original sampling grid g usually refers to the coordinates of the upsampling points arranged according to rules. The addition operation here is to add the offset corresponding to each upsampling point to the original regular position coordinates to form a new set of sampling point coordinates.

[0030] Furthermore, for the Shape-IoU loss function, calculating the loss using the shape and scale of the bounding box makes the bounding box regression more accurate, effectively improving model detection accuracy and reducing mismatched feature points in the target detection box. The ground truth box B of Shape-IoU... gt And prediction box B p like Figure 4 As shown, the specific formula is as follows: Where scale is the scaling factor, which is related to the scale of the target in the dataset; ww and hh are the weight coefficients in the horizontal and vertical directions, respectively, and their values ​​are related to the shape of the ground truth box; w, h: the width and height of the predicted box; x, y: the coordinates of the center point of the predicted box. , The width and height of the actual bounding box; , : Center coordinates of the ground truth bounding box; c: Normalized length; scale: Scale factor, which controls the influence of large and small targets; θ: Exponential hyperparameter, usually set to 4; (t∈{w,h}) reflects the relative difference between the predicted bounding box and the ground truth bounding box in the corresponding dimension (width or height); : Overall measure of shape difference; IoU: Intersection over Union (IoU) ratio between predicted bounding boxes and ground truth bounding boxes; Position error term; Overall regression loss.

[0031] In step S2 above, the RAFT-Stereo stereo matching algorithm, which is a stereo matching algorithm based on deep learning, is used to optimize the disparity map calculation in binocular vision localization. It includes three core components: a feature extractor, a correlation pyramid, and an iterative update operator based on a gated recurrent unit (GRU). Figure 5The computational process is intuitively demonstrated: First, key features are extracted from the corrected left and right images, and a correlation pyramid is constructed to capture multi-scale correlations. Simultaneously, a context encoder is used to extract image context features and initialize the hidden state, with the disparity map preset to zero. Subsequently, during iterative optimization, the GRU(s) module, i.e., the gated recurrent unit, intelligently samples relevant features from the correlation pyramid based on the current disparity estimate. Combining the initial image features, the current hidden state, and the correlation features, it generates an updated hidden state and disparity estimate. Through multi-layer cyclic field transformation, RAFT-Stereo gradually refines the localization of corresponding points in the left and right images, ultimately outputting a smooth and accurate disparity map. The disparity calculation mechanism of RAFT-Stereo ultimately yields a disparity map, but it cannot directly provide the target's depth information; it requires calculation of the target's depth information using parameters from a stereo camera. Figure 6 As shown, depth information can be derived from the principle of similar triangles, and the specific calculation formula is as follows: Where B is the baseline distance, which is the horizontal distance between the optical centers of the left and right cameras; f is the focal length of the camera. , Let P be the horizontal pixel coordinates of point P in the left and right images; Z is the pixel disparity value between the left and right cameras after RAFT-Stereo optimization, representing the horizontal offset of point P's projection in the left and right pixel coordinate systems; Z is the depth coordinate of point P in the camera coordinate system. The horizontal and vertical coordinates (X, Y) of point P can be calculated as follows: Where (u, v) are the pixel coordinates of point P in the image; , P is the principal point of the image; X is the horizontal coordinate of point P in the camera coordinate system; Y is the vertical coordinate of point P in the camera coordinate system.

[0032] In step S3 above, the interquartile range optimization of the depth values ​​obtained from YOLO-CDS and RAFT-Stereo matching is specifically performed by optimizing the detected depth values ​​based on the aforementioned detection and localization process. A pear depth value calculation method based on (IQR) dynamic threshold optimization is introduced. The interquartile method involves arranging all values ​​from smallest to largest and dividing them into four equal parts. The values ​​at the three dividing points are the quartiles. Figure 7As shown in b, there are three quartiles. The first quartile, commonly referred to as the lower quartile, the second quartile, is the median, and the third quartile, called the upper quartile, are denoted by Q1, Q2, and Q3, respectively. Q1, Q2, and Q3 represent the 25th, 50th, and 75th percentiles of all values ​​in the sample, arranged from smallest to largest. The IQR is the difference between the third quartile (Q3) and the first quartile (Q1) in the dataset. Based on the IQR, Q1, and Q3, upper and lower thresholds can be set to filter and remove data points below the lower limit or above the upper limit. The specific formulas for calculating the upper and lower thresholds are: In the formula This indicates the lower limit of the depth value threshold; This represents the upper limit of the depth value threshold; k is the threshold coefficient, usually taken as 1.5. The filtering formula is: Ze = {Ze1, Ze2, ..., Zen} represents the set of effective depth points. For different pear targets, this method dynamically adjusts the upper and lower limits of the threshold, thereby optimizing target depth estimation through adaptive thresholding.

[0033] from Figure 10 As can be seen from a, in the detection box of the pear target, the coverage area of ​​pear pixels is usually much larger than that of non-pear pixels, and the difference in depth value between the occlusion or background and the pear target is generally large. Therefore, the interquartile range (IQR) dynamic thresholding optimization method can effectively remove a large number of non-pear pixels, ensuring that the final calculated target box depth Zbox is closer to the true value.

[0034] The following is a specific experimental analysis based on the above method: 1. Experimental environment and parameter settings: The model training platform and parameter settings of this invention are as follows: The training platform is an AMD Ryzen 7 7745HX CPU computer with a main frequency of 3.6GHz, a maximum turbo frequency of 5.1GHz, 16GB of RAM, and an NVIDIA RTX 4060 LapTop graphics card. The operating system environment is Windows 11, and the software includes CUDA 12.1, cuDNN 8.8.1, Python 3.9.18, and PyTorch 2.1.2. The training parameters were set as follows: the model input size was 640×640, three epochs were used for warm-up, the warm-up momentum was 0.8, the batch size was 16, the number of multilinear processes (num workers) was 8, stochastic gradient descent (SGD) was used to optimize the network parameters, the initial learning rate was 0.01, cosine annealing was used to optimize the learning rate, the momentum was 0.937, the weight decay was 0.0005, the number of training epochs was 1000, and the early stopping mechanism (patience) was set to 50. If the model did not improve after 50 epochs of training, training was stopped early.

[0035] Location Experiment Platform and Environment: Due to the difficulty and significant measurement error in accurately locating the true coordinates of pears in an orchard environment, this invention verifies the proposed depth optimization method through indoor testing and the construction of a horizontal trellis experimental platform. The equipment required for the depth optimization experiment includes a ZED2I binocular camera, a camera tripod, a laser rangefinder, a laptop computer, a USB data transmission cable, and a simulated horizontal trellis (including branches, leaves, and pear models). Figure 9 As shown in Figure a, the trellis is 1500mm high, 1000mm wide, and 2000mm long; the simulated pear fruit has a diameter between 60-80mm; the camera equipment is mounted on a tripod to ensure stability and accuracy. Figure 9 b-9d represent front lighting, backlighting, and nighttime supplemental lighting test scenarios, respectively.

[0036] Harvesting Test Platform and Orchard Environment: To verify the reliability of the detection and positioning algorithm of this invention in actual harvesting scenarios, this invention was deployed on a self-made pear harvesting robot platform for orchard testing. Unobstructed labels (NO) were set as the harvesting targets. The test platform mainly consisted of a tracked chassis, a Jetson AGX Orin control system, a Reimann six-DOF robotic arm, a vision system, and an end effector system. The test base was the horizontal trellis pear orchard of the Zhenjiang Academy of Agricultural Sciences in Jiangsu Province (coordinates). Specific agronomical parameters were as follows: row spacing 4000mm, plant spacing 4000mm, trellis height 1800mm, trunk height within the range of 1100-1200mm, and mature pears distributed within an area 1450mm to 1850mm above the ground. The test platform and orchard were as follows: Figure 10 As shown.

[0037] 2. Construction of a pear fruit dataset in a complex environment: The pear fruit dataset of this invention was collected from several representative pear orchards, including the pear orchard of Jiangsu Academy of Agricultural Sciences, the Gaochun (Pear) Extension and Demonstration Base of Jiangsu Modern Agricultural Industry System, the pear orchard of Zhenjiang Academy of Agricultural Sciences, and the Xuzhou (Pear) Extension and Demonstration Base of Jiangsu Modern Agricultural Industry System. The dataset covers three main cultivation modes: central leader, Y-shaped trellis, and horizontal trellis, and includes two pear varieties: Su Cui and Qiu Yue.

[0038] Data collection time: two harvest seasons in 2022 and 2023, including pear images under different weather conditions and time periods such as cloudy days, sunny days, mornings, noon, and evenings. Images were acquired at night with the help of auxiliary light sources to ensure the potential for all-weather operation.

[0039] Data acquisition equipment: Sony FDR-AX60 4K camera, Redmi K40, Huawei P40, iPhone 14, etc., to increase data diversity and improve robustness. The shooting distance is set to 0.3-1.5m according to the robot's operation.

[0040] To quantify the impact of different distances on background complexity, this invention categorizes the acquired image background into three levels based on the distance between the camera and the fruit: simple background (distance between 0.3m and 0.5m), general background (distance between 0.5m and 1m), and complex background (distance exceeding 1m). A pear fruit dataset is provided as an example. Figure 8 As shown. A total of 850 images were selected from the dataset and stored in JPEG format. 750 images were used as the training and validation set, divided in an 8:2 ratio, and underwent image enhancement processing. 100 images were used as the model test set, which consists of model test images that are different from the training samples and are not enhanced. The aim is to evaluate the model's generalization ability in real-world scenes without enhancement processing.

[0041] 3. Test evaluation indicators To verify the effectiveness of the improved detection model, this invention uses precision (P), recall (R), average precision (AP), computational complexity Floating Point Operations (FLOPs), and Frames Per Second (FPS) to evaluate the effectiveness of the model. Precision represents the ratio of correct detection results to total detection results, and the calculation formula is shown in (1). Recall represents the proportion of correct detection results among all true results, and the calculation formula is shown in (2). Average precision (AP) is the area enclosed by the PR curve in two-dimensional coordinates, which reflects the overall performance of the model. The average precision for multiple categories is mAP. This paper uses mAP with an IoU of 0.5 as the main evaluation criterion, and the calculation formula is shown in (3).

[0042] Wherein, TP (True Positives) represents the number of true samples correctly classified as true samples by the model; FP (False Positives) represents the number of false samples incorrectly detected as true samples by the model; FN (False Negatives) represents the number of true samples incorrectly detected as false samples by the model; Precision: represents the proportion of correct results in the model's detection results; Recall: represents the proportion of positive samples detected by the model from all positive samples; N represents the number of classes; represents the average accuracy for the i-th class; mAP: the mean of the average accuracy across all classes, which is one of the main performance metrics for object detection. In terms of efficiency and speed, we use FLOPs to evaluate the computational cost and complexity of the model, and FPS to evaluate the inference speed.

[0043] 4. Ablation experiment of the YOLO-CDS algorithm model proposed in this invention. This invention conducted detection ablation experiments on a dataset specifically designed for multi-class pear fruit recognition to evaluate the contribution of the proposed improvements to the overall model performance. Three sets of control experiments were set up. In experiment T1, all C2f modules of YOLOv8s were replaced with C2f-EMBC modules. In experiment T2, all upsampling modules in the YOLOv8s model were replaced with DySample modules. Finally, replacing CIoU in experiment T2 with Shape-IoU yielded the complete YOLO-CDS network. The specific test results for these three sets of experiments are shown in Table 1. Table 1 Ablation Test Figure 11 'a' shows the comparison results of each model on the mAP index in the ablation experiment, which intuitively reflects the improvement effect of the proposed model on the overall performance. Figure 11 Figure b presents a comparison of the AP values ​​of the improved YOLO-CDS and YOLOv8s models for different categories. As can be seen from the figure, YOLO-CDS outperforms YOLOv8s in AP detection across all categories, further demonstrating the comprehensive advantages of YOLO-CDS in the multi-class pear fruit classification task. This indicates that the proposed improved strategy exhibits excellent detection results in the multi-class pear fruit detection task in the complex environment of orchards. More accurate detection results can reduce non-target feature points in the detection bounding box, contributing to improved accuracy in binocular pear fruit localization.

[0044] 5. Comparison Experiment of the YOLO-CDS Algorithm Model Proposed in This Invention To further verify the performance advantages of the YOLO-CDS algorithm model proposed in this invention in the pear detection task, this paper conducted a comparative experiment with other mainstream object detection network algorithms, using mAP and FPS with a confidence threshold of 0.5 as evaluation indicators, as shown in Table 2.

[0045] Table 2 Performance index values ​​for different models Based on the data in Table 2, it can be concluded that YOLO-CDS achieved the best mAP and relatively high FPS in the comparative experiment, and also showed good overall detection results in terms of multi-class detection accuracy (AP). Compared with the original network, although the detection speed decreased slightly, YOLO-CDS's mAP was significantly improved, increasing by 1.9 percentage points. Compared with SSD, Faster R-CNN, and YOLOv9c, YOLO-CDS showed the best performance in both mAP and FPS. This indicates that the proposed YOLO-CDS model can maintain a high detection speed while exhibiting the best detection performance when detecting pears in natural environments. This can reduce the localization error of the binocular camera to a certain extent, which is beneficial to improving the working efficiency of pear-picking robots in complex environments.

[0046] 6. Experiment on pear fruit detection using the YOLO-CDS algorithm model proposed in this invention. To further verify the superiority of the model, a comparative experiment was conducted on the detection performance of YOLOv8s and YOLO-CDS in a test set different from the training samples. The detection results of the test set are shown in Table 3. In the seven categories of pear fruit, the detection results of YOLO-CDS were superior to those of YOLOv8s. The AP of YOLO-CDS for NO, OL, OB, OF, OW, OC, and OS were 1.2, 1.5, 2.9, 2.2, 2.1, 1.5, and 2.1 percentage points higher than those of YOLOv8s, respectively.

[0047] Table 3. Results of YOLOv8s and YOLO-CDS for detecting seven types of pears. YOLOv8s and YOLO-CDS detection results in complex environments are as follows: Figure 12 As shown in the figure, manually drawn black rectangles mark fruit examples incorrectly detected by the two models, while manually drawn white rectangles mark fruit examples correctly detected by the two models. NO, OL, OB, OF, OW, OC, and OS are detected in red, orange, yellow, green, blue, purple, and pink boxes, respectively. The figure shows that YOLOv8s incorrectly detected four fruits, while YOLO-CDS only missed one. This indicates that the YOLO-CDS model can better distinguish different pear categories in multi-class pear detection tasks in complex orchard environments. This helps prevent the robot from detecting OL, OB, OF, OW, OC, or OS as pickable targets, reducing damage to the robot from obstacles during harvesting and improving harvesting efficiency.

[0048] 7. Comparison experiment between the RAFT-Stereo algorithm and the SGBM algorithm proposed in this invention. This invention compares and evaluates the disparity maps generated by the SGBM algorithm and the RAFT-Stereo algorithm in different scenes for images with a resolution of 640 pixels. The detection results are as follows: Figure 13 As shown. During testing in an indoor environment ( Figure 13 a) Due to minimal interference from natural light, pears possess distinct visual characteristics, allowing both the SGBM and RAFT-Stereo algorithms to accurately delineate their outlines. When tested outdoors... Figure 13 (b) Under natural light interference, the performance of the SGBM algorithm deteriorates significantly, failing to fully delineate the pear's outline, while the RAFT-Stereo algorithm can still clearly delineate the pear's outline. In outdoor multi-target and occlusion situations ( Figure 13 c) Compared to the SGBM algorithm, the RAFT-Stereo algorithm can still completely outline the shape of the pear. This is especially true when facing backlighting conditions outdoors. Figure 13 d) The SGBM algorithm is completely unable to reproduce the shape and boundary of the pear, but the RAFT-Stereo algorithm can still completely and clearly outline the pear's contour. These experiments show that under the influence of multiple interference factors such as low light environment, natural light interference (front lighting, back lighting), same color scheme, and occlusion, the RAFT-Stereo algorithm still has extremely strong stability, can accurately capture the contour details of the pear, and effectively generate disparity maps. This can meet the needs of pear target recognition tasks in complex orchard environments, thereby achieving accurate pear localization.

[0049] 8. Comparison experiment before and after the proposed interquartile range (IQR) optimization algorithm. This invention conducted experiments on pears at different distances (0.3m, 0.6m, and 1.0m) under indoor conditions. Ten sets of data were collected for each distance, and the mean depth value of each set was calculated. The experimental results are shown in Table 4. The comparative data in the table show that after optimization based on the interquartile range (IQR), the accuracy of pear depth measurement was improved at different distances, and the positioning accuracy increased with increasing measurement distance. Through this optimization, the algorithm can maintain high accuracy even at long distances, significantly improving the overall positioning accuracy and providing a more reliable technical guarantee for pear depth measurement.

[0050] Table 4 Comparison of pear fruit positioning depth values ​​before and after optimization 9. Test experiments of the RAFT-stereo algorithm based on IQR optimization proposed in this invention. The stability and robustness of the IQR-optimized RAFT-stereo algorithm under varying lighting conditions were tested. Positioning experiments were conducted on targets ranging from 300-1000mm under front lighting, backlighting, and nighttime supplemental lighting conditions. The test scenarios are as follows: Figure 14 As shown in Tables 5, 6, and 7, the pear fruit positioning results are presented in Tables 5, 6, and 7. The experimental data from these tables indicate that the improved pear fruit positioning method described in this paper has an average relative error of less than 3% within a distance of 300-1000 mm, meeting the spatial positioning requirements of the pear fruit harvesting robot.

[0051] Table 5. Results of three-dimensional coordinate detection of pear fruit under direct lighting conditions Table 6 Results of 3D coordinate detection of pear fruit under backlight conditions Table 7 Results of 3D coordinate detection of pear fruit under nighttime conditions 10. Orchard deployment experiment of the identification and positioning algorithm proposed in this invention. To verify the reliability of the detection and localization algorithm of this invention in actual harvesting scenarios, an orchard harvesting experiment was conducted using this method. Considering issues such as robot harvesting efficiency and robot hardware damage, this invention only sets the unobstructed label NO as the harvesting target to avoid hardware damage to the robot caused by orchard obstacles such as branches and wire mesh. Table 8 shows the test results of the orchard detection and localization effect of the harvesting robot. The detection and localization success rate was 100%, but the harvesting success rate was only 91.93%. This is mainly because in actual orchards, when the harvesting robot detects the same target from different angles, it will generate different labels. For example, as shown in 15a, when the robot detects from directly below, it is a harvestable target NO, but if the robot detects from directly in front, it is an unharvestable target OB. When the robot detects from directly below and performs the harvesting task, due to the randomness of the robotic arm path planning, the robot's end effector may collide with the pear target, branches, and other obstacles, causing the pear's position to change or the grasping process to be obstructed. Figure 15 As shown in b, 15c and 15d. Figure 15 In step b, the end of the branch is pushed, causing the pear fruit to change position and the branch to obstruct the end from grasping the target; Figure 15 In c, the random path planning of the robotic arm causes the end effector to collide with the pear fruit target while moving to the target position; Figure 15 If the pears encounter a wire during harvesting, they will fall to the ground.

[0052] Table 8 Results of orchard location and harvesting experiment Orchard trials show that the pear fruit positioning method proposed in this invention can effectively complete the task of detecting and locating harvestable target pear fruits in complex orchards. Combined with the robot harvesting strategy, it can reduce robot hardware damage to a certain extent and help improve the operating efficiency of the pear fruit harvesting robot.

[0053] In summary, addressing the interference caused by obstacles such as wires and ropes in modern orchards on pear identification, and the insufficient robustness of semi-global matching (SGM) algorithms in binocular vision localization systems under complex scenarios, this invention proposes a binocular localization method for pear-picking robots based on the collaborative optimization of the YOLO-CDS depth detection network and RAFT-Stereo. In the YOLO-CDS depth detection network, based on the YOLOv8s model, the original BottleNeck module is replaced by a redesigned C2f-EMBC module, significantly enhancing the ability to capture key features of different pear categories. Simultaneously, a DySample dynamic upsampling module is introduced to accurately capture the feature details of the pears, and the Shape-IoU loss function is used instead of the CIoU loss function to further optimize the regression accuracy of the target prediction box and reduce the number of mismatched feature points in the detection box, thereby significantly improving the model's detection accuracy in complex environments. In the localization module, RAFT-Stereo stereo matching technology replaces the traditional SGM algorithm, utilizing a deep learning model to capture pixel-level correspondences between image pairs and generate a high-precision disparity map, significantly improving the robustness of binocular visual localization and depth estimation accuracy. Furthermore, by introducing the interquartile range (IQR) optimization algorithm, depth calculation errors caused by mismatched points are effectively eliminated, further optimizing depth value calculation and significantly improving the accuracy and reliability of the entire localization system. Experimental results show that the YOLO-CDS model achieves an average precision (mAP@50) of 97.5% on multi-class pear fruit datasets, a 1.9 percentage point improvement compared to the original YOLOv8s, while reducing model complexity by 5G. Within the range of 300mm to 1000mm, the average relative error of pear fruit coordinates is less than 3%, meeting the high-precision spatial localization requirements of the pear fruit picking robot.

[0054] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A binocular localization method for a pear-picking robot based on YOLO-CDS depth detection network and RAFT-Stereo collaboration, characterized in that, include: The binocular camera acquires color images from the left eye and inputs them into the YOLO-CDS depth detection network. Based on the different occlusion conditions of pears in the natural environment of the orchard, the pears are classified into multiple categories. The channel weights are dynamically adjusted through the C2f-EMBC module to optimize the sampling process. The RAFT-Stereo algorithm is used to replace the SGM algorithm of the stereo camera for disparity map calculation, which improves the robustness of stereo visual localization and the accuracy of depth estimation. The interquartile range optimization is performed on the depth values ​​obtained by YOLO-CDS and RAFT-Stereo matching to remove mismatched feature points in the background and foreground, and the optimal depth information of the target region is calculated. Based on the obtained optimal depth information, and combined with the transformation relationship between the camera coordinate system and three-dimensional space, the precise three-dimensional spatial coordinates of the pear fruit are calculated.

2. The binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration as described in claim 1, characterized in that, The steps to construct the YOLO-CDS depth detection network include: Based on the YOLOv8 model, the structure and feature extraction capabilities of the BottleNeck module of YOLOv8 were analyzed. The EMBC module was designed to replace the BottleNeck module, and the improved module was named C2f-EMBC to improve the ability to capture key features of pear fruit targets of different classifications. Replace all upsampling modules in YOLOv8s with the DySample module, which can dynamically adjust sampling offset and grouping strategy, to more accurately capture feature details of different pear categories; The Shape-IoU loss function, which takes into account the shape and scale of the real bounding box, is used to replace the CIoU loss function, making the regression of the target predicted box more accurate and reducing the number of mismatched feature points in the detected box. The improved network was named YOLO-CDS, where C, D, and S represent the first letters of C2f-EMBC, DySample, and Shape-IoU, respectively.

3. The binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration as described in claim 1, characterized in that, The usage process of the C2f-EMBC module includes: The EMBC module first performs feature fusion and enhancement on the input feature map using a 1×1 ordinary convolution, and then increases the dimensionality of the image. Then, Depthwise Conv is used to reduce the number of model parameters and computational cost. In order to further compensate for the loss of feature information that may be caused by the lack of channel fusion in Depthwise Conv, the Effective SE module is used to dynamically adjust the weights of each channel, optimize the feature expression ability of the neural network, and make the diverse features more informative while reducing information loss. Then, a 1×1 ordinary convolution is used again to fuse and reduce the dimensionality of the feature maps; Finally, through the Dropout layer, some activated neurons are randomly selected to stop working with a certain probability, thereby achieving regularization, model averaging, and feature independence enhancement, and improving the generalization of the model. After passing through the Shortcut structure, the feature map processed by the EMBC module is concatenated with the input feature map. In the entire structure, the SiLu activation function is used to improve the model's expressive efficiency and convergence speed.

4. The binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration as described in claim 1, characterized in that, The DySample module is a dynamic upsampling module. DySample is designed from the perspective of point sampling. It dynamically generates an upsampling kernel by generating an offset, replacing the fixed kernel of traditional bilinear interpolation, and achieving the effect of dynamic upsampling.

5. The binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration as described in claim 1, characterized in that, The Shape-IoU loss function is defined as follows: Where scale is the scaling factor, which is related to the scale of the target in the dataset; ww and hh are the weight coefficients in the horizontal and vertical directions, respectively, and their values ​​are related to the shape of the ground truth box; w, h: the width and height of the predicted box; x, y: the coordinates of the center point of the predicted box; , The width and height of the actual bounding box; , : Center coordinates of the ground truth bounding box; c: Normalized length; scale: Scale factor, which controls the influence of large and small targets; θ: Exponential hyperparameter, usually set to 4; (t∈{w,h}) reflects the relative difference between the predicted bounding box and the ground truth bounding box in the corresponding dimension (width or height); : Overall measure of shape difference; IoU: Intersection over Union (IoU) ratio between predicted bounding boxes and ground truth bounding boxes; Position error term; Overall regression loss.

6. The binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration as described in claim 1, characterized in that, The RAFT-Stereo stereo matching algorithm includes a feature extractor, a correlation pyramid, and an iterative update operator based on gated recurrent units. The computation process includes: Key features are extracted from the corrected left and right images, and a correlation pyramid is constructed to capture multi-scale correlations. The context encoder is used to extract image context features and initialize the hidden state, and the disparity map is preset to zero. Subsequently, during the iterative optimization process, the gated loop unit intelligently samples relevant features from the relevant pyramid based on the current disparity estimate, and combines the initial image features, the current hidden state, and the relevant features to generate an updated hidden state and disparity estimate; Through multi-layer cyclic field transformation, RAFT-Stereo gradually refines the positioning of corresponding points in the left and right images, and finally outputs a smooth and accurate disparity map, which, together with the relevant parameters of the stereo camera, calculates the depth information of the target.

7. The binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration as described in claim 1, characterized in that, The depth information of the target is calculated using relevant parameters from the stereo camera, including: Depth information can be derived from the principle of similar triangles, and the specific calculation formula is as follows: Where B is the baseline distance, which is the horizontal distance between the optical centers of the left and right cameras; f is the focal length of the camera. , Let P be the horizontal pixel coordinates of point P in the left and right images; Z is the pixel disparity value between the left and right cameras after RAFT-Stereo optimization, representing the horizontal offset of point P's projection in the left and right pixel coordinate systems; Z is the depth coordinate of point P in the camera coordinate system, and the horizontal and vertical coordinates (X, Y) of point P can be calculated as follows: Where (u, v) are the pixel coordinates of point P in the image; , P is the principal point of the image; X is the horizontal coordinate of point P in the camera coordinate system; Y is the vertical coordinate of point P in the camera coordinate system.

8. The binocular localization method for pear-picking robots based on YOLO-CDS depth detection network and RAFT-Stereo collaboration as described in claim 1, characterized in that, The depth values ​​obtained by YOLO-CDS and RAFT-Stereo matching are optimized using interquartile range (IQR) to remove mismatched feature points in the background and foreground. In calculating the optimal depth information of the target region, the IQR method involves arranging all values ​​from smallest to largest and dividing them into four equal parts. The values ​​at the three dividing points are called quartiles. There are three quartiles: the first quartile is commonly referred to as the lower quartile, the second quartile is the median, and the third quartile is called the upper quartile, denoted by Q1, Q2, and Q3, respectively. Q1, Q2, and Q3 represent the 25th, 50th, and 75th percentiles of all values ​​in the sample after arranging them from smallest to largest, respectively. IQR is the difference between the third quartile and the first quartile in the dataset. At this point, upper and lower thresholds can be set based on IQR, Q1, and Q3 to filter and remove data points that are below the lower limit or above the upper limit. The specific calculation formulas for the upper and lower thresholds are as follows: In the formula This indicates the lower limit of the depth value threshold; This represents the upper limit of the depth value threshold; k is the threshold coefficient, usually taken as 1.5; the filtering formula is: Z e ={Z e1 Z e2 ,...,Z en } represents the set of valid depth points.

Images