Robot identification and positioning method for picking fruit medicinal materials

Abstract

The invention discloses a robot identification and positioning method for fruit medicinal material picking, and the method comprises the following steps: S1, integrating a near infrared spectrum, multispectral vision and flexible touch triple sensor module at the tail end of a mechanical arm, and synchronously collecting three types of data; s2, extracting exclusive feature sub-vectors based on specific components and physical characteristics of medicinal materials, and dynamically fusing through an attention mechanism to obtain fruit feature vectors; s3, constructing a point cloud cluster in combination with multi-source data, segmenting clustered fruits by adopting a density clustering and morphological feature fusion algorithm, and realizing coordinate calibration and dynamic compensation through inertial measurement and frame difference updating; and S4, constructing a maturity evaluation model based on the feature vector, outputting a grade and determining a picking priority. The method breaks through the limitation of single perception, improves the feature discrimination and positioning precision, solves the problems of difficult segmentation of clustered fruits and dynamic positioning drift, gives consideration to the picking efficiency and the quality of medicinal materials, and provides support for intelligent harvesting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fruit medicinal material picking, and particularly relates to a robot recognition and positioning method for fruit medicinal material picking. BACKGROUND

[0002] The picking recognition and positioning of fruit medicinal materials are key links of standardized planting and harvesting of traditional Chinese medicinal materials, and directly affect the quality, effective component content and large-scale production efficiency of medicinal materials. With the development of agricultural intelligence, picking robots are gradually applied to the harvesting of fruit crops, but there are still many problems to be solved in the existing technology in view of the specific needs of fruit medicinal materials.

[0003] The perception system of the existing fruit medicinal material picking robot mainly depends on a single sensor, such as only using a visual sensor or a single spectrum sensor, which cannot comprehensively obtain the complete information of the fruit. The visual sensor can only capture the external morphological characteristics, which is easily disturbed by environmental factors such as light changes and branch and leaf shielding, resulting in fluctuations in recognition accuracy. Although the spectrum sensor can reflect part of the internal components, it lacks the perception of the physical properties and spatial morphology of the fruit surface. The application of the tactile sensor is limited to surface property detection, and cannot associate the internal components with the maturity, so the data dimension of the single sensor is limited, the effective information coverage of the initial perception data is low, the data reliability is insufficient, and it is difficult to support the subsequent accurate feature extraction and recognition positioning, which becomes the primary bottleneck restricting the performance improvement of the picking robot.

[0004] In the aspect of feature extraction and fusion, the existing technology mostly adopts a general feature extraction scheme, without fully considering the specificity of different fruit medicinal materials and the differences in physical properties. The maturity of Fructus Aurantii is closely related to the content of characteristic volatile oil components, and the core quality indicator of Prunus mume is the characteristic organic acid component. The general feature extraction does not capture these specific indicators, resulting in insufficient discrimination of the feature vector. At the same time, the feature fusion mostly adopts a simple superposition method with fixed weights, which cannot dynamically adjust the weight distribution according to the effective information content of the features in different scenes, and the redundant information and invalid interference signals are difficult to be suppressed, so the features of different fruit medicinal materials are easily confused, the maturity judgment error is large, and the accuracy of recognition is seriously affected.

[0005] The segmentation and dynamic positioning of clustered fruits present another technical challenge. Fruit-bearing medicinal materials, such as trifoliate orange, often grow in clusters with tightly adhered fruits. Existing technologies can easily generate fruit point cloud clusters that contain non-fruit impurities such as branches, leaves, and soil, resulting in low purity. Segmentation of clustered fruits often employs single morphological algorithms or traditional clustering methods, lacking a comprehensive consideration of fruit morphological characteristics and density distribution. This makes it difficult to accurately delineate the spatial boundaries of individual fruits, leading to incomplete or incorrect segmentation. Furthermore, factors such as wind disturbance and robot movement in the field environment can cause dynamic displacement of the fruits. Existing positioning technologies lack effective sensor attitude calibration and dynamic compensation mechanisms, making it easy for the spatial coordinates of the fruits to drift. The positioning error of the optimal picking contact point is relatively large, failing to meet the stringent positioning accuracy requirements for medicinal material harvesting.

[0006] In terms of harvesting decisions, existing robots mostly adopt a random harvesting mode of picking any fruit they see, or simply judge the maturity based on the appearance and color of the fruit, without establishing a correlation model between maturity and the content of effective internal components. This approach results in a low harvesting rate of mature fruits, the misharvesting of immature fruits leading to resource waste, and the rotting and spoilage of overripe fruits due to failure to harvest them in time. This not only reduces the overall quality and average content of effective components of medicinal materials, but also affects the efficiency of large-scale operations, which contradicts the industry demand for high-quality and precise harvesting in the standardized planting and harvesting of fruit-based medicinal materials.

[0007] Therefore, this application proposes a robot recognition and localization method for harvesting fruit-type medicinal materials. Summary of the Invention

[0008] One objective of this invention is to propose a robot identification and localization method for harvesting fruit-type medicinal materials. This invention can achieve accurate identification, maturity determination, and spatial positioning of fruit-type medicinal materials such as trifoliate orange and green plum. Through multimodal data collaborative acquisition and intelligent fusion, it breaks through the limitations of traditional single perception, general feature extraction, and static positioning. It efficiently solves technical problems such as difficulty in segmenting clustered fruits, positioning drift in dynamic scenes, and confusion in medicinal material identification, significantly improving the accuracy of fruit-type medicinal material identification and the precision and stability of positioning. At the same time, through maturity-oriented harvesting priority decision-making, it achieves the harvesting of superior fruits, taking into account both the efficiency of large-scale harvesting and the control of medicinal material quality, providing reliable technical support for the intelligent and standardized harvesting of fruit-type medicinal materials.

[0009] A robot identification and localization method for harvesting fruit-type medicinal materials according to an embodiment of the present invention includes the following steps:

[0010] S1. Multimodal perception data acquisition: Near-infrared spectroscopy, multispectral vision and flexible tactile sensor modules are integrated at the end of the robotic arm of the picking robot to simultaneously collect near-infrared spectral data, multispectral vision data and fruit pre-contact tactile data of the target area.

[0011] S2. Extraction of medicinal material-specific features: Based on the components and physical properties of bitter orange peel and green plum, specific feature sub-vectors are extracted from spectral data, visual data, and tactile data respectively, and then fused through an attention mechanism to obtain the fruit feature vector;

[0012] The formula for the process of fusing feature vectors using the attention mechanism is as follows: ,and ;

[0013] in, The fused fruit feature vector is the core input for subsequent identification and maturity determination. , , These are the weight coefficients for the spectral feature sub-vector, visual feature sub-vector, and tactile feature sub-vector, respectively. The weight values ​​are dynamically adjusted based on the fruit's feature recognition accuracy. For spectral feature vectors, For visual feature sub-vectors, For tactile feature sub-vectors;

[0014] The core algorithm formula for fusing feature vectors using the attention mechanism is as follows: ;

[0015] in, The final fused fruit feature vector has a dimension of , These correspond to spectral, visual, and tactile feature sub-vectors, respectively. For spectral feature vectors, For visual feature sub-vectors, These are tactile feature sub-vectors, all three having a dimension of 1. ;

[0016] For the first The attention weights of the class feature subvectors are output by the feature importance evaluation module, and the calculation formula is as follows:

[0017]

[0018] in, For the first Information entropy of class feature vectors;

[0019] S3. Dynamic spatial positioning modeling: Combine multi-source data to construct fruit point cloud clusters, use density clustering and morphological feature fusion algorithms to segment clustered fruits, and achieve accurate calibration and dynamic compensation of fruit spatial coordinates through inertial measurement and frame difference update.

[0020] S4. Picking priority decision: Based on feature vectors, a maturity assessment model is constructed to output the fruit maturity level and determine the picking priority, thus completing the identification and location of the target fruit.

[0021] Preferably, the triple sensor module in step S1 includes a near-infrared spectral acquisition unit, a multispectral visual acquisition unit, and a flexible tactile acquisition unit. The three units work together to simultaneously acquire spectral, visual, and tactile perception data of the target area and the fruit surface.

[0022] Preferably, the method for extracting the spectral feature sub-vector in step S2 is to smooth and preprocess the derivative of the near-infrared spectral data, and extract the absorbance values ​​of the corresponding characteristic peaks based on the characteristic volatile oil components of the trifoliate orange and the characteristic organic acid components of the green plum, to form the spectral feature sub-vector.

[0023] The SG smoothing formula used in the smoothing and derivative preprocessing is:

[0024]

[0025] in, For the smoothed first Spectral values ​​at each wavelength point To smooth out half the window width, The smoothing coefficient is SG. The first in the original spectrum Spectral values ​​at each wavelength point;

[0026] The formula for processing the first derivative in the smoothing and derivative preprocessing is as follows:

[0027]

[0028] in, For the first First derivative spectral values ​​at each wavelength point The smoothed first , Spectral values ​​at each wavelength point The interval between adjacent wavelength points, the characteristic peak absorbance value is When taking the extreme value, the corresponding value.

[0029] Preferably, the method for extracting visual feature sub-vectors in step S2 is to use an improved U-Net network to segment the fruit region in the multispectral image, extract the hue features, vegetation index features and contour morphology features of the fruit, and form visual feature sub-vectors.

[0030] The loss function of the improved U-Net network is calculated as follows:

[0031]

[0032] in, This is the total loss value. The cross-entropy loss is calculated using the following formula:

[0033]

[0034] in, Image pixel coordinates, For pixel-based true labels, To predict the probability that this pixel represents a fruit for the network. , The Dice coefficient loss is calculated using the following formula:

[0035]

[0036] The vegetation index calculation formula for the improved U-Net network is as follows:

[0037]

[0038] in, The pixel grayscale values ​​in the near-infrared band of a multispectral image. This represents the pixel grayscale value in the red light band. The value range is [-1, 1], for fruits and branches. The difference is ≥0.3.

[0039] Preferably, in step S2, the extraction of tactile feature sub-vectors is achieved by filtering the pre-contact pressure data to extract pressure distribution features and texture features that reflect the roughness of the fruit surface, thus forming tactile feature sub-vectors.

[0040] Preferably, in step S3, the fruit point cloud cluster is constructed by acquiring initial point cloud data using binocular vision, supplementing distance information using a depth camera, and removing non-fruit point clouds by combining spectral feature vectors, thus retaining a pure fruit point cloud cluster.

[0041] Preferably, the clustered fruit segmentation algorithm in step S3 adopts a density-based clustering algorithm, combined with the fruit outline morphology features, to segment the point cloud cluster of clustered trifoliate orange fruits and clarify the spatial boundary of individual fruits.

[0042] The core judgment formula of the density-based clustering algorithm is:

[0043] Key point judgment:

[0044] in, For a point in a point cloud cluster, For Centered on Let be the set of points contained within the neighborhood of the radius. The number of points in this set. The minimum number of neighborhood points, points that satisfy this condition. As the core point;

[0045] Density reachability criterion: Existence of a sequence of points ,in , .

[0046] Preferably, the spatial coordinate calibration method in step S3 is to establish a world coordinate system based on the robot base, calibrate the sensor attitude in real time through the inertial measurement unit, convert the centroid coordinates of the fruit point cloud into world coordinates, and determine the optimal picking contact point in combination with the fruit shape.

[0047] The core formula for segmenting clustered fruits based on density-based clustering algorithm in step S3 is:

[0048]

[0049] in, Let be the local density of the i-th point in the point cloud cluster, used to determine whether this point is the cluster center. Let i be the set of all points within the neighborhood of the i-th point. Let be the Euclidean distance between the i-th point and the j-th point. The neighborhood radius, This is an indicator function; when the value inside the parentheses is ≤0... =1, otherwise =0, meaning only distances ≤ The adjacent points are used to calculate the local density.

[0050] Preferably, in step S3, the dynamic compensation mechanism updates the spatial coordinates of the fruit in real time through difference analysis of adjacent data frames, compensates for the dynamic displacement of the fruit in the field environment, and ensures the continuity of positioning.

[0051] Preferably, in step S4, the maturity assessment model takes spectral features and visual hue features as input, outputs the fruit maturity level through a classification algorithm, and plans the harvesting priority accordingly.

[0052] The beneficial effects of this invention are:

[0053] 1. In this invention, a triple sensor module of near-infrared spectroscopy, multispectral vision, and flexible touch is integrated at the end of the robotic arm. Through a synchronous triggering mechanism, the spatiotemporal consistency of the three types of data is achieved, which not only covers the internal components and external morphology of the fruit, but also takes into account the surface physical characteristics. This completely solves the defects of traditional single sensors that can only obtain local information and are easily affected by environmental interference. The complementary verification of the three types of data greatly improves the effective information coverage of the initial sensing data, laying a full-dimensional data foundation for subsequent feature extraction and recognition positioning.

[0054] 2. This invention targets the specific components and physical properties of bitter orange peel and green plum, and extracts exclusive feature sub-vectors such as spectral characteristic peak absorbance, visual vegetation index, and tactile texture. This avoids the neglect of the specificity of medicinal materials by general feature extraction. At the same time, it realizes the dynamic allocation of feature weights through a dual attention fusion formula, uses information entropy to quantify the effective information content of features, and can automatically amplify the weight ratio of high-value features. This allows the fused fruit feature vector to focus on core information and suppress redundant interference, thereby greatly improving the distinguishability of feature vectors for fruit categories and maturity, increasing the accuracy of medicinal material identification, reducing the error in maturity judgment, and solving the technical problem of feature overlap and identification confusion among different fruit types of medicinal materials.

[0055] 3. In this invention, a high-purity fruit point cloud cluster is constructed using binocular vision, a depth camera, and spectral screening. Interference points such as branches, leaves, and soil are removed. A density clustering and morphological feature fusion algorithm is used, combined with the core point judgment formula, the local density calculation formula, and the density reachability judgment rule, to achieve precise segmentation of clustered and adhering fruits such as trifoliate oranges. The spatial boundaries of individual fruits are clearly defined. Then, the sensor attitude is calibrated in real time through an inertial measurement unit. Combined with a dynamic compensation mechanism for frame difference updates, the displacement of fruits caused by field wind and robot movement is effectively compensated. Ultimately, the spatial coordinate positioning accuracy of fruits is high, and the positioning error of the optimal picking contact point is low, which completely solves the industry pain points of difficult segmentation of clustered fruits and positioning drift in dynamic scenes.

[0056] 4. This invention constructs a maturity assessment model based on fused feature vectors, using spectral features and visual hue features as core inputs. It outputs three levels of maturity and determines the harvesting priority, enabling the robot to prioritize the harvesting of first-level mature fruits with peak effective ingredient content. This design significantly improves the harvesting rate of mature fruits, avoids resource waste caused by harvesting immature fruits and losses from the rotting of overripe fruits, and greatly increases the average content of effective ingredients in harvested medicinal materials compared to random harvesting. At the same time, the harvesting efficiency is significantly improved, achieving precise harvesting and high-quality harvesting. It balances the efficiency of large-scale operations with the control of medicinal material quality, meeting the industry needs for standardized planting and harvesting of fruit-based medicinal materials. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0058] Fig. 1 This is a schematic diagram of the overall process of a robot recognition and positioning method for harvesting fruit-type medicinal materials proposed in this invention;

[0059] Fig. 2 This diagram illustrates the detailed steps of extracting medicinal plant-specific features in a robot recognition and localization method for harvesting fruit-type medicinal plants proposed in this invention. Detailed Implementation

[0060] To make the technical means and objectives and effects of the present invention easier to understand, the embodiments of the present invention will be described in detail below with reference to specific illustrations.

[0061] Please see Figs. 1-2 This invention proposes a robot recognition and localization method for harvesting fruit-type medicinal materials, comprising the following steps:

[0062] S1. Multimodal perception data acquisition: Near-infrared spectroscopy, multispectral vision and flexible tactile sensor modules are integrated at the end of the robotic arm of the picking robot to simultaneously collect near-infrared spectral data, multispectral vision data and fruit pre-contact tactile data of the target area.

[0063] Specifically, the triple sensor module includes a near-infrared spectral acquisition unit, a multispectral visual acquisition unit, and a flexible tactile acquisition unit. Through the preset synchronous triggering command of the robot control system, the three units work together to ensure that the acquisition timestamps are consistent. They synchronously acquire near-infrared spectral data and multispectral visual data of the target area, as well as tactile perception data of the fruit surface acquired by the robotic arm driving the sensor module to make pre-contact with the fruit surface. This achieves the synchronous acquisition and storage of the three types of perception data. After the data acquisition is completed, the step of extracting the medicinal material's specific features can be entered.

[0064] S2. Extraction of Specific Features for Medicinal Herbs: Based on the components and physical properties of Citrus aurantium and Prunus mume, specific feature vectors are extracted from spectral data, visual data, and tactile data, respectively. For the extraction of spectral feature vectors, the collected near-infrared spectral data is first smoothed and preprocessed using derivatives. The smoothing process uses the SG smoothing method, the formula of which is:

[0065]

[0066] in, For the smoothed first Spectral values ​​at each wavelength point To smooth out half the window width, The smoothing coefficient is SG. The first in the original spectrum Spectral values ​​at each wavelength point;

[0067] The derivative preprocessing uses the first derivative, and the formula is as follows:

[0068]

[0069] in, For the first First derivative spectral values ​​at each wavelength point The smoothed first , Spectral values ​​at each wavelength point The interval between adjacent wavelength points, the characteristic peak absorbance value is When taking the extreme value, the corresponding value;

[0070] Based on the above preprocessing results, the absorbance values ​​of the characteristic peaks corresponding to the characteristic volatile oil components of Citrus aurantium and the characteristic organic acid components of Prunus mume were extracted to form spectral feature subvectors. .

[0071] For the extraction of visual feature sub-vectors, an improved U-Net network is used to segment the fruit region in the multispectral image. The loss function of this improved U-Net network is calculated as follows:

[0072]

[0073] in, This is the total loss value. The cross-entropy loss is calculated using the following formula:

[0074]

[0075] in, Image pixel coordinates, For pixel-based true labels, To predict the probability that this pixel represents a fruit for the network. , The Dice coefficient loss is calculated using the following formula:

[0076]

[0077] Meanwhile, the vegetation index calculation formula for the improved U-Net network is:

[0078]

[0079] in, The pixel grayscale values ​​in the near-infrared band of a multispectral image. This represents the pixel grayscale value in the red light band. The value range is [-1, 1], for fruits and branches. If the difference is ≥0.3, based on the network segmentation results, the hue features, vegetation index features, and contour morphology features of the fruit are extracted to form a visual feature sub-vector. .

[0080] For the extraction of tactile feature sub-vectors, the collected pre-contact pressure data is filtered to remove environmental interference noise, and then the pressure distribution features and texture features reflecting the roughness of the fruit surface are extracted to form tactile feature sub-vectors. .

[0081] After extracting the three feature vectors, they are fused using an attention mechanism. The formula for the attention mechanism to fuse feature vectors is as follows:

[0082]

[0083] and ,in The fused fruit feature vector is the core input for subsequent identification and maturity determination. , , These are the weight coefficients for the spectral feature sub-vector, visual feature sub-vector, and tactile feature sub-vector, respectively, and the weight values ​​are dynamically adjusted according to the feature recognition of the fruit.

[0084] Meanwhile, the core algorithm formula for the attention mechanism to fuse feature vectors is:

[0085]

[0086] and ;in, The final fused fruit feature vector has a dimension of , These correspond to spectral, visual, and tactile feature sub-vectors, respectively. For spectral feature vectors, For visual feature sub-vectors, These are tactile feature sub-vectors, all three having a dimension of 1. ;

[0087] The attention weight for the i-th class feature sub-vector is output by the feature importance evaluation module, and its calculation formula is as follows:

[0088]

[0089] in, For the first Information entropy of class feature vectors.

[0090] The next step is dynamic spatial positioning modeling:

[0091] S3. Dynamic spatial positioning modeling: Combine multi-source data to construct fruit point cloud clusters, use density clustering and morphological feature fusion algorithms to segment clustered fruits, and achieve accurate calibration and dynamic compensation of fruit spatial coordinates through inertial measurement and frame difference update.

[0092] Specifically, firstly, a fruit point cloud cluster is constructed by combining multi-source data. This involves acquiring initial point cloud data of the target area using a binocular vision system, supplementing distance information by collecting data using a depth camera, and then filtering the initial point cloud data by combining the extracted spectral feature vectors to remove point cloud data that does not correspond to fruit, thus retaining a pure fruit point cloud cluster.

[0093] Next, a density clustering and morphological feature fusion algorithm was used to segment the clustered fruits. The clustered fruit segmentation algorithm adopted a density-based clustering algorithm, combined with the fruit outline morphological features, to segment the point cloud cluster of clustered trifoliate orange fruits and clarify the spatial boundary of individual fruits.

[0094] The core judgment formula of this density-based clustering algorithm includes core point judgment and density reachability judgment. The core point judgment formula is as follows:

[0095]

[0096] in, For a point in a point cloud cluster, For Centered on Let be the set of points contained within the neighborhood of the radius. The number of points in this set. The minimum number of neighborhood points, points that satisfy this condition. As the core point;

[0097] Density reachability is determined by the existence of a sequence of points. ,in, , ;

[0098] Meanwhile, the core formula for density-based clustering algorithms to segment clustered fruits is:

[0099]

[0100] in, Let be the local density of the i-th point in the point cloud cluster, used to determine whether this point is the cluster center. Let i be the set of all points within the neighborhood of the i-th point. Let be the Euclidean distance between the i-th point and the j-th point. The neighborhood radius, This is an indicator function; when the value inside the parentheses is ≤0... =1, otherwise =0, meaning only distances ≤ The adjacent points are used to calculate the local density.

[0101] Then, precise spatial coordinate calibration is performed. A world coordinate system is established with the robot base as the reference. The attitude data of the sensor module is collected in real time through the inertial measurement unit. The attitude of the sensor is calibrated in real time. The centroid coordinates of the fruit point cloud are converted into coordinates in the world coordinate system. The optimal picking contact point is determined by combining the fruit morphological characteristics analysis.

[0102] Finally, a dynamic compensation mechanism is used to ensure the continuity of positioning. This mechanism analyzes the differences in point cloud data of adjacent data frames, updates the spatial coordinates of the fruit in real time, and compensates for the dynamic displacement of the fruit caused by factors such as wind and robot movement in the field environment, thus ensuring the continuity and accuracy of positioning.

[0103] Finally, the harvesting priority decision-making step is executed:

[0104] S4. Picking priority decision: Based on feature vectors, a maturity assessment model is constructed to output the fruit maturity level and determine the picking priority, thus completing the identification and location of the target fruit.

[0105] Specifically, a maturity assessment model is constructed based on the fruit feature vector fused in step S2. This maturity assessment model takes the component content information corresponding to the absorbance value of the feature peak in the spectral feature sub-vector and the hue feature in the visual feature sub-vector as input, evaluates the maturity of the fruit through a preset classification algorithm, outputs the fruit maturity level, and determines the picking priority according to the maturity level. The higher the maturity level, the higher the picking priority, thereby completing the identification and positioning of the target fruit and providing accurate target information for the subsequent picking action.

[0106] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A robot recognition and positioning method for fruit medicinal material picking, characterized in that, The method comprises the following steps: S1, multi-modal sensing data acquisition, integrating a near-infrared spectrum, multi-spectral vision and flexible touch triple sensor module at the end of the picking robot mechanical arm, and synchronously collecting near-infrared spectrum data, multi-spectral vision data and fruit pre-contact touch data of the target region; S2, medicine special feature extraction, based on the ingredients and physical properties of Fructus Aurantii and Prunus mume, special feature sub-vectors are extracted from spectrum data, vision data and touch data respectively, and fruit feature vectors are obtained through attention mechanism fusion; The process formula for fusing the feature sub-vectors by the attention mechanism is , and ; wherein, is the fruit feature vector after fusion, which is the core input for subsequent recognition and maturity determination, , , are the weight coefficients of the spectral feature sub-vector, the visual feature sub-vector, and the tactile feature sub-vector, respectively, and the weight values are dynamically adjusted according to the feature recognition degree of the fruit, is the spectral feature sub-vector, is the visual feature sub-vector, is the tactile feature sub-vector. The core algorithm formula of the attention mechanism fusion feature sub-vector is ; wherein, is the final fused fruit feature vector, with dimension , corresponding to the spectral, visual, and tactile feature sub-vectors, respectively, is the spectral feature sub-vector, is the visual feature sub-vector, is the tactile feature sub-vector, all with dimension ; For the first The attention weight of the feature sub-vector is output by the feature importance evaluation module, and the calculation formula is: wherein, is the first information entropy of the class feature sub-vector; S3, dynamic spatial positioning modeling, combining multi-source data to construct fruit point cloud clusters, adopting a density clustering and morphological feature fusion algorithm to segment cluster growth fruits, and realizing accurate calibration and dynamic compensation of fruit spatial coordinates through inertial measurement and frame difference update; S4, picking priority decision, based on the feature vector, a maturity evaluation model is constructed, the maturity grade of the fruit is outputted, and the picking priority is determined, so as to complete the identification and positioning of the target fruit. 2.The method for fruit medicine picking robot recognition and positioning according to claim 1, characterized in that, The triple sensor module in step S1 comprises a near-infrared spectrum acquisition unit, a multi-spectral vision acquisition unit and a flexible touch acquisition unit, and the three units work cooperatively to synchronously acquire spectrum, vision and fruit surface touch sensing data of the target region. 3.The method for fruit medicine picking robot recognition and positioning according to claim 1, characterized in that, In step S2, the extraction method of the spectrum feature sub-vector is to perform smoothing and derivative preprocessing on the near-infrared spectrum data, based on the characteristic volatile oil components of Fructus Aurantii and the characteristic organic acid components of Prunus mume, the absorbance values of the corresponding characteristic peaks are extracted to form a spectrum feature sub-vector; In the smoothing and derivative preprocessing, the SG smoothing formula is: wherein, is the smoothed spectrum value of the wavelength point, is the smoothed window half-width, is the SG smoothing coefficient, is the spectrum value of the wavelength point, in the original spectrum;​ In the smoothing and derivative preprocessing, the first derivative processing formula is: in, For the first First derivative spectral values ​​at each wavelength point The smoothed first , Spectral values ​​at each wavelength point The interval between adjacent wavelength points, the characteristic peak absorbance value is When taking the extreme value, the corresponding value. 4.The method for fruit medicine picking robot recognition and positioning according to claim 1, characterized in that, In step S2, the extraction method of the vision feature sub-vector is to adopt an improved U-Net network to segment the fruit region in the multi-spectral image, extract the hue feature, vegetation index feature and contour morphological feature of the fruit, and form a vision feature sub-vector; The loss function calculation formula of the improved U-Net network is: wherein, is the total loss value, is the cross-entropy loss, calculated as: where, is the image pixel coordinate, is the pixel ground truth label, is the probability that the network predicts the pixel as fruit, , is the Dice coefficient loss, calculated as: The vegetation index calculation formula of the improved U-Net network is: wherein, is the pixel gray value of the near-infrared waveband in the multispectral image, is the pixel gray value of the red waveband, the value range is [-1, 1], the difference between the fruit and the branches and leaves the difference is greater than or equal to 0.

3. 5.The method for fruit medicine picking robot recognition and positioning according to claim 1, characterized in that, In step S2, the touch feature sub-vector is extracted by filtering the pre-contact pressure data, and the pressure distribution feature and texture feature reflecting the roughness of the fruit surface are extracted to form a touch feature sub-vector. 6.The method for fruit medicine picking robot recognition and positioning according to claim 1, characterized in that, In step S3, the fruit point cloud cluster is constructed by using binocular vision to acquire initial point cloud data, supplementing distance information through a depth camera, and removing non-fruit point clouds based on the spectrum feature sub-vector to retain pure fruit point cloud clusters. 7.The method for fruit medicine picking robot recognition and positioning according to claim 4, characterized in that, In step S3, the cluster growth fruit segmentation algorithm adopts a density-based clustering algorithm, which combines the fruit contour morphological feature to segment the cluster growth Fructus Aurantii point cloud cluster, and clearly defines the spatial boundary of a single fruit; The core judgment formula of the density-based clustering algorithm is: Core point determination: wherein, is a point in the point cloud cluster, is a neighborhood with as the center, as the radius, is the number of points in the set, is the minimum neighborhood point number, and the point satisfies the condition; Density achievable determination: presence of point sequence wherein , . 8.The method for fruit medicine picking robot recognition and positioning according to claim 1, characterized in that, In step S3, the method for spatial coordinate calibration is to establish a world coordinate system with the robot base as the reference, to real-time calibrate the sensor posture through an inertial measurement unit, to convert the fruit point cloud centroid coordinates into world coordinates, and to determine the optimal picking contact point in combination with the fruit morphology; The core formula of the density-based clustering algorithm for segmenting cluster growth fruits in step S3 is: wherein, is the local density of the i-th point in the point cloud cluster, which is used to determine whether the point is a cluster center, is the set of all points within the neighborhood range centered at the i-th point, is the Euclidean distance between the i-th point and the j-th point, is the neighborhood radius, is an indicator function, when the value in the parentheses ≤ 0 = 1, otherwise = 0, i.e., only the neighboring points with distance ≤ are counted to calculate the local density. 9.The method for fruit medicine picking robot recognition and positioning according to claim 1, characterized in that, The dynamic compensation mechanism in the step S3 compensates the dynamic displacement of the fruits in the field environment by real-time updating the spatial coordinates of the fruits through difference analysis of adjacent data frames, thereby ensuring the continuity of the positioning. 10.The method for fruit medicine picking robot recognition and positioning according to claim 4, characterized in that, The maturity evaluation model in the step S4 takes the spectral features and the visual hue features as inputs, and outputs the maturity grade of the fruits through a classification algorithm, thereby planning the picking priority.

Images