Ultrasonic fracturing rubber tapping device based on multi-mode perception and control method

Abstract

The invention discloses an ultrasonic fracturing rubber tapping device based on multi-mode perception, which comprises an ultrasonic probe control module mounted on one side of a replaceable rubber tapping knife module, a depth camera module, a three-axis force sensor module, a data processing module, a data processing module, a data processing module, a data processing module and a data processing module, the data processing module is connected with a mechanical arm of the self-propelled rubber tapping robot, and switching of rubber tapping objects is achieved through a movable chassis of the self-propelled rubber tapping robot. The invention further discloses a control method of the ultrasonic fracturing rubber tapping device based on multi-mode sensing. According to the rubber tree bark cutting device, efficient and accurate cutting of rubber tree barks can be achieved, damage to rubber trees is effectively reduced, then the rubber yield is increased, and the service life of the trees is prolonged.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent agricultural equipment, and specifically relates to an ultrasonic fracturing rubber tapping device based on multimodal sensing. The present invention also relates to a control method of the ultrasonic fracturing rubber tapping device based on multimodal sensing. Background Art

[0002] Existing rubber tapping technologies primarily include the following approaches. The first is traditional manual tapping, where workers use handheld tapping knives to tap rubber at specific angles and depths, depending on the tree's growth and personal experience. The second is electric tapping knives, which use electricity to drive the blades to rotate or reciprocate. The third is fixed tapping machines, which are permanently installed on each rubber tree and utilize control technologies such as a control panel and wireless communications to complete the tapping operation. The fourth is mobile tapping robots. One approach uses deep learning and machine vision to plan the tapping trajectory, allowing the robots to autonomously move to the rubber trees and conduct the tapping. Another approach involves some robots employing technologies such as LiDAR and GPS for positioning and navigation, working in conjunction with robotic arms and tapping mechanisms to achieve automated tapping. The fifth is laser tapping, which utilizes the high energy density of lasers to instantly vaporize the rubber tree bark, achieving the desired tapping effect. The sixth is the compound motion tapping machine, for example, a compound motion natural rubber automatic tapping machine, which consists of a frame, a centering clamping device, a radial motion device, a circumferential motion device and an end processor. It is driven by a motor and a gear transmission, so that the elliptical groove tapping knife on the end processor moves in a coordinated manner in the radial and circumferential directions to achieve tapping.

[0003] Traditional rubber tapping methods, due to their reliance on manual labor, suffer from drawbacks such as low efficiency, high labor intensity, and difficulty in precisely controlling cutting depth and angle. This can easily cause excessive damage to rubber trees, thereby impacting rubber yield and tree lifespan. Furthermore, the tapping process is susceptible to environmental factors such as light and humidity, making it difficult to ensure consistent cutting. While existing automated tapping equipment has improved, significant deficiencies remain in its ability to detect the internal structure of the bark and dynamically adjust the cutting path. Summary of the Invention

[0004] The purpose of the present invention is to provide an ultrasonic cracking and tapping device based on multimodal sensing, which can achieve efficient and precise cutting of rubber tree bark, effectively reduce damage to rubber trees, and thus increase rubber production and extend the life of trees.

[0005] Another object of the present invention is to provide a control method for an ultrasonic fracturing rubber tapping device based on multimodal sensing.

[0006] The first technical solution adopted by the present invention is an ultrasonic cracking rubber tapping device based on multimodal perception, which includes an ultrasonic probe control module installed on one side of a replaceable tapping knife module. The ultrasonic probe control module is connected to a data processing module, and also includes a depth camera module and a three-axis force sensor module. The data processing module is connected to the robotic arm of a self-propelled tapping robot, and the switching of tapping objects is realized through the mobile chassis of the self-propelled tapping robot.

[0007] The first technical solution of the present invention is also characterized in that: The ultrasonic probe control module includes an ultrasonic transmitter and an ultrasonic receiver. The ultrasonic transmitter is used to transmit ultrasonic waves adjustable in a specific frequency range, and the ultrasonic receiver is used to receive ultrasonic signals in a specific frequency range. Both the ultrasonic transmitter and the ultrasonic receiver are connected to the data processing module using a coaxial cable.

[0008] The replaceable tapping knife module includes a replaceable tapping knife, a tapping knife holder, a tapping knife holder cover and a knife holder connector; wherein, The replaceable tapping knife is used to cut the bark of rubber trees. The blades of different specifications can be replaced as needed. The initial cutting depth is set according to the tree species and age. The tapping knife holder and the tapping knife holder cover are used to fix the replaceable tapping knife to ensure that the tapping knife will not slide or dislocate during the tapping process; The rubber tapping tool holder is fixed to the top end of the tool holder connecting piece, and the tool holder connecting piece is connected to the force-bearing end of the three-axis force sensor through bolts.

[0009] The three-axis force sensor is used to monitor the three-axis resistance changes on the knife holder connecting piece during the rubber tapping process in real time, and then convert it into the force conditions on the replaceable rubber tapping knife; The triaxial force sensor is connected to the data processing module via a coaxial cable.

[0010] The depth camera captures the surface of the rubber bark and the depth image of the cutting path to ensure uniform cutting depth; The depth camera is connected to the data processing module via a USB bus.

[0011] The data processing module is used to fuse ultrasonic data, resistance feedback and depth images, and dynamically adjust cutting parameters and paths; according to the output of the data processing module, the movement of the replaceable tapping knife and the working status of the ultrasonic transmitting end are controlled.

[0012] The second technical solution adopted by the present invention is a control method for an ultrasonic fracturing rubber tapping device based on multimodal sensing, which is specifically implemented according to the following steps: Step 1, the ultrasonic probe control module performs frequency sweep detection on the rubber bark area to be cut, and transmits it back to the data processing module to obtain the natural vibration frequency of the rubber bark area to be cut, and uses the natural vibration frequency to perform ultrasonic fracturing on the rubber bark area to be cut; Step 2: The three-axis force sensor and the depth camera respectively collect the force conditions on the replaceable tapping knife and the depth images of the surface and cutting path of the rubber bark, and are connected to the data processing module via a coaxial cable and a USB respectively; Step 3: The data processing module fuses the ultrasonic signal of the ultrasonic probe control module with the point cloud data of the depth camera to obtain the rough surface characteristics of the tapping area; and fuses the three-axis force signal of the three-axis force sensor 8 with the RGB visual image of the depth camera to obtain the material hardness characteristics of the tapping area; Step 4: The data processing module then obtains the tapping depth parameter correction value and the tapping speed correction value by integrating and analyzing the rough surface characteristics and material hardness characteristics of the tapping area, and then dynamically adjusts the tapping parameters; Step 5: The data processing module uses the path correction strategy based on the changes in the three-axis force signal of the three-axis force sensor to adjust the tapping path in real time according to the data prediction; Step 6: The data processing module exchanges data with other modules via high-speed communication interfaces (coaxial and USB) to ensure efficient system operation. Based on the data processing results, the data processing module coordinates the movements of the ultrasonic probe control module, the replaceable tapping knife, and the depth camera to ensure stability and consistency throughout the tapping process. Motion synchronization ensures the coordinated movements of the ultrasonic probe control module, the replaceable tapping knife, and the depth camera. Parameter coordination dynamically adjusts the parameters of each module based on real-time feedback. The system features a self-calibration function that regularly calibrates the depth camera and triaxial force sensor to ensure data accuracy. The calibration process uses standard calibration tools to calibrate the sensors and dynamically adjusts sensor parameters based on the calibration results to ensure data accuracy.

[0013] The second technical solution of the present invention is also characterized in that: Before step 1, the following steps are also included: During the system initialization phase, the ultrasonic transmitter, ultrasonic receiver, depth camera, three-axis force sensor, and data processing module must be started first, and the communication link must be tested using a coaxial cable and USB. The depth camera and three-axis force sensor must then be calibrated, and the ultrasonic probe must be preheated for 10-20 seconds at a frequency of 20-25kHz and a power of 10-15W to ensure stable operation.

[0014] Step 1 is as follows: The actual installation situation is as follows Figure 2 , in the process of implementing ultrasonic cracking technology ( Figure 3), the ultrasonic probe control module (including the ultrasonic transmitter and the ultrasonic receiver) is installed on the side of the replaceable tapping knife, and has good contact with the surface bark of the rubber tree. The module first performs a frequency sweep detection on the rubber tree bark, setting the sweep frequency range to 20kHz-100kHz, the sweep frequency step to 1kHz, the transmission power to 10-40W, and the initial transmission power to 10W. After the ultrasonic transmitter transmits the sweep frequency signal, the ultrasonic receiver collects the reflected signal. In the data processing module, the frequency response curve is generated by fast Fourier transform FFT to extract the natural frequency of the bark. After determining the natural frequency, the ultrasonic transmitter transmits the same frequency ultrasonic wave for a duration of 10-30 seconds. The transmission parameters are dynamically adjusted according to the thickness and hardness of the bark. The specific adjustment formula is as follows:

[0015]

[0016]

[0017] where K p is the power coefficient, t is the duration, P is the final transmitted power, T is the bark thickness, H is the bark hardness, P base is the initial transmission power; the final transmission power and duration are dynamically adjusted according to the detected bark thickness and hardness.

[0018] During transmission, the ultrasonic reflection signal is monitored in real time, and the transmission power is dynamically adjusted. The real-time signal-to-noise ratio (SNR) of the monitored signal is used as a basis. If the SNR is less than 10dB, the output power is increased by 10% based on the current transmission power and transmission continues, but the total transmission power is limited to 40W to ensure uniform distribution of microcracks and avoid localized excessive cracking. The distribution of microcracks is verified using ultrasonic data and depth images. If the distribution is uneven or insufficient, the system automatically adjusts the transmission parameters and re-transmits the ultrasound until the desired cracking effect is achieved.

[0019] Step 4 is as follows: During the cutting execution phase, the ultrasonic transmitter emits ultrasonic waves of the same frequency for 10-30 seconds, and uses the reflected signal to verify the uniformity of the crack distribution. If the local crack density is 0-5 / cm 2 Within the range, the transmission parameters are readjusted to re-send ultrasonic waves. The tapping knife can be replaced to cut into the bark at a preset angle of 25°-30°, with a cutting speed of 5-10mm / s and a cutting depth of 1.4-1.8mm. The system dynamically adjusts parameters based on real-time feedback to ensure that the cutting depth fluctuation is less than ±0.2mm. During the system integration and collaborative control phase, the data processing module coordinates the actions of various components, ensuring synchronization errors of less than 10ms. If the depth camera or three-axis force sensor data is abnormal (such as the depth camera is out of focus or the force sensor is out of range), the system will enter safety mode, cut off the ultrasonic emission, and retract the blade. The multimodal fusion perception stage involves estimating depth information features and mechanical information features. During the multimodal data acquisition and fusion phase, the ultrasonic transmitter continuously emits ultrasonic waves of the same frequency (e.g., 50kHz, 30W) and monitors the distribution of microcracks in real time. The data update frequency ranges from 97-102Hz. The three-axis force sensor collects the resistance components of the replaceable rubber tapping knife 4 in the X / Y / Z directions. The resistance change rate (ΔF / Δt) is calculated using a 10ms sliding window to determine the cutting depth deviation. The depth camera captures the cutting path at a resolution of 1280×720 and generates point cloud data. In terms of depth information feature estimation, the point cloud data obtained by the depth camera is normalized to the spatial point cloud coordinates. The normalization method is as follows:

[0020] in, p i ’ is the coordinate of the space point after transformation, p i are the coordinates of the spatial point to be transformed, p min is the point with the smallest spatial distance, p max It is the point with the largest spatial distance; Then use 3D convolution kernel to extract local features:

[0021] in, W(i,j,l) is the weight parameter of the three-dimensional convolution kernel, I(x+I,y+j,z+l) is the weight coefficient of the three-dimensional convolution kernel, b is the bias term; Use the ReLU function for pooling to amplify features and reduce computational complexity:

[0022] Finally, the geometric features of the bark depression are extracted through global average pooling to generate a 128-dimensional spatial feature vector:

[0023]

[0024] in, H, W, D Represents input features Figure 3 The resolution in dimensions, Wx+b is a fully connected layer; At the same time, the ultrasonic signal obtained by ultrasonic ranging is preprocessed, normalized using a sliding window, and then processed by STFT to obtain time-frequency information:

[0025]

[0026] Among them, normalization uses Z-score standardization. μ is the mean of the data in the window, σ is the standard deviation of the data in the window, w(n) is the window function; After CNN processing to extract feature information, a 64-dimensional ultrasound feature vector is finally generated in the fully connected layer. Through cross-attention weight calculation, the final 192-dimensional depth information feature vector containing ultrasound information and depth point cloud data is obtained:

[0027]

[0028]

[0029] in, Q is the query matrix from the target sequence, K is the key matrix of the source sequence, V is the value matrix of the source sequence; f out is the final depth information feature vector; In terms of mechanical information feature estimation, each channel of the triaxial force signal is first normalized and divided into processing windows:

[0030]

[0031] Among them, T is the window size, x min is the minimum value in the window, x max is the maximum value in the window; The triaxial force signal is feature extracted using a 1D convolutional layer + LSTM composite form. First, a 1D convolutional layer is used to extract the local pattern, and then the pooling layer is used to reduce the dimension:

[0032] in K is the convolution kernel size, w is the convolution kernel weight,

[0033] in sis the pooling window size; Then, LSTM is used to obtain the intrinsic temporal dependency and output a 64-dimensional mechanical temporal feature vector; The input gate of LSTM is:

[0034] The forget gate of LSTM is:

[0035] The candidate memory units of LSTM are:

[0036] The memory unit of LSTM is updated as follows:

[0037] The output gate of LSTM is:

[0038] The hidden state of LSTM is:

[0039] in, x t is the current input, h t-1 is the hidden state at the previous moment, W and b are weight and bias parameters respectively, σ is a sigmoid function, and ⊙ represents element-wise multiplication. A single-layer LSTM structure is used to model the short-term and long-term dependencies of force signals through the above-mentioned gating mechanism. The implementation of the dynamic adjustment mechanism relies on the analysis of the comprehensive perception information by the data processing module to dynamically adjust the cutting speed, depth and angle. The mechanism mainly uses three layers of LSTM units to predict the dynamic parameters of the main body and extract deeper temporal features. The output of each layer is h t (l) As input to the next layer x t (l+1) , where L∈{1,2,3} represents the layer index, and the first layer input is the original sequence x t (1) =x t , each layer is represented as follows: The input gate of the Lth layer is:

[0040] The forget gate of the Lth layer is:

[0041] The candidate memory units in the Lth layer are:

[0042] The memory unit of the Lth layer is updated as follows:

[0043] The output gate of the Lth layer is:

[0044] The hidden state of the Lth layer is:

[0045] The state transfer of the Lth layer is:

[0046] The final output of the three-layer LSTM unit is:

[0047] At the input end, the rough surface feature vector and the material hardness feature vector are fused and spliced, and the fused features are convolved through the Causal CNN to ensure y [ t ] only by x [ t ] ，x [ t-1 ] ，···，x [ t-K+1 ]calculate;

[0048] Given the strong coupling relationship between cutting depth, speed and energy consumption, these relationships can be explicitly modeled through energy equation constraints. Therefore, a physical constraint correction part is introduced after the predicted value is output at the output end to ensure that the output parameters are always in a reasonable range and can be forced to be corrected to a safe range under extreme working conditions to prevent dangerous operations.

[0049]

[0050] Where is the total energy consumed in cutting, is the cutting force, is the predicted cutting speed, is the learnable stiffness coefficient, is the estimated material deformation, is the kinetic energy consumed during the cutting process, and It is the accumulation of kinetic energy during the cutting process.

[0051] The beneficial effects of the present invention are as follows: technical effectiveness verification verifies the system's advantages in improving tapping efficiency by comparing the tapping efficiency of traditional tapping methods and the system. Efficiency verification includes time measurement and yield comparison, recording the time required to complete the same tapping task and comparing the rubber yield of the traditional method and the system. The system's advantages in improving tapping quality are verified by analyzing the uniformity of cutting depth and rubber quality. Quality verification includes depth image analysis of cutting depth uniformity and testing rubber purity and yield. The system's advantages in reducing tree damage are verified by observing the growth and extent of damage to rubber trees. Damage verification includes examining cut marks and damage on the bark, as well as long-term monitoring of rubber tree growth and yield changes.

[0052] Through the above-mentioned specific implementation process, the present invention achieves intelligent, automated, and efficient rubber tapping, providing an innovative solution for the field of intelligent agricultural equipment technology. The system's design and implementation process complies with relevant technical standards and specifications, ensuring the feasibility and reliability of the technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the overall structure of the system; Figure 2 This is a schematic diagram of installing the system on an existing self-propelled rubber tapping robot; Figure 3 It is a flow chart of ultrasonic cracking treatment; Figure 4 It is the logic block diagram of the data processing module, including ultrasound data processing, resistance data processing, depth image processing and data fusion modules; Figure 5 It is a rough surface feature fusion diagram; Figure 6 It is the material hardness feature fusion diagram; Figure 7 This is the block diagram of the LSTM dynamic parameter prediction system.

[0054] In the figure, 1. Ultrasonic transmitting end, 2. Ultrasonic receiving end, 3. Tapping knife holder cover, 4. Replaceable tapping knife, 5. Tapping knife holder, 6. Depth camera, 7. Data processing module, 8. Three-axis force sensor, 9. Knife holder connector, 10. Robotic arm, 11. Mobile chassis. DETAILED DESCRIPTION

[0055] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] The ultrasonic cracking rubber tapping device based on multi-modal sensing of the present invention has a structure as follows Figure 1 As shown, combined Figure 2, including an ultrasonic probe control module installed on one side of the replaceable tapping knife module, the ultrasonic probe control module is connected to the data processing module 7, and also includes a depth camera module and a three-axis force sensor module. The data processing module 7 is connected to the mechanical arm 10 of the self-propelled tapping robot, and the switching of the tapping object is realized through the mobile chassis 11 of the self-propelled tapping robot.

[0057] The ultrasonic probe control module includes an ultrasonic transmitting end 1 and an ultrasonic receiving end 2. The ultrasonic transmitting end 1 is used to transmit ultrasonic waves that are adjustable in a specific frequency range, and the ultrasonic receiving end 2 is used to receive ultrasonic signals in a specific frequency range. The ultrasonic transmitting end 1 and the ultrasonic receiving end 2 are both connected to the data processing module 7 using a coaxial cable.

[0058] The replaceable tapping knife module includes a replaceable tapping knife 4, a tapping knife holder 5, a tapping knife holder cover 3 and a knife holder connector 9; wherein, The replaceable tapping knife 4 is used to cut the rubber bark. The blades of different specifications can be replaced as needed. The initial cutting depth is set according to the tree species and age; The tapping knife holder 5 and the tapping knife holder cover 3 are used to fix the replaceable tapping knife 4 to ensure that the tapping knife will not slide or dislocate during the tapping process; The rubber tapping tool holder 5 is fixed to the top end of the tool holder connecting member 9, and the tool holder connecting member 9 is connected to the force-bearing end of the three-axis force sensor 8 by bolts.

[0059] The three-axis force sensor 8 is used to monitor the three-axis resistance changes on the tool holder connector 9 during the rubber tapping process in real time, and then convert it into the force conditions on the replaceable rubber tapping knife 4; The triaxial force sensor 8 is connected to the data processing module 7 via a coaxial cable.

[0060] The depth camera 6 captures the surface of the rubber bark and the depth image of the cutting path to ensure uniform cutting depth; The depth camera 6 is connected to the data processing module 7 via a USB bus.

[0061] The data processing module 7 is used to fuse the ultrasonic data, resistance feedback and depth image, and dynamically adjust the cutting parameters (cutting speed, cutting depth) and path; according to the output of the data processing module 7, the action of the replaceable tapping knife 4 and the working state of the ultrasonic transmitting end 1 are controlled.

[0062] The control method of the ultrasonic cracking rubber tapping device based on multimodal sensing of the present invention is specifically implemented according to the following steps: Step 1, the ultrasonic probe control module performs frequency sweep detection on the rubber bark area to be cut, and transmits it back to the data processing module 7 to obtain the natural vibration frequency of the rubber bark area to be cut, and uses the natural vibration frequency to perform ultrasonic fracturing on the rubber bark area to be cut; Before step 1, the following steps are also included: During the system initialization phase, the ultrasonic transmitter 1, ultrasonic receiver 2, depth camera 6, triaxial force sensor 8, and data processing module 7 must be started first, and the communication link must be tested using a coaxial cable and USB. The depth camera 6 and triaxial force sensor 8 are then calibrated, and the ultrasonic probe is preheated for 10-20 seconds at a frequency of 20-25kHz and a power of 10-15W to ensure stable operation.

[0063] Step 1 is as follows: The actual installation situation is as follows Figure 2 , during the implementation of ultrasonic cracking technology ( Figure 3 ), the ultrasonic probe control module (including the ultrasonic transmitter 1 and the ultrasonic receiver 2) is installed on one side of the replaceable tapping knife 4, and has good contact with the surface bark of the rubber tree. The module first performs a frequency sweep detection on the rubber tree bark, setting the frequency sweep range to 20kHz-100kHz, the frequency sweep step to 1kHz, the transmission power to 10-40W, and the initial transmission power to 10W. After the ultrasonic transmitter 1 transmits the frequency sweep signal, the ultrasonic receiver 2 collects the reflected signal. In the data processing module 7, the frequency response curve is generated by fast Fourier transform FFT to extract the natural frequency of the bark. After determining the natural frequency, the ultrasonic transmitter 1 transmits the same frequency ultrasonic wave for a duration of 10-30 seconds. The transmission parameters are dynamically adjusted according to the thickness and hardness of the bark. The specific adjustment formula is as follows:

[0064]

[0065]

[0066] where K p is the power coefficient, t is the duration, P is the final transmitted power, T is the bark thickness, H is the bark hardness, P base is the initial transmission power; the final transmission power and duration are dynamically adjusted according to the detected bark thickness and hardness.

[0067] During transmission, the ultrasonic reflection signal is monitored in real time, and the transmission power is dynamically adjusted. The real-time signal-to-noise ratio (SNR) of the monitored signal is used as a basis. If the SNR is less than 10dB, the output power is increased by 10% based on the current transmission power and transmission continues, but the total transmission power is limited to 40W to ensure uniform distribution of microcracks and avoid localized excessive cracking. The distribution of microcracks is verified using ultrasonic data and depth images. If the distribution is uneven or insufficient, the system automatically adjusts the transmission parameters and re-transmits the ultrasound until the desired cracking effect is achieved.

[0068] Step 2, the three-axis force sensor 8 and the depth camera 6 respectively collect the force conditions on the replaceable tapping knife 4 and the depth image of the surface and cutting path of the rubber bark, and are connected to the data processing module 7 via a coaxial cable and a USB respectively; Step 3: The data processing module 7 fuses the ultrasonic signal of the ultrasonic probe control module with the point cloud data of the depth camera 6 to obtain the rough surface characteristics of the tapping area; and fuses the three-axis force signal of the three-axis force sensor 8 with the RGB visual image of the depth camera 6 to obtain the material hardness characteristics of the tapping area; Step 4: The data processing module 7 obtains the tapping depth parameter correction value and the tapping speed correction value by fusing and analyzing the rough surface characteristics and material hardness characteristics of the tapping area, thereby dynamically adjusting the tapping parameters; Step 4 is as follows: During the cutting execution phase, the ultrasonic transmitter 1 emits ultrasonic waves of the same frequency for 10-30 seconds, and uses the reflected signal to verify the uniformity of the crack distribution. If the local crack density is 0-5 / cm 2 Within the range, the transmission parameters are readjusted and the ultrasonic wave is re-sent ( Figure 3 ), the replaceable tapping knife 4 cuts into the bark at a preset angle of 25°-30°, with a cutting speed of 5-10mm / s and a cutting depth of 1.4-1.8mm. The system dynamically adjusts parameters based on real-time feedback to ensure that the cutting depth fluctuation is less than ±0.2mm; During the system integration and coordinated control phase, the data processing module 7 coordinates the actions of various components and synchronizes them to ensure that the error is less than 10ms. When the data from the depth camera 6 or the three-axis force sensor 8 is abnormal (such as the depth camera is out of focus or the force sensor is out of range), the system will enter a safe mode, cut off the ultrasonic emission, and retract the blade. The multimodal fusion perception stage involves the estimation of depth information features and mechanical information features (such as Figure 4During the multimodal data acquisition and fusion phase, the ultrasonic transmitter 1 continuously emits ultrasonic waves of the same frequency (e.g., 50kHz, 30W) and monitors the distribution of microcracks in real time. The data update frequency ranges from 97-102Hz. The triaxial force sensor 8 collects the resistance components of the replaceable rubber tapping knife 4 in the X / Y / Z directions and calculates the resistance change rate (ΔF / Δt) using a 10ms sliding window to determine the cutting depth deviation. The depth camera 6 captures the cutting path at a resolution of 1280×720 and generates point cloud data. Depth information feature estimation ( Figure 3 ), the point cloud data obtained by the depth camera 6 is normalized to the spatial point cloud coordinates. The normalization method is as follows:

[0069] in, p i ’ is the coordinate of the space point after transformation, p i are the coordinates of the spatial point to be transformed, p min is the point with the smallest spatial distance, p max It is the point with the largest spatial distance; Then use 3D convolution kernel to extract local features:

[0070] in, W(i,j,l) is the weight parameter of the three-dimensional convolution kernel, I(x+I,y+j,z+l) is the weight coefficient of the three-dimensional convolution kernel, b is the bias term; Use the ReLU function for pooling to amplify features and reduce computational complexity:

[0071] Finally, the geometric features of the bark depression are extracted through global average pooling to generate a 128-dimensional spatial feature vector:

[0072]

[0073] in, H, W, D Represents input features Figure 3 The resolution in dimensions, Wx+b is a fully connected layer; At the same time, the ultrasonic signal obtained by ultrasonic ranging is preprocessed, normalized using a sliding window, and then processed by STFT to obtain time-frequency information:

[0074]

[0075] Among them, normalization uses Z-score standardization. μ is the mean of the data in the window, σ is the standard deviation of the data in the window, w(n) is the window function; After CNN processing to extract feature information, a 64-dimensional ultrasound feature vector is finally generated in the fully connected layer. Through cross-attention weight calculation, the final 192-dimensional depth information feature vector containing ultrasound information and depth point cloud data is obtained:

[0076]

[0077]

[0078] in, Q is the query matrix from the target sequence, K is the key matrix of the source sequence, V is the value matrix of the source sequence; f out is the final depth information feature vector; Mechanical information feature estimation ( Figure 4 ), first normalize each channel of the triaxial force signal separately and divide the processing window:

[0079]

[0080] Among them, T is the window size, x min is the minimum value in the window, x max is the maximum value in the window; The triaxial force signal is feature extracted using a 1D convolutional layer + LSTM composite form. First, a 1D convolutional layer is used to extract the local pattern, and then the pooling layer is used to reduce the dimension:

[0081] in K is the convolution kernel size, w is the convolution kernel weight,

[0082] in s is the pooling window size; Then, LSTM is used to obtain the intrinsic temporal dependency and output a 64-dimensional mechanical temporal feature vector; The input gate of LSTM is:

[0083] The forget gate of LSTM is:

[0084] The candidate memory units of LSTM are:

[0085] The memory unit of LSTM is updated as follows:

[0086] The output gate of LSTM is:

[0087] The hidden state of LSTM is:

[0088] in, x t is the current input, h t-1 is the hidden state at the previous moment, W and b are weight and bias parameters respectively, σ is a sigmoid function, and ⊙ represents element-wise multiplication. A single-layer LSTM structure is used to model the short-term and long-term dependencies of force signals through the above-mentioned gating mechanism. The implementation of the dynamic adjustment mechanism relies on the analysis of the comprehensive perception information by the data processing module 7 to dynamically adjust the cutting speed, depth and angle. The mechanism mainly uses the three-layer LSTM unit dynamic parameter prediction body to extract deeper temporal features ( Figure 5 ). The output of each layer h t (l) As input to the next layer x t (l+1) , where L∈{1,2,3} represents the layer index, and the first layer input is the original sequence x t (1) =x t , each layer is represented as follows: The input gate of the Lth layer is:

[0089] The forget gate of the Lth layer is:

[0090] The candidate memory units in the Lth layer are:

[0091] The memory unit of the Lth layer is updated as follows:

[0092] The output gate of the Lth layer is:

[0093] The hidden state of the Lth layer is:

[0094] The state transfer of the Lth layer is:

[0095] The final output of the three-layer LSTM unit is:

[0096] At the input end, the rough surface feature vector and the material hardness feature vector are fused and spliced, and the fused features are convolved through the Causal CNN to ensure y [ t ] only by x [ t ] ，x [ t-1 ] ，···，x [ t-K+1 ]calculate;

[0097] Given the strong coupling relationship between cutting depth, speed and energy consumption, these relationships can be explicitly modeled through energy equation constraints. Therefore, a physical constraint correction part is introduced after the predicted value is output at the output end to ensure that the output parameters are always in a reasonable range and can be forced to be corrected to a safe range under extreme working conditions to prevent dangerous operations.

[0098]

[0099] in is the total energy consumed in cutting, is the cutting force, is the predicted cutting speed, is the learnable stiffness coefficient, is the estimated material deformation, is the kinetic energy consumed during the cutting process, and It is the accumulation of kinetic energy during the cutting process.

[0100] Step 5: The data processing module 7 uses a path correction strategy based on the changes in the three-axis force signals of the three-axis force sensor 8 to adjust the tapping path in real time according to data prediction; In step 6, the data processing module 7 interacts with other modules via high-speed communication interfaces (coaxial and USB) to ensure efficient system operation. Based on the data processing results, the data processing module 7 coordinates the actions of the ultrasonic probe control module, the replaceable tapping knife 4, and the depth camera 6 to ensure stability and consistency throughout the tapping process. Synchronization ensures the coordinated actions of the ultrasonic probe control module, the replaceable tapping knife, and the depth camera. Parameter coordination dynamically adjusts the parameters of each module based on real-time feedback data. The system has a self-calibration function that regularly calibrates the depth camera and triaxial force sensor to ensure data accuracy. The calibration process uses standard calibration tools to calibrate the sensors and dynamically adjusts sensor parameters based on the calibration results to ensure data accuracy.

[0101] Example 1 The ultrasonic cracking rubber tapping device based on multi-modal sensing of the present invention has a structure as follows Figure 1 As shown, combined Figure 2 , including an ultrasonic probe control module installed on one side of the replaceable tapping knife module, the ultrasonic probe control module is connected to the data processing module 7, and also includes a depth camera module and a three-axis force sensor module. The data processing module 7 is connected to the mechanical arm 10 of the self-propelled tapping robot, and the switching of the tapping object is realized through the mobile chassis 11 of the self-propelled tapping robot.

[0102] Example 2 The ultrasonic cracking rubber tapping device based on multi-modal sensing of the present invention has a structure as follows Figure 1 As shown, combined Figure 2 , including an ultrasonic probe control module installed on one side of the replaceable tapping knife module, the ultrasonic probe control module is connected to the data processing module 7, and also includes a depth camera module and a three-axis force sensor module. The data processing module 7 is connected to the mechanical arm 10 of the self-propelled tapping robot, and the switching of the tapping object is realized through the mobile chassis 11 of the self-propelled tapping robot.

[0103] The ultrasonic probe control module includes an ultrasonic transmitting end 1 and an ultrasonic receiving end 2. The ultrasonic transmitting end 1 is used to transmit ultrasonic waves that are adjustable in a specific frequency range, and the ultrasonic receiving end 2 is used to receive ultrasonic signals in a specific frequency range. The ultrasonic transmitting end 1 and the ultrasonic receiving end 2 are both connected to the data processing module 7 using a coaxial cable.

[0104] The replaceable tapping knife module includes a replaceable tapping knife 4, a tapping knife holder 5, a tapping knife holder cover 3 and a knife holder connector 9; wherein, The replaceable tapping knife 4 is used to cut the rubber bark. The blades of different specifications can be replaced as needed. The initial cutting depth is set according to the tree species and age; The tapping knife holder 5 and the tapping knife holder cover 3 are used to fix the replaceable tapping knife 4 to ensure that the tapping knife will not slide or dislocate during the tapping process; The rubber tapping tool holder 5 is fixed to the top end of the tool holder connecting member 9, and the tool holder connecting member 9 is connected to the force-bearing end of the three-axis force sensor 8 by bolts.

[0105] The three-axis force sensor 8 is used to monitor the three-axis resistance changes on the tool holder connector 9 during the rubber tapping process in real time, and then convert it into the force conditions on the replaceable rubber tapping knife 4; The triaxial force sensor 8 is connected to the data processing module 7 via a coaxial cable.

[0106] The depth camera 6 captures the surface of the rubber bark and the depth image of the cutting path to ensure uniform cutting depth; The depth camera 6 is connected to the data processing module 7 via a USB bus.

[0107] The data processing module 7 is used to fuse the ultrasonic data, resistance feedback and depth image, and dynamically adjust the cutting parameters (cutting speed, cutting depth) and path; according to the output of the data processing module 7, the action of the replaceable tapping knife 4 and the working state of the ultrasonic transmitting end 1 are controlled.

[0108] Example 3 The control method of the ultrasonic cracking rubber tapping device based on multimodal sensing of the present invention is specifically implemented according to the following steps: Step 1, the ultrasonic probe control module performs frequency sweep detection on the rubber bark area to be cut, and transmits it back to the data processing module 7 to obtain the natural vibration frequency of the rubber bark area to be cut, and uses the natural vibration frequency to perform ultrasonic fracturing on the rubber bark area to be cut; Step 2, the three-axis force sensor 8 and the depth camera 6 respectively collect the force conditions on the replaceable tapping knife 4 and the depth image of the surface and cutting path of the rubber bark, and are connected to the data processing module 7 via a coaxial cable and a USB respectively; Step 3: The data processing module 7 fuses the ultrasonic signal of the ultrasonic probe control module with the point cloud data of the depth camera 6 to obtain the rough surface characteristics of the tapping area; and fuses the three-axis force signal of the three-axis force sensor 8 with the RGB visual image of the depth camera 6 to obtain the material hardness characteristics of the tapping area; Step 4: The data processing module 7 obtains the tapping depth parameter correction value and the tapping speed correction value by fusing and analyzing the rough surface characteristics and material hardness characteristics of the tapping area, thereby dynamically adjusting the tapping parameters; Step 5: The data processing module 7 uses a path correction strategy based on the changes in the three-axis force signals of the three-axis force sensor 8 to adjust the tapping path in real time according to data prediction; In step 6, the data processing module 7 interacts with other modules via high-speed communication interfaces (coaxial and USB) to ensure efficient system operation. Based on the data processing results, the data processing module 7 coordinates the actions of the ultrasonic probe control module, the replaceable tapping knife 4, and the depth camera 6 to ensure stability and consistency throughout the tapping process. Synchronization ensures the coordinated actions of the ultrasonic probe control module, the replaceable tapping knife, and the depth camera. Parameter coordination dynamically adjusts the parameters of each module based on real-time feedback data. The system has a self-calibration function that regularly calibrates the depth camera and triaxial force sensor to ensure data accuracy. The calibration process uses standard calibration tools to calibrate the sensors and dynamically adjusts sensor parameters based on the calibration results to ensure data accuracy.

[0109] Example 4 The control method of the ultrasonic cracking rubber tapping device based on multimodal sensing of the present invention is specifically implemented according to the following steps: Step 1, the ultrasonic probe control module performs frequency sweep detection on the rubber bark area to be cut, and transmits it back to the data processing module 7 to obtain the natural vibration frequency of the rubber bark area to be cut, and uses the natural vibration frequency to perform ultrasonic fracturing on the rubber bark area to be cut; Before step 1, the following steps are also included: During the system initialization phase, the ultrasonic transmitter 1, ultrasonic receiver 2, depth camera 6, triaxial force sensor 8, and data processing module 7 must be started first, and the communication link must be tested using a coaxial cable and USB. The depth camera 6 and triaxial force sensor 8 are then calibrated, and the ultrasonic probe is preheated for 10-20 seconds at a frequency of 20-25kHz and a power of 10-15W to ensure stable operation.

[0110] Step 1 is as follows: The actual installation situation is as follows Figure 2 , in the process of implementing ultrasonic cracking technology ( Figure 3), the ultrasonic probe control module (including the ultrasonic transmitter 1 and the ultrasonic receiver 2) is installed on one side of the replaceable tapping knife 4, and has good contact with the surface bark of the rubber tree. The module first performs a frequency sweep detection on the rubber tree bark, setting the frequency sweep range to 20kHz-100kHz, the frequency sweep step to 1kHz, the transmission power to 10-40W, and the initial transmission power to 10W. After the ultrasonic transmitter 1 transmits the frequency sweep signal, the ultrasonic receiver 2 collects the reflected signal. In the data processing module 7, the frequency response curve is generated by fast Fourier transform FFT to extract the natural frequency of the bark. After determining the natural frequency, the ultrasonic transmitter 1 transmits the same frequency ultrasonic wave for a duration of 10-30 seconds. The transmission parameters are dynamically adjusted according to the thickness and hardness of the bark. The specific adjustment formula is as follows:

[0111]

[0112]

[0113] where K p is the power coefficient, t is the duration, P is the final transmitted power, T is the bark thickness, H is the bark hardness, P base is the initial transmission power; the final transmission power and duration are dynamically adjusted according to the detected bark thickness and hardness.

[0114] During transmission, the ultrasonic reflection signal is monitored in real time, and the transmission power is dynamically adjusted. The real-time signal-to-noise ratio (SNR) of the monitored signal is used as a basis. If the SNR is less than 10dB, the output power is increased by 10% based on the current transmission power and transmission continues, but the total transmission power is limited to 40W to ensure uniform distribution of microcracks and avoid localized excessive cracking. The distribution of microcracks is verified using ultrasonic data and depth images. If the distribution is uneven or insufficient, the system automatically adjusts the transmission parameters and re-transmits the ultrasound until the desired cracking effect is achieved.

[0115] Step 2, the three-axis force sensor 8 and the depth camera 6 respectively collect the force conditions on the replaceable tapping knife 4 and the depth image of the surface and cutting path of the rubber bark, and are connected to the data processing module 7 via a coaxial cable and a USB respectively; Step 3: The data processing module 7 fuses the ultrasonic signal of the ultrasonic probe control module with the point cloud data of the depth camera 6 to obtain the rough surface characteristics of the tapping area; and fuses the three-axis force signal of the three-axis force sensor 8 with the RGB visual image of the depth camera 6 to obtain the material hardness characteristics of the tapping area; Step 4: The data processing module 7 obtains the tapping depth parameter correction value and the tapping speed correction value by fusing and analyzing the rough surface characteristics and material hardness characteristics of the tapping area, thereby dynamically adjusting the tapping parameters; Step 5: The data processing module 7 uses a path correction strategy based on the changes in the three-axis force signals of the three-axis force sensor 8 to adjust the tapping path in real time according to data prediction; In step 6, the data processing module 7 interacts with other modules via high-speed communication interfaces (coaxial and USB) to ensure efficient system operation. Based on the data processing results, the data processing module 7 coordinates the actions of the ultrasonic probe control module, the replaceable tapping knife 4, and the depth camera 6 to ensure stability and consistency throughout the tapping process. Synchronization ensures the coordinated actions of the ultrasonic probe control module, the replaceable tapping knife, and the depth camera. Parameter coordination dynamically adjusts the parameters of each module based on real-time feedback data. The system has a self-calibration function that regularly calibrates the depth camera and triaxial force sensor to ensure data accuracy. The calibration process uses standard calibration tools to calibrate the sensors and dynamically adjusts sensor parameters based on the calibration results to ensure data accuracy.

[0116] Example 5 The control method of the ultrasonic cracking rubber tapping device based on multimodal sensing of the present invention is specifically implemented according to the following steps: Step 1, the ultrasonic probe control module performs frequency sweep detection on the rubber bark area to be cut, and transmits it back to the data processing module 7 to obtain the natural vibration frequency of the rubber bark area to be cut, and uses the natural vibration frequency to perform ultrasonic fracturing on the rubber bark area to be cut; Step 2, the three-axis force sensor 8 and the depth camera 6 respectively collect the force conditions on the replaceable tapping knife 4 and the depth image of the surface and cutting path of the rubber bark, and are connected to the data processing module 7 via a coaxial cable and a USB respectively; Step 3: The data processing module 7 fuses the ultrasonic signal of the ultrasonic probe control module with the point cloud data of the depth camera 6 to obtain the rough surface characteristics of the tapping area; and fuses the three-axis force signal of the three-axis force sensor 8 with the RGB visual image of the depth camera 6 to obtain the material hardness characteristics of the tapping area; Step 4: The data processing module 7 obtains the tapping depth parameter correction value and the tapping speed correction value by fusing and analyzing the rough surface characteristics and material hardness characteristics of the tapping area, thereby dynamically adjusting the tapping parameters; Step 4 is as follows: During the cutting execution phase, the ultrasonic transmitter 1 emits ultrasonic waves of the same frequency for 10-30 seconds, and uses the reflected signal to verify the uniformity of the crack distribution. If the local crack density is 0-5 / cm 2 Within the range, the transmission parameters are readjusted and the ultrasonic wave is re-sent ( Figure 3 ), the replaceable tapping knife 4 cuts into the bark at a preset angle of 25°-30°, with a cutting speed of 5-10mm / s and a cutting depth of 1.4-1.8mm. The system dynamically adjusts parameters based on real-time feedback to ensure that the cutting depth fluctuation is less than ±0.2mm; Example 6 The control method of the ultrasonic cracking rubber tapping device based on multimodal sensing of the present invention is specifically implemented according to the following steps: Step 1, the ultrasonic probe control module performs frequency sweep detection on the rubber bark area to be cut, and transmits it back to the data processing module 7 to obtain the natural vibration frequency of the rubber bark area to be cut, and uses the natural vibration frequency to perform ultrasonic fracturing on the rubber bark area to be cut; Before step 1, the following steps are also included: During the system initialization phase, the ultrasonic transmitter 1, ultrasonic receiver 2, depth camera 6, triaxial force sensor 8, and data processing module 7 must be started first, and the communication link must be tested using a coaxial cable and USB. The depth camera 6 and triaxial force sensor 8 are then calibrated, and the ultrasonic probe is preheated for 10-20 seconds at a frequency of 20-25kHz and a power of 10-15W to ensure stable operation.

[0117] Step 1 is as follows: The actual installation situation is as follows Figure 2 , in the process of implementing ultrasonic cracking technology ( Figure 3 ), the ultrasonic probe control module (including the ultrasonic transmitter 1 and the ultrasonic receiver 2) is installed on one side of the replaceable tapping knife 4, and has good contact with the surface bark of the rubber tree. The module first performs a frequency sweep detection on the rubber tree bark, setting the frequency sweep range to 20kHz-100kHz, the frequency sweep step to 1kHz, the transmission power to 10-40W, and the initial transmission power to 10W. After the ultrasonic transmitter 1 transmits the frequency sweep signal, the ultrasonic receiver 2 collects the reflected signal. In the data processing module 7, the frequency response curve is generated by fast Fourier transform FFT to extract the natural frequency of the bark. After determining the natural frequency, the ultrasonic transmitter 1 transmits the same frequency ultrasonic wave for a duration of 10-30 seconds. The transmission parameters are dynamically adjusted according to the thickness and hardness of the bark. The specific adjustment formula is as follows:

[0118]

[0119]

[0120] where K p is the power coefficient, t is the duration, P is the final transmitted power, T is the bark thickness, H is the bark hardness, P base is the initial transmission power; the final transmission power and duration are dynamically adjusted according to the detected bark thickness and hardness.

[0121] During transmission, the ultrasonic reflection signal is monitored in real time, and the transmission power is dynamically adjusted. The real-time signal-to-noise ratio (SNR) of the monitored signal is used as a basis. If the SNR is less than 10dB, the output power is increased by 10% based on the current transmission power and transmission continues, but the total transmission power is limited to 40W to ensure uniform distribution of microcracks and avoid localized excessive cracking. The distribution of microcracks is verified using ultrasonic data and depth images. If the distribution is uneven or insufficient, the system automatically adjusts the transmission parameters and re-transmits the ultrasound until the desired cracking effect is achieved.

[0122] Step 2, the three-axis force sensor 8 and the depth camera 6 respectively collect the force conditions on the replaceable tapping knife 4 and the depth image of the surface and cutting path of the rubber bark, and are connected to the data processing module 7 via a coaxial cable and a USB respectively; Step 3: The data processing module 7 fuses the ultrasonic signal of the ultrasonic probe control module with the point cloud data of the depth camera 6 to obtain the rough surface characteristics of the tapping area; and fuses the three-axis force signal of the three-axis force sensor 8 with the RGB visual image of the depth camera 6 to obtain the material hardness characteristics of the tapping area; Step 4: The data processing module 7 obtains the tapping depth parameter correction value and the tapping speed correction value by fusing and analyzing the rough surface characteristics and material hardness characteristics of the tapping area, thereby dynamically adjusting the tapping parameters; Step 4 is as follows: During the cutting execution phase, the ultrasonic transmitter 1 emits ultrasonic waves of the same frequency for 10-30 seconds, and uses the reflected signal to verify the uniformity of the crack distribution. If the local crack density is 0-5 / cm 2 Within the range, the transmission parameters are readjusted and the ultrasonic wave is re-sent ( Figure 3 ), the replaceable tapping knife 4 cuts into the bark at a preset angle of 25°-30°, with a cutting speed of 5-10mm / s and a cutting depth of 1.4-1.8mm. The system dynamically adjusts parameters based on real-time feedback to ensure that the cutting depth fluctuation is less than ±0.2mm; During the system integration and coordinated control phase, the data processing module 7 coordinates the actions of various components and synchronizes them to ensure that the error is less than 10ms. When the data from the depth camera 6 or the three-axis force sensor 8 is abnormal (such as the depth camera is out of focus or the force sensor is out of range), the system will enter a safe mode, cut off the ultrasonic emission, and retract the blade. The multimodal fusion perception stage involves the estimation of depth information features and mechanical information features (such as Figure 4 During the multimodal data acquisition and fusion phase, the ultrasonic transmitter 1 continuously emits ultrasonic waves of the same frequency (e.g., 50kHz, 30W) and monitors the distribution of microcracks in real time. The data update frequency ranges from 97-102Hz. The triaxial force sensor 8 collects the resistance components of the replaceable rubber tapping knife 4 in the X / Y / Z directions and calculates the resistance change rate (ΔF / Δt) using a 10ms sliding window to determine the cutting depth deviation. The depth camera 6 captures the cutting path at a resolution of 1280×720 and generates point cloud data. Depth information feature estimation ( Figure 3 ), the point cloud data obtained by the depth camera 6 is normalized to the spatial point cloud coordinates. The normalization method is as follows:

[0123] in, p i ’ is the coordinate of the space point after transformation, p i are the coordinates of the spatial point to be transformed, p min is the point with the smallest spatial distance, p max It is the point with the largest spatial distance; Then use 3D convolution kernel to extract local features:

[0124] in, W(i,j,l) is the weight parameter of the three-dimensional convolution kernel, I(x+I,y+j,z+l) is the weight coefficient of the three-dimensional convolution kernel, b is the bias term; Use the ReLU function for pooling to amplify features and reduce computational complexity:

[0125] Finally, the geometric features of the bark depression are extracted through global average pooling to generate a 128-dimensional spatial feature vector:

[0126]

[0127] in, H, W, D Represents input features Figure 3 The resolution in dimensions, Wx+b is a fully connected layer; At the same time, the ultrasonic signal obtained by ultrasonic ranging is preprocessed, normalized using a sliding window, and then processed by STFT to obtain time-frequency information:

[0128]

[0129] Among them, normalization uses Z-score standardization. μ is the mean of the data in the window, σ is the standard deviation of the data in the window, w(n) is the window function; After CNN processing to extract feature information, a 64-dimensional ultrasound feature vector is finally generated in the fully connected layer. Through cross-attention weight calculation, the final 192-dimensional depth information feature vector containing ultrasound information and depth point cloud data is obtained:

[0130]

[0131]

[0132] in, Q is the query matrix from the target sequence, K is the key matrix of the source sequence, V is the value matrix of the source sequence; f out is the final depth information feature vector; Mechanical information feature estimation ( Figure 4 ), first normalize each channel of the triaxial force signal separately and divide the processing window:

[0133]

[0134] Among them, T is the window size, x min is the minimum value in the window, x max is the maximum value in the window; The triaxial force signal is feature extracted using a 1D convolutional layer + LSTM composite form. First, a 1D convolutional layer is used to extract the local pattern, and then the pooling layer is used to reduce the dimension:

[0135] in K is the convolution kernel size, w is the convolution kernel weight,

[0136] in s is the pooling window size; Then, LSTM is used to obtain the intrinsic temporal dependency and output a 64-dimensional mechanical temporal feature vector; The input gate of LSTM is:

[0137] The forget gate of LSTM is:

[0138] The candidate memory units of LSTM are:

[0139] The memory unit of LSTM is updated as follows:

[0140] The output gate of LSTM is:

[0141] The hidden state of LSTM is:

[0142] in, x t is the current input, h t-1 is the hidden state at the previous moment, W and b are weight and bias parameters respectively, σ is a sigmoid function, and ⊙ represents element-wise multiplication. A single-layer LSTM structure is used to model the short-term and long-term dependencies of force signals through the above-mentioned gating mechanism. The implementation of the dynamic adjustment mechanism relies on the analysis of the comprehensive perception information by the data processing module 7 to dynamically adjust the cutting speed, depth and angle. The mechanism mainly uses the three-layer LSTM unit dynamic parameter prediction body to extract deeper temporal features ( Figure 5 ). The output of each layer h t (l) As input to the next layer x t (l+1) , where L∈{1,2,3} represents the layer index, and the first layer input is the original sequence x t (1) =x t , each layer is represented as follows: The input gate of the Lth layer is:

[0143] The forget gate of the Lth layer is:

[0144] The candidate memory units in the Lth layer are:

[0145] The memory unit of the Lth layer is updated as follows:

[0146] The output gate of the Lth layer is:

[0147] The hidden state of the Lth layer is:

[0148] The state transfer of the Lth layer is:

[0149] The final output of the three-layer LSTM unit is:

[0150] At the input end, the rough surface feature vector and the material hardness feature vector are fused and spliced, and the fused features are convolved through the Causal CNN to ensure y [ t ]Only by x [ t ] ，x [ t-1 ] ，···，x [ t-K+1 ]calculate;

[0151] Given the strong coupling relationship between cutting depth, speed and energy consumption, these relationships can be explicitly modeled through energy equation constraints. Therefore, a physical constraint correction part is introduced after the predicted value is output at the output end to ensure that the output parameters are always in a reasonable range and can be forced to be corrected to a safe range under extreme working conditions to prevent dangerous operations.

[0152]

[0153] in is the total energy consumed in cutting, is the cutting force, is the predicted cutting speed, is the learnable stiffness coefficient, is the estimated material deformation, is the kinetic energy consumed during the cutting process, and It is the accumulation of kinetic energy during the cutting process.

[0154] Step 5: The data processing module 7 uses a path correction strategy based on the changes in the three-axis force signals of the three-axis force sensor 8 to adjust the tapping path in real time according to data prediction; In step 6, the data processing module 7 interacts with other modules via high-speed communication interfaces (coaxial and USB) to ensure efficient system operation. Based on the data processing results, the data processing module 7 coordinates the actions of the ultrasonic probe control module, the replaceable tapping knife 4, and the depth camera 6 to ensure stability and consistency throughout the tapping process. Synchronization ensures the coordinated actions of the ultrasonic probe control module, the replaceable tapping knife, and the depth camera. Parameter coordination dynamically adjusts the parameters of each module based on real-time feedback data. The system has a self-calibration function that regularly calibrates the depth camera and triaxial force sensor to ensure data accuracy. The calibration process uses standard calibration tools to calibrate the sensors and dynamically adjusts sensor parameters based on the calibration results to ensure data accuracy.

Claims

1. An ultrasonic fracturing rubber tapping device based on multimodal sensing, characterized in that: The invention comprises an ultrasonic probe control module installed on one side of a replaceable rubber tapping knife module, the ultrasonic probe control module being connected to a data processing module (7), and further comprising a depth camera module and a three-axis force sensor module. The data processing module (7) is connected to a mechanical arm (10) of a self-propelled rubber tapping robot, and switching of the tapping object is achieved through a mobile chassis (11) of the self-propelled rubber tapping robot.

2. The ultrasonic fracturing rubber tapping device based on multimodal perception according to claim 1, wherein The ultrasonic probe control module comprises an ultrasonic transmitting end (1) and an ultrasonic receiving end (2), wherein the ultrasonic transmitting end (1) is used to transmit ultrasonic waves adjustable in a specific frequency range, and the ultrasonic receiving end (2) is used to receive ultrasonic signals in the specific frequency range; the ultrasonic transmitting end (1) and the ultrasonic receiving end (2) are both connected to the data processing module (7) using a coaxial cable.

3. The ultrasonic fracturing rubber tapping device based on multimodal perception according to claim 2, wherein The replaceable tapping knife module comprises a replaceable tapping knife (4), a tapping knife holder (5), a tapping knife holder cover (3) and a knife holder connector (9); wherein, The replaceable rubber tapping knife (4) is used for cutting the rubber bark; The tapping knife holder (5) and the tapping knife holder cover (3) are used to fix the replaceable tapping knife (4); The rubber tapping knife holder (5) is fixed to the top end of the knife holder connecting member (9), and the knife holder connecting member (9) is connected to the force-bearing end of the three-axis force sensor (8) through bolts; The three-axis force sensor (8) is used to monitor the three-axis resistance changes on the knife holder connecting member (9) during the rubber tapping process in real time, and then convert it into the force conditions on the replaceable rubber tapping knife (4); The three-axis force sensor (8) is connected to the data processing module (7) via a coaxial cable; The depth camera (6) captures the depth image of the surface of the rubber bark and the cutting path to ensure uniform cutting depth; The depth camera (6) is connected to the data processing module (7) via a USB bus; The data processing module (7) is used to perform data fusion on the ultrasonic data, resistance feedback and depth image, and dynamically adjust the cutting parameters and path; and according to the output of the data processing module (7), the action of the replaceable rubber tapping knife (4) and the working state of the ultrasonic transmitting end (1) are controlled.

4. A control method for an ultrasonic fracturing rubber tapping device based on multimodal sensing, based on the ultrasonic fracturing rubber tapping device based on multimodal sensing according to claim 3, characterized in that: Please follow the steps below to implement it: Step 1, the ultrasonic probe control module performs a frequency sweep detection on the rubber bark area to be cut, and transmits the frequency sweep detection back to the data processing module (7) to obtain the natural vibration frequency of the rubber bark area to be cut, and uses the natural vibration frequency to perform ultrasonic fracturing treatment on the rubber bark area to be cut; Step 2, the three-axis force sensor (8) and the depth camera (6) respectively collect the force conditions on the replaceable rubber tapping knife (4) and the depth image of the surface and cutting path of the rubber bark, and are connected to the data processing module (7) via a coaxial cable and a USB respectively; Step 3, the data processing module (7) fuses the ultrasonic signal of the ultrasonic probe control module with the point cloud data of the depth camera (6) to obtain the rough surface features of the tapping area; fusing the three-axis force signal of the three-axis force sensor (8) with the RGB visual image of the depth camera (6) to obtain the material hardness characteristics of the tapping area; Step 4, the data processing module (7) obtains the tapping depth parameter correction value and the tapping speed correction value by integrating and analyzing the rough surface characteristics and material hardness characteristics of the tapping area, and then dynamically adjusts the tapping parameters; Step 5, the data processing module (7) uses the change of the three-axis force signal of the three-axis force sensor (8) to adopt a path correction strategy and adjust the tapping path in real time according to the data prediction; Step 6: The data processing module (7) and other modules realize data interaction through a high-speed communication interface. The data processing module (7) coordinates the actions of the ultrasonic probe control module, the replaceable rubber tapping knife (4) and the depth camera (6) according to the data processing results.

5. The control method of the ultrasonic fracturing rubber tapping device based on multimodal perception according to claim 4, characterized in that, The following steps are also included before step 1: During the system initialization phase, the ultrasonic transmitter 1, ultrasonic receiver (2), depth camera (6), three-axis force sensor (8) and data processing module (7) must be started first, and the communication link detection must be performed using a coaxial cable and USB. The depth camera (6) and three-axis force sensor (8) must then be calibrated, and the ultrasonic probe must be preheated for 10-20 seconds at a frequency of 20-25kHz and a power of 10-15W to ensure a stable working state.

6. The control method of the ultrasonic fracturing rubber tapping device based on multimodal perception according to claim 5, wherein The step 1 is specifically as follows: The ultrasonic probe control module is installed on one side of the replaceable rubber tapping knife (4) and contacts the surface bark of the rubber tree. The module first performs a frequency sweep detection on the rubber tree bark, sets the frequency sweep range to 20kHz-100kHz, the frequency sweep step to 1kHz, the transmission power to 10-40W, and the initial transmission power to 10W. After the ultrasonic transmitter (1) transmits the frequency sweep signal, the ultrasonic receiver (2) collects the reflected signal, and generates a frequency response curve through fast Fourier transform FFT in the data processing module (7), thereby extracting the natural frequency of the bark. After determining the natural frequency, the ultrasonic transmitter (1) transmits the same frequency ultrasonic wave for a duration of 10-30 seconds. The transmission parameters are dynamically adjusted according to the thickness and hardness of the bark. The specific adjustment formula is as follows: where K p is the power coefficient, t is the duration, P is the final transmitted power, T is the bark thickness, H is the bark hardness, P base is the initial transmission power; the final transmission power and duration are dynamically adjusted according to the detected bark thickness and hardness; During transmission, the ultrasonic reflection signal is monitored in real time and the transmission power is adjusted dynamically. The signal-to-noise ratio (SNR) of the real-time monitoring signal is used as a basis. If the SNR is less than 10dB, the output power is increased by 10% based on the current transmission power and the transmission is continued. However, the total transmission power is limited to 40W. The distribution of microcracks is verified by ultrasonic data and depth images. If the distribution is uneven or insufficient, the system will automatically adjust the transmission parameters and re-emit the ultrasonic wave until the expected cracking effect is achieved.

7. The control method of the ultrasonic fracturing rubber tapping device based on multimodal perception according to claim 6, wherein The step 4 is specifically as follows: During the cutting execution phase, the ultrasonic transmitter (1) emits ultrasonic waves of the same frequency for 10-30 seconds, and uses the reflected signal to verify the uniformity of the crack distribution. If the local crack density is 0-5 / cm 2 Within the range, the transmission parameters are readjusted to re-send the ultrasonic wave, and the rubber tapping knife (4) can be replaced to cut into the bark at a preset angle of 25°-30°, with a cutting speed between 5-10mm / s and a cutting depth between 1.4-1.8mm. The system dynamically adjusts the parameters based on real-time feedback to ensure that the cutting depth fluctuation is less than ±0.2mm.

8. The control method of the ultrasonic fracturing rubber tapping device based on multimodal perception according to claim 7, wherein The step 4 further comprises the following steps: During the system integration and collaborative control phase, the data processing module (7) coordinates the actions of each component and synchronizes to ensure that the error is less than 10ms. When the data from the depth camera (6) or the three-axis force sensor (8) is abnormal, the system will enter a safe mode, cut off the ultrasonic emission, and retract the blade. The multimodal fusion perception stage involves the estimation of depth information features and mechanical information features. In the multimodal data acquisition and fusion stage, the ultrasonic transmitter (1) continuously emits ultrasonic waves of the same frequency and monitors the distribution of microcracks in real time. The data update frequency range is 97-102Hz. The three-axis force sensor (8) collects the resistance components of the replaceable rubber tapping knife (4) in the X / Y / Z directions and calculates the resistance change rate through a 10ms sliding window to determine the cutting depth deviation. The depth camera (6) captures the cutting path at a resolution of 1280×720 and generates point cloud data. In terms of depth information feature estimation, the point cloud data obtained by the depth camera (6) is normalized to the spatial point cloud coordinates. The normalization method is as follows: in, p i ’ is the coordinate of the space point after transformation, p i are the coordinates of the spatial point to be transformed, p min is the point with the smallest spatial distance, p max It is the point with the largest spatial distance; Then use 3D convolution kernel to extract local features: in, W(i,j,l) is the weight parameter of the three-dimensional convolution kernel, I(x+I,y+j,z+l) is the weight coefficient of the three-dimensional convolution kernel, b is the bias term; Use the ReLU function for pooling to amplify features and reduce computational complexity: Finally, the geometric features of the bark depression are extracted through global average pooling to generate a 128-dimensional spatial feature vector: in, H, W, D represents the resolution of the three dimensions of the input feature map, Wx+b is a fully connected layer; At the same time, the ultrasonic signal obtained by ultrasonic ranging is preprocessed, normalized using a sliding window, and then processed by STFT to obtain time-frequency information: Among them, normalization uses Z-score standardization. μ is the mean of the data in the window, σ is the standard deviation of the data in the window, w(n) is the window function; After CNN processing to extract feature information, a 64-dimensional ultrasound feature vector is finally generated in the fully connected layer. Through cross-attention weight calculation, the final 192-dimensional depth information feature vector containing ultrasound information and depth point cloud data is obtained: in, Q is the query matrix from the target sequence, K is the key matrix of the source sequence, V is the value matrix of the source sequence; f out is the final depth information feature vector; In terms of mechanical information feature estimation, each channel of the triaxial force signal is first normalized and divided into processing windows: Among them, T is the window size, x min is the minimum value in the window, x max is the maximum value in the window; The triaxial force signal is feature extracted using a 1D convolutional layer + LSTM composite form. First, a 1D convolutional layer is used to extract the local pattern, and then the pooling layer is used to reduce the dimension: in K is the convolution kernel size, w is the convolution kernel weight, in s is the pooling window size; Then, LSTM is used to obtain the intrinsic temporal dependency and output a 64-dimensional mechanical temporal feature vector; The input gate of LSTM is: The forget gate of LSTM is: The candidate memory units of LSTM are: The memory unit of LSTM is updated as follows: The output gate of LSTM is: The hidden state of LSTM is: in, x t is the current input, h t-1 is the hidden state at the previous moment, W and b are weight and bias parameters respectively, σ is a sigmoid function, ⊙ represents element-wise multiplication, and a single-layer LSTM structure is used to model the short-term and long-term dependencies of force signals through the above-mentioned gating mechanism; The dynamic adjustment mechanism uses three layers of LSTM units to predict the dynamic parameters of the main body and extract deeper temporal features. The output of each layer h t (l) As input to the next layer x t (l+1) , where L∈{1,2,3} represents the layer index, and the first layer input is the original sequence x t (1) =x t , each layer is represented as follows: The input gate of the Lth layer is: The forget gate of the Lth layer is: The candidate memory units in the Lth layer are: The memory unit of the Lth layer is updated as follows: The output gate of the Lth layer is: The hidden state of the Lth layer is: The state transfer of the Lth layer is: The final output of the three-layer LSTM unit is: At the input end, the rough surface feature vector and the material hardness feature vector are fused and spliced, and the fused features are convolved through the Causal CNN to ensure y [ t ]Only by x [ t ] ，x [ t-1 ] ，···，x [ t-K+1 ]calculate; After outputting the predicted value at the output end, a physical constraint correction part is introduced to ensure that the output parameters are always within a reasonable range and can be forced to correct to a safe range under extreme working conditions; in is the total energy consumed in cutting, is the cutting force, is the predicted cutting speed, is the learnable stiffness coefficient, is the estimated material deformation, is the kinetic energy consumed during the cutting process, and It is the accumulation of kinetic energy during the cutting process.

Images