1621 results about "Tactile sense" patented technology

The invention relates to the technical field of robot decision making, and provides a robot multi-modal fusion autonomous decision making method and system based on a large language model.The method comprises the steps that a robot obtains multi-modal environment information through a visual sensor, a touch sensor, an auditory sensor and a laser radar which are carried by the robot; performing preliminary filtering and noise reduction processing on the original sensor data, and synchronously recording all the sensor data by timestamps; performing space-time semantic alignment on the preprocessed multi-modal data, mapping pixel coordinates of a target in a visual target coordinate quantization original image to a robot coordinate system, performing uncertainty evaluation on a multi-modal signal through a dynamic Bayesian network, and taking entropy or variance as an uncertainty quantitative evaluation index. According to the method, the information quality is improved from a data fusion source, accurate and reliable basic support is provided for subsequent decision making, and decision making errors caused by data deviation are greatly reduced.

The invention relates to the technical field of soft robots and intelligent control, in particular to an intelligent control method based on the Internet of Things. The method comprises the following steps: carrying out multi-modal data acquisition and preprocessing on a contact process of a gripper and an object to obtain a time synchronization data set; constructing a tactile feature tensor including pressure, friction force and strain force according to the time synchronization data set; performing stress gradient calculation and mapping according to the tactile feature tensor to obtain a stress gradient field and a region threshold mapping table; performing stress gradient threshold adaptive adjustment according to the regional threshold mapping table and the stress gradient field to obtain an adaptive threshold distribution map; and performing stress risk prediction according to the adaptive threshold distribution map to obtain a stress risk map. According to the method, the potential damage risk is predicted by sensing the contact state of the gripper and stress concentration, the adaptive capacity, robustness and success rate of the grabbing process are remarkably improved, and the method is particularly suitable for grabbing tasks of fragile and irregular objects which are difficult to prejudge.

The invention relates to the technical field of body-equipped intelligent robots and intelligent equipment, in particular to a self-adaptive flexible tightening body-equipped intelligent robot which comprises a body-equipped moving platform, the top of the body-equipped moving platform is provided with a body-equipped lifting mechanism, and the top of the body-equipped moving platform is provided with a body-equipped articulated arm module. And a tightening actuator and a clamping jaw actuator are arranged at the tail end of the outer part of the body articulated arm module. According to the self-adaptive flexible tightening intelligent robot and the tightening process method, sensing and self-adaptive adjustment of external environment changes are achieved through multi-module integration, and the tightening process method based on a multi-mode sensing system and an intelligent control system is constructed; vision, force sense and touch sense data are fused, real-time monitoring and dynamic adjustment of the tightening process are achieved through a machine learning algorithm, and the intelligent level and quality stability of tightening operation are effectively improved.

The invention relates to the technical field of artificial intelligence, can be applied to service scenes such as pension service, financial science and technology and medical health, and discloses a grabbing control method, device and equipment based on collaborative grabbing characteristics and a medium. Generating collaborative grabbing features and feature weight vectors, constructing a grabbing model in combination with historical grabbing data, and determining an initial grabbing strategy; and in combination with an initial grabbing strategy and real-time tactile feedback data, a grabbing control sequence and a cooperative adaptation coefficient are generated through constraint optimization, grabbing operation is executed, real-time feedback is monitored, the grabbing force and posture are dynamically adjusted according to deviation and the cooperative adaptation coefficient, and accurate control is achieved. According to the method, the problem that in the prior art, grabbing response is not flexible is solved by constructing the collaborative grabbing features and the grabbing model and fusing real-time feedback and control strategies, dynamic self-adaption and fine adjustment in the grabbing process of different objects are achieved, and the damage risk is reduced.

The invention discloses a joint denoising method for robot visual motion prediction, and the method comprises the steps: constructing a unified generative model through fusing an image and a depth map collected by a depth camera, motion data collected by CAN line communication of a Piper mechanical arm, and a tactile image collected by a Gelsight Mini tactile sensor; the method comprises two steps of data acquisition and input coding, and joint denoising and generation: firstly, multi-modal data are coded into low-dimensional potential representation, and then future images, depth maps, tactile data and robot actions are cooperatively predicted through a joint denoising framework based on Transform. A mask self-attention mechanism is innovatively introduced, information interaction between modes is dynamically adjusted, action generation is guided through tactile feedback, and the force control precision is improved. The model adopts a de-noising diffusion probability loss function to jointly optimize multi-modal prediction, so that the output consistency is ensured. According to the method, the robustness and the accuracy of flexible operation of the robot are remarkably improved.

The invention discloses a flexible magnetic tactile sensing device based on a microstructure elastic layer, a three-dimensional force measuring device and a measuring method.The flexible magnetic tactile sensing device comprises a flexible magnetic film, the microstructure elastic layer, a Hall sensor, a PCB and a bottom shell, the Hall sensor is installed on the PCB, and the flexible magnetic film, the microstructure elastic layer, the PCB and the bottom shell are sequentially stacked from top to bottom; the microstructure elastic layer is of a non-planar structure, when the flexible magnetic film deforms due to stress, deformation of the flexible magnetic film is transmitted to the microstructure elastic layer, the change of the three-axis magnetic field intensity is measured through the Hall sensor and uploaded to a PC terminal for data processing, and three-dimensional force measurement is achieved. Three-dimensional force measurement can be achieved, the flexible magnetic film can generate local deformation in the normal direction when normal force is applied, the flexible magnetic film can generate tangential displacement when tangential force is applied, and application scenes suitable for different microstructure elastic layers are provided according to quantitative analysis results.

The invention relates to a robot dynamic grabbing control method based on visual sense and tactile sense depth fusion. The method comprises the following steps that firstly, based on visual information, surface geometric features of a target object are extracted, grabbing adaptability scores are calculated, an optimal grabbing point is selected, and a collision-free approaching track is planned; secondly, in a contact establishment stage, detecting a contact event through a touch sensor, fusing vision and touch data to unify a coordinate system, calculating a vision-touch consistency score, and applying an initial grabbing force; and thirdly, in the stable holding stage, slip detection is conducted through wavelet packet energy entropy and pressure gradient, and the grabbing force and impedance parameters are dynamically adjusted in combination with self-adaptive impedance control. According to the method, intelligent control over the whole process from approaching to contact to stable holding is achieved, and the adaptability, stability and safety of robot grabbing in the dynamic environment are improved.

The invention discloses a surgical operation system with a training or auxiliary function, particularly relates to the field of intelligent surgical operation systems, and cooperatively works through five modules including multi-modal sensing, layered AI decision making, dynamic feedback, closed-loop learning and an expansion interface. MEMS pressure sensing and biological signal fusion sensing are adopted, strategic layer three-dimensional navigation and execution layer magneto-rheological damping control are combined, and an operation whole-process optimization system is constructed; a multi-source data persistent evolution model is integrated through federal learning, and precise man-machine interaction is realized by matching eye movement tracking AR and a variable stiffness touch glove. According to the system, the intraoperative risk identification accuracy is improved, the operation efficiency is improved, third-party equipment extension and doctor skill quantitative evaluation are supported, and the operation safety and the training standardization level are remarkably improved.

The invention relates to the technical field of electromyographic signal monitoring, and discloses a guided monitoring method and system based on a wireless surface electromyographic analysis feedback instrument. The method comprises the following steps: acquiring electromyographic signal time sequence data and limb movement track space coordinate information when a target muscle group moves through a multi-channel wireless surface electromyographic sensor; muscle activation intensity and fatigue change features are extracted, a joint activity angle change sequence is calculated, and a muscle-joint collaborative activity feature matrix is generated through space-time alignment fusion; analyzing signal transduction delay and amplitude matching degree in the matrix, and evaluating the nerve driving coordination level; according to the deviation from the preset standard mode, generating a real-time biological feedback signal containing an action correction instruction and a muscle activation intensity adjustment parameter; and converting into a tactile vibration mode and a visual guidance mark to be output. According to the method, synchronous acquisition and fusion analysis of multi-dimensional motion information are realized, and visual guidance is provided through multi-modal feedback.

The invention discloses a touch feedback enhanced multi-mode robot grabbing control system and method. The touch feedback enhanced multi-mode robot grabbing control system comprises a touch feedback enhancement module, a visual touch language large model semantic generation module and a touch enhancement diffusion control optimization module. The tactile feedback enhancement module is based on a multi-agent cooperation system, performs semantic level and signal level processing on tactile signals, and realizes upper and lower layer coordination through a feedback routing mechanism. And the visual touch language large model module adopts a touch encoder based on CLIP to realize multi-mode semantic conversion and material attribute identification. And the tactile enhancement diffusion control module adopts a BRIDGeR style interpolation diffusion technology, and combines visual embedding and tactile constraint to realize high-frequency optimization of an action track. According to the method, the material identification accuracy, the precision operation success rate and the complex scene self-adaptive capability are remarkably improved.

A robot control method based on tactile prediction pre-training comprises the following steps: acquiring and generating a human playing data set consisting of three-channel image tensors in an offline stage, and training a constructed conditional diffusion model comprising a tactile encoder, a tactile decoder and an action and visual encoder; in the online stage, the trained conditional diffusion model is integrated into a standard imitation learning strategy network, and an action instruction of the robot is generated according to the state of the robot, the current visual features and the tactile feature vectors extracted by the imitation learning strategy network. According to the method, a specific agent task is completed by training a deep neural network model, that is, a future tactile signal sequence is predicted according to historical information and future action intentions; the model is enabled to characterize generic haptic features contacting physical dynamic laws for further migration into downstream robot control tasks.

The invention relates to the technical field of robot control, and discloses an industrial robot adaptive control method and system based on multi-modal sensor fusion, and the method comprises the steps: collecting multi-modal original data, and carrying out the time-space alignment; capturing space-time semantic association of visual textures, tactile pressure distribution and force sense fluctuation in the multi-modal data through a multi-head attention mechanism guided by a physical model, and performing space-time registration; a CNN-LSTM hybrid model is adopted to extract visual texture features and time sequence tactile features in the physical information enhanced multi-modal feature matrix; and carrying out dynamic weight distribution on the fusion feature vectors with physical consistency by utilizing a weight distribution model driven by element reinforcement learning to generate dynamic weighted fusion features. According to the method, the spatial positioning precision of the industrial robot in a precise assembly scene is greatly improved, the contact force control stability is greatly improved, and the control robustness in a complex operation scene is remarkably enhanced.

In an ophthalmic surgical laser system, a patient interface device for coupling a patient's eye to the laser system includes a lens cone with a frustoconical shaped shell for coupling to the laser system and a suction ring for coupling to the patient's eye, the lens cone and the suction ring being joined together by clamping. A cap is provided for use with the lens cone as an installation aid. In the configuration supplied to the user, the lens cone is partially embedded in and snapped to the cap. The cap has a portion with a relatively large diameter and multiple ribs for easy handling. The user holds the cap to install the lens cone on the laser system, and pulls the cap to unsnap it from the lens cone. The lens cone is attached to the laser system with a bayonet mount that provides tactile feedback to the user.

The invention discloses a method and system for evaluating infusion operation of intern medical staff, and belongs to the field of infusion, and the method comprises the steps that the operation of the intern medical staff is collected in multiple modes, and the multiple modes comprise a visual mode, a touch mode and a physiological signal mode; inputting the collected operation of the intern medical staff into the multi-modal infusion evaluation model, performing feature extraction by the multi-modal infusion evaluation model in a modal manner, and performing cross-modal attention fusion on feature vectors of different modals; and finally, comparing the calculated total feature tensor Hfusion with a standard total feature tensor Hfusion ', and outputting a final score of the practice medical personnel according to a preset scoring standard. According to the scheme, multi-dimensional data such as visual, tactile and physiological signals are synchronously collected through a multi-modal fusion method, operation details are comprehensively covered, comprehensive evaluation is carried out, and evaluation objectivity is improved.

The invention relates to the technical field of blind guiding robot interactive navigation, and particularly discloses a blind guiding robot interactive navigation system combining vision, inertial navigation and voice, which is characterized in that environmental vision, movement and voice data are synchronously acquired through a multi-modal perception data acquisition module, and are analyzed through a situation-user joint state feature extraction module; a user potential intention deduction module is combined to predict a user intention and generate a self-adaptive guide strategy; a self-adaptive guiding path optimization module adjusts a navigation path according to a strategy to ensure safety and high efficiency; finally, the situational multi-mode interaction instruction output module provides voice and tactile feedback, and interaction experience and navigation autonomy are enhanced. According to the method, the travel problem of the visually impaired person in a complex environment is solved, the individuation, context perception capability and safety of blind guiding are improved, and more natural and efficient navigation assistance is brought to the visually impaired person.

The invention relates to the field of robot intelligent control and man-machine collaboration, in particular to a multi-sensing collaborative robot control method, which comprises the following steps: synchronously filtering mechanical, tactile, visual and physiological signals to generate a unified observation frame, and calculating power flow and energy in combination with a joint state to form an execution vector; extracting a topological descriptor by using persistent coherence, and outputting an interaction potential field and a human body intention by using the topological descriptor as a conditional driving diffusion model; then, power density is constructed based on the potential field gradient and intention, joint torque and impedance are generated through entropy regular optimal transmission and Jacobian and microoperator kernel mapping, and an energy Lagrange multiplier is updated in real time; and if the consistency error continuously exceeds a threshold value, estimating a Koopman lifting model on line by using a causal weight, completing model prediction control under the energy, conformal security and topology consistency constraint, outputting an update instruction and performing closed-loop feedback. The method improves the cooperation precision and safety, reduces the energy consumption, and is suitable for a high-dynamic man-machine cooperation scene.

The invention relates to the technical field of robot control, in particular to a robot object grabbing adaptive learning system based on visual and tactile feedback. The robot object grabbing adaptive learning system based on visual and tactile feedback comprises a data acquisition module used for acquiring adaptive control data of a robot; the joint adjusting module is used for obtaining the joint compensation amount of the robot according to the adaptive control data of the robot and adjusting the grabbing motion of the robot to obtain the final joint adjustment degree; the visual feedback module is used for obtaining a rough grabbing force estimation value of the visual information by using a visual feedback formula according to the visual data of the robot; and the tactile feedback module is used for obtaining correction force of tactile feedback by using a tactile feedback formula according to the tactile data of the robot. Through multi-modal information fusion and modular design, high-precision and self-adaptive grabbing of the thermochromic object by the robot is realized.

The invention relates to the technical field of vehicles, in particular to an intelligent cabin man-machine cooperative control method and device, a vehicle, a medium and a product. Performing fusion processing on the voice signal data, the visual signal data and the tactile signal data of the current driver by using a preset dynamic weight distribution model to obtain a multi-modal fusion control instruction; and under the condition that the multi-mode fusion control instruction meets a preset sight-voice dual-mode condition, synchronously issuing the multi-mode fusion control instruction to a plurality of target execution devices in the intelligent cabin so as to drive each target execution device to perform cooperative control. Therefore, by constructing a cooperative control mechanism of multi-mode sensing and dynamic weight distribution, the problems that the false wake-up rate is high, the voice and graphical interface cooperative efficiency is low and the interaction mode cannot adapt to a complex driving environment due to a single or separated interaction mode in the related technology are effectively solved.

The invention discloses an operation AR intelligent auxiliary management system based on voice AI interaction driving, which is applied to the field of intelligent warehousing and comprises an environment perception type voice interaction module, an AR multi-mode interaction compensation module, a dynamic noise robustness data processing module, an operation management module and a database module. The environment perception type voice interaction module is used for collecting voice signals of operating personnel and inhibiting environment noise; the dynamic noise robustness data processing module is used for carrying out noise reduction processing, semantic understanding and closed-loop optimization on the voice signal and generating feedback information and a control instruction; the AR multi-modal interaction compensation module is used for receiving feedback information of the dynamic noise robustness data processing module and providing visual and tactile interaction compensation, and the job management module is used for scheduling management equipment according to a control instruction; according to the method, the operation efficiency can be effectively improved, the accuracy is high, the real-time performance is high, and interaction experience optimization can be realized.

The invention relates to the technical field of mechanical arms, in particular to a chemical emergency disposal-oriented manipulator cross-modal sensing control system which comprises an environment scanning module, a visual positioning module, an obstacle avoidance planning module, a pose compensation module, a decision mapping module and a grabbing updating module. Obtaining environment state information; an operation view angle image of the manipulator is collected, visual features of the operation view angle image are solved, and pose information is obtained; obstacle avoidance correction is conducted on the motion trail of the mechanical arm, and a collision-free approaching path is obtained; the operation view angle image and the multi-mode micro-touch signal are fused, error compensation is carried out on the pose information, and an accurate grabbing pose is obtained; mapping object surface physical attributes sensed by the multi-mode micro-tactile signals to obtain a self-adaptive grabbing instruction; the manipulator is controlled to conduct grabbing through the self-adaptive grabbing instruction, and environment state information is updated; according to the invention, the accuracy of cross-modal sensing control of the manipulator can be improved.

The invention relates to the technical field of data processing, in particular to a method for driving emotion interaction of an intelligent device based on multi-modal understanding, and aims to eliminate illumination and noise interference and output a standardized face video stream, an effective voice segment and a touch thermodynamic diagram through an environment adaptive acquisition module. The feature extraction module extracts facial action optical flow features, voice Mel-frequency cepstral coefficient vectors and tactile pressure gradient parameters. The cross-modal correlation model adopts a tensor decomposition algorithm to calculate a space-time correlation matrix of visual and voice features, and the tactile feature weight is dynamically adjusted in combination with environmental parameters. According to the response strategy, an intervention scheme is retrieved based on a graph database, emotion confirmation statements, guide statements and behavior suggestions are fused to generate multi-mode response, and PID adjustment of the temperature control device and tactile pulse output of the vibration device are synchronously driven. And the feedback evaluation module verifies the emotion recognition consistency through a Pearson's correlation coefficient, triggers conflict sample separation storage and model increment training, and realizes closed-loop optimization.

The embodiment of the invention provides a multi-mode fusion man-machine interaction control method, system and device and a storage medium, and relates to the technical field of intelligent driving, and the method comprises the steps: obtaining vehicle driving data and driver state data; based on the vehicle driving data and the driver state data, calculating fusion weights of a visual mode, an auditory mode and a tactile mode to obtain a multi-mode fusion weight matrix; generating a multi-modal interaction signal according to the multi-modal fusion weight matrix, wherein the multi-modal interaction signal comprises a visual signal, an auditory signal and a tactile signal; and triggering corresponding visual warning, auditory warning and tactile warning according to the multi-mode interaction signal. In this way, multi-mode fusion is conducted on vision, hearing and touch, the output intensity of visual warning, hearing warning and touch warning is adjusted according to the weight of each mode of vision, hearing and touch, a driver is helped to make the most urgent operation at present according to the warning of each mode, and the driving safety of man-machine interaction in the driving process is improved.

The invention relates to the technical field of artificial intelligence, and discloses an artificial limb driving method based on a spiking neural network, which comprises the following steps: acquiring a multi-modal signal, and preprocessing the multi-modal signal; converting the preprocessed multi-mode signal into a pulse signal; inputting the pulse signal into a pulse neural network for sensing fusion processing, and outputting a control strategy; dynamically adjusting a control strategy based on pulse reinforcement learning according to the behavior feedback information through a reward function, and generating a learning result; and generating a bionic motion instruction according to the adjusted control strategy and the learning result, and driving an artificial limb execution mechanism to act according to the bionic motion instruction. According to the method, the electromyographic signals, the tactile pressure signals and the inertial measurement data are uniformly coded into the pulse sequence, the pulse neural network is utilized to realize multi-modal sensing fusion, the pulse reinforcement learning algorithm is introduced, the artificial limb control strategy is optimized in real time according to user behavior feedback, and the accurate control of the artificial limb joint is improved.

The invention provides a multi-modal fusion fatigue screen monitoring recognition and reminding method and system, and the method comprises the steps: data collection: collecting the visual data of a screen monitoring person in real time, and synchronously collecting the physiological data; multi-modal data fusion: adopting a feature weighted fusion method based on an entropy weight method to adaptively distribute weights and generate a fusion fatigue index FFI by quantifying dynamic information entropy of each visual data and physiological data; fatigue grade classification: realizing three-level fatigue state judgment based on a fusion fatigue index FFI obtained by multi-modal data fusion and a dynamic decision tree model; and dynamic intervention: based on a fatigue grade classification result, adopting a double-channel intervention mechanism of bracelet touch alarm and automatic telephone call value length. When fatigue and inattention of a monitoring screen watchman occur, the monitoring screen watchman can be timely and accurately identified and a reminding intervention mechanism is started, so that the continuity and the safety of the monitoring screen work are ensured, and bad safety production events caused by human reasons are avoided.

The embodiment of the invention provides a dynamic tactile rendering method based on a two-dimensional image and a finger motion track. The method comprises the following steps: acquiring an original two-dimensional image; converting a target object in the original two-dimensional image into a three-dimensional structure point cloud; respectively mapping the three-dimensional structure point cloud and the predicted complete trajectory to a unified coordinate system to obtain a corresponding point cloud data point set and a trajectory data point set; performing interpolation smoothing on the matching point set to construct a continuous geometric profile curve; calculating a first-order derivative of each section point in the geometric profile curve, and taking the first-order derivative as a stimulation value of the virtual tactile sense; converting the stimulation value into a driving voltage signal, and controlling the operation of the electrostatic adhesion tactile feedback device through the driving voltage signal; according to the dynamic tactile rendering generation method based on the two-dimensional image and the finger motion trail, the limitation that traditional tactile feedback depends on a static template and special sampling data is broken through, and automatic modeling and real-time response from two-dimensional visual input to physical tactile output are achieved.

The invention discloses a quadruped vehicle sensing and decision-making system based on reinforcement learning. The quadruped vehicle sensing and decision-making system comprises a multi-mode sensing module, a dynamic fusion module, a reinforcement learning decision-making module and an execution module. The multi-modal sensing module is used for collecting multi-modal environment information including visual, tactile and inertial data; the dynamic fusion module carries out dynamic fusion on the multi-modal data by using a Transform-based feature extraction and time sequence alignment method to generate a unified environment representation; the reinforcement learning decision-making module is used for analyzing the multi-modal sensing result on the basis of a Proximal Policy Optimization (PPO) algorithm, and generating an optimized path planning and dynamic obstacle avoidance instruction; and the execution module generates a gait and motion control command according to the decision instruction to ensure the autonomous navigation and environment adaptability of the quadruped robot. According to the system, the sensing precision and the decision-making efficiency of the quadruped robot in a complex scene are remarkably improved, and the system has a wide application prospect.

The invention discloses a robot system with tactile perception, and belongs to the technical field of tactile robots. The problem that an existing robot is poor in environment pressure sensing capacity is solved. Comprising a head module, an integrated vision-inertial navigation sensing module and a head controller, two side surfaces are provided with a pair of rotatable connecting ends, and the outer surface is provided with a tactile sensor; the head controller transmits data to the upper computer through the communication module; the driving body module comprises a plurality of modularized joints which are connected end to end, touch sensors are arranged on the two side faces of each modularized joint, and the adjacent modularized joints are orthogonally connected; the modularized joint is integrated with a joint controller and a joint steering engine; the tail module integrates a tail controller and a tail steering engine, and tactile sensors are arranged on the two side faces of the tail module. The tail controller carries out data transmission with the head controller through the adjacent previous joint controller in sequence, and the head controller receives a control instruction of the upper computer and carries out data transmission with all other controllers. According to the invention, the environmental pressure can be sensed in all directions.

The invention provides an intelligent tactile stick interaction method and system based on multi-modal perception and edge calculation, and relates to the technical field of intelligent auxiliary equipment. The method comprises the following steps: acquiring environment and position data through a camera, a GPS and an inertial measurement unit; performing multi-modal data fusion and deep learning identification by using an edge calculation module to generate a dynamic environment map; performing real-time path planning and obstacle avoidance based on a map and a destination; analyzing a user instruction through voice recognition and interactively adjusting a path; haptic and auditory bimodal feedback is provided through a vibration module and a voice synthesis module according to a navigation result; and when an emergency obstacle is detected or a user actively alarms, a local and remote alarm mechanism is triggered. The system correspondingly comprises an intelligent tactile stick terminal and a cloud server. According to the invention, accurate and real-time perception and intelligent navigation of the environment are realized, visual and natural interaction experience is provided, an effective emergency help channel is established, and the safety, independence and convenience of travel of visually impaired people are significantly improved.

The invention discloses a green plum fruit picking robot based on multi-source data fusion and a picking method thereof, and relates to the technical field of agricultural intelligent equipment. The robot comprises a crawler-type self-adaptive chassis, a multi-degree-of-freedom mechanical arm, a multi-source sensing module, a bionic flexible end effector, an intelligent control unit and a fruit collecting box, and all the parts are electrically connected with the intelligent control unit. The multi-source sensing module synchronously collects vision, spectrum, touch, acoustics and environment data, and accurate recognition and positioning of mature fruits are achieved through hierarchical weighted attention fusion algorithm processing. And fruit damage is reduced in combination with a path planning algorithm introducing a flexible obstacle avoidance coefficient and a flexible picking execution mechanism. The self-adaptive chassis can adapt to complex terrains, and parameters are dynamically optimized through reinforcement learning in the picking process. The green plum picking machine solves the problems that traditional picking is low in efficiency and high in damage rate, and existing equipment is poor in adaptability, and the intelligent level and operation quality of green plum picking are improved.

A sensor module for an end effector of a robot is described. The sensor module includes a substrate having formed thereon, a set of proximity sensors and a set of pressure sensors, the set of proximity sensors and the set of pressure sensors configured to have overlapping sensing regions, and a cover coupled to the substrate, the cover comprising a material that permits transmission of signals from the set of proximity sensors through the material.