194 results about "Touch Perception" patented technology

The invention discloses a manipulator grabbing planning system and method based on visual identification, and relates to the technical field of robots, the manipulator grabbing planning system comprises a visual perception module, a coordinate conversion module, a motion planning module, a control execution module, a tail end perception module and a system integration and communication module; the visual perception module realizes multi-view target detection, semantic segmentation and three-dimensional pose estimation through a binocular depth camera; the coordinate conversion module is used for converting a target pose under a camera coordinate system into a world coordinate under a mechanical arm base coordinate system; the motion planning module is responsible for generating a mechanical arm grabbing path, optimizing a strategy and supporting generalization migration of a multi-form mechanical arm; the control execution module drives the six-axis cooperative mechanical arm to complete the grabbing action, and vision-force mixed feedback closed-loop control is achieved. The tail end sensing module monitors the grabbing state in real time through a touch sensor and dynamically adjusts a grabbing strategy; and the system integration and communication module is used for realizing real-time data interaction and cooperative work among the modules of the system.

The invention relates to a deformable object interactive operation control method based on a visual touch-language-action multi-mode model, and the method comprises the steps: carrying out the image, touch and language data coding of a deformable object, and obtaining visual, touch and language features; performing cross-modal feature alignment processing on the visual features, the tactile features and the language features to obtain multi-modal fusion features; inputting the multi-modal fusion features into a large model for environment understanding; iteratively performing action planning and execution by adopting a'thinking-decision 'planning mode; and repeatedly executing the steps until the current deformable object interaction operation task is completed. Compared with the prior art, the multi-modal feature alignment capability, the action planning precision and the task adaptability are improved, efficient recognition and interaction of the robot on the deformable object can be achieved, deformation and state changes of the object can be effectively dealt with especially in a complex environment, the operation strategy can be dynamically adjusted, and the operation efficiency is improved. And more intelligent and accurate deformable object operation is realized.

The invention provides a dexterous hand multi-object stable grabbing method and system based on vision and touch perception. The dexterous hand multi-object stable grabbing method comprises the steps that RGB images and depth maps of target objects are collected through a monocular RGB-D camera, and dense point cloud data are generated; geometric features are extracted by using a three-dimensional point cloud neural network, and composite object distribution features are analyzed in combination with visual information; an initial grasping strategy is generated based on the features, actions are executed, and the joint state and contact force data are synchronously obtained in real time through body sensing and a fingertip force sensor; in the carrying process, multi-dimensional force perception and the joint state are fused to analyze object stress distribution and center-of-gravity shift; vision and force sensing data are used as input. Vision, body sensing and force sensing data are fused, the problems of three-dimensional structure recognition and dynamic stress regulation and control in multi-object composite grabbing are solved, compared with a single sensing scheme, the grabbing planning precision and the self-adaptive capacity in a complex scene are remarkably improved, and the method is particularly suitable for stable carrying of combinations of loose objects such as dinner plates carrying tableware.

The invention discloses a multi-curvature high-resolution bionic miniature visual tactile sensor for fingertips, which relates to the technical field of sensors and comprises a fingertip structure, a transparent support structure, a flexible silica gel sensing layer and an image acquisition module. A cavity is formed in the fingertip structure; the transparent supporting structure is clamped in the cavity; the flexible silica gel sensing layer is adhered to the transparent supporting structure, and black circular array mark points are attached to the surface of the flexible silica gel sensing layer; the image acquisition module is arranged on the fingertip structure, the acquisition end of the image acquisition module is arranged above the transparent supporting structure, and the view field area of the acquisition end covers the whole flexible silica gel sensing layer area; compared with other fingertip tactile sensors, the visual tactile sensor is a human fingertip simulated visual tactile sensor with the highest spatial resolution, the smallest overall size and the lowest manufacturing cost.

The invention discloses a digital twin interaction control system based on robot vision and tactile fusion, and the system comprises a data collection module which is used for collecting visual image data and tactile sensing data; the preprocessing module is used for preprocessing; the multi-mode perception modeling module is used for executing spatial perception modeling and tactile state modeling; the fusion state vector construction module is used for generating a fusion state vector; the digital twinborn mapping module is used for constructing a digital twinborn body; the simulation interaction control module is used for executing interactive action control and generating a control instruction parameter group; the action execution and feedback module is used for controlling the robot to execute the actual action and updating the fusion state vector; and the dynamic evaluation and self-adaptive adjustment module is used for dynamically optimizing and adjusting the fusion state vector. Visual and touch information of the robot is fused, a digital twinning synchronous control mechanism is constructed, and the interactive optimization method which is fast in operation response, high in execution precision and stable in control process is achieved.

A visual tactile sensor, a manipulator and a control method thereof, the visual tactile sensor comprising: a contact layer, at least a portion of which is transparent; when the contact layer is subjected to external force, at least one part of a plurality of markers in the marker array generates displacement and / or deformation; and the image acquisition unit is used for performing image acquisition on the marker array and performing image acquisition on the outside through the transparent part of the contact layer. The visual tactile sensor provided by the invention not only can capture the external visual information before operation to identify the operation object, but also can provide the tactile information in the operation process, so that the perception capability of an operation system is enhanced, and the operation success rate is improved.

The invention provides a touch and visual perception integrated micro-nano robot magnetic field control system and method, and the system comprises a camera which collects the image of a micro-nano robot in a pipeline in real time; the magnetic field generating device generates a gradient magnetic field for driving the micro-nano robot to move through a permanent magnet; the tail end of the mechanical arm is connected with the permanent magnet and adjusts the three-dimensional pose to change the magnetic field distribution; the force feedback equipment is used for providing a touch signal for the micro-nano robot to be in contact with the pipeline boundary; and the computer is configured to identify the aggregation state of the micro-nano robot according to the camera image, increase the resistance of the force feedback equipment and reduce the rotation speed of the permanent magnet when the dispersion state is detected, and reduce the resistance and recover the conventional magnetic field control after the aggregation state is recovered. According to the system, accurate control and abnormal state self-adaptive response of the micro-nano robot in a complex experiment scene are realized through touch-vision collaborative perception and dynamic magnetic field regulation and control.

The invention discloses a wearable path guiding system based on visual-tactile cross-modal mapping. The wearable path guiding system adopts a three-module architecture, namely a visual environment sensing module, a simplified path planning module and a tactile guiding module. The environment perception module constructs an incremental scene map with semantic annotations based on a depth camera. And the simplified path planning module is used for generating a steering vector containing gait information and a steering angle according to the advancing direction and simplified path pointing and performing real-time iterative optimization to realize real-time matching of the advancing direction of the user and the target direction. And the tactile guidance module is used for realizing synchronization of gait and tactile stimulation, generating priority gradient intensity field tactile feedback with a direction indication characteristic according to path deviation information, correcting the advancing direction, and starting the voice auxiliary module when multiple times of tactile correction do not reach the expectation. The method is suitable for a visual signal limited scene, the visual signal is mapped into the tactile signal to realize self-adaptive synchronization with the motion rhythm, and path guidance is completed.

The invention discloses a multi-mode tactile and visual fusion identification method for medicine box braille salient point characters. The method comprises the following steps: acquiring three-dimensional morphology and pressure data of Braille salient points through a tactile sensor, acquiring an image and the like through a visual sensor, and performing preprocessing (improved U-Net segmentation, illumination correction and the like); hardware is utilized to trigger synchronization, an ICP algorithm is improved to complete space-time alignment, and a feature level (attention mechanism) and a decision level (D-S evidence theory) are fused; character semantics are analyzed by means of a multi-mode neural network, and multi-channel feedback is visualized through voice, tactile vibration and AR. A dynamic optimization mechanism is set, and sensor scheduling, incremental learning and safety verification are covered. The system comprises a tactile / visual perception module, a data processing module (FPGA + NPU), a feedback execution module and a communication module. According to the scheme, the problems of low traditional recognition precision and poor environmental adaptability are solved, high-precision medicine box braille recognition is achieved, and a reliable scheme suitable for multiple scenes is provided for safe medicine use of visually impaired people.

The embodiment of the invention discloses a visual tactile sensor, a clamping force control method and device and related equipment, a clamping device is controlled to clamp an object to move, and the visual tactile sensor is arranged on the clamping device; in the process of controlling the clamping device to clamp the object to move, the clamping force of the clamping device is adjusted in response to event information output by an event camera on the visual touch sensor. Therefore, through the visual tactile sensor arranged on the clamping device, in the process of controlling the clamping device to clamp the object to move, the clamping force of the clamping device can be adjusted in time according to the event information output by the event camera on the visual tactile sensor, so that the object is prevented from sliding or even falling off, and the user experience is improved.

The invention provides a control method and system of a double-arm clothes folding robot, and relates to the technical field of robot control, and the control method constructs a perceived control closed loop by fusing visual, force and tactile multi-modal data. The touch sense and the force sense make up the blind area of the vision in the self-shielding scene, and grabbing, sliding and pleating perception is improved. Based on clothing joint characterization of a visual semantic network, geometric morphology, semantic components and confidence coefficient are structured and coded, a robust state fusing prior and real-time observation is provided for planning, and decision errors caused by deformation and noise are reduced. Long-time-history folding is decomposed into sub-target sequences through a hierarchical strategy, action closed-loop correction is conducted through multi-modal data, and cooperative control over grabbing, folding and pleating is achieved. And finally, a torque instruction is generated based on a control algorithm, interaction compliance and safety are ensured, and the folding success rate and operation stability of the double-arm clothes folding robot under different materials, sizes and initial forms are integrally improved.

The invention relates to the technical field of image analysis, and discloses an end-to-end visual tactile perception method and system based on a morphology-force field analysis model, a terminal and a storage medium, and the method comprises the steps: employing a morphology reconstruction module to achieve the analysis of a micron-order contact surface, employing a force field analysis model to achieve the precise analysis of a force field in each direction, and taking the two results as a data set, inputting and mapping the image into contact morphology, normal and shear force distribution through an end-to-end network, integrating the high efficiency of data driving and the reliability of physical consistency, and finally introducing partial differential equation residual error sum into a loss function to carry out optimization. And the morphology and the force field meet the set physical consistency on the whole. According to the method, micron-sized morphology analysis and multi-axial force synchronous estimation are realized, the real-time robust performance under a high-frequency dynamic task is realized, the cost of tactile perception is reduced, and meanwhile, the accuracy of a tactile perception result is improved.

The invention discloses a double-stage object grabbing method and system based on visual touch feature fusion, and the method comprises the steps: carrying out the visual touch feature fusion processing of a to-be-grabbed object based on a visual touch fusion network model, and obtaining a visual touch fusion feature vector of the object; performing pose estimation on a to-be-grabbed object based on the attention mechanism residual convolutional network model to obtain a grabbing rotation angle and a grabbing mass score; the to-be-grabbed object is pre-grabbed based on the grabbing rotation angle and the grabbing mass fraction, and physical parameter information of the to-be-grabbed object is obtained; and in combination with the visual touch fusion feature vector of the object and the physical parameter information of the object, performing slip detection on the to-be-grabbed object. According to the method, the visual features and the tactile features of the to-be-grabbed object can be fully fused, the pose information of the grabbed object can be effectively predicted, and the object grabbing accuracy is improved. The two-stage object grabbing method and system based on visual touch feature fusion can be widely applied to the technical field of machine vision.

The invention discloses a robot finger tactile perception method and system based on artificial intelligence, and the method comprises the following steps: S1, obtaining visual data and tactile perception initial parameters of a to-be-interacted object of a robot finger, carrying out the preprocessing of the visual data, and extracting the appearance features of the object, performing initial configuration on the touch sensing initial parameters; and S2, object recognition is conducted on the preprocessed visual data through a machine learning model, and the material category, morphological characteristics and surface attributes of an object are determined. According to the robot finger tactile perception method and system based on artificial intelligence, by combining a visual system and tactile sensor data and utilizing a convolutional neural network and Transform fusion architecture to extract local textures and global morphological features of an object, high-precision recognition of material categories, morphological features and surface attributes of the object is realized.

The invention provides an isomorphic sensor dexterous hand-tactile glove system and a data acquisition method, and belongs to the technical field of tactile sensing, the system comprises a tactile data acquisition module which comprises a glove main body and a fixed flange arranged on the glove main body, a tactile sensor array is arranged on the palm side of the fixed flange and used for collecting tactile data of interaction between the glove body and an object; the visual perception module comprises an infrared camera array and an RGB (Red, Green and Blue) camera array, and is used for tracking marking balls arranged on the fixed flanges so as to solve pose data of the fixed flanges and collecting RGB images of interaction between the glove main body and an object; and the multi-modal data processing module is used for receiving the tactile data, the pose data and the RGB image, realizing space-time alignment of the data and generating a standardized data set. The system is simple in structure, rapid in deployment and high in acquisition safety, and can be suitable for tactile big data acquisition under multi-scene and multi-task conditions.

The invention provides a multi-modal intelligent interaction optimization method and system, and the method comprises the steps: obtaining text instruction data according to a voice instruction of a user, and calculating the voice confidence; identifying and positioning an object in an image shot by the camera to obtain visual data, and calculating visual confidence; tactile data of the intelligent device and the contacted object are obtained according to the tactile sensor, and the tactile confidence coefficient is calculated; and then, adjusting the weight of each perception in a preset decision model according to the confidence of the three perceptions. And finally, inputting the text instruction data, the visual data and the tactile data into the decision model after weight adjustment to obtain a task execution strategy. According to the method and the system disclosed by the invention, voice, vision and touch perception can be combined, meanwhile, a confidence evaluation mechanism is introduced, multiple verification is performed on task execution, and dynamic adjustment is ensured to be made according to accuracy and reliability of different perception data.

The invention relates to the technical field of sensors, and particularly discloses an active omnidirectional dynamic feedback self-calibration visual tactile sensor, which comprises a flexible tactile sensing module, an imaging module, a driving mechanism and a supporting module, the driving mechanism comprises a lifting motion module, a yawing motion module and a pitching motion module; the pitching motion module is arranged on the yawing motion module, and the yawing motion module is used for driving the pitching motion module to do yawing motion; the imaging module is arranged on the pitching motion module, and the pitching motion module is used for driving the imaging module to do pitching motion; the lifting motion module is used for driving the yawing motion module to do lifting motion; the active omni-directional dynamic feedback self-calibration visual tactile sensor has three-degree-of-freedom active motion capabilities of lifting, pitching and turnover, and the active motion mechanism can dynamically adjust the imaging visual angle of the camera and effectively cover a tactile sensing blind area formed by mechanical constraint solidification of the camera in a traditional scheme.

The invention discloses a cognitive impairment risk intelligent matching intervention system based on multi-modal data, and belongs to the technical field of cognitive impairment, and the cognitive impairment risk intelligent matching intervention system specifically comprises the following steps: collecting a tactile interaction sequence and a face video data stream when a user executes a cognitive intervention task, and constructing an original multi-modal data set; extracting tactile operation track features and reconstructing a heart rate variability sequence, and generating a synchronous multi-mode feature set; identifying a difference interval of state fluctuation and extracting behavior and physiological features in the interval to form an instant physical and mental load parameter set; in combination with an interaction efficiency index in the task execution record, generating a joint state feature vector, and outputting a personalized adaptation instruction through a pre-trained sub-state-parameter association model; and constructing an adaptive strategy model by using the multi-modal features and an adaptive instruction to realize millisecond-level dynamic adjustment from the real-time multi-modal features to task parameters. According to the method, real-time perception and accurate matching of the cognitive emotion load of the user are realized, and the individuation degree and instantaneity of intervention are improved.

The invention relates to a visual sense and tactile sense fusion object identification method and system based on wavelet enhancement Transform. The method comprises the following steps: acquiring an original visual image and a tactile sense time sequence of an object to be identified; generating a visual token sequence and reconstructing visual features through visual wavelet enhancement processing based on the original visual image, and generating a tactile token sequence and reconstructing a tactile time sequence through tactile wavelet enhancement processing based on the tactile time sequence; based on the visual token sequence and the reconstructed visual features, generating a two-dimensional visual feature map by using a visual Transform encoder; based on the tactile token sequence and the reconstructed tactile time sequence, generating a tactile feature map by using a tactile Transform encoder; semantic alignment and fusion are carried out based on the two-dimensional visual feature map and the tactile feature map to obtain fusion features, and object recognition is realized based on the fusion features; the system is used for implementing the method. Compared with the prior art, the method improves the recognition precision.

The application belongs to the technical field of two-dimensional material devices, and discloses a strain-enhanced multi-sensory fusion optoelectronic synapse device and a preparation method and application thereof. The optoelectronic synapse device comprises, from bottom to top, an insulating layer, an electrode and a functional layer. The material of the functional layer is an in-plane polarization two-dimensional ferroelectric material, and the functional layer is applied with a tensile strain along the direction of its own polar axis. The optoelectronic synapse device simulates the perception of visual information through the photoconductivity effect of the functional layer and simulates the sense of touch through the tensile strain. The synergistic effect of the photoconductivity effect and the tensile strain significantly enhances the persistent photoconductivity (PPC) of the synapse device, and the enhancement processing of a fingerprint image is realized through the characteristics of the device. At the same time, by applying the tensile strain along a specific direction, the in-plane polarization intensity is significantly improved, the constraint of the depolarization field is avoided, the stability and performance of the ferroelectric device are improved, and a new way for improving the performance of the optoelectronic synapse device is provided.

The utility model relates to the technical field of sensors, in particular to a visual touch sensor. According to the utility model, the colloid on the transparent plate is in contact with an object, the single-sided mirror is arranged opposite to the colloid on the opening, the camera component performs visual detection on the deformation condition of the colloid reflected by the single-sided mirror, and the effect of clamping different objects is achieved through the deformation of the colloid, so that the tactile perception capability of the visual tactile sensor is realized; the robot is simple in structure, low in cost and high in adaptability, and can be widely applied to the fields of object grabbing, flexible operation, touch perception in rehabilitation prostheses and the like; and meanwhile, the number of parts required by the visual tactile sensor is reduced, the working stability of the visual tactile sensor can be effectively improved, the failure rate is low, and the reliability is relatively high.

The invention provides an object sensing method and system based on proximity vision and tactile deformation, which are applied to the technical field of tactile sensing, and the method comprises the steps: obtaining a binocular image collected by a visual tactile sensor after a target object is in contact with a preset visual tactile sensor; performing visual feature extraction based on the binocular image to obtain a binocular feature point two-dimensional coordinate of the target object; extracting a tactile mark based on the binocular image to obtain a two-dimensional coordinate of a tactile mark point of the target object; through a three-dimensional reconstruction model based on ray tracing, based on the two-dimensional coordinates of the binocular feature points, determining approximate three-dimensional visual perception of the target object in the non-contact area; and determining contact three-dimensional deformation perception of the target object in the contact area through a three-dimensional reconstruction model based on ray tracing and based on the two-dimensional coordinates of the binocular feature points. According to the invention, the perception capability of the visual tactile sensor on the object state and the interaction degree can be improved.

The invention particularly relates to an intelligent garbage classifying and sorting system based on machine vision, and relates to the technical field of intelligent garbage classification and industrial automation. A digital twin construction module; an adaptive control and optimization module; and a predictive maintenance and health management module. According to the method, vision-touch-acoustics multi-mode perception is deeply fused, so that the identification limitation of a traditional single vision sorting system is solved; after the visual identification and the tactile feature are subjected to weighted fusion, the material classification accuracy is improved, and the problems of misjudgment and missed judgment under the scenes of oil contamination, light reflection, form deformation and the like are effectively avoided.

The invention relates to a food sensory evaluation method and device based on an attention mechanism, equipment and a medium. The method comprises the following steps: acquiring sensory information such as vision, smell, taste and touch of a food sample through multi-modal data acquisition equipment, and performing standardization processing to obtain a sensory feature matrix; generating a sensory feature vector set by using a pre-established multi-modal feature extraction model; modeling each sensory dimension relationship by means of an attention layer in a deep learning network, and calculating to obtain an attention weight of each sensory feature; and performing weighted combination on the sensory feature vector set based on the weight to generate a comprehensive sensory feature basic representation. According to the method, the association among the multi-modal sensory characteristics is effectively modeled through an attention mechanism, and a scientific and efficient solution is provided for food sensory evaluation.

The invention discloses a visual tactile sensor fused with an optical waveguide and a preparation method thereof, and belongs to the technical field of intelligent tactile sensing. The visual tactile sensor comprises a fixed shell, and the top end of the fixed shell is open; the composite tactile sensing layer is arranged at the opening position of the fixed shell, and the composite tactile sensing layer is sequentially composed of a shading layer, a reflecting layer, a red-green spaced fluorescent layer, an elastic deformation layer and a supporting layer from top to bottom; and the optical lighting and sensing component comprises a blue LED light bar, an array light intensity sensor and a camera, the blue LED light bar and the array light intensity sensor are arranged on the two corresponding sides of the composite touch sensing layer respectively, and the camera is arranged at the bottom in the fixed shell. Through fusion of rapid and stable light intensity detection characteristics of optical waveguide sensing and pressure distribution detection capability of the visual tactile sensor, high-resolution space and texture identification and high-sensitivity mechanical response detection are simultaneously realized in a single sensor architecture.

The invention discloses a visual tactile sensor, an electronic skin and a robot, relates to the field of sensors, and is used for improving the comprehensiveness and sensitivity of stress detection. The sensor comprises a magnetic film covering the top surface and the side surface, and the magnetic film is prepared by mixing magnetic powder and silicone rubber; the support supports the magnetic thin film, a magnetic display card and a light source are arranged in the support, the visual touch collector collects magnetic induction data and development data of stress of the magnetic thin film, and then the multi-mode pressure detection model analyzes the stress position and / or the stress size. The stress detection sensitivity is high, comprehensive detection of the top face and the side faces can be achieved, and the detection precision is high.

The invention provides an industrial robot self-adaptive grabbing method based on multi-modal sensing, which comprises the following steps of: 1, acquiring multi-modal data of an object and an environment through a multi-modal sensing module; the multi-mode sensing module comprises at least two sensing modules of a visual sensing unit, a touch sensing unit, a force sensing unit and an auditory sensing unit; 2, preprocessing and feature extraction are carried out on the multi-modal data, feature information of different modals is fused through a multi-modal fusion algorithm, and comprehensive object description information is generated; 3, generating an optimal grabbing strategy based on the comprehensive object description information; wherein the grabbing strategy comprises the position and posture of a grabbing point, a grabbing path and grabbing force; and fourthly, the industrial robot executes the optimal grabbing strategy, the first step to the third step are repeated, and the grabbing strategy is adjusted in a self-adaptive mode till grabbing is completed. Stable and efficient grabbing can be achieved.

An embodiment of the utility model discloses a visual tactile sensor, which relates to the technical field of sensors and comprises a base, a flexible plate, an image acquisition device and a plane mirror. The flexible plate is arranged on the base, the image acquisition device is arranged on the base, the image acquisition device is located on one side of the flexible plate, and the plane mirror is obliquely arranged on the upper side of the flexible plate. The layout between the flexible plate and the image acquisition device can be changed through the plane mirror, the size of the visual tactile sensor in the height direction is greatly reduced, and the size of the visual tactile sensor is reduced. According to the scheme, through the innovative structural design of reflection imaging of the plane mirror, on the premise that the tactile perception precision is ensured, the problem that the size of a traditional visual tactile sensor is too large due to vertical imaging layout is solved, meanwhile, multiple technical requirements of environmental adaptability, cost controllability and the like are taken into consideration, and the visual tactile sensor is suitable for popularization and application. And better hardware support is provided for scenes such as refined operation and man-machine interaction of the robot.

The invention discloses a vision-based high-density three-dimensional force sensing array system and a three-dimensional force distribution identification method thereof. The system comprises a first force sensing layer, a second force sensing layer, a support force bearing layer, an LED lighting unit, a light-transmitting reference plate, a camera and a supporting shell box body. The second force sensing layer is a high-density array formed by independently distributed cylinder and hemisphere elastic structure units; and the light-transmitting reference plate, the LED lighting unit and the camera form an optical imaging assembly. When an external object is in contact with the surface of the array, each structural unit deforms independently, a camera captures a deformation image, features are extracted through an image processing algorithm, and three-dimensional force distribution of a contact area is solved according to a pre-calibrated image-mechanics mapping model. The system can reconstruct the three-dimensional contour form of a contact object in real time and perceive the force distribution characteristics of the contact object, and a high-precision and low-cost force touch perception solution is provided for the fields of robot touch recognition, human-computer interaction, safety monitoring and the like.

The invention discloses an in-cabin grabbing target characterization method based on visual touch fusion. The in-cabin grabbing target characterization method comprises the steps that an in-cabin grabbing target three-dimensional model reconstruction data set based on visual touch fusion information is acquired; training an initial three-dimensional model reconstruction network by using the data set to obtain an in-cabin target grabbing three-dimensional model reconstruction network based on visual touch fusion information; and the in-cabin target single-visual-angle visual point cloud and the visual field blind area touch point cloud of which the three-dimensional model needs to be reconstructed are input into the in-cabin grabbing target three-dimensional model reconstruction network based on the visual touch fusion information, and the in-cabin grabbing target three-dimensional model is obtained. According to the method, the in-cabin grabbing target three-dimensional model is reconstructed with high precision in a visual touch fusion mode, the defects of a robot vision-based grabbing target three-dimensional model reconstruction method are overcome, the precision of robot grabbing target reconstruction is improved, and then the reliability of a robot in-cabin grabbing task in a space station is guaranteed.