189 results about "Robot perception" patented technology

Provided is a method for token-based representation and learning of robotic perception data based on a graph neural network, comprising: obtaining a plurality of types of perception data of a robot; performing token-based representation according to types of the plurality of types of perception data; constructing an initial feature graph based on the plurality of types of perception data after the token-based representation; learning a compact representation of the initial feature graph based on an autoencoder and reconstructing a graph structure; after the autoencoder completes learning of the graph structure, fixing the graph structure; and converting the plurality of types of perception data into node feature vectors, constructing a feature graph based on the graph structure, and performing numerical encoding on each of the node feature vectors by utilizing the graph neural network to obtain a representation of high-dimensional feature vectors of the plurality of types of perception data.

The invention discloses an autonomous charging dynamic control system for an inspection robot, and relates to the technical field of inspection robot management, and the system comprises a condition triggering module, a robot sensing module, a task decision module, a dynamic charging control module, a safety monitoring module and a log recording module. When a new inspection task packet is issued to a waiting or charging robot, a condition triggering module starts judgment, and a robot sensing module collects real-time electric quantity, task attributes and a charging pile state and encapsulates data; the task decision module calculates a task emergency degree and constructs a priority, and compares a charging basic priority to trigger a temporary or safe charging strategy; the dynamic charging control module dynamically adjusts charging parameters or calls safety parameters according to the dynamic charging parameters; the safety monitoring module monitors and intervenes in the charging process in real time, the log recording module records key operation and abnormity, and the system can dynamically adapt to task requirements, improves response efficiency and safety and is suitable for complex inspection scenes.

The invention relates to a robot navigation method and system based on a visual language large model and experience memory. The method comprises the following steps: firstly, generating a semantic map with labels by utilizing an SLAM technology through multi-modal environment data sensed by a robot; receiving an instruction, and analyzing the instruction through a large language model to obtain a structured task representation; constructing an experience library based on historical data, and generating an experience package in combination with task representation; planning a path based on a semantic map, a structured representation and an experience package, obtaining an alternative path set, evaluating confidence, determining whether to trigger a clarification action, and outputting an optimal path; and finally, robot navigation is controlled according to the optimal path, and the experience library is updated. Compared with the prior art, the method has the advantages of improving the accuracy and interpretability of autonomous navigation of the robot, improving the navigation success rate of the robot in a complex environment and the like.

The invention relates to the field of robot perception and intelligent man-machine interaction, an equipment platform and a perception module, a system is provided with an Intel RealSense D435i depth camera which is used for acquiring an RGB-D image sequence containing depth information in real time, and meanwhile, an advanced YOLOv8-Pose deep learning model is embedded, so that the depth information of the RGB-D image sequence is acquired in real time. High-precision detection, natural interaction, a gesture guidance mechanism, direction inference and target candidate aggregation, interactive target screening and judgment, a robustness enhancement strategy, deep deletion and noise robustness and an incomplete target completion mechanism of human upper body skeleton key points in a front scene are realized; the man-machine collaborative target object identification and positioning method based on depth vision, attitude estimation and semantic understanding is mainly realized. The technology can be suitable for service robots, intelligent assistant robots and other practical application scenes with high requirements for autonomous perception and natural interaction, and man-machine co-fusion and intelligent environment construction are further promoted.

The invention discloses a non-modal segmentation method and system based on RGB-D and a 6D object grabbing method and system, and relates to the technical field of computer vision and robot perception. According to the method, RGB and depth modal features are extracted through a double-branch backbone, and cross-channel image features are output; channel alignment and gating selection are carried out through a mutual attention fusion module; and finally, generating four branches of occlusion object segmentation, visible area segmentation, complete non-modal mask complementation and occlusion relation classification through a visible part guide non-modal multi-branch decoding module, and generating a complete target mask with consistent semantics. Three-dimensional geometric information of a target object is extracted based on a complete non-modal mask, a 6D grabbing posture is output through a posture estimation network and converted into a mechanical arm motion instruction to complete grabbing, the problems of serious shielding, sensor noise and cross-modal dislocation can be effectively solved, the non-modal instance segmentation precision is improved, and the grabbing efficiency is improved. The method can be widely applied to high-precision robot operation scenes such as autonomous storage and industrial operation.

The invention discloses a multi-camera radar-inertia-visual odometer method fusing a sonar image, and belongs to the technical field of robot perception and navigation. According to the method, environment perception is realized through cooperative work of multiple sensors; in the air, a multi-camera system and a conventional laser radar are taken as main modes; in water, an active imaging sonar and an underwater laser radar are used as main modes. The system detects the medium of the aircraft according to the resistivity, dynamically switches the main mode and fuses the auxiliary mode information. Through a tight coupling optimization framework, observation data of vision, sonar, laser radar and an inertial measurement unit are uniformly modeled as residual constraints, a nonlinear least square problem is constructed and solved, and efficient fusion of multi-source information is realized. The method has robustness in low-texture, low-illumination and underwater turbid environments, and the positioning precision and stability in a cross-medium scene are remarkably improved.

The invention discloses an immersive interactive system and method integrating pet robot perception and AR rendering, and the system carries out the visual-inertial combined mapping and anchor point generation, the coding and network transmission of visual frames and spatial data, the synchronous decoding and coordinate system alignment of a terminal side, virtual object anchoring, and spatial mapping and AR rendering through a robot side. User intention analysis, control issuing and robot control, and low-delay synchronization, timeline alignment and closed-loop consistent maintenance are carried out; according to the method, the first visual angle image collected by the front camera of the pet robot can be utilized, the augmented reality technology is combined, and the user is substituted into the visual angle of the robot, so that the user can'personally feel 'the observation and action process of the pet robot, and the immersion and immediacy of human-pet interaction are remarkably enhanced; and human-machine-virtual object ternary space interaction is realized.

The invention discloses a three-dimensional point cloud processing method, system, terminal and medium for anti-wrinkle interference clothes for a massage robot, and relates to the technical field of robot perception and point cloud processing, and the method comprises the steps: obtaining the three-dimensional point cloud data of the surface of the clothes based on a depth camera, and carrying out the cutting processing, and obtaining the point cloud data after cutting; preprocessing the cut point cloud data to obtain target point cloud data, and performing cascade filtering processing to obtain optimized point cloud data; and based on the optimized point cloud data, calculating a normal vector of each point, and determining to carry out massage path planning so as to control the massage robot to execute a massage action based on the planned massage path. The method can remarkably reduce the influence of clothes wrinkles on massage direction recognition of the massage robot, and can adapt to interference of clothes made of different materials. In addition, according to the cascade filtering processing method, the calculation burden is greatly reduced while the effectiveness of normal vector estimation is ensured.

The invention discloses an SLAM-oriented multi-sensor space-time synchronization and online calibration method, and relates to the technical field of computer vision and robot perception. Comprising the steps that S100, initialization is carried out, and an initial pose and an external parameter transformation matrix initial value are obtained through coarse alignment; s200, data acquisition and preprocessing: acquiring IMU data, LiDAR data and Odom data in real time, and performing data filtering, distortion removal, time sequence correction and point cloud downsampling processing on the acquired data; and S300, performing time synchronization processing, estimating the time offset of the LiDAR and Odom data in real time, and performing unified correction to generate a unified timestamp. According to the method, an online time offset estimation and compensation mechanism with an IMU as a main clock is established, accurate synchronization of sensor data of different frequencies is achieved, and the problem of data alignment caused by sampling frequency difference and clock drift is effectively solved; and meanwhile, tight coupling fusion is realized by using an IMU pre-integration factor, a LiDAR residual factor and an Odom constraint factor, so that the space-time consistency of multi-sensor data is remarkably improved.

The invention relates to the technical field of robot perception and data generation technologies, in particular to a first-view-angle drilling method and device based on memory enhancement and a storage medium, and the method comprises the steps: extracting a plurality of target frames from a historical video stream stored in a space intelligent machine, obtaining a plurality of memory elements based on the plurality of target frames, performing three-dimensional reconstruction on each target frame, constructing a target world model, planning a first visual angle track of the target robot in the target world model, performing imaging simulation on the target robot, performing real-time rendering on a first visual angle video of the target robot, and performing real-time rendering on a second visual angle video of the target robot based on the same time axis and action script as the first visual angle video. A third-person video corresponding to the space intelligent machine is generated, and a drilling result is output; according to the method, the long-term memory data of the space intelligent machine is ingeniously used, high-quality drilling data can be quickly generated, the reliability of the first-view video is improved, and a reliable basis is provided for training and testing of a robot algorithm.

The invention belongs to the technical field of robot perception and decision making, and discloses a multi-modal fused intelligent target perception method and system, and the method comprises the steps: obtaining multi-modal data in a distribution network operation scene, carrying out the preprocessing, fusing the knowledge of the distribution network operation field, and generating knowledge-enhanced multi-modal features; carrying out cross-modal alignment processing on the knowledge-enhanced multi-modal features, carrying out graph structure-based joint semantic and space alignment on isomorphic modals, and carrying out feature projection-based binding alignment on heterogeneous modals to obtain consistent aligned multi-modal features in a shared semantic space; based on a dynamic adaptive strategy, screening and fusing the aligned multi-modal features to generate unified fusion features; and inputting the fusion features into a target perception model, and outputting an image segmentation result and a point cloud semantic segmentation result of the distribution network operation target. According to the method, high-precision segmentation and positioning of the target in a complex distribution network scene are realized, and safe and efficient operation of the robot is effectively supported.

The invention provides a robot environment sensing method and device based on semantic completion, which are used for improving the accurate sensing ability of a robot with a body to obstacles in a complex environment. The method comprises the following steps: S1, preprocessing collected original three-dimensional environment data to obtain standardized input data; s2, inputting the standardized input data into a semantic completion model for reasoning, and generating a three-dimensional semantic voxel graph; s3, extracting a voxel region representing a barrier category according to the semantic tag, and fusing the voxel region with historical perception information to generate barrier region representation; and S4, mapping the obstacle region representation to a robot sensing structure, and constructing an occupation graph or an obstacle code for supporting path planning and obstacle avoidance control. According to the method, multi-scale sparse voxel representation and a semantic completion strategy are combined, the continuity, robustness and scene adaptability of obstacle recognition are remarkably improved, and the method is suitable for various application scenes such as navigation of a robot with a body, obstacle avoidance of a service robot and the like.

The invention discloses a humanoid robot self-supervised state estimation method based on a variational auto-encoder, and relates to the technical field of robot perception and control, and the method constructs a state estimation model of an encoder-decoder architecture, and carries out training based on a self-supervised learning method. Historical observation information of a robot is input, a state coding module extracts potential feature representation in an observation sequence, supervised training is carried out in combination with privilege information, and a coded submerged space vector is divided into an explicit variable and an implicit variable. The explicit variable is used for calculating a loss function with privilege information, and the implicit variable is used as the input of the state decoding module to reconstruct original observation data, so that the accurate prediction of the floating base speed, position, posture and foot end contact state of the robot is realized. In the reasoning stage, the model carries out state estimation by using real-time observation information and provides state feedback for a downstream motion control module, so that closed-loop control is realized.

The embodiment of the invention provides a robot and an operation method and device thereof, a storage medium and a program product. The method comprises the steps that natural language instruction information and multiple robot sensing data are received; performing spatial feature extraction processing based on at least part of the robot sensing data to obtain spatial feature information of the corresponding robot sensing data; performing time sequence feature fusion based on the spatial feature information of the perception data of the at least two robots to generate space-time semantic implicit representation information; performing cross-modal interaction based on the natural language instruction information and the space-time semantic implicit representation information to generate space target representation information; and path planning and / or action control are / is carried out based on the spatial target representation information, so that the robot is controlled to execute operation corresponding to the natural language instruction information. Therefore, high information integrity can be kept under the condition of low computing power, serious information loss caused by white list detection is reduced, and the universality of robot operation control is remarkably improved.

The invention provides a microgravity environment flying robot sensing and scene understanding method and system, and belongs to the field of robot intelligent sensing and scene understanding. The problems that in an indoor working scene under the microgravity environment, facility layout is crowded, space illumination is complex, and floating objects are blocked, so that image feature extraction is difficult, the sight line of a sensor is blocked, and the three-dimensional environment modeling precision and the target recognition accuracy are affected are solved. A lightweight convolutional neural network structure is adopted, so that the calculation burden is reduced; rGB images and data of a laser ranging sensor are combined in the aspect of multi-modal data fusion, a multi-modal information fusion module is added in target detection and pose estimation, and the method is suitable for recognition and positioning of a target object needed by high-precision operation; in the aspect of task adaptive semantic segmentation, aiming at a specific task scene in a microgravity environment, performing network training based on a specific data set; and through a transfer learning mode, the model has higher adaptability.

The invention provides an acquisition-training-evaluation integrated method and system for a humanoid robot. The method comprises the following steps: acquiring operation data of an operator by using master-slave control teleoperation equipment, and optimizing the operation data; the mechanical arm is controlled to respond, and data collection is conducted in the movement process of the mechanical arm; processing and storing the acquired data; applying the processed required information to a reasoning training process of the robot to form a reasoning model; a reasoning training result is deployed to the robot, so that the robot can start to execute a test task; real data of the robot in the execution process are collected, and the motion effect of robot training is evaluated and judged; a robot training reasoning model which does not conform to an evaluation standard is optimized, a data acquisition process and a training reasoning process are purposefully optimized from an evaluation result, and a'training-adjusting-correcting 'whole-process data active discovery and utilization mechanism is formed, so that the construction of a high-quality and multi-modal universal intelligent data set with a body is accelerated, and the accuracy and the reliability of the system are improved. Dependence on real data acquisition is reduced, and robot perception, understanding, reasoning and general generalization capabilities are enhanced. The problems that in the robot training process, training data and evaluation indexes are separated and separated, so that an integrated closed loop cannot be achieved in the collection-training-evaluation process, the evaluation result cannot act on the robot training improvement process, and the virtuous circle that the evaluation result counteracts on robot training cannot be achieved are solved.

The invention provides a sensor system of a logistics robot and a data processing method of the sensor system. The method belongs to the cross technical field of intelligent logistics, robot sensing and artificial intelligence signal processing. The method comprises the steps that multi-source data of the logistics robot in the moving process are collected in real time through a multi-source sensor, and a multi-source sensor original data set is generated; performing data fusion preprocessing on the original data set of the multi-source sensor to generate a fused synchronous data stream; based on the synchronous data flow, through a gravity center inversion algorithm, in combination with the cargo three-dimensional model and the pressure distribution characteristics, performing inversion to obtain cargo dynamic gravity center position data, synchronously recording corresponding timestamp information, and generating cargo dynamic gravity center position-time sequence data; through fusion of multi-source sensor data, the dynamic center-of-gravity position of the cargo can be accurately inverted in real time, and the sensing ability of the robot to the center-of-gravity change is greatly improved.

The invention discloses a robot data analysis decision method and system based on a neural network, and relates to the technical field of robot production, and the method comprises the steps: obtaining the state data of a local robot; acquiring state data of other local robots, and selecting the robot with the highest comprehensive score as an auxiliary robot; collecting local data and auxiliary data; performing modal noise reduction, feature extraction and fusion on the local data and the auxiliary data by using the neural network preprocessing layer to obtain an environment representation vector; determining a bottom risk by using a Bayesian network; analyzing a risk evolution trend by using a knowledge graph; and inputting the underlying risk and the risk evolution trend into the reinforcement learning model, and outputting an optimal decision. According to the method, the deficiency of single-robot sensing is made up through multi-robot cooperative data acquisition, noise reduction and deep feature extraction of interfered data are realized by means of neural network preprocessing, and the environment sensing accuracy, risk assessment precision and decision reliability of the robot in the interference environment are improved.

The invention relates to the field of video prediction, in particular to a video sequence prediction method and system based on object segmentation guidance, and the prediction method comprises the steps: receiving a historical video frame sequence, and carrying out the video object segmentation and tracking processing of the historical video frame sequence, generating structural representation information of each object in each frame and distributing a continuous and unique tracking ID for each object; encoding the structural representation information and the tracking ID into an object-level structured feature sequence; inputting the object-level structured feature sequence into a conditional diffusion model as a guide condition, and generating a potential spatial intermediate feature representing a future video frame through an iterative denoising process; the potential spatial intermediate features are decoded into a pixel-level sequence of future video frames. According to the method, the problems of the existing video prediction technology in the aspects of object consistency, physical authenticity, error accumulation and the like are solved, the application potential in a complex scene is expanded, and support is provided for video prediction in the fields of automatic driving, robot perception, content creation and the like.

The invention discloses a cross-scene self-adaption oriented on-line optimization method for perception parameters of a body-equipped robot, and relates to the technical field of robot perception control. The method comprises the steps of obtaining a historical scene type and environment data of the robot with the body, identifying a current scene type, and if the scene types are the same, collecting voice data and calling a special environment analysis model to determine current environment data; and if not, directly matching standard sensing parameters, calculating an environment change degree based on historical and current environment data, and constructing a sensing parameter optimization strategy, including determining an optimization depth and a parameter adjustment space, and finally performing iterative optimization by taking a minimum sensing deviation as a target, evaluating parameter performance by using an integrated prediction plug-in, and outputting an optimal adaptive parameter. According to the method, adaptive switching of the sensing parameters among different scenes and dynamic optimization of the sensing parameters in the same scene are realized, and the sensing robustness and task execution reliability of the robot in a complex environment are improved.

The invention discloses a robot perception enhancement method based on a vision-touch fusion codec, and belongs to the technical field of robot perception and artificial intelligence. Aiming at the problems that a single visual perception scheme cannot directly observe contact information, the robustness is poor under environment change and multi-modal fusion is difficult, the method is characterized by comprising the following steps: synchronously collecting visual and tactile data, and carrying out time sequence alignment and preprocessing; respectively converting the image blocks and the tactile signals into feature sequences and adding position codes; feature interaction is carried out through cross-modal attention in a unified encoder, and training is carried out by adopting a joint random masking and reconstruction mechanism to obtain shared representation; the shared characterization is used for a downstream robotic operation task. According to the method, the observability of the contact behavior is realized, the perception robustness under shielding and illumination changes is improved, and more stable perception input is provided for a smart operation task.

The invention relates to the technical field of three-dimensional vision and robot perception, in particular to a belt conveying line multi-camera point cloud real-time splicing method and system based on GPU acceleration. The point cloud real-time splicing method comprises the following steps of S1, collecting point cloud data of objects on a belt conveying line through a plurality of depth cameras; s2, performing spatial range filtering of the point clouds in parallel by using a GPU, and removing invalid points; s3, point cloud coordinate transformation is executed in parallel by using a GPU, and the point clouds of all the cameras are uniformly converted into the same world coordinate system; s4, performing triple filtering processing on the transformed point cloud to realize noise suppression, downsampling and density equalization; s5, fusing the filtered point clouds to form a complete scene point cloud; and S6, publishing the fused point cloud data through an ROS mechanism. GPU parallel computing is utilized, multi-camera point cloud high-speed real-time splicing is achieved, denoising equalization is effectively achieved through triple filtering, the processing efficiency and precision are remarkably improved, and the method is suitable for long-distance industrial monitoring.

The invention discloses a robot sensing method based on field landform semantics and topographic geometric features, and the method comprises the steps: firstly building and training a semantic segmentation network, and obtaining the semantic information of each landform in an environment; and laser point cloud information is obtained. Secondly, fusing landform semantics with laser radar point cloud information to obtain semantic point cloud, rasterizing the semantic point cloud, determining semantic features of grids according to semantic information in the semantic point cloud after rasterizing, and constructing a semantic feature cost map and a terrain geometric cost map; and finally, based on the terrain geometric cost map, tracking the change of the dynamic obstacle in time by using a ray tracing method, eliminating false height caused by the dynamic obstacle, fusing the semantic feature cost map and the terrain geometric cost map by using an environment traffic cost model, and quantifying the traffic cost of each position in the field complex environment. According to the invention, the robot can better understand the field environment, and the perception is more accurate.

The invention discloses a self-supervised monocular depth estimation method based on an enhanced multi-scale pose network, and relates to the field of computer vision. The problems that in an existing self-supervision monocular depth estimation method, the pose network structure is simple, the time sequence modeling capacity is insufficient, and geometric constraints are missing are solved. According to the method, a self-supervision joint training framework composed of a depth estimation sub-network and an enhanced pose estimation sub-network is constructed; wherein the pose estimation sub-network extracts multi-scale spatial structure features through a layer-by-layer feature fusion encoder, and adopts a context fusion decoder based on time sequence attention to model a motion dependency relationship between continuous frames; meanwhile, a self-supervised pose consistency loss function is introduced, geometric continuity of a camera track is enhanced through forward and reverse transformation consistency constraint and closed-loop geometric constraint, and collaborative optimization of depth prediction and pose estimation is realized. The method is also suitable for the application fields of automatic driving, robot perception, augmented reality and the like.

The invention discloses a robot sensing and autonomous navigation method and system for a three-dimensional topological map, and the method comprises the steps: obtaining the three-dimensional point cloud data of a surrounding environment, the acceleration and angular velocity data of a robot, and predicting the pose of the robot; identifying a free space and an obstacle area, determining an occupation state of each point cloud voxel, and generating a three-dimensional occupation map; dividing a free space into a plurality of convex clusters through neighborhood search based on a DBSCAN algorithm, and constructing a three-dimensional topological graph; identifying a vertical channel, and setting corresponding edges to be connected with portal nodes to form a global three-dimensional topological graph; and according to the global three-dimensional topological graph, determining a global path and a local obstacle avoidance path, and combining the global path and the local obstacle avoidance path to obtain a global optimization path. The technical bottlenecks that a two-dimensional map lacks a vertical dimension and a traditional voxel method is insufficient in real-time performance are broken through on the whole, and an efficient, safe and coherent solution is provided for autonomous navigation of a robot in a complex three-dimensional environment.

The invention belongs to the technical field of robot sensing and control, and particularly relates to a full-area sensing dexterous hand based on multi-mode sensing fusion. A multi-dimensional force sensing assembly is arranged at the wrist of the palm structure; the palm center visual touch sensor is arranged in the palm center of the palm structure; the number of the fingertip vision tactile sensors is matched with that of the fingers of the palm structure, and the fingertip vision tactile sensors are arranged at the finger pulps of the fingers of the palm structure; the magnetic flexible touch skin wraps the outer side of the palm structure; the control assembly is arranged in the palm structure, and the control assembly is electrically connected with the palm center visual tactile sensor, the fingertip visual tactile sensor and the magnetic flexible tactile skin; the control assembly is used for controlling finger movement and receiving feedback data of the palm center visual tactile sensor, the fingertip visual tactile sensor and the magnetic flexible tactile skin.

The invention relates to the field of computer vision and artificial intelligence, in particular to an image event multi-mode semantic segmentation method, device and equipment, which can be applied to scenes such as automatic driving, robot perception and intelligent traffic. Time slices are divided through a fixed time window, event information is accumulated, and asynchronous event streams are converted into T * H * W voxel tensors; a selective state scanning mechanism of a Mama framework is used for replacing a self-attention mechanism of a traditional Transform, the calculation complexity is reduced while modeling global feature dependence is achieved, and the problems of video memory and delay in a Transform high-resolution scene are solved; besides, image textures and event edges are aligned through cross-space interaction, an event dynamic time sequence is captured through cross-time interaction, and modal inherent characteristics are retained through residual connection, so that feature degradation caused by excessive fusion is effectively avoided; finally, the image segmentation precision is effectively improved, and the processing efficiency and the model robustness are improved at the same time.

A robot control apparatus can include a memory that stores computer-executable instructions and at least one processor that executes the instructions by accessing the memory. The at least one processor can apply event information about activity of a user, which can be identified from user activity data perceived from a robot, and context information about time and space in which the activity occurs, to a knowledge graph formed by a relation between an event instance regarding the event information and a context instance regarding the context information, obtain user intent data regarding intent of the activity by applying the event instance among instances included in the knowledge graph to a rule creation model for creating information about the intent of the activity, and control the robot such that the robot performs a target task related to expected activity, which can follow the activity, based on the user intent data.

The invention discloses a real-time AR data superposition and interaction system for an industrial inspection robot, and relates to the technical field of industrial man-machine interaction, and the system comprises the steps: collecting multi-dimensional inspection context data obtained by a sensor carried by the inspection robot, and building a unified space reference system; updating of the AR terminal in the industrial scene is completed; and the corresponding control command is issued to the inspection robot for execution. According to the invention, high-precision, low-delay and semantic-consistent closed-loop coordination between data sensed by the inspection robot and AR terminal display is realized, and the information presentation intuition, the man-machine coordination fluency and the field decision accuracy in the industrial inspection process are improved; the defects of interaction lag, virtual-real dislocation, operation interruption and the like existing in a traditional system are effectively overcome, and the urgent requirement for efficient and immersive man-machine interaction equipment in a high-risk complex industrial scene is met.

The invention discloses a three-dimensional occupancy perception method and system suitable for multiple tasks, and belongs to the field of robot perception, and the method comprises the steps: carrying out the manual labeling of point cloud data and image data, obtaining a point-by-point label and a pixel-by-pixel label, carrying out the data verification, converting the point cloud data with the labels into a 3D semantic occupancy label of a scene through 3D semantic reconstruction, and carrying out the recognition of the 3D semantic occupancy label. Forming sample data; training a three-dimensional occupancy perception model by using the sample data and performing three-dimensional occupancy perception prediction; sequentially performing feature enhancement extraction, feature dimension conversion, feature time sequence fusion and feature compensation correction on the image data based on a feature extraction module to obtain multi-scale voxel features; the feature information of the multi-scale voxel features is independently adjusted through a semantic segmentation head and a target detection head, then a 3D semantic occupancy result and an instance detection result are output, the two detection results are converted into an instance segmentation result and a target tracking result through a post-processing module, and low-cost and high-efficiency multi-task three-dimensional occupancy perception is achieved.