Operation environment and state sensing method and device based on robot and electronic equipment

Abstract

The invention discloses a robot-based working environment and state sensing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring multi-source heterogeneous data of a target robot; determining the current state of the target robot according to the multi-source heterogeneous data; constructing a working environment map according to the position data of the target robot and the multi-source heterogeneous data; and planning an obstacle avoidance path of the target robot according to the working environment map. By detecting the current state of the target robot, the stability of the target robot can be ensured; by constructing the working environment map, the working environment map of the target robot is automatically updated, so that changes in the operating environment of the target robot can be found in time; the obstacle avoidance path of the target robot is planned based on the working environment map, the driving path of the target robot can be corrected in real time, and the stability and reliability of the target robot are improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence processing technology, and specifically to a robot-based working environment and state perception method, device, electronic device and storage medium. Background Art

[0002] When flooding occurs in confined spaces such as underground parking lots, subway stations, and culverts, emergency drainage often requires equipment such as drainage pumps. However, these confined spaces are complex environments: dimly lit, narrow, and filled with various fixed obstacles (pillars, equipment) and floating debris. Furthermore, they can pose risks such as electric shock and collapse, making manual entry extremely dangerous. Traditional drainage methods rely on manual labor carrying pumping equipment deep into the flooded areas, which is not only inefficient but also fails to ensure personnel safety.

[0003] In recent years, some remote-controlled or autonomous drainage robots have emerged. However, due to the lack of a complete environmental perception system, it is difficult for them to accurately locate themselves, identify environmental obstacles, and detect their own working status in complex and confined spaces. These deficiencies in existing technologies have led to drainage robots still being highly dependent on manual visual inspection and experience-based operations in actual applications, making it difficult to ensure operational safety and efficiency in harsh environments.

[0004] Therefore, the existing technology has the problem that it is difficult to ensure the safe and effective operation of the robot due to the low detection efficiency of the robot. Summary of the Invention

[0005] The embodiments of the present application provide a robot-based working environment and state perception method, device, electronic device and storage medium, which aim to realize automatic generation of text based on pictures.

[0006] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0007] Acquire multi-source heterogeneous data of the target robot;

[0008] Determining a current state of the target robot based on the multi-source heterogeneous data;

[0009] Constructing an operating environment map based on the position data of the target robot and the multi-source heterogeneous data;

[0010] Plan an obstacle avoidance path for the target robot according to the operating environment map.

[0011] Optionally, the target robot includes multiple data acquisition modules, and the multiple data acquisition modules include multiple types of laser radar, infrared camera, inertial measurement unit, ultrasonic sensor, posture sensor, temperature sensor, oil pressure sensor, oil level sensor and electromagnetic flowmeter; the multi-source heterogeneous data includes multiple types of point cloud data, laser odometry, image frame, inertial data, ultrasonic sensor data, posture information, temperature, oil pressure, oil level and flow, and the data collected by the multiple data acquisition modules respectively correspond to the data in the multi-source heterogeneous data; the obtaining of the multi-source heterogeneous data of the target robot includes:

[0012] Collecting initial multi-source heterogeneous data of the target robot in the working environment;

[0013] The initial multi-source heterogeneous data is synchronously fused to obtain the multi-source heterogeneous data.

[0014] Optionally, performing synchronous fusion processing on the initial multi-source heterogeneous data to obtain the multi-source heterogeneous data includes:

[0015] Aligning the initial multi-source heterogeneous data based on a unified timestamp to obtain initial multi-source heterogeneous data with consistent timeliness;

[0016] fusing the initial point cloud data, the initial laser odometry data, and the initial inertial data through an extended Kalman filter to obtain the point cloud data, the laser odometry data, and the inertial data;

[0017] Smoothing the initial temperature, initial oil pressure, initial oil level, and initial flow rate by using mean filtering to obtain the temperature, the oil pressure, the oil level, and the flow rate;

[0018] The outlier noise in the initial ultrasonic sensor data and the initial posture information is removed by median filtering to obtain the ultrasonic sensor data and the posture information.

[0019] Optionally, the multi-source heterogeneous data includes oil pressure, oil temperature, oil temperature duration, and oil level; and determining the current state of the target robot based on the multi-source heterogeneous data includes:

[0020] determining whether the oil pressure state of the target robot is abnormal based on a relationship between the oil pressure and a preset oil pressure safety threshold;

[0021] determining whether the oil temperature state of the target robot is abnormal based on a relationship between the oil temperature and a preset oil temperature safety threshold, and determining that the safety state of the target robot is abnormal when the oil temperature duration of the abnormal oil temperature is greater than a preset safety time threshold;

[0022] According to the relationship between the oil level and a preset oil level threshold, it is determined whether the oil level state of the target robot is abnormal.

[0023] Optionally, the multi-source heterogeneous data includes point cloud data and image frames; the position data includes angular velocity and acceleration of the target robot; and constructing the operating environment map based on the position data of the target robot and the multi-source heterogeneous data includes:

[0024] Iteratively calculating the point cloud data to obtain the position and posture data of the target robot;

[0025] Determining key image features of the image frame according to a fast feature point extraction algorithm, and performing loop detection on the target robot based on the key image features to obtain loop closure data of the operating environment;

[0026] Performing global error correction on the loop closure data and the pose data using a graph optimization method to obtain a global operating environment map;

[0027] The global operating environment map is fused and compensated according to the angular velocity and the acceleration to obtain the operating environment map.

[0028] Optionally, the multi-source heterogeneous data includes image frames and ranging data; and constructing a work environment map based on the position data of the target robot and the multi-source heterogeneous data includes:

[0029] Identifying obstacles in the image frame based on a well-trained target obstacle detection model;

[0030] Determining the distance and orientation between the obstacle and the target robot according to the ranging data;

[0031] Obstacle coordinates are calculated according to the position data, the distance, and the orientation, and the obstacle coordinates are marked on the initial working environment map to obtain a three-dimensional working environment map with obstacles.

[0032] Optionally, planning an obstacle avoidance path for the target robot according to the operating environment map includes:

[0033] Obtaining an initial path of the target robot;

[0034] Based on the initial path, determining whether there is an obstacle within a first distance threshold in the forward direction of the target robot;

[0035] If yes, controlling the target robot to perform emergency braking;

[0036] If not, calculating the number and area of ​​obstacles within a second distance threshold in the forward direction of the target robot based on the initial path;

[0037] When the number of obstacles is greater than a preset obstacle number threshold, and / or the area of ​​the obstacles is greater than a preset obstacle area threshold, replanning a collision-free path based on the operating environment map;

[0038] When the number of obstacles is not greater than the preset obstacle number threshold and the obstacle area is not greater than the preset obstacle area threshold, the obstacle avoidance decision of the target robot is triggered.

[0039] A robot-based operating environment and state perception device, comprising:

[0040] Data acquisition module, used to obtain multi-source heterogeneous data of the target robot;

[0041] a current state determination module, configured to determine the current state of the target robot based on the multi-source heterogeneous data;

[0042] An operating environment map construction module, configured to construct an operating environment map based on the position data of the target robot and the multi-source heterogeneous data;

[0043] The obstacle avoidance module is used to plan the obstacle avoidance path of the target robot according to the working environment map.

[0044] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

[0045] Acquire multi-source heterogeneous data of the target robot;

[0046] Determining a current state of the target robot based on the multi-source heterogeneous data;

[0047] Constructing an operating environment map based on the position data of the target robot and the multi-source heterogeneous data;

[0048] Plan an obstacle avoidance path for the target robot according to the operating environment map.

[0049] A computer-readable storage medium stores a computer program, which is loaded by a processor to execute the steps in the above-mentioned robot-based working environment and state perception method.

[0050] In an embodiment of the present application, by determining the current state of the target robot based on multi-source heterogeneous data, the operating status of the target robot itself is detected, which can ensure the stability of the target robot itself; by constructing an operating environment map based on the position data of the target robot and multi-source heterogeneous data, the operating environment map of the target robot is automatically updated according to the multi-source heterogeneous data and position data acquired in real time, so as to timely discover changes in the operating environment of the target robot; by planning the obstacle avoidance path of the target robot based on the operating environment map, the driving path of the target robot can be corrected in real time, thereby improving the stability and reliability of the target robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of a scenario of an embodiment of a robot-based working environment and state perception system provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of another embodiment of the robot-based working environment and state perception system provided in an embodiment of the present application;

[0054] Figure 3 A flowchart of an embodiment of a method for robot-based working environment and state perception provided in an embodiment of the present application;

[0055] Figure 4 A schematic structural diagram of an embodiment of a target robot provided in an embodiment of the present application;

[0056] Figure 5 A flowchart of an embodiment of a target robot's operating environment and state perception process according to an embodiment of the present application;

[0057] Figure 6 A schematic structural diagram of an embodiment of a robot-based working environment and state perception device provided in an embodiment of the present application;

[0058] Figure 7 It is a structural diagram of an embodiment of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0060] In the following description, the specific embodiments of the present application will be described with reference to steps and symbols performed by one or more computers, unless otherwise stated. Therefore, these steps and operations will be mentioned several times as being performed by a computer, and the computer execution referred to herein includes the operation of a computer processing unit by an electronic signal representing data in a structured form. This operation converts the data or maintains it at a location in the computer's memory system, which can be reconfigured or otherwise change the operation of the computer in a manner familiar to testers in the field. The data structure in which the data is maintained is a physical location in the memory, which has specific characteristics defined by the data format. However, the principles of the present application are described in the above text, which does not represent a limitation, and testers in the field will understand that the various steps and operations described below can also be implemented in hardware.

[0061] As used herein, the terms "module" or "unit" may be considered software objects executed on the computing system. The various components, modules, engines, and services described herein may be considered implementation objects on the computing system. While the devices and methods described herein are preferably implemented in software, they may also be implemented in hardware and are within the scope of protection of this application.

[0062] The embodiments of the present application provide a robot-based working environment and state perception method, device, electronic device and storage medium.

[0063] See also Figure 1 , Figure 1 Schematic diagram of a scenario of an embodiment of a robot-based working environment and state perception system provided in an embodiment of the present application. The robot-based working environment and state perception system may include a client 100 and a server 200. The client 100 and the server 200 are connected via a network. The server 200 is integrated with a robot-based working environment and state perception device. The server 200 may be a working platform server (i.e., a server loaded with a working platform). Figure 1 The client 100 can access the server 200. In the embodiment of the present application, the server 200 is mainly used to obtain multi-source heterogeneous data of the target robot; determine the current state of the target robot based on the multi-source heterogeneous data; construct an operating environment map based on the position data of the target robot and the multi-source heterogeneous data; and plan an obstacle avoidance path for the target robot based on the operating environment map.

[0064] In an embodiment of the present application, the server 200 may be an independent server or a server network or server cluster composed of servers. For example, the server 200 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. Wherein, the cloud server is composed of a large number of computers or network servers based on cloud computing. In an embodiment of the present application, communication between the server and the client can be achieved through any communication method, including but not limited to mobile communications based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), Worldwide Interoperability for Microwave Access (WiMAX), or computer network communications based on the TCP / IP Protocol Suite (TCP / IP), User Datagram Protocol (UDP), etc.

[0065] It is understood that the client 100 used in the embodiments of the present application can be understood as a client device, which includes both receiving and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such client devices may include: cellular or other communication devices with a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. The specific client 100 can be a desktop terminal or a mobile terminal, and can be a mobile phone, a tablet computer, a laptop computer, etc.

[0066] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer servers, or server network connection relationships, as shown in Figure 1 Only one server and two clients are shown. It can be understood that the robot-based working environment and state perception system can also include one or more other servers, or / and one or more clients connected to the server network, which is not limited here.

[0067] In some embodiments of the present application, the work platform may be an enterprise office platform, such as enterprise WeChat. Taking server 200 as an example, it may further include an enterprise office platform contact server, an enterprise office platform configuration management server, and a Web management server. Enterprise users or developers may access the Web management server by using a Web browsing terminal to configure the field configuration information on the enterprise office platform configuration management server, and set and store the enterprise user information of enterprise employees of the enterprise office platform on the enterprise office platform contact server.

[0068] In addition, if Figure 2 As shown, Figure 2 This is a scene diagram of another embodiment of the robot-based working environment and state perception system provided in the embodiment of the present application. The robot-based working environment and state perception system may also include a storage end 300 for storing data, such as a storage object database. The object database stores object data. The object data may include application templates (such as approval templates, punch-in templates and other application templates), file data (such as Word files, Excel files or PPT files and other formats of files), image data (such as jpg, png, bmp and other formats of images) and other data. Correspondingly, the object database can also be divided into multiple types of data, such as an application database, a file database or a picture database.

[0069] It should be noted that Figure 1-2 The scenario diagram of the robot-based working environment and state perception system shown is merely an example. The robot-based working environment and state perception system and the scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the robot-based working environment and state perception system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present application is also applicable to similar technical problems.

[0070] The following describes it in detail with reference to specific embodiments.

[0071] In this embodiment, the description is made from the perspective of a robot-based working environment and a state perception device, and the robot-based working environment and the state perception device can be specifically integrated into the server 200.

[0072] This application provides a robot-based working environment and state perception method, please refer to Figure 3 , Figure 3 The flowchart of an embodiment of a robot-based working environment and state perception method provided in an embodiment of the present application includes:

[0073] S301: Acquire multi-source heterogeneous data of the target robot;

[0074] In one embodiment, the target robot is an intelligent agent capable of perceiving its environment and autonomously taking actions to achieve a specific goal. The target robot includes multiple data acquisition modules, including a laser radar, an infrared camera, an inertial measurement unit, an ultrasonic sensor, an attitude sensor (IMU), a temperature sensor, an oil pressure sensor, an oil level sensor, and an electromagnetic flowmeter. The target robot can collect a variety of data from its environment.

[0075] Multi-source heterogeneous data refers to data collected by the target robot. Multi-source heterogeneous data includes point cloud data, image frames, inertial data, ultrasonic sensor data, posture information, temperature, oil pressure, oil level and flow. Obviously, multi-source heterogeneous data comes from a variety of different and independent acquisition devices, and has different forms, structures, accuracy, update frequency and semantic meanings.

[0076] S302: Determine the current state of the target robot based on multi-source heterogeneous data;

[0077] The current state of the target robot refers to the current operating state of the target robot, specifically the real-time health status of the robot's own core working system. Based on the current operating state of the target robot, its internal fault risks can be diagnosed, safety hazards can be warned, and the stable and reliable operation of the robot body can be ensured.

[0078] S303: Constructing an operating environment map based on the target robot's position data and multi-source heterogeneous data;

[0079] The position data of the target robot refers to data collected in real time and used to describe the positioning and motion status of the target robot itself in three-dimensional space. It can be collected in real time by the target robot itself or by external equipment, and there is no restriction here.

[0080] The operating environment map is a computable model that incorporates spatial structure, obstacle information, and semantic features, based on the target robot's current location data and combined with multi-source heterogeneous sensor data. Specifically, it can be the real-time environmental perception results of the target robot while operating in a confined space, such as an underground garage or culvert.

[0081] S304: Planning an obstacle avoidance path for the target robot according to the operating environment map.

[0082] The obstacle avoidance path refers to a moving route that is generated through planning based on the working environment map and enables the target robot to safely avoid various obstacles in the working environment while performing its tasks.

[0083] In this embodiment, by determining the current state of the target robot based on multi-source heterogeneous data, the operating status of the target robot itself is detected, which can ensure the stability of the target robot itself; by constructing an operating environment map based on the position data of the target robot and multi-source heterogeneous data, the operating environment map of the target robot is automatically updated according to the multi-source heterogeneous data and position data acquired in real time, so as to timely discover changes in the operating environment of the target robot; by planning the obstacle avoidance path of the target robot based on the operating environment map, the driving path of the target robot can be corrected in real time, thereby improving the stability and reliability of the target robot.

[0084] In a specific embodiment, in S301, the target robot is integrated with multiple types of laser radar, infrared camera, inertial measurement unit, ultrasonic sensor, posture sensor, temperature sensor, oil pressure sensor, oil level sensor and electromagnetic flowmeter; the multi-source heterogeneous data includes multiple types of point cloud data, image frames, inertial data, ultrasonic sensor data, posture information, temperature, oil pressure, oil level and flow; the data collected by the multiple data acquisition modules correspond to the data in the multi-source heterogeneous data, that is, each type of multi-source heterogeneous data is collected by one of the multiple data acquisition modules, such as: point cloud data is collected by laser radar, image frames are collected by infrared camera, inertial data is collected by inertial measurement unit, ultrasonic sensor data is collected by ultrasonic sensor, posture information is collected by posture sensor, temperature is collected by temperature sensor, oil pressure is collected by oil pressure sensor, oil level is collected by oil level sensor, and flow is collected by electromagnetic flowmeter.

[0085] Obviously, multi-source heterogeneous data come from different acquisition devices. Therefore, there may be data disorder between different multi-source heterogeneous data, and direct data interaction is impossible. Therefore, first, the initial multi-source heterogeneous data of the target robot in the working environment is collected; then, the initial multi-source heterogeneous data is synchronously fused to obtain multi-source heterogeneous data.

[0086] In a specific embodiment, the sensors in the various data acquisition modules of the target robot are arranged on the body of the target robot according to their functions. In order to intuitively represent the structure of the target robot, as shown in FIG. Figure 4 As shown, Figure 4 This is a structural schematic diagram of an embodiment of the target robot provided in an embodiment of the present application. The target robot includes multiple data acquisition modules (lidar, infrared camera, inertial measurement unit, ultrasonic sensor, attitude sensor, temperature sensor, oil pressure sensor, oil level sensor and electromagnetic flowmeter), a vehicle-mounted control box, a wireless communication module and a remote control.

[0087] The vehicle-mounted control box collects and manages data from various data acquisition modules, and sends the processed key environment and status data to the remote control through the wireless communication module. At the same time, the vehicle-mounted control box also receives external control instructions and controls the operation of the target robot based on the control instructions, realizing two-way information interaction.

[0088] The main functions of the sensors in various data acquisition modules are as follows:

[0089] LiDAR: This system uses a rotating scanning LiDAR, utilizing the principle of laser time-of-flight ranging to obtain distance point cloud data of the surrounding environment, thereby mapping the planar geometry of the environment. In this application, LiDAR is the core sensor for environmental mapping and obstacle detection, providing highly accurate spatial structure information.

[0090] Infrared camera: Installed on the front of the target robot, it captures visible light and infrared images of the work site. The infrared camera has low-light imaging capabilities, enabling clear video capture even in dark underground environments. This application utilizes it for visual recognition (such as obstacle detection) and remote inspection footage, helping to improve the robot's environmental perception and obstacle recognition capabilities.

[0091] Ultrasonic sensor: Installed in front of or to the side of the target robot, low to the ground. Ultrasonic sensors measure distance by emitting ultrasonic pulses and receiving echoes reflected from obstacles or liquid surfaces. They are suitable for detecting obstacles at close range (typically 0.2 to 5 meters). This application utilizes ultrasonic sensors to compensate for the blind spots of lidar at specific angles and low altitudes, ensuring detection of close-range obstacles (such as potholes and rocks) and water levels.

[0092] Oil pressure sensor: A strain gauge or piezoresistive oil pressure sensor converts the pressure of the liquid in the pipeline into an electrical signal. The oil pressure sensor is used to detect changes in engine oil pressure to prevent damage due to oil shortage.

[0093] Temperature sensor: It is placed in the hydraulic oil tank of the robot. Its function is to monitor the temperature of the hydraulic oil and prevent equipment failure due to overheating.

[0094] Oil Level Sensor: Installed in the robot's fuel tank, it monitors the fuel level. Using a float-type level gauge or ultrasonic level sensor, it converts the fuel level into an electrical signal. The oil level sensor monitors energy / hydraulic reserves in real time to prevent robot shutdown or hydraulic system damage due to fuel depletion or insufficient hydraulic fluid. This application uses oil level sensor information to alert operators to timely refuel or refuel, ensuring the smooth completion of long, continuous drainage tasks.

[0095] An inertial measurement unit (IMU) is installed in the center of the robot's body and consists of a three-axis gyroscope, a three-axis accelerometer, and a magnetometer. The IMU measures the acceleration and angular velocity of each axis of the robot in real time, and calculates the target robot's attitude angle (pitch angle, roll angle) and heading. In this application, the attitude sensor is used to detect the target robot's tilt while driving, preventing the risk of overturning due to uneven ground or obstacles. It also provides inertial data for the SLAM algorithm, combining laser and visual information to improve positioning accuracy.

[0096] Electromagnetic flowmeter: Installed on the drainage pump suction pipe, used to measure the volume flow of water inhaled by the target robot. The electromagnetic flowmeter uses Faraday's electromagnetic induction principle. When a conductive liquid passes through an external magnetic field, it will induce an electromotive force proportional to the flow rate. The flow rate and instantaneous flow rate can be calculated by measuring the voltage. This sensor is used to determine the drainage flow rate. When the water flow rate suddenly drops sharply, it indicates that the water level has dropped below the drainage pump suction port, and the target robot needs to continue to move forward to continue the flood discharge operation. Through the electromagnetic flowmeter, this application can quantify the flood discharge efficiency and help staff evaluate the on-site drainage progress and equipment working status.

[0097] In a specific embodiment, in order to ensure the rationality of data fusion, the initial multi-source heterogeneous data is synchronously fused to obtain multi-source heterogeneous data, which specifically includes:

[0098] Align the initial multi-source heterogeneous data based on a unified timestamp to obtain the initial multi-source heterogeneous data with consistent timeliness;

[0099] The initial point cloud data and the initial inertial data are fused through the extended Kalman filter to obtain point cloud data and inertial data;

[0100] The initial temperature, initial oil pressure, initial oil level and initial flow are smoothed by mean filtering to obtain the temperature, oil pressure, oil level and flow;

[0101] The outlier noise in the initial ultrasonic sensor data and initial posture information is removed by median filtering to obtain the ultrasonic sensor data and posture information.

[0102] In this embodiment, the initial multi-source heterogeneous data are aligned based on a unified timestamp, the initial multi-source heterogeneous data are time-series standardized, the time dimension deviation of the multi-source data is eliminated, and the timeliness of the initial multi-source heterogeneous data is strictly synchronized; the initial point cloud data and the initial inertial data are deeply fused by the extended Kalman filter algorithm to reduce the cumulative error and significantly improve the accuracy of spatial positioning and motion state estimation; the initial temperature, oil pressure, oil level and flow data are dynamically smoothed by mean filtering to reduce the reading fluctuation caused by the instantaneous jitter of the sensor, effectively suppress the measurement noise interference, and enhance the stability and continuity of the time series data; the initial ultrasonic sensor data and the initial posture information are filtered out of outliers by the median filter algorithm to remove occasional jump values, and significantly improve the anti-interference ability of the data set while retaining effective features; in summary, high-quality and highly consistent multi-source heterogeneous data are obtained by systematically fusing the multi-source heterogeneous data.

[0103] Furthermore, in S302, based on the fused multi-source heterogeneous data, in order to determine the current state of the target robot, specifically, extracting the oil pressure, oil temperature, oil temperature duration, and oil level from the multi-source heterogeneous data, the step of determining the current state of the target robot includes:

[0104] Determine whether the oil pressure state of the target robot is abnormal based on the relationship between the oil pressure and a preset oil pressure safety threshold;

[0105] Determine whether the oil temperature state of the target robot is abnormal based on the relationship between the oil temperature and a preset oil temperature safety threshold, and determine that the safety state of the target robot is abnormal when the oil temperature duration of the abnormal oil temperature is greater than a preset safety time threshold;

[0106] According to the relationship between the oil level and a preset oil level threshold, it is determined whether the oil level state of the target robot is abnormal.

[0107] In a specific embodiment, the oil pressure safety threshold T is set according to the normal oil pressure range of the target robot and the test results of the on-site working conditions. press (Minimum safety oil pressure), when it is detected that the current oil pressure p is lower than the safety threshold (ie p <T press ), it is determined that the oil pressure status is abnormal.

[0108] Furthermore, when the oil pressure status is determined to be abnormal, since the abnormal oil pressure will seriously affect the safe operation of the robot, an alarm is triggered to remind the operator, and emergency shutdown measures can also be taken directly to prevent mechanical parts from being damaged due to lack of oil.

[0109] In a specific embodiment, since the hydraulic oil temperature should not exceed a certain temperature value for a long time, this value is set as the oil temperature safety threshold T temp, when the target robot's hydraulic oil temperature t exceeds the oil temperature safety threshold (t>T temp ), it is determined that the oil temperature state of the target robot is abnormal; since the temperature of the hydraulic oil may have a sudden change at a certain moment, after determining that the oil temperature state of the target robot is abnormal, it is also necessary to detect the oil temperature duration h of the abnormal oil temperature. When the oil temperature duration h is greater than the preset safety time threshold T duration (h>T duration ), the safety status of the target robot is determined to be abnormal.

[0110] Furthermore, since overheating of the hydraulic oil will seriously affect the safety of the robot, an alarm is triggered to remind the operator, and a shutdown operation can be directly executed to avoid the expansion of the fault.

[0111] In a specific embodiment, different robots include different fuel tank models. In order to standardize the setting of the fuel level threshold, the fuel level threshold T is set as a percentage of the fuel tank capacity. v , when it is detected that the current fuel level v is lower than the fuel level threshold T v (v <T v ), the target robot is judged to be low on fuel and an alarm is immediately triggered. Since a low fuel level has little impact on the robot's safe operation in the short term, a load reduction operation is implemented, appropriately reducing the power output of the robot's non-critical functional components. This conserves fuel and extends the robot's remaining operating time, ensuring that the robot has ample time to reach the maintenance location.

[0112] Specifically, the oil level threshold T v It can be set to 1 / 5, and can be adjusted to other values ​​according to actual needs. There is no limit here.

[0113] In this embodiment, the actual operating parameters (oil pressure, oil temperature, oil level) of the target robot are compared with the preset safety thresholds (oil pressure safety threshold, oil temperature safety threshold, oil level threshold, safety time threshold) in real time to detect whether the system is within the allowable safety boundary; by comprehensively judging the parameters of three specific dimensions and their mutual relationships, it is systematically detected and diagnosed whether the target robot is currently facing abnormal working conditions or safety hazards (such as abnormal oil pressure, continuous high temperature, low oil level, etc.), thereby realizing the state perception of the target robot, so as to ensure the operation safety of the target robot.

[0114] In a specific embodiment, in S303, after ensuring that the target robot can operate normally, the target robot is required to integrate data of its operating environment in order to construct an operating environment map. Obviously, in the process of constructing the operating environment map, multi-source heterogeneous data technology is also required. Specifically, first, point cloud data and image frames in the multi-source heterogeneous data are required. Then, in order to ensure the accuracy of the operating environment map, the constructed map data needs to be detected and corrected based on the position data of the target robot. Then, constructing the operating environment map based on the position data of the target robot and the multi-source heterogeneous data includes:

[0115] Perform iterative calculations on the point cloud data to obtain the position and posture data of the target robot;

[0116] The key features of the image frame are determined by a fast feature point extraction algorithm, and loop closure detection is performed on the target robot based on the key features of the image to obtain loop closure data of the working environment;

[0117] The global error correction of loop closure data and pose data is performed through graph optimization methods to obtain a global working environment map.

[0118] The global working environment map is fused and compensated according to the angular velocity and acceleration to obtain the working environment map.

[0119] The fast feature point extraction algorithm specifically refers to the ORB (Oriented FAST and Rotated BRIEF) feature extraction algorithm, which is a feature point extraction and matching algorithm commonly used in computer vision.

[0120] Loop closure detection is a core technology in visual navigation systems (such as SLAM). Its main function is to detect when a robot or camera revisits a previously explored scene. Through loop closure detection, accumulated errors can be corrected and the pose graph can be optimized, thereby improving the global consistency and accuracy of the SLAM system.

[0121] Graph optimization (also known as pose graph optimization) is a method for expressing and solving the robot positioning (SLAM) problem and is a mainstream method for SLAM backend optimization. Specifically, graph optimization utilizes a graph model to fuse various measurement constraints and minimizes the overall prediction error through numerical optimization. Its core function is to efficiently correct the accumulated odometry drift through global (or large-scale local) optimization, especially by utilizing loop closure constraints, thereby obtaining a globally consistent, high-precision robot trajectory and environmental map.

[0122] In a specific embodiment, by introducing the RTAB-Map (Real-Time Appearance-Based Mapping) algorithm, the lidar point cloud data and infrared camera image information are combined, so that the fused global working environment map has high-precision and multimodal perception capabilities, which can effectively cope with underground space working environments with insufficient lighting and complex structures.

[0123] Specifically, during the movement of the target robot, the laser radar continuously performs 360° plane scanning to obtain two-dimensional point cloud data of the surrounding environment. Each frame of laser point cloud is aligned with the previous frame through the iterative closest point algorithm (ICP), and the relative motion pose is calculated to generate a laser odometry trajectory. At the same time, the infrared camera collects image sequences in real time, extracts key image features through the ORB (OrientedFAST and Rotated BRIEF, fast feature point) feature extraction algorithm, and performs inter-frame feature matching to estimate visual motion information. RTAB-Map stores image frames and their poses as key frame nodes in the internal graph database, and periodically performs loop detection - by comparing the feature similarity between the current image and the historical key frames to determine whether it has returned to the visited area. When a loop closure is detected, the graph optimization method is used to globally adjust the entire trajectory, correct the accumulated error, and optimize the ground. Figure 1 Consistency.

[0124] In addition, to enhance the robustness of the system, this application also integrates the angular velocity and acceleration information measured by the IMU (inertial measurement unit). Specifically, the angular velocity and acceleration are input into the extended Kalman filter (EKF) optimization framework as a prediction model, so that stable estimation can be maintained when the infrared image temporarily fails or the laser data is sparse.

[0125] In this embodiment, the point cloud data is iteratively calculated, the point cloud differences of consecutive frames are matched, the pose transformation between adjacent frames is iteratively optimized, and the position and posture of the target robot are estimated in real time; a fast feature point extraction algorithm is used to identify significant feature points from the image frame, and feature matching is used to detect whether the target robot revisits historical scenes, to obtain loop closure data of the operating environment, and generate loop closure constraints to eliminate the positioning drift of the target robot; a pose graph model is constructed by using pose data and loop closure data as edges, the global pose distribution is adjusted, and the consistency of the global operating environment map is improved; based on angular velocity and acceleration information, the motion distortion and short-term positioning noise of the global operating environment map are compensated, thereby enhancing the robustness and accuracy of the operating environment map in a dynamic environment.

[0126] Obviously, by deeply integrating lidar, infrared camera and IMU, this application does not need to rely on external positioning signals (such as GPS), and can ensure the autonomous navigation capability of the target robot in underground garages, tunnels, culverts and other spaces, significantly improving positioning accuracy, environmental adaptability and system robustness.

[0127] Furthermore, the target robot's operating environment changes in real time and may include obstacles. Therefore, in order to ensure the real-time effectiveness of the operating environment map, it is necessary to identify obstacles in the operating environment to improve the integrity of the operating environment map. Specifically, image frames and ranging data from multi-source heterogeneous data are extracted. Then, the operating environment map is constructed based on the target robot's position data and multi-source heterogeneous data. The following steps are also included:

[0128] Identify obstacles in image frames based on a well-trained object obstacle detection model;

[0129] Determine the distance and orientation between the obstacle and the target robot based on the ranging data;

[0130] The obstacle coordinates are calculated based on the position data, distance and orientation, and the obstacle coordinates are marked on the initial working environment map to obtain a three-dimensional working environment map with obstacles.

[0131] The obstacle detection model is a convolutional neural network (CNN) model. To obtain the target obstacle detection model, a diverse training sample set is required. Specifically, a large number of on-site images are collected as training samples, targeting obstacles that may appear in underground confined spaces (such as scattered branches, parked vehicles, fallen equipment, and floating garbage). The locations and categories of the obstacles are manually annotated to construct an initial image dataset. To improve the generalization ability of the recognition model, various data augmentation methods are used to expand the training data. These include random rotation, scaling, and translation of the original images, adjustments to image brightness, contrast, and color to simulate different lighting environments, and the addition of random noise, blurring, and occlusion. Through data augmentation techniques, a diverse training sample set is generated. The massive preprocessed samples are then fed into the CNN model for training iterations, with parameters continuously tuned to obtain the target obstacle detection model.

[0132] Furthermore, to obtain a three-dimensional map of the obstacle-laden work environment, the infrared camera continuously captures images while the target robot performs its drainage task and feeds them into the target obstacle detection model, outputting obstacle detection results in real time. These detection results include the obstacle's category label and its position in the image. Subsequently, the visual recognition results are combined with ranging data from ultrasonic and camera depth sensors to calculate the distance and orientation of the obstacle relative to the target robot. Combined with the target robot's own pose information, the obstacle position is converted to a global coordinate system. Therefore, the visual recognition module not only provides obstacle recognition and detection but also participates in achieving three-dimensional obstacle positioning. Finally, the identified obstacle location data is added to the constructed map, resulting in a three-dimensional map of the obstacle-laden work environment.

[0133] In this embodiment, by identifying obstacles in image frames based on the target obstacle detection model, obstacles in the working environment can be obtained in real time. The distance and orientation between the obstacle and the target robot can be determined based on the ranging data, and the position of the obstacle relative to the target robot can be determined. Then, combined with the position data of the target robot, the specific position of the obstacle can be determined. By marking the obstacle coordinates on the initial working environment map, a three-dimensional working environment map with obstacles is obtained, which provides the target robot with an accurate, objective and continuously updated spatial representation of the obstacles in the working environment, greatly improving the target robot's collision avoidance ability and its operation ability in complex terrain / changing environments.

[0134] In another specific embodiment, other data in the multi-source heterogeneous data, such as point cloud data, can be combined to perform a secondary analysis of the shape, size, properties, material, etc. of the obstacles, and the analysis results can be marked on the working environment map to help the target robot better avoid obstacles in the working environment. This will not be elaborated here.

[0135] In a specific embodiment, after obtaining the operating environment map in S304, since the target robot still needs to complete the task in the real-time changing operating environment, to ensure the operation safety of the target robot, it is also necessary to plan the target robot's obstacle avoidance path based on the operating environment map, which specifically includes:

[0136] Get the initial path of the target robot;

[0137] Based on the initial path, determine whether there is an obstacle within a first distance threshold in the forward direction of the target robot;

[0138] If so, control the target robot to perform emergency braking;

[0139] If not, calculate the number and area of ​​obstacles within a second distance threshold in the forward direction of the target robot based on the initial path;

[0140] When the number of obstacles is greater than a preset obstacle number threshold, and / or the area of ​​the obstacles is greater than a preset obstacle area threshold, a collision-free path is replanned based on the work environment map;

[0141] When the number of obstacles is not greater than the preset obstacle number threshold and the obstacle area is not greater than the preset obstacle area threshold, the obstacle avoidance decision of the target robot is triggered.

[0142] In a specific embodiment, the decision rules of the decision algorithm for planning the obstacle avoidance path of the target robot are as follows:

[0143]

[0144] Where ΔD is the distance between the target robot and the obstacle; T L is the first distance threshold; T D is the second distance threshold; N b is the number of obstacles; T N is the preset obstacle quantity threshold; S b is the obstacle area; T S is the preset obstacle area threshold; R1 is the first obstacle avoidance rule, which specifically refers to replanning the collision-free path based on the working environment map; R2 is the second obstacle avoidance rule, which specifically refers to triggering the obstacle avoidance decision of the target robot, that is, moving along the initial path and avoiding detected obstacles in the process of moving forward; R3 is the third obstacle avoidance rule, which specifically refers to controlling the target robot to emergency brake.

[0145] Obviously, when the distance between the target robot and the obstacle is greater than the second distance threshold, it means that there is no obstacle on the initial path, or even if there is an obstacle, it does not affect the current target robot's progress and its impact does not need to be considered.

[0146] In this embodiment, the target robot's predetermined moving direction is determined based on the initial path; by detecting whether an emergency collision may occur within a short distance (first threshold) in front of the target robot, a rapid response to sudden obstacles is ensured, and the target robot's movement is immediately stopped when a close-range obstacle is detected to avoid collision risks and ensure safety; by evaluating the density of obstacles within a longer distance (second threshold) in front of the target robot, the potential risk level of the path is judged, and a new global path is generated in the obstacle-dense area to bypass the high-risk area and ensure the feasibility of the path; when the number of obstacles does not exceed the threshold, a dynamic obstacle avoidance strategy (such as detour, deceleration) is activated to maintain local optimization of the path. In summary, planning the target robot's obstacle avoidance path based on the operating environment map can ensure that the target robot can move safely and efficiently in the operating environment, improve the robustness and adaptability of path planning, ensure that the robot can reliably complete tasks in complex environments, and minimize path interruptions and operational efficiency losses.

[0147] In a specific embodiment, the specific process of the target robot performing operating environment and state perception is as follows: Figure 5 As shown, Figure 5 The present application provides a flowchart of an embodiment of the target robot's operating environment and state perception process. After collecting data from various sensors, all sensor data are first synchronously preprocessed to achieve standardized processing of all sensor data. Then, on the one hand, the current positioning of the target robot is determined based on the standardized sensor data, and a map of the target robot's current environment is constructed in combination with SLAM technology. On the other hand, the sensor data is visually recognized according to the obstacle recognition model to obtain obstacles in the target robot's current environment. Finally, the decision to avoid obstacles is executed based on the identified obstacles and the newly constructed map, so as to ultimately control the robot to bypass the obstacles.

[0148] To facilitate better implementation of the robot-based working environment and state perception method provided in the embodiments of the present application, the embodiments of the present application also provide a device based on the aforementioned robot-based working environment and state perception method. The meanings of the terms herein are the same as those in the aforementioned robot-based working environment and state perception method. For specific implementation details, please refer to the description in the method embodiment.

[0149] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an embodiment of a robot-based working environment and state perception device provided in an embodiment of the present application, wherein the robot-based working environment and state perception device 600 may include:

[0150] Data acquisition module 601, used to acquire multi-source heterogeneous data of the target robot;

[0151] A current state determination module 602 is used to determine the current state of the target robot based on multi-source heterogeneous data;

[0152] An operating environment map construction module 603 is used to construct an operating environment map based on the position data of the target robot and multi-source heterogeneous data;

[0153] The obstacle avoidance module 604 is used to plan the obstacle avoidance path of the target robot according to the working environment map.

[0154] The present application also provides an electronic device, such as Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Specifically:

[0155] The electronic device may include one or more processing core processors 701, one or more computer-readable storage media memories 702, a power supply 703, an input unit 704 and other components. Those skilled in the art will understand that Figure 7 The electronic device structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0156] in:

[0157] The processor 701 is the control center of the electronic device. It connects the various parts of the entire electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in the memory 702 and calling data stored in the memory 702, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operation of storage media, user interface, and application programs, and the modem processor mainly handles wireless communications. It is understood that the above-mentioned modem processor may not be integrated into the processor 701.

[0158] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store operating storage media, applications required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0159] The electronic device also includes a power supply 703 for supplying power to various components. Preferably, the power supply 703 can be logically connected to the processor 701 via a power management storage medium, thereby implementing functions such as charging, discharging, and power consumption management through the power management storage medium. The power supply 703 can also include one or more DC or AC power supplies, a recharge storage medium, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0160] The electronic device may further include an input unit 704, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0161] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application programs stored in the memory 702 to implement various functions as follows:

[0162] Acquire multi-source heterogeneous data of the target robot; determine the current state of the target robot based on the multi-source heterogeneous data; construct an operating environment map based on the target robot's position data and the multi-source heterogeneous data; and plan an obstacle avoidance path for the target robot based on the operating environment map.

[0163] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0164] To this end, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which is loaded by a processor to execute the steps of any of the robot-based working environment and state perception methods provided in the embodiments of the present application. For example, the computer program loaded by the processor may execute the following steps:

[0165] Acquire multi-source heterogeneous data of the target robot; determine the current state of the target robot based on the multi-source heterogeneous data; construct an operating environment map based on the target robot's position data and the multi-source heterogeneous data; and plan an obstacle avoidance path for the target robot based on the operating environment map.

[0166] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0167] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0168] Since the computer program stored in the computer-readable storage medium can execute the steps in any one of the robot-based working environment and state perception methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the robot-based working environment and state perception methods provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0169] The above is a detailed introduction to the robot-based working environment and state perception method, device, electronic device and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A robot-based working environment and state perception method, characterized in that: include: Acquire multi-source heterogeneous data of the target robot; Determining a current state of the target robot based on the multi-source heterogeneous data; Constructing an operating environment map based on the position data of the target robot and the multi-source heterogeneous data; Plan an obstacle avoidance path for the target robot according to the operating environment map.

2. The robot-based working environment and state perception method according to claim 1, characterized in that: The target robot includes multiple data acquisition modules, and the multiple data acquisition modules include multiple types of laser radar, infrared camera, inertial measurement unit, ultrasonic sensor, posture sensor, temperature sensor, oil pressure sensor, oil level sensor and electromagnetic flowmeter; the multi-source heterogeneous data includes multiple types of point cloud data, laser odometer, image frame, inertial data, ultrasonic sensor data, posture information, temperature, oil pressure, oil level and flow, and the data collected by the multiple data acquisition modules respectively correspond to the data in the multi-source heterogeneous data; the multi-source heterogeneous data of the target robot is obtained, including: Collecting initial multi-source heterogeneous data of the target robot in the working environment; The initial multi-source heterogeneous data is synchronously fused to obtain the multi-source heterogeneous data.

3. The robot-based working environment and state perception method according to claim 2, characterized in that: The synchronous fusion processing of the initial multi-source heterogeneous data to obtain the multi-source heterogeneous data includes: Aligning the initial multi-source heterogeneous data based on a unified timestamp to obtain initial multi-source heterogeneous data with consistent timeliness; fusing the initial point cloud data, the initial laser odometry data, and the initial inertial data through an extended Kalman filter to obtain the point cloud data, the laser odometry data, and the inertial data; Smoothing the initial temperature, initial oil pressure, initial oil level, and initial flow rate by using mean filtering to obtain the temperature, the oil pressure, the oil level, and the flow rate; The outlier noise in the initial ultrasonic sensor data and the initial posture information is removed by median filtering to obtain the ultrasonic sensor data and the posture information.

4. The robot-based working environment and state perception method according to claim 1, characterized in that: The multi-source heterogeneous data includes oil pressure, oil temperature, oil temperature duration and oil level; Determining the current state of the target robot according to the multi-source heterogeneous data includes: determining whether the oil pressure state of the target robot is abnormal based on a relationship between the oil pressure and a preset oil pressure safety threshold; determining whether the oil temperature state of the target robot is abnormal based on a relationship between the oil temperature and a preset oil temperature safety threshold, and determining that the safety state of the target robot is abnormal when the oil temperature duration of the abnormal oil temperature is greater than a preset safety time threshold; According to the relationship between the oil level and a preset oil level threshold, it is determined whether the oil level state of the target robot is abnormal.

5. The robot-based working environment and state perception method according to claim 1, characterized in that: The multi-source heterogeneous data includes point cloud data and image frames; the position data includes the angular velocity and acceleration of the target robot; and constructing an operating environment map based on the position data of the target robot and the multi-source heterogeneous data includes: Iteratively calculating the point cloud data to obtain the position and posture data of the target robot; Determining key image features of the image frame according to a fast feature point extraction algorithm, and performing loop detection on the target robot based on the key image features to obtain loop closure data of the operating environment; Performing global error correction on the loop closure data and the pose data using a graph optimization method to obtain a global operating environment map; The global operating environment map is fused and compensated according to the angular velocity and the acceleration to obtain the operating environment map.

6. The robot-based working environment and state perception method according to claim 1, characterized in that: The multi-source heterogeneous data includes image frames and ranging data; and constructing an operating environment map based on the position data of the target robot and the multi-source heterogeneous data includes: Identifying obstacles in the image frame based on a well-trained target obstacle detection model; Determining the distance and orientation between the obstacle and the target robot according to the ranging data; Obstacle coordinates are calculated according to the position data, the distance, and the orientation, and the obstacle coordinates are marked on the initial working environment map to obtain a three-dimensional working environment map with obstacles.

7. The robot-based working environment and state perception method according to claim 1, characterized in that: Planning an obstacle avoidance path for the target robot according to the operating environment map includes: Obtaining an initial path of the target robot; Based on the initial path, determining whether there is an obstacle within a first distance threshold in the forward direction of the target robot; If yes, controlling the target robot to perform emergency braking; If not, calculating the number and area of ​​obstacles within a second distance threshold in the forward direction of the target robot based on the initial path; When the number of obstacles is greater than a preset obstacle number threshold, and / or the area of ​​the obstacles is greater than a preset obstacle area threshold, replanning a collision-free path based on the operating environment map; When the number of obstacles is not greater than the preset obstacle number threshold and the obstacle area is not greater than the preset obstacle area threshold, the obstacle avoidance decision of the target robot is triggered.

8. A robot-based working environment and state perception device, characterized in that: include: Data acquisition module, used to obtain multi-source heterogeneous data of the target robot; a current state determination module, configured to determine the current state of the target robot based on the multi-source heterogeneous data; An operating environment map construction module, configured to construct an operating environment map based on the position data of the target robot and the multi-source heterogeneous data; The obstacle avoidance module is used to plan the obstacle avoidance path of the target robot according to the working environment map.

9. An electronic device, characterized in that: The system comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the following steps: Acquire multi-source heterogeneous data of the target robot; Determining a current state of the target robot based on the multi-source heterogeneous data; Constructing an operating environment map based on the position data of the target robot and the multi-source heterogeneous data; Plan an obstacle avoidance path for the target robot according to the operating environment map.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the robot-based working environment and state perception method according to any one of claims 1 to 8.

Images