Path planning method and device based on multi-source data, electronic equipment, medium and program product

Abstract

The invention discloses a path planning method and device based on multi-source data, electronic equipment, a medium and a program product, and relates to the technical field of robots. The path planning method based on the multi-source data comprises the following steps: acquiring sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor and a temperature sensor which are arranged on a robot; determining the position of a fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor; and determining a target path based on the current position of the robot, the fire source position and an environment obstacle determined according to the sensor data. According to the technical scheme, the problem that the optimal action path is difficult to efficiently and accurately determine by a current inspection strategy is solved.

Description

Technical Field

[0001] The present application relates to the field of robotics, and in particular to a path planning method, device, electronic device, computer-readable storage medium, and computer program product based on multi-source data. Background Art

[0002] Traditional inspection and fire-fighting strategies have many shortcomings when facing modern fires. First, the response speed is slow. The early stage of a fire is a critical moment to control the spread of the fire, but traditional inspection methods often cannot respond quickly in a short period of time to determine the location of the fire source, which causes the fire to spread and increases the difficulty of rescue and fire fighting. Since the location of the fire source usually relies on manual experience to judge, the accuracy is poor. In addition, the fire scene environment is complex, and factors such as smoke, toxic gases, and high temperatures put rescuers at extremely high risks. Failure to quickly determine the best path of action to reach the fire source will make the fire uncontrollable and affect the efficiency of rescue and fire fighting.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a path planning method, device, electronic device, computer-readable storage medium and computer program product based on multi-source data, aiming to solve the technical problem that the current inspection strategy is difficult to efficiently and accurately determine the optimal action path.

[0005] To achieve the above objectives, the present application proposes a path planning method based on multi-source data, which is applied to a robot. The path planning method based on multi-source data includes:

[0006] Obtain sensor data from infrared cameras, lidars, ultrasonic sensors, gas sensors, and temperature sensors installed on the robot;

[0007] Determine the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor;

[0008] A target path is determined based on the current position of the robot, the location of the fire source, and environmental obstacles determined according to the sensor data.

[0009] In one embodiment, the step of determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor includes:

[0010] Determine the location of a high temperature area having a temperature higher than a preset temperature threshold according to the temperature conditions of each area in the infrared image data;

[0011] According to the carbon monoxide concentration of each area in the carbon monoxide concentration data, determine the location of a high-concentration area where the carbon monoxide concentration is higher than a preset concentration threshold;

[0012] The location of the fire source is determined based on the location of the high temperature area and the location of the high concentration area.

[0013] In one embodiment, the step of determining a target path based on the current position of the robot, the fire source position, and environmental obstacles determined according to the sensor data comprises:

[0014] Initializing the current position into an open list to obtain a current node, wherein the open list includes a plurality of neighboring nodes adjacent to the current node in the area where the robot is located;

[0015] Determine the cost values ​​corresponding to the current position and each neighbor node in the open list, and set the neighbor node with the lowest cost value as the updated current node;

[0016] When the current node is not the fire source location, returning to the step of determining the cost values ​​corresponding to the current location to each of the neighboring nodes in the open list, and selecting the neighboring node with the lowest cost value to update the current node;

[0017] When the current node is the fire source location, a target path is generated according to each current node generated historically;

[0018] As the robot moves, the target path is updated in real time based on environmental obstacles determined by the sensor data.

[0019] In one embodiment, the step of updating the target path in real time based on the environmental obstacles determined by the sensor data comprises:

[0020] Predicting multiple motion trajectories of the robot within a preset time period in the future based on the current speed and acceleration of the robot and obstacles;

[0021] Calculate the cost value corresponding to each trajectory, and select the trajectory with the lowest cost value as the update path within the preset time length;

[0022] The update path is used for the robot to generate corresponding motion control instructions to update the current speed, acceleration and current position of the robot.

[0023] In one embodiment, after the step of acquiring sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor, and a temperature sensor provided on the robot, the method further comprises:

[0024] Removing noise from each of the sensor data by using a preset bandpass filtering algorithm and a mean filtering algorithm;

[0025] Calibrate the infrared camera, laser radar, ultrasonic sensor, gas sensor and temperature sensor respectively to obtain corresponding deviation values;

[0026] Adjusting the denoised sensor data based on the deviation values ​​respectively corresponding to the sensors to obtain updated sensor data;

[0027] The step of determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor is performed based on the updated sensor data.

[0028] In one embodiment, the sensor data includes at least temperature data and carbon monoxide concentration data;

[0029] After the step of determining a target path based on the current position of the robot, the location of the fire source, and environmental obstacles determined according to the sensor data, the method further includes:

[0030] Acquiring the temperature data and the change of the carbon monoxide concentration;

[0031] When the change amplitude of the temperature data and / or the carbon monoxide concentration is greater than a preset change threshold, it is determined that the fire in the area where the robot is located is intensifying;

[0032] Adjust the fire extinguishing parameters of the robot, wherein the fire extinguishing parameters include at least one of the distance from the fire source, the type of fire extinguisher, the spray intensity of the fire extinguisher, the moving speed or the number of robots.

[0033] In addition, to achieve the above purpose, the present application also proposes a path planning device based on multi-source data, and the path planning device based on multi-source data includes:

[0034] A data acquisition module, used to acquire sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor, and a temperature sensor provided on the robot;

[0035] A fire source determination module, used to determine the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor;

[0036] A path determination module is used to determine a target path based on the current position of the robot, the fire source position, and environmental obstacles determined according to the sensor data.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the path planning method based on multi-source data as described above.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the path planning method based on multi-source data as described above are implemented.

[0039] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the path planning method based on multi-source data as described above.

[0040] This application proposes a path planning method based on multi-source data, firstly obtaining sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor and a temperature sensor provided on a robot, then determining the location of the fire source based on the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor, and finally determining the target path based on the current position of the robot, the location of the fire source and the environmental obstacles determined based on the sensor data. The technical solution of this application obtains multi-source sensor data through a variety of sensors provided on the robot, and then determines the location of the fire source based on the data that can reflect the temperature and carbon monoxide conditions in the area where the robot is located, thereby realizing rapid and automatic positioning of the fire source, and then automatically planning the action path based on the current position of the robot, the location of the fire source and the obstacles in the environment, which improves the efficiency of path planning compared to the traditional method of manually planning the action path, allowing the robot to reach the fire source location faster to extinguish the fire, effectively improving the efficiency of rescue and fire extinguishing, and ensuring the safety of life and property of the affected people. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1A schematic diagram of a process flow of a path planning method based on multi-source data in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of a process for determining a target path based on the current position of the robot, the location of the fire source, and environmental obstacles determined according to sensor data in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the entire process of implementing inspection strategy decision support using robots and sensors in an embodiment of the present application is feasible;

[0046] Figure 4 This is a schematic diagram of the structure of a path planning device based on multi-source data in an embodiment of the present application;

[0047] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the path planning method based on multi-source data in the embodiment of the present application.

[0048] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0051] Traditional inspection strategies have many shortcomings when facing modern fires. First, slow response speed is a significant problem. The early stage of a fire is a critical moment to control the spread of the fire, but traditional inspection methods often cannot respond quickly during this period, which causes the fire to spread and increases the difficulty of rescue. Secondly, the inspection and fire-fighting strategies are single and usually rely on manual judgment and operation. They cannot be adjusted in real time according to changes in the fire, resulting in low fire-fighting efficiency. In addition, the safety of rescuers cannot be fully guaranteed. The fire scene environment is complex, and factors such as smoke, toxic gases, and high temperatures put rescuers at extremely high risks, while existing equipment and technical means cannot provide adequate protection. Based on the existence of these problems and the increase in safety needs, this field urgently needs a system that can respond quickly and make intelligent decisions to improve rescue efficiency and safety. Therefore, a more intelligent and efficient solution is needed to meet these challenges.

[0052] In order to achieve the above objectives, an embodiment of the present application provides a path planning method based on multi-source data applied to a robot, wherein the robot is used for inspection and has motion functions. The robot is provided with sensors such as infrared cameras, lidars, ultrasonic sensors, gas sensors, and temperature sensors, as well as a processor. The processor is used to analyze and process the received sensor data, so as to determine the location of the fire source, plan the action path, and perform other automatic inspection strategy adjustment tasks.

[0053] Figure 1 This is a flow chart of a first embodiment of a path planning method based on multi-source data of the present application. The path planning method based on multi-source data includes:

[0054] Step S10, acquiring sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor, and a temperature sensor provided on the robot;

[0055] In an embodiment of the present application, when the robot enters the fire area, the infrared camera, lidar, ultrasonic sensor, gas sensor and temperature sensor installed on the robot start working to conduct in-depth exploration of environmental information (including temperature, gas concentration, obstacle location and map information, etc.).

[0056] For example, the robot integrates multiple sensors such as infrared cameras, LiDAR, ultrasonic sensors, gas sensors and temperature sensors for environmental perception and fire source detection. It is also equipped with a water gun or dry powder fire extinguisher for fire fighting.

[0057] Step S20, determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor;

[0058] It is understandable that in a fire scene, the temperature and carbon monoxide concentration at the fire source location are usually higher than those at other locations. Therefore, in the acquired infrared image data, the direction of the fire source location can be determined based on the position with the highest temperature in the infrared image, and the robot is moved in that direction. If the carbon monoxide concentration detected by the gas sensor is constantly rising at this time, and the carbon monoxide concentration reaches a peak at the target position with the highest temperature in the infrared image, then the target position can be determined to be the fire source location.

[0059] Step S30, determining a target path based on the current position of the robot, the location of the fire source, and environmental obstacles determined according to sensor data.

[0060] In the embodiment of the present application, after the location of the fire source is determined, the location information of the environmental obstacles can be determined according to the data of each sensor, and the map information of the current area can also be determined according to the location information of each obstacle. Among them, the location information of the environmental obstacles can be determined by laser radar and ultrasonic sensors. The laser radar data and ultrasonic sensor data can calculate the distance between the robot and each obstacle according to the emitted laser or ultrasonic wave, so as to draw the map information of the current area, which is convenient for determining which unobstructed paths can be taken from the current location to the location of the fire source.

[0061] In another feasible embodiment, the robot can pre-store a regional map corresponding to the current area. After acquiring the data from each sensor, it can construct a small-scale map information of the current location, and then match the small-scale map information with the pre-stored regional map to determine the robot's own location in the regional map. In the subsequent movement process, the robot can update its real-time position in the regional map in a timely manner according to its own speed and acceleration, and can more quickly plan the action path between the current location and the fire source location according to the regional map.

[0062] Specifically, in the process of path determination, the path planning based on the A* algorithm (A-star Algorithm) and the dynamic window method path planning algorithm can be integrated to enable the robot to reach the target point as quickly as possible, determine the target path with the lowest cost, and improve the efficiency of fire fighting and rescue. The cost can be determined by time and safety, etc. The shorter the time, the higher the safety and the lower the cost.

[0063] The embodiment of the present application proposes a path planning method based on multi-source data, firstly obtaining sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor and a temperature sensor provided on the robot, then determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor, and finally determining the target path based on the current position of the robot, the location of the fire source and the environmental obstacles determined according to the sensor data. The technical solution of the embodiment of the present application obtains multi-source sensor data by means of a variety of sensors provided on the robot, and then determines the location of the fire source according to the data that can reflect the temperature and carbon monoxide conditions in the area where the robot is located, thereby realizing rapid and automatic positioning of the fire source, and then automatically planning the action path based on the current position of the robot, the location of the fire source and the obstacles in the environment, which improves the efficiency of path planning compared to the traditional method of manually planning the action path, allowing the robot to reach the location of the fire source faster to extinguish the fire, effectively improving the efficiency of rescue and fire extinguishing, and ensuring the safety of life and property of the affected people.

[0064] Furthermore, in a feasible embodiment, the step of determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor includes:

[0065] Step S21, determining the location of a high temperature area having a temperature higher than a preset temperature threshold according to the temperature conditions of each area in the infrared image data;

[0066] Step S22, determining the location of a high-concentration area where the carbon monoxide concentration is higher than a preset concentration threshold according to the carbon monoxide concentration of each area in the carbon monoxide concentration data;

[0067] Step S23, determining the fire source location according to the high temperature area location and the high concentration area location.

[0068] The embodiment of the present application provides a method for determining the location of a fire source based on infrared image data and carbon monoxide concentration data. First, a preset temperature threshold and a preset concentration threshold set according to the experience of experts in the field are pre-stored in the robot, wherein the preset temperature threshold is used to indicate that when a certain location is higher than the preset temperature threshold, it is very likely to be the location of the fire source. In different fire environments, the preset temperature threshold may be different. For example, when there are more flammable materials or the air circulation is good, the preset temperature threshold is higher; the preset concentration threshold is used to indicate that when the carbon monoxide concentration at a certain location is higher than the preset concentration threshold, the location is very likely to be the location of the fire source. In the case of good air circulation in the fire environment, the preset concentration threshold may be lower.

[0069] Exemplarily, after the robot has moved in a fire environment for a certain period of time, the temperature data (represented by infrared image data) and carbon monoxide concentration data of each location are collected. The high-temperature area locations and high-concentration locations can be screened out based on the preset temperature threshold and preset concentration threshold, and then the overlapping area of ​​the two locations can be determined as the fire source location.

[0070] In the embodiment of the present application, the specific location of the fire source is determined by combining the high temperature area in the infrared image and the high concentration of carbon monoxide detected by the gas sensor. The automation of fire source positioning is realized, the efficiency of fire source location determination is effectively improved, and it is convenient to formulate a reasonable inspection strategy and improve the success rate of rescue and fire extinguishing.

[0071] In one possible embodiment, referring to Figure 2 The step of determining a target path based on the current position of the robot, the fire source position, and environmental obstacles determined according to the sensor data comprises:

[0072] Step S31, initializing the current position into an open list to obtain the current node, wherein the open list includes a plurality of neighboring nodes adjacent to the current node in the area where the robot is located;

[0073] Step S32, determining the cost values ​​corresponding to the current position to each neighbor node in the open list, and setting the neighbor node with the lowest cost value as the updated current node;

[0074] Step S33, when the current node is not the fire source location, return to the step of determining the cost values ​​corresponding to each neighbor node in the open list from the current location, and select the neighbor node with the lowest cost value to update the current node;

[0075] Step S34, when the current node is the fire source location, generate a target path based on each current node generated historically;

[0076] Step S35, when the robot moves, the target path is updated in real time based on the environmental obstacles determined by the sensor data.

[0077] The embodiment of the present application provides a method for path planning specifically through the A* algorithm. Specifically, first initialize an open list, set the current position in the open list as the current node, and initialize other positions adjacent to the current position as neighbor nodes. In the process of path planning, it is first necessary to calculate the cost function (total cost = known cost + expected cost) from the current node to each neighbor node, wherein the cost may include time, risk (affected by temperature and terrain drop), robot loss, etc. Then select the neighbor node with the lowest total cost from the open list as the updated current node, and further expand the current node, and the expansion method is consistent with the step of step S32. First calculate the total cost from the updated current node to each neighbor node, and then select the neighbor node with the smallest total cost as the next current node, and so on, until the target node (such as the fire source location) is updated, and the search for all path points is completed. Finally, connect the current nodes determined in the aforementioned process to obtain the target path.

[0078] For example, the main idea of ​​the A* algorithm is to select the node with the smallest total cost from the open list as the current node. Then expand each neighbor node of the current node, calculate the total cost corresponding to its cost function, and add it to the open list. If the current node is the target node, backtrack from the target node to generate a target path from the starting point to the target point.

[0079] Since the total cost between each node is calculated step by step during the planning process of the target path, and the neighboring node with the minimum total cost is selected, the target path finally obtained is also the optimal path with the minimum total cost.

[0080] In addition, although the optimal target path has been planned, the fire environment changes rapidly. Therefore, in the subsequent process of moving the robot using the target path, it is also necessary to determine the environmental obstacles in the path based on the real-time collected sensor data, and update the target path in a timely manner to ensure that the robot moves to the fire source in the shortest time and at the lowest cost, so as to avoid new obstacles blocking its progress and affecting the efficiency of rescue and fire fighting.

[0081] Further, in a feasible embodiment, the step of updating the target path in real time based on the environmental obstacles determined by the sensor data includes:

[0082] Step S341, predicting multiple motion trajectories of the robot within a preset time period in the future according to the current speed and acceleration of the robot and obstacles;

[0083] Step S342, calculating the cost value corresponding to each trajectory, and selecting the trajectory with the lowest cost value as the update path within the preset time length;

[0084] The update path is used for the robot to generate corresponding motion control instructions to update the current speed, acceleration and current position of the robot.

[0085] An embodiment of the present application provides a method for updating a path in real time based on a dynamic window path planning algorithm.

[0086] In a dynamic environment, the robot needs to update the path in real time to avoid new dynamic obstacles in the fire environment. For example, first, based on the current speed and acceleration of the robot, its possible movement trajectory in the map in the future can be predicted. Then, the cost value of each predicted trajectory is calculated through the cost function to select the optimal trajectory with the smallest cost value, wherein the cost function is consistent with the cost function between the current node and each neighbor determined in step S32. Finally, the optimal trajectory selection is generated, and the trajectory with the smallest cost function value is selected as the updated path to generate the current motion instruction of the robot. For example, if the updated path needs to move to the right relative to the target path before the update, a rightward acceleration control instruction is generated to move the robot toward the updated path and update the real-time position.

[0087] In a feasible embodiment, after the step of acquiring sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor, and a temperature sensor provided on the robot, the method further includes:

[0088] Step S11, removing noise from each sensor data by using a preset bandpass filtering algorithm and a mean filtering algorithm;

[0089] Step S12, calibrating the infrared camera, laser radar, ultrasonic sensor, gas sensor and temperature sensor respectively to obtain corresponding deviation values;

[0090] Step S13, adjusting the denoised sensor data based on the deviation values ​​corresponding to the sensors to obtain updated sensor data;

[0091] Step S14, based on the updated sensor data, executes the step of determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor.

[0092] In the embodiment of the present application, the collected sensor data is mainly preprocessed. Specifically, the noise in the sensor data can be removed by using a bandpass filtering algorithm, and the mean filtering denoising technology can be used to smooth the data to eliminate the influence of random noise on the data; and the sensor is calibrated to correct the measurement deviation. Among them, the sensor calibration process can be adjusted based on a conventional reference standard to determine the deviation value of the sensor, and the sensor data can be updated based on the deviation value to improve the accuracy of the sensor data.

[0093] In another feasible embodiment, data from different sensors can be integrated through multi-sensor fusion to achieve more accurate environmental perception and decision support. Specifically, the Kalman filter algorithm can be used to perform algorithm fusion processing on the data collected by multiple sensors, establish a linear system model, and describe the dynamic changes of the robot state (including position, speed, acceleration, ambient temperature, and gas concentration, etc.). Based on the fused data, the robot can be assisted to make more effective decisions, identify the source of fire, assess the fire, and formulate fire extinguishing strategies.

[0094] For example, a decision support system (DSS) based on sensor fusion data can be pre-built in the robot to help the robot make intelligent decisions at the fire scene. The system can determine the specific location of the fire source by combining the high-temperature area in the infrared image and the high concentration of carbon monoxide detected by the gas sensor; by analyzing the temperature data and gas concentration, it can determine whether the fire is intensifying and whether the fire-fighting strategy needs to be adjusted; using the three-dimensional map and real-time sensor data provided by the lidar, it can plan the optimal path from the robot's current position to the fire source; considering the obstacles in the environment, it can generate a safe and effective route based on the path planning algorithm based on the fusion of the A* algorithm and the dynamic obstacle avoidance algorithm; and select the appropriate type of fire extinguisher and spray intensity for different fire intensities.

[0095] In another feasible embodiment, the sensor data includes at least temperature data and carbon monoxide concentration data;

[0096] After the step of determining a target path based on the current position of the robot, the location of the fire source, and environmental obstacles determined according to the sensor data, the method further includes:

[0097] Step S40, obtaining temperature data and changes in carbon monoxide concentration;

[0098] Step S50, when the change amplitude of the temperature data and / or the carbon monoxide concentration is greater than a preset change threshold, it is determined that the fire in the area where the robot is located is intensifying;

[0099] Step S60, adjusting the fire extinguishing parameters of the robot, wherein the fire extinguishing parameters include at least one of the distance from the fire source, the type of fire extinguisher, the spray intensity of the fire extinguisher, the moving speed or the number of robots.

[0100] Compared with the traditional patrol strategy formulation scheme, the embodiment of the present application uses robots to detect fire environments, plan paths and perform fire extinguishing tasks, which has the function of more scientific and accurate path planning. In addition, the robot can also accurately judge the changes in the current fire situation based on the acquired sensor data.

[0101] It is understandable that when the fire intensifies, the ambient temperature and carbon monoxide concentration will rise sharply. When the robot detects that the temperature and / or carbon monoxide concentration in the environment has risen above the preset change threshold (which can be in percentage or numerical form), it can determine that the fire in the fire environment is intensifying. In this case, it is necessary to adjust the inspection strategy in time to prevent the fire from spreading further. When the fire intensifies, it is necessary to improve the efficiency of fire extinguishing. At the same time, in order to protect its own safety, it is necessary to increase the safe distance between the robot and the fire source when extinguishing the fire.

[0102] In addition, different types of fire extinguishers are used for different combustibles. For example, the combustion of grease mainly produces carbon dioxide and water vapor, while the combustion of wood may produce carbon monoxide, methane, ethylene and other combustible gases, as well as solid or semi-solid products such as tar and charcoal, in addition to carbon dioxide and water vapor, showing a clear difference in the combustion products between the two. When the robot determines the combustibles in the current fire environment based on the data collected by the gas sensor (including the concentration of various gases), it is necessary to select the corresponding type of fire extinguisher. For example, if the main combustible in the fire environment changes from wood to grease, it is necessary to switch from a water gun to a dry powder fire extinguisher or a foam fire extinguisher to achieve a better fire extinguishing effect.

[0103] For example, when the fire intensifies, the safety distance can be increased, the spray intensity of the fire extinguisher can be increased, the moving speed can be increased, or the staff can be prompted to increase the number of robots to control the increasing fire in a timely manner and reduce the losses caused by the fire.

[0104] In the embodiment of the present application, the robot can respond quickly and sensitively to changes in the fire based on sensor data, achieving rapid response and intelligent decision-making, and effectively improving rescue efficiency and safety.

[0105] For ease of understanding, based on the aforementioned application embodiments, refer to Figure 3 A feasible full process of using robots and sensors to realize patrol strategy decision support includes: first, sensor data is obtained based on lidar, ultrasonic sensor, gas sensor and temperature sensor to realize sensor fusion; then data preprocessing is performed, including bandpass filter denoising, mean filter denoising and sensor data calibration; then data fusion processing is performed based on the Kalman filter algorithm, and the fused data is used for fire source detection and environmental perception, including fire source location and fire intensity assessment; then path planning based on the A* algorithm and DWA (Dynamic Window Approac, local path planning algorithm) are used for path planning and navigation, including initialization and cost calculation, node selection and expansion, path generation and backtracking, trajectory prediction, trajectory evaluation and selection, real-time update and adjustment; finally, the robot's decision support functions include: fire source location determination, optimal path planning, fire intensity judgment and measurement, fire extinguisher selection and spray intensity adjustment.

[0106] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the path planning method based on multi-source data of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0107] The present application also provides a path planning device for multi-source data, referring to Figure 4 , the path planning device based on multi-source data includes:

[0108] The data acquisition module 10 is used to acquire sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor, and a temperature sensor provided on the robot;

[0109] A fire source determination module 20, for determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor;

[0110] The path determination module 30 is used to determine a target path based on the current position of the robot, the fire source position and environmental obstacles determined according to the sensor data.

[0111] In one embodiment, the fire source determination module 20 is further configured to:

[0112] Determine the location of a high temperature area having a temperature higher than a preset temperature threshold according to the temperature conditions of each area in the infrared image data;

[0113] According to the carbon monoxide concentration of each area in the carbon monoxide concentration data, determine the location of a high-concentration area where the carbon monoxide concentration is higher than a preset concentration threshold;

[0114] The location of the fire source is determined based on the location of the high temperature area and the location of the high concentration area.

[0115] In one embodiment, the path determination module 30 is further configured to:

[0116] Initializing the current position into an open list to obtain a current node, wherein the open list includes a plurality of neighboring nodes adjacent to the current node in the area where the robot is located;

[0117] Determine the cost values ​​corresponding to the current position and each neighbor node in the open list, and set the neighbor node with the lowest cost value as the updated current node;

[0118] When the current node is not the fire source location, returning to the step of determining the cost values ​​corresponding to the current location to each of the neighboring nodes in the open list, and selecting the neighboring node with the lowest cost value to update the current node;

[0119] When the current node is the fire source location, a target path is generated according to each current node generated historically;

[0120] As the robot moves, the target path is updated in real time based on environmental obstacles determined by the sensor data.

[0121] In one embodiment, the path determination module 30 is further configured to:

[0122] Predicting multiple motion trajectories of the robot within a preset time period in the future based on the current speed and acceleration of the robot and obstacles;

[0123] Calculate the cost value corresponding to each trajectory, and select the trajectory with the lowest cost value as the update path within the preset time length;

[0124] The update path is used for the robot to generate corresponding motion control instructions to update the current speed, acceleration and current position of the robot.

[0125] In one embodiment, the path planning device based on multi-source data further includes a preprocessing module, and the preprocessing module is used to:

[0126] Removing noise from each of the sensor data by using a preset bandpass filtering algorithm and a mean filtering algorithm;

[0127] Calibrate the infrared camera, laser radar, ultrasonic sensor, gas sensor and temperature sensor respectively to obtain corresponding deviation values;

[0128] Adjusting the denoised sensor data based on the deviation values ​​respectively corresponding to the sensors to obtain updated sensor data;

[0129] The step of determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor is performed based on the updated sensor data.

[0130] In one embodiment, the sensor data includes at least temperature data and carbon monoxide concentration data, and the path planning device based on multi-source data further includes a strategy adjustment module, which is used to:

[0131] Acquiring the temperature data and the change of the carbon monoxide concentration;

[0132] When the change amplitude of the temperature data and / or the carbon monoxide concentration is greater than a preset change threshold, it is determined that the fire in the area where the robot is located is intensifying;

[0133] Adjust the fire extinguishing parameters of the robot, wherein the fire extinguishing parameters include at least one of the distance from the fire source, the type of fire extinguisher, the spray intensity of the fire extinguisher, the moving speed or the number of robots.

[0134] The path planning device based on multi-source data provided by the present application adopts the path planning method based on multi-source data in the above embodiment, which can solve the technical problem that the current inspection strategy is difficult to efficiently and accurately determine the optimal action path. Compared with the prior art, the beneficial effects of the path planning device based on multi-source data provided by the present application are the same as the beneficial effects of the path planning method based on multi-source data provided by the above embodiment, and the other technical features of the path planning device based on multi-source data are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0135] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the path planning method based on multi-source data in the above-mentioned embodiment 1.

[0136] Reference below Figure 5, which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0137] like Figure 5 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0138] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0139] The electronic device provided by the present application adopts the path planning method based on multi-source data in the above embodiment, which can solve the technical problem that the current inspection strategy is difficult to efficiently and accurately determine the optimal action path. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the path planning method based on multi-source data provided in the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0140] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0142] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the path planning method based on multi-source data in the above-mentioned embodiment.

[0143] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0144] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0145] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: obtains sensor data from an infrared camera, a lidar, an ultrasonic sensor, a gas sensor and a temperature sensor provided on the robot; determines the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor; determines the target path based on the current position of the robot, the location of the fire source and the environmental obstacles determined according to the sensor data.

[0146] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0149] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned path planning method based on multi-source data, and can solve the technical problem that the current inspection strategy is difficult to efficiently and accurately determine the optimal action path. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the path planning method based on multi-source data provided by the above-mentioned embodiment, and will not be repeated here.

[0150] The present application also provides a computer program product, including a computer program, which implements the steps of the path planning method based on multi-source data as described above when executed by a processor.

[0151] The computer program product provided by this application can solve the technical problem that the current inspection strategy is difficult to efficiently and accurately determine the optimal action path. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the path planning method based on multi-source data provided by the above embodiment, which will not be repeated here.

[0152] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A path planning method based on multi-source data, characterized in that: Applied to robots, the path planning method based on multi-source data includes: Obtain sensor data from infrared cameras, lidars, ultrasonic sensors, gas sensors, and temperature sensors installed on the robot; Determine the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor; A target path is determined based on the current position of the robot, the location of the fire source, and environmental obstacles determined based on the sensor data.

2. The path planning method based on multi-source data according to claim 1, characterized in that: The step of determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor comprises: Determine the location of a high temperature area having a temperature higher than a preset temperature threshold according to the temperature conditions of each area in the infrared image data; According to the carbon monoxide concentration of each area in the carbon monoxide concentration data, determine the location of a high-concentration area where the carbon monoxide concentration is higher than a preset concentration threshold; The location of the fire source is determined based on the location of the high temperature area and the location of the high concentration area.

3. The path planning method based on multi-source data according to claim 1, characterized in that: The step of determining a target path based on the current position of the robot, the fire source position, and environmental obstacles determined according to the sensor data comprises: Initializing the current position into an open list to obtain a current node, wherein the open list includes a plurality of neighboring nodes adjacent to the current node in the area where the robot is located; Determine the cost values ​​corresponding to the current position and each neighbor node in the open list, and set the neighbor node with the lowest cost value as the updated current node; When the current node is not the fire source location, returning to the step of determining the cost values ​​corresponding to the current location to each of the neighboring nodes in the open list, and selecting the neighboring node with the lowest cost value to update the current node; When the current node is the fire source location, a target path is generated according to each current node generated historically; As the robot moves, the target path is updated in real time based on environmental obstacles determined by the sensor data.

4. The path planning method based on multi-source data as claimed in claim 3, characterized in that: The step of updating the target path in real time based on the environmental obstacles determined by the sensor data comprises: Predicting multiple motion trajectories of the robot within a preset time period in the future based on the current speed and acceleration of the robot and obstacles; Calculate the cost value corresponding to each trajectory, and select the trajectory with the lowest cost value as the update path within the preset time length; The update path is used for the robot to generate corresponding motion control instructions to update the current speed, acceleration and current position of the robot.

5. The path planning method based on multi-source data according to claim 1, characterized in that: After the step of acquiring sensor data from the infrared camera, laser radar, ultrasonic sensor, gas sensor and temperature sensor provided on the robot, the method further comprises: Removing noise from each of the sensor data by using a preset bandpass filtering algorithm and a mean filtering algorithm; Calibrate the infrared camera, laser radar, ultrasonic sensor, gas sensor and temperature sensor respectively to obtain corresponding deviation values; Adjusting the denoised sensor data based on the deviation values ​​respectively corresponding to the sensors to obtain updated sensor data; The step of determining the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor is performed based on the updated sensor data.

6. The path planning method based on multi-source data according to claim 1, characterized in that: The sensor data includes at least temperature data and carbon monoxide concentration data; After the step of determining a target path based on the current position of the robot, the location of the fire source, and environmental obstacles determined according to the sensor data, the method further includes: Acquiring the temperature data and the change of the carbon monoxide concentration; When the change amplitude of the temperature data and / or the carbon monoxide concentration is greater than a preset change threshold, it is determined that the fire in the area where the robot is located is intensifying; Adjust the fire extinguishing parameters of the robot, wherein the fire extinguishing parameters include at least one of the distance from the fire source, the type of fire extinguisher, the spray intensity of the fire extinguisher, the moving speed or the number of robots.

7. A path planning device based on multi-source data, characterized in that: The path planning device based on multi-source data includes: A data acquisition module, used to acquire sensor data from an infrared camera, a laser radar, an ultrasonic sensor, a gas sensor, and a temperature sensor provided on the robot; A fire source determination module, used to determine the location of the fire source according to the infrared image data corresponding to the infrared camera and the carbon monoxide concentration data corresponding to the gas sensor; A path determination module is used to determine a target path based on the current position of the robot, the fire source position, and environmental obstacles determined according to the sensor data.

8. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the path planning method based on multi-source data as claimed in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the path planning method based on multi-source data as described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the path planning method based on multi-source data are implemented as claimed in any one of claims 1 to 6.

Images