Emergency response method based on robot, electronic equipment and storage medium

Abstract

The invention discloses an emergency event response method based on a robot, electronic equipment and a storage medium, and relates to the technical field of robots. The fitness represents the suitability of the emergency robot for executing the emergency task, and then the emergency task is allocated to the target emergency robot with the highest fitness for execution. Therefore, each emergency task is ensured to be matched with the robot executing the task, that is, each emergency task can be ensured to be quickly and correctly executed as much as possible. The robot-based emergency response method provided by the invention improves the autonomous decision making and complex environment adaptability of the emergency system, thereby ensuring that the emergency system can quickly and efficiently complete each emergency task in emergency situations such as fire, earthquake and the like.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a robot-based emergency response method, electronic device, and storage medium. Background Art

[0002] The current emergency response system has certain limitations when facing disasters. Although traditional alarm systems, automatic fire extinguishing devices, simple patrol robots and other equipment can respond to certain emergencies, they lack flexibility, intelligence and real-time response capabilities. Traditional systems lack the ability to make autonomous decisions and adapt to complex environments, and are unable to quickly and efficiently evacuate personnel, distribute relief supplies and perform simple rescue tasks in emergency situations such as fires and earthquakes. Summary of the invention

[0003] The main purpose of this application is to provide a robot-based emergency response method, system, equipment and medium, aiming to solve the technical problems that traditional emergency systems lack autonomous decision-making and complex environment adaptability and are unable to cope with complex emergency rescue work.

[0004] To achieve the above objectives, the present application proposes a robot-based emergency response method, which is applied to an emergency response system. The robot-based emergency response method comprises the following steps:

[0005] For any emergency task to be assigned under the emergency event, determine the adaptability between the emergency task and each emergency robot in the emergency response system;

[0006] The emergency task is assigned to the target emergency robot with the highest fitness among the emergency robots for execution.

[0007] Optionally, the step of determining the adaptability between the emergency task and each emergency robot in the emergency response system includes:

[0008] For any one of the emergency robots, obtaining a first indicator score according to a moving distance between a destination in the emergency task and the emergency robot;

[0009] Determining a second indicator score according to the resource consumption of the emergency task and the idle resources of the emergency robot;

[0010] Determining a third indicator score according to the task response delay of the emergency robot with respect to the emergency task;

[0011] The fitness between the emergency task and the emergency robot is generated based on the first indicator score, the second indicator score, and the third indicator score.

[0012] Optionally, the step of obtaining a first indicator score according to a moving distance between a destination in the emergency task and the emergency robot comprises:

[0013] In the case where the emergency robot has been assigned tasks to be performed, determining the stop position of the emergency robot according to the execution order of the assigned tasks to be performed;

[0014] Planning a moving path for the emergency robot when it moves from the stop position to the destination, and determining the moving distance through the moving path;

[0015] The first indicator score is calculated by the moving distance, wherein the first indicator score is negatively correlated with the moving distance.

[0016] Optionally, the resource consumption includes power consumption and material consumption, and the step of determining the second indicator score according to the resource consumption of the emergency task and the idle resources of the emergency robot includes:

[0017] When the emergency robot has been assigned a task to be performed, predict the amount of electricity and materials required to complete the task to be performed;

[0018] Subtracting the required power from the remaining power of the emergency robot to obtain idle power, and subtracting the required materials from the remaining materials of the emergency robot to obtain idle materials, wherein the idle power and the idle materials constitute the idle resources;

[0019] The second indicator score is calculated based on a first difference between the idle power and the power consumption, and a second difference between the idle materials and the material consumption, wherein the second indicator score is positively correlated with the first difference and the second difference, respectively.

[0020] Optionally, the step of determining a third indicator score according to the task response delay of the emergency robot with respect to the emergency task includes:

[0021] In the case where the emergency robot has been assigned tasks to be performed, determining a priority task among the tasks to be performed whose priority is higher than or equal to the priority of the emergency task;

[0022] The estimated time required to complete the priority task is used as the task response delay, and the third indicator score is determined according to the task response delay, wherein the third indicator score is negatively correlated with the task response delay.

[0023] Optionally, after the step of allocating the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution, the method further includes:

[0024] In the case that the execution condition of the emergency task changes, the emergency task is restored to an emergency task to be assigned and reallocated.

[0025] Optionally, the method further comprises:

[0026] For any detection robot in the emergency response system, the detection robot is an emergency robot that moves at the location corresponding to the emergency event, and the visual scene map and infrared scene map of the location are obtained by the detection robot;

[0027] Fusing the visual scene image and the infrared scene image to obtain a fused image;

[0028] Performing target detection on the fused image by using a preset target detection model to obtain a target detection result;

[0029] Based on the target detection result, feature enhancement is performed on the fused image to obtain an enhanced image;

[0030] An emergency response task is generated based on the enhanced image.

[0031] Optionally, the step of performing feature enhancement on the fused image based on the target detection result to obtain an enhanced image includes:

[0032] For any detected target in the target detection results, feature enhancement is performed on a region or an edge of the detected target in the fused image, and the type of the detected target is marked to obtain the enhanced image.

[0033] In addition, to achieve the above objectives, the present application also proposes an emergency response system, the emergency response system comprising:

[0034] A determination module, for determining, for any emergency task to be assigned under the emergency event, the adaptability between the emergency task and each emergency robot in the emergency response system;

[0035] The allocation module is used to allocate the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution.

[0036] In addition, to achieve the above-mentioned objectives, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the robot-based emergency response method as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the robot-based emergency response method as described above are implemented.

[0038] In addition, to achieve the above objectives, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the robot-based emergency response method as described above.

[0039] One or more technical solutions proposed in this application have at least the following technical effects:

[0040] An embodiment of the present application provides a robot-based emergency event response method, which is applied to an emergency response system. For any emergency task to be assigned under the emergency event, the fitness between the emergency task and the emergency robots in the emergency response system is determined; and the emergency task is assigned to a target emergency robot with the highest fitness among the emergency robots for execution.

[0041] That is, when an emergency occurs, it is usually accompanied by multiple emergency tasks. For example, when a fire occurs, there may be multiple people in different locations at the scene of the accident waiting for rescue, that is, multiple emergency tasks. For any emergency task that needs to be assigned, in the embodiment of the present application, the emergency system will determine the fitness of each emergency robot at the scene of the accident to perform the emergency task, and the fitness characterizes the suitability of the emergency robot to perform the emergency task, and then assign the emergency task to the target emergency robot with the highest fitness to perform. Thereby ensuring that each emergency task is matched with the robot that performs the task, that is, ensuring that each emergency task can be executed quickly and accurately as much as possible. Therefore, the robot-based emergency response method provided in the present application improves the autonomous decision-making and complex environment adaptability of the emergency system, thereby ensuring that the emergency system can quickly and efficiently complete emergency tasks such as personnel evacuation, rescue material distribution and simple rescue in emergency situations such as fires and earthquakes. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 This is a flowchart of the first embodiment of the robot-based emergency response method of the present application;

[0045] Figure 2 This is a flow chart of the second embodiment of the robot-based emergency response method of the present application;

[0046] Figure 3 A schematic diagram of a scenario of an emergency response system in a robot-based emergency response method of this application;

[0047] Figure 4 A schematic diagram of the framework of the emergency response system in the robot-based emergency response method of this application;

[0048] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the robot-based emergency response method in the embodiment of the present application.

[0049] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The current emergency response system has certain limitations when facing disasters. Although traditional alarm systems, automatic fire extinguishing devices, simple patrol robots and other equipment can respond to certain emergencies, they lack flexibility, intelligence and real-time response capabilities. Traditional systems lack the ability to make autonomous decisions and adapt to complex environments, and are unable to quickly and efficiently evacuate personnel, distribute relief supplies and perform simple rescue tasks in emergency situations such as fires and earthquakes.

[0053] The main solution of the embodiment of the present application is: to provide a robot-based emergency event response method, applied to the emergency response system, for any emergency task to be assigned under the emergency event, determine the fitness between the emergency task and the emergency robots in the emergency response system; assign the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution.

[0054] That is, when an emergency occurs, it is usually accompanied by multiple emergency tasks. For example, when a fire occurs, there may be multiple people in different locations at the scene of the accident waiting for rescue, that is, multiple emergency tasks. For any emergency task that needs to be assigned, in the embodiment of the present application, the emergency system will determine the fitness of each emergency robot at the scene of the accident to perform the emergency task, and the fitness characterizes the suitability of the emergency robot to perform the emergency task, and then assign the emergency task to the target emergency robot with the highest fitness to perform. Thereby ensuring that each emergency task is matched with the robot that performs the task, that is, ensuring that each emergency task can be executed quickly and accurately as much as possible. Therefore, the robot-based emergency response method provided in the present application improves the autonomous decision-making and complex environment adaptability of the emergency system, thereby ensuring that the emergency system can quickly and efficiently complete emergency tasks such as personnel evacuation, rescue material distribution and simple rescue in emergency situations such as fires and earthquakes.

[0055] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a cloud server, a robot, or an electronic device that can realize the above functions.

[0056] Based on the above introduction, the present application embodiment provides a robot-based emergency response method, referring to Figure 1 , which is a flow chart of the first embodiment of the robot-based emergency response method of the present application.

[0057] In this embodiment, applied to an emergency response system, the robot-based emergency response method includes steps S10-S20:

[0058] Step S10: for any emergency task to be assigned under the emergency event, determine the adaptability between the emergency task and each emergency robot in the emergency response system.

[0059] It should be noted that the emergency response method of the above-mentioned robot in this application can be applied to the emergency response system. Among them, the emergency response system can be composed of a cloud server and a plurality of emergency robots, and the cloud server is connected to each emergency robot in communication, or, it is composed of a plurality of emergency robots, and each robot communicates with each other, so as to collaborate to complete the emergency task. The above-mentioned emergency events usually refer to events such as fire, earthquake or chemical leakage. It is worth noting that the robot can be equipped with sensors such as vision, infrared, thermal imaging, smoke detection, gas sensor, ultrasonic radar and LIDAR (laser radar) for comprehensive monitoring of the environment around the emergency robot, detection or identification of personnel and objects on the scene, and generation of emergency tasks through the identified personnel and objects. For example, if it is identified that a person is under an object, or the person's posture is lying down or sitting, etc., emergency tasks such as personnel evacuation or personnel rescue tasks can be generated. In addition, in addition to being generated by the robot itself, emergency tasks can also be set by rescuers and published to the emergency event response system, and the emergency response system will then assign emergency tasks to appropriate emergency robots. It is understandable that, since each emergency task to be assigned is substantially the same during the assignment process, in this embodiment, one of the emergency tasks to be assigned will be taken as an example for description.

[0060] Exemplarily, for any emergency task to be assigned under the emergency event, the emergency response system will determine the adaptability between the emergency task and each emergency robot in the emergency response system. The adaptability between the emergency task and the emergency robot can be determined from multiple dimensions. For example, the dimension can be the distance between the destination of the emergency task and the emergency robot. Generally, the closer the distance, the higher the adaptability. In addition, the dimension can also be the matching degree between the resource requirements of the emergency task and the resources owned by the emergency robot. The resources can be the power of the robot, rescue materials (such as masks, rescue bags, etc.) and the functional resources configured by the robot. Similarly, the higher the matching degree, the higher the adaptability.

[0061] Step S20: Allocate the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution.

[0062] Exemplarily, after determining the fitness between the emergency task and each emergency robot, the emergency response system selects the emergency robot with the highest fitness as the target robot, and the target robot performs the emergency task. For example, the target robot can plan its own moving path according to the task destination in the emergency task, and then go to the task destination according to the moving path, and complete the emergency task at the destination.

[0063] In this embodiment, for any emergency task to be assigned under the emergency event, the emergency response system will determine the fitness between the emergency task and the emergency robots in the emergency response system; and assign the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution.

[0064] That is, when an emergency occurs, it is usually accompanied by multiple emergency tasks. For example, when a fire occurs, there may be multiple people in different locations at the scene of the accident waiting for rescue, that is, multiple emergency tasks. For any emergency task that needs to be assigned, in the embodiment of the present application, the emergency system will determine the fitness of each emergency robot at the scene of the accident to perform the emergency task, and the fitness characterizes the suitability of the emergency robot to perform the emergency task, and then assign the emergency task to the target emergency robot with the highest fitness to perform. Thereby ensuring that each emergency task is matched with the robot that performs the task, that is, ensuring that each emergency task can be executed quickly and accurately as much as possible. Therefore, the robot-based emergency response method provided in the present application improves the autonomous decision-making and complex environment adaptability of the emergency system, thereby ensuring that the emergency system can quickly and efficiently complete emergency tasks such as personnel evacuation, rescue material distribution and simple rescue in emergency situations such as fires and earthquakes.

[0065] In a feasible implementation manner, the step S20 of determining the adaptability between the emergency task and each emergency robot in the emergency response system includes steps S21-S24:

[0066] Step S21: for any one of the emergency robots, a first indicator score is obtained according to a moving distance between a destination in the emergency task and the emergency robot.

[0067] Step S22: determining a second indicator score according to the resource consumption of the emergency task and the idle resources of the emergency robot.

[0068] Step S23, determining a third indicator score according to the task response delay of the emergency robot with respect to the emergency task.

[0069] Step S24: generating the fitness between the emergency task and the emergency robot based on the first indicator score, the second indicator score, and the third indicator score.

[0070] It should be noted that the process of determining the adaptability between each emergency robot and the above-mentioned emergency tasks is basically similar, so in this embodiment, one of the emergency robots will be taken as an example for description.

[0071] Exemplarily, for any one of the emergency robots, in this embodiment, the adaptability between the emergency robot and the above-mentioned emergency task will be determined through three indicator dimensions. For the dimension of distance, that is, the first indicator dimension, the emergency response system obtains the first indicator score according to the moving distance between the destination in the emergency task and the emergency robot. When the emergency robot moves in the corresponding place of the emergency event, it can use laser radar and camera to perform real-time environment modeling and build a three-dimensional map, and each emergency robot can share the local three-dimensional map constructed by each other, so as to form a complete three-dimensional map of the corresponding place of the emergency event. On the basis of the shared map, the robot can use the reinforcement learning algorithm for path planning, can adaptively avoid obstacles, and generate the shortest path to the destination, and then calculate the moving distance through the path and generate the first indicator score, and the first indicator score can be inversely proportional to the moving distance, that is, the shorter the moving distance, the higher the first indicator score. For the indicator dimension of resources, that is, the second indicator dimension, the emergency response system can determine it according to the resource consumption of the emergency task and the idle resources of the emergency robot. It is worth noting that, under normal circumstances, robots with idle resources greater than resource consumption will be screened to generate the second indicator score. And the more idle resources there are, the higher the second indicator score is. As for the indicator dimension of task response delay, that is, the third indicator dimension, it can be determined according to the emergency tasks that have been assigned to the emergency robot. For example, the time required to complete the emergency tasks that have been assigned to the emergency robot can be determined, and this time is used as the above-mentioned task response delay, and then the third indicator score is determined according to the task response delay, wherein the smaller the task response delay, the higher the third indicator score. After determining the first indicator score, the second indicator score and the third indicator score, the fitness between the emergency task and the emergency robot is calculated by combining the first indicator score, the second indicator score and the third indicator score. For example, a pre-set calculation formula can be used to perform a weighted sum calculation on the first to third indicator scores to obtain the fitness between the emergency task and the emergency robot. For example, the calculation formula can be:

[0072] Fitness = α1·A+α2·B+α3·C

[0073] In the formula, α1 to α3 are weights, A is the score of the first indicator, B is the score of the second indicator, and C is the score of the third indicator.

[0074] In a feasible implementation manner, the step S21 of obtaining a first indicator score according to the moving distance between the destination in the emergency task and the emergency robot includes steps S211-S213:

[0075] Step S211, when the emergency robot has been assigned tasks to be performed, the stopping position of the emergency robot is determined according to the execution order of the assigned tasks to be performed.

[0076] Step S212: planning a moving path of the emergency robot when it moves from the stop position to the destination, and determining the moving distance through the moving path.

[0077] Step S213: obtaining the first indicator score by calculating the moving distance, wherein the first indicator score is negatively correlated with the moving distance.

[0078] It should be noted that when calculating the first indicator score, it can be set according to different situations. For example, if the emergency robot has been assigned a task to be performed, that is, if there are still unfinished emergency tasks (tasks to be performed) on the emergency robot, the stop position of the emergency robot can be determined according to the execution order of the assigned tasks to be performed. For example, according to the execution order, the destination of the last task to be performed is used as the stop position of the emergency robot. On the contrary, if the emergency robot is not assigned a task to be performed, the location of the emergency robot can be directly used as the above-mentioned stop position. According to the three-dimensional map of the corresponding place of the emergency event, the moving path of the emergency robot when it moves from the stop position to the destination is planned, so as to obtain the moving distance required by the emergency robot, wherein the path planning part of the robot is currently relatively mature, so it will not be repeated here. After obtaining the moving distance, the first indicator score is calculated according to the moving distance, and the first indicator score is negatively correlated with the moving distance, that is, the shorter the moving distance, the higher the first indicator score.

[0079] In a feasible implementation manner, the resource consumption includes power consumption and material consumption, and the step S22 of determining the second indicator score according to the resource consumption of the emergency task and the idle resources of the emergency robot includes steps S221-S223:

[0080] Step S221, when the emergency robot has been assigned a task to be performed, predict the amount of electricity and materials required to complete the task to be performed.

[0081] Step S222, subtracting the required power from the remaining power of the emergency robot to obtain idle power, and subtracting the required supplies from the remaining supplies of the emergency robot to obtain idle supplies, wherein the idle power and the idle supplies constitute the idle resources.

[0082] Step S223, calculating the second indicator score according to the first difference between the idle power and the power consumption, and the second difference between the idle materials and the material consumption, wherein the second indicator score is positively correlated with the first difference and the second difference, respectively.

[0083] It should be noted that in this embodiment, resource consumption can be divided into power consumption and material consumption, such as material consumption can include first aid kits, fire extinguishers, oxygen cylinders, etc., which will not be repeated here. Similarly, when the emergency robot has been assigned a task to be performed, the power and materials required to complete the task to be performed are predicted. For example, the required power can be estimated by the moving path planned for the task to be performed, and the longer the total moving path of the task to be performed, the more power is required. As for the required materials, the required materials can be obtained by the sum of the material consumption of each task in the task to be performed. The idle power can be obtained by subtracting the required power from the remaining power of the above-mentioned emergency robot. Then, the idle materials can be obtained by subtracting the above-mentioned required materials from the remaining materials of the emergency robot. The idle power and idle materials obtained can constitute the above-mentioned idle resources. On the contrary, if the emergency robot is not assigned a task to be performed, the remaining power of the emergency robot can be directly used as the idle power and the remaining materials as the idle materials.

[0084] The first difference is obtained by subtracting the determined idle power from the power consumption, and the second difference is obtained by subtracting the determined idle materials from the material consumption. The second indicator score is then calculated based on the first difference and the second difference, which can be substituted into the preset second indicator score calculation formula. Among them, the second indicator score is proportional to the first difference and the second difference, respectively, that is, the larger the first difference and the second difference, the higher the second indicator score. It is worth noting that in actual applications, emergency robots whose first difference and second difference are both greater than 0 can be selected to calculate the second indicator score, so as to ensure that the above emergency tasks can be executed and completed.

[0085] In a feasible implementation manner, the step S23 of determining the third indicator score according to the task response delay of the emergency robot to the emergency task includes steps S231-S232:

[0086] Step S231, when the emergency robot has been assigned tasks to be executed, determine a priority task among the tasks to be executed whose priority is higher than or equal to the priority of the emergency task.

[0087] Step S232: taking the estimated duration for completing the priority task as the task response delay, and determining the third indicator score according to the task response delay, wherein the third indicator score is negatively correlated with the task response delay.

[0088] Similarly, in this embodiment, it will also be set according to the situation. For example, when the emergency robot has been assigned a task to be performed, a priority task with a priority higher than or equal to the priority of the emergency task is determined. It should be noted that in this embodiment, there can be many types of tasks to be performed, such as personnel evacuation, fire fighting, emergency material distribution, obstacle removal, and personnel rescue, and different types of tasks can be set with different priorities. For example, in practical applications, the priority of personnel rescue can be set higher. The estimated time to complete the priority task is used as the task response delay. For example, for a certain priority task, the moving time can be calculated based on the moving distance of the priority task and then added to the operation time to obtain the estimated time, wherein the operation time can be obtained by averaging the time required for the emergency robot to complete the operation in history. Then the estimated time to complete the priority task is used as the above-mentioned task response delay. In addition, if the emergency robot is not assigned a task to be performed, the task response delay can be directly determined to be zero. Finally, the third indicator score is calculated based on the task response delay, and the third indicator score is negatively correlated with the task response delay, that is, the lower the task response delay, the higher the third indicator score.

[0089] In a feasible implementation manner, after the step S20 of allocating the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution, the method further includes step S30:

[0090] Step S30: when the execution condition of the emergency task changes, the emergency task is restored to an emergency task to be assigned and reallocated.

[0091] It should be noted that after an emergency task is assigned to an emergency robot, the emergency robot may be unable to complete the emergency task due to various reasons, or because obstacles unexpectedly appear on the emergency robot's moving path, assigning the emergency task to the emergency robot may not be the best choice. Therefore, when the execution conditions of the emergency task change, the emergency robot will be restored to the emergency task to be assigned and reallocated.

[0092] The types of changes in the above-mentioned execution conditions include at least one of the following: the target emergency robot that performs the emergency task fails or changes in state; the task route of the emergency task changes, wherein the change in the task route includes changes caused by the appearance of new obstacles in the corresponding place of the emergency event, and changes caused by changes in the order of task execution; the execution time of the emergency task has timed out. It is understandable that if the target emergency robot fails or changes in state (such as low power), it means that the target emergency robot may not be able to perform the emergency task normally, so it is necessary to allocate other robots to perform. If the task route of the emergency task changes, it means that the emergency task may not be allocated to the target emergency robot in the best way, and accordingly, it needs to be reallocated. In addition, when the execution time of the emergency task is overtime, if the execution time exceeds the predicted execution time, it means that unknown abnormalities may occur in the execution process of the emergency task, so the emergency task also needs to be allocated. It is understandable that for the specific content of the change in the execution conditions, those skilled in the art can refer to the above examples for setting, so it will not be repeated here.

[0093] Reference Figure 2 , is a flow chart of a second embodiment of the robot-based emergency response method of the present application, which is proposed based on the first embodiment of the robot-based emergency response method of the present application. In this embodiment, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be repeated later. The method also includes steps S100-S500:

[0094] Step S100, for any detection robot in the emergency response system, the detection robot is an emergency robot moving at the place corresponding to the emergency event, and the visual scene map and infrared scene map of the place are obtained by the detection robot;

[0095] Step S200, fusing the visual scene image and the infrared scene image to obtain a fused image;

[0096] Step S300, performing target detection on the fused image by using a preset target detection model to obtain a target detection result;

[0097] Step S400, based on the target detection result, feature enhancement is performed on the fused image to obtain an enhanced image;

[0098] Step S500: Generate an emergency response task based on the enhanced image.

[0099] It should be noted that, in this embodiment, the emergency response system also includes a detection robot, which refers to an emergency robot that moves to the corresponding place of the above-mentioned emergency event, such as a robot that specifically performs detection tasks, or an emergency robot that performs emergency tasks in the corresponding place of the emergency event. Since the detection process of each detection robot is basically the same, in this embodiment, the execution process of one of the detection robots will be used as an example for description.

[0100] Exemplarily, for any detection robot of the emergency response system, the detection robot can be used to obtain a visual scene map and an infrared scene map of the site. For example, the visual scene map can be obtained by a visual camera configured on the detection machine, and the infrared scene map can be collected by a configured infrared sensor.

[0101] The acquired visual scene map and infrared scene image are fused, that is, the features on the two images are fused to ensure that no features are lost in a complex scene environment. For example, a fire scene may be accompanied by a large amount of smoke and block the scene, so that a simple visual scene map may not be able to identify the desired content. For example, people who need rescue may be blocked by thick smoke, resulting in the inability to identify the people. The infrared scene image acquired by the infrared sensor can include the features of the blocked people to a certain extent, thereby compensating for the information that may be blocked on the visual scene map.

[0102] It should be noted that before fusion, the visual image and the infrared image can be denoised and standardized to reduce sensor noise. At the same time, the two images can be aligned to ensure that the visual scene map and the infrared scene map are completely matched in space and eliminate the deviation caused by different sensor angles. For example, the image registration algorithm (such as feature point-based method, mutual information method, etc.) is used to register the visual and infrared images so that the two images are consistent in spatial position, and geometric transformation is performed according to the different viewing angles of the sensor to ensure the synchronous alignment of the visual image and the infrared image.

[0103] After alignment, the aligned visual image and infrared image are fused in the channel dimension to generate a multi-channel image, retaining their respective important features. Image fusion can be achieved using a simple weighted method (i.e., weighted sum of the pixel values ​​of the pixels at the same position in the two images) or a more complex convolutional neural network (CNN). For example, a convolutional neural network can be used to extract features from the fused multi-channel image to obtain the key features of each image (such as edges, corners, textures, etc.). The feature fusion is further optimized through the fusion network to ensure that the data from different sensors can complement each other and improve the image quality, thereby ensuring that the target can be accurately detected.

[0104] After obtaining the fused image, the preset target detection model is used to perform target detection on the fused image. For example, the preset target detection model can be a deep learning model such as YOLO (You Only Look Once, target detection algorithm) and Faster R-CNN (Fast Region-based Convolutional Neural Network, target detection model), etc. The fused image is then input into the preset target detection model, and target detection is performed on the fused image to identify people, obstacles and other key objects in the image, and the target is classified (such as people, fire extinguishers, etc.) to obtain the target detection result.

[0105] The target detection results obtained can be used to enhance the features of the fused image again to obtain an enhanced image. For example, the enhancement process can include post-processing the image according to the target detection and classification results (such as edge enhancement, brightness adjustment, etc.) to optimize the visual effect, and can also label the detected targets, such as marking a certain area on the fused image as a person and a certain area as an obstacle. The enhanced image and the corresponding scene template (for example, the scene template can be, there is a fire at present, if you are a rescuer, what should you do when facing the scene in the picture. At the same time, the scene template can also provide several emergency tasks for the large model to choose) can be input into the large language model, so that the large language model can describe the content in the image and give the emergency response task. At the same time, the enhanced image of the monitored personnel can also be synchronously output to the relevant rescue personnel, and the relevant rescue personnel will formulate emergency tasks.

[0106] Reference Figure 3 , is a schematic diagram of the emergency response system in this application. Figure 3 As shown, the figure includes multiple emergency robots. Each emergency robot can take images of the scene and upload them to the cloud server. The cloud server can generate emergency tasks based on the received images and dispatch the emergency tasks to each emergency robot for execution.

[0107] In a feasible implementation manner, the step S400 performs feature enhancement on the fused image based on the target detection result to obtain an enhanced image, including step S410:

[0108] Step S410 , for any detected target in the target detection result, feature enhancement is performed on the region or edge of the detected target in the fused image, and the type of the detected target is marked to obtain the enhanced image.

[0109] Exemplarily, for any detected target in the target detection results, the features of the region or contour edge of the detected target in the fused image can be enhanced. For example, the region or contour edge can be highlighted, and the type of the detected target can be marked, such as personnel or obstacles, so that the large model can accurately generate emergency tasks based on the enhanced image.

[0110] This application also provides an emergency response system, referring to Figure 4 , the emergency response system comprises:

[0111] A determination module 10 is used to determine the adaptability between any emergency task to be assigned under the emergency event and each emergency robot in the emergency response system;

[0112] The allocation module 20 is used to allocate the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution.

[0113] Optionally, the determining module 10 is further configured to:

[0114] For any one of the emergency robots, obtaining a first indicator score according to a moving distance between a destination in the emergency task and the emergency robot;

[0115] Determining a second indicator score according to the resource consumption of the emergency task and the idle resources of the emergency robot;

[0116] Determining a third indicator score according to the task response delay of the emergency robot with respect to the emergency task;

[0117] The fitness between the emergency task and the emergency robot is generated based on the first indicator score, the second indicator score, and the third indicator score.

[0118] Optionally, the determining module 10 is further configured to:

[0119] In the case where the emergency robot has been assigned tasks to be performed, determining the stop position of the emergency robot according to the execution order of the assigned tasks to be performed;

[0120] Planning a moving path for the emergency robot when it moves from the stop position to the destination, and determining the moving distance through the moving path;

[0121] The first indicator score is calculated by the moving distance, wherein the first indicator score is negatively correlated with the moving distance.

[0122] Optionally, the resource consumption includes power consumption and material consumption, and the determining module 10 is further used to:

[0123] When the emergency robot has been assigned a task to be performed, predict the amount of electricity and materials required to complete the task to be performed;

[0124] Subtracting the required power from the remaining power of the emergency robot to obtain idle power, and subtracting the required materials from the remaining materials of the emergency robot to obtain idle materials, wherein the idle power and the idle materials constitute the idle resources;

[0125] The second indicator score is calculated based on a first difference between the idle power and the power consumption, and a second difference between the idle materials and the material consumption, wherein the second indicator score is positively correlated with the first difference and the second difference, respectively.

[0126] Optionally, the determining module 10 is further configured to:

[0127] In the case where the emergency robot has been assigned tasks to be performed, determining a priority task among the tasks to be performed whose priority is higher than or equal to the priority of the emergency task;

[0128] The estimated time required to complete the priority task is used as the task response delay, and the third indicator score is determined according to the task response delay, wherein the third indicator score is negatively correlated with the task response delay.

[0129] Optionally, the allocation module 20 is further used for:

[0130] In the case that the execution condition of the emergency task changes, the emergency task is restored to an emergency task to be assigned and reallocated.

[0131] Optionally, the emergency response system further includes a generating module 30, and the generating module 30 is used to:

[0132] For any detection robot in the emergency response system, the detection robot is an emergency robot that moves at the location corresponding to the emergency event, and the visual scene map and infrared scene map of the location are obtained by the detection robot;

[0133] Fusing the visual scene image and the infrared scene image to obtain a fused image;

[0134] Performing target detection on the fused image by using a preset target detection model to obtain a target detection result;

[0135] Based on the target detection result, feature enhancement is performed on the fused image to obtain an enhanced image;

[0136] An emergency response task is generated based on the enhanced image.

[0137] Optionally, the generating module 30 is further used for:

[0138] For any detected target in the target detection results, feature enhancement is performed on a region or an edge of the detected target in the fused image, and the type of the detected target is marked to obtain the enhanced image.

[0139] The emergency response system provided by the present application adopts the emergency event response method of the robot in the above embodiment, which can solve the technical problem that the traditional emergency system lacks autonomous decision-making and complex environment adaptability and cannot cope with complex emergency rescue work. Compared with the prior art, the beneficial effects of the emergency response system provided by the present application are the same as the beneficial effects of the emergency event response method of the robot provided by the above embodiment, and the other technical features of the emergency response system are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0140] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the robot-based emergency response method in the above-mentioned embodiment one.

[0141] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to cloud servers, robots, and computers. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0142] like Figure 5As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0143] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0144] The electronic device provided by the present application adopts the robot-based emergency response method in the above embodiment, which can solve the technical problem that the traditional emergency system lacks autonomous decision-making and complex environment adaptability and cannot cope with complex emergency rescue work. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the robot-based emergency response method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0145] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0146] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0147] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the robot-based emergency response method in the above-mentioned embodiment.

[0148] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0149] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0150] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device:

[0151] For any emergency task to be assigned under the emergency event, determine the adaptability between the emergency task and each emergency robot in the emergency response system;

[0152] The emergency task is assigned to the target emergency robot with the highest fitness among the emergency robots for execution.

[0153] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0155] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0156] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned robot-based emergency response method, which can solve the technical problem that the traditional emergency system lacks autonomous decision-making and complex environment adaptability and cannot cope with complex emergency rescue work. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the robot-based emergency response method provided by the above-mentioned embodiment, and will not be repeated here.

[0157] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned robot-based emergency response method when executed by a processor.

[0158] The computer program product provided by this application can solve the technical problem that the traditional emergency system lacks autonomous decision-making and complex environment adaptability and cannot cope with complex emergency rescue work. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the robot-based emergency response method provided by the above embodiment, which will not be repeated here.

[0159] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A robot-based emergency response method, characterized in that: Applied to an emergency response system, the robot-based emergency response method includes: For any emergency task to be assigned under the emergency event, determine the adaptability between the emergency task and each emergency robot in the emergency response system; The emergency task is assigned to the target emergency robot with the highest fitness among the emergency robots for execution.

2. The robot-based emergency response method according to claim 1, characterized in that: The step of determining the adaptability between the emergency task and each emergency robot in the emergency response system comprises: For any one of the emergency robots, obtaining a first indicator score according to a moving distance between a destination in the emergency task and the emergency robot; Determining a second indicator score according to the resource consumption of the emergency task and the idle resources of the emergency robot; Determining a third indicator score according to the task response delay of the emergency robot with respect to the emergency task; The fitness between the emergency task and the emergency robot is generated based on the first indicator score, the second indicator score, and the third indicator score.

3. The robot-based emergency response method according to claim 2, characterized in that: The step of obtaining a first indicator score according to the moving distance between the destination in the emergency task and the emergency robot comprises: In the case where the emergency robot has been assigned tasks to be performed, determining the stop position of the emergency robot according to the execution order of the assigned tasks to be performed; Planning a moving path for the emergency robot when it moves from the stop position to the destination, and determining the moving distance through the moving path; The first indicator score is calculated by the moving distance, wherein the first indicator score is negatively correlated with the moving distance.

4. The robot-based emergency response method according to claim 2, characterized in that: The resource consumption includes power consumption and material consumption. The step of determining the second indicator score according to the resource consumption of the emergency task and the idle resources of the emergency robot includes: When the emergency robot has been assigned a task to be performed, predict the amount of electricity and materials required to complete the task to be performed; Subtracting the required power from the remaining power of the emergency robot to obtain idle power, and subtracting the required materials from the remaining materials of the emergency robot to obtain idle materials, wherein the idle power and the idle materials constitute the idle resources; The second indicator score is calculated based on a first difference between the idle power and the power consumption, and a second difference between the idle materials and the material consumption, wherein the second indicator score is positively correlated with the first difference and the second difference, respectively.

5. The robot-based emergency response method according to claim 2, characterized in that: The step of determining the third indicator score according to the task response delay of the emergency robot to the emergency task comprises: In the case where the emergency robot has been assigned tasks to be performed, determining a priority task among the tasks to be performed whose priority is higher than or equal to the priority of the emergency task; The estimated time required to complete the priority task is used as the task response delay, and the third indicator score is determined according to the task response delay, wherein the third indicator score is negatively correlated with the task response delay.

6. The robot-based emergency response method according to claim 1, characterized in that: After the step of allocating the emergency task to the target emergency robot with the highest fitness among the emergency robots for execution, the method further includes: In the case that the execution condition of the emergency task changes, the emergency task is restored to an emergency task to be assigned and reallocated.

7. The robot-based emergency response method according to claim 1, characterized in that: The method further comprises: For any detection robot in the emergency response system, the detection robot is an emergency robot that moves at the location corresponding to the emergency event, and the visual scene map and infrared scene map of the location are obtained by the detection robot; Fusing the visual scene image and the infrared scene image to obtain a fused image; Performing target detection on the fused image by using a preset target detection model to obtain a target detection result; Based on the target detection result, feature enhancement is performed on the fused image to obtain an enhanced image; An emergency response task is generated based on the enhanced image.

8. The robot-based emergency response method according to claim 7, characterized in that: The step of performing feature enhancement on the fused image based on the target detection result to obtain an enhanced image comprises: For any detected target in the target detection results, feature enhancement is performed on a region or an edge of the detected target in the fused image, and the type of the detected target is marked to obtain the enhanced image.

9. An electronic device, characterized in that: The electronic device includes: a memory, a processor, and a robot-based emergency response program stored in the memory and executable on the processor. When the robot-based emergency response program is executed by the processor, the steps of the robot-based emergency response method as described in any one of claims 1 to 8 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium is a computer-readable storage medium, on which a robot-based emergency response program is stored. When the robot-based emergency response program is executed by a processor, the steps of the robot-based emergency response method as described in any one of claims 1 to 8 are implemented.

Images