Remote and semi-autonomous method and system for controlling firefighting and emergency robot

Abstract

The invention discloses a remote semi-automatic control method and system for a fire-fighting emergency robot, and the method comprises the steps: responding to a search instruction issued by an operator through a control end, and issuing a search task package to the fire-fighting emergency robot; the fire-fighting emergency robot executes a search action according to the search task package and returns scene state information to the control end for display; when the fire-fighting emergency robot detects any task object, marking the task object in the scene state information and displaying the task object to an operator; when receiving a task instruction of an operator for a task object, the control end selects an initial task package, requests task conditions from the operator, constructs a target task package according to the task conditions, and issues the target task package to the fire-fighting emergency robot; and the fire-fighting emergency robot autonomously plans and executes the target action according to the target task package. Therefore, semi-automatic control over the fire-fighting emergency robot is achieved, and the response flexibility of the robot in an emergency scene is effectively improved through the mode of combining user decision making with automatic planning execution of the robot.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202411230134.2 (22) Application Date 2024.09.03 (30) Priority Data 63 / 641,662 2024.05.02 US (71) Applicant: Hong Kong Polytechnic University Address: Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong, China (72) Inventors: Huang Xinyan, Wang Meng, Zhang Xiaoning, Zhang Yuxin (74) Patent Agency: Beijing J&J Intellectual Property Agency Co., Ltd. 11227 Patent Attorney: Hu Qingxin (51) Int.Cl. B25J 9 / 16 (2006.01) (54) Invention Title: A Remote Semi-Automatic Control Method and System for a Fire Emergency Robot (57) Abstract: This invention discloses a remote semi-automatic control method and system for a fire emergency robot. The control terminal responds to search commands issued by the operator and sends a search task package to the fire emergency robot. The fire emergency robot executes search actions according to the search task package and returns scene status information to the control terminal for display. When the fire emergency robot detects any task object, it marks the task object in the scene status information and displays it to the operator. When the control terminal receives a task command from the operator for a task object, it selects an initial task package and requests task conditions from the operator. Based on the task conditions, it constructs a target task package and sends it to the fire emergency robot. The fire emergency robot autonomously plans and executes the target actions according to the target task package. This achieves semi-automatic control of the fire emergency robot, effectively improving the robot's flexibility in emergency situations through a combination of user decision-making and automatic robot planning and execution. Claims 3 pages, Description 13 pages, Drawings 6 pages, CN 120886234 A 2025.11.04 CN 1 20 88 62 34 A 1. A remote semi-automatic control method for a fire emergency robot, characterized in that it involves a control terminal and a fire emergency robot connected by remote communication, the method comprising: the control terminal responding to a search command and sending a search task package to the fire emergency robot; the fire emergency robot performing a search action according to the search task package and returning scene status information to the control terminal for display; when the fire emergency robot detects any task object, marking the task object in the scene status information; when the control terminal receives a task package command for the task object, selecting an initial task package and setting task conditions, constructing a target task package and sending it to the fire emergency robot; the fire emergency robot performing a target action according to the target task package. 2. The method according to claim 1, characterized in that the method further comprises: the control terminal responding to a task package creation command and selecting multiple task actions from a preset action library;The control terminal responds to the requirement setting instruction, selects a pending action matching the requirement setting instruction from multiple task actions, and creates a condition setting page; the control terminal responds to the sorting operation instruction, sorts all the task actions and the pending actions, and generates a candidate task package. 3. The method according to claim 1, characterized in that, when the control terminal receives a task package instruction for the task object, it selects an initial task package and sets task conditions, constructs a target task package and sends it to the fire emergency robot, including: when the control terminal receives a task package instruction for the task object, it selects the initial task package specified by the task package instruction from multiple candidate task packages; the control terminal displays the condition setting page corresponding to the initial task package in a predetermined display area of ​​the scene status information; the control terminal responds to the condition information entered on the condition setting page, generates a target task package and sends it to the fire emergency robot. 4. The method according to claim 3, characterized in that, if the task object is a fire source target, the initial task package is a fire extinguishing task package, and the control terminal responds to the condition information input on the condition setting page, generates a target task package and sends it to the fire emergency robot, including: the control terminal responds to the first input trigger, selects the corresponding fire extinguishing agent limitation condition on the condition setting page, and generates a target task package; the control terminal sends the target task package to the fire emergency robot, so that the fire emergency robot selects a fire extinguishing agent that meets the fire extinguishing agent limitation condition and performs a fire extinguishing action. 5. The method according to claim 4, characterized in that, the control terminal is further provided with a manipulation device, further including: generating a jet parabolic trajectory corresponding to the fire emergency robot in the scene status information through the manipulation device responding to the first pointing action; updating the existing target task package through the manipulation device responding to the determination instruction of the jet parabolic trajectory; the control terminal sends the updated target task package to the fire emergency robot, so that the fire emergency robot selects the fire extinguishing agent and performs a fire extinguishing action according to the jet parabolic trajectory. Claims 1 / 3 Page 2 CN 120886234 A 6. The method according to claim 3, characterized in that the control terminal is further provided with a manipulation device, if the task object is a moving target, then the initial task package is a moving task package, the control terminal responds to the condition information input on the condition setting page, generates a target task package and sends it to the fire emergency robot, including: responding to the second pointing action of the manipulation device at the optical axis specified position of the scene status information, selecting the primary moving target and the moving destination position to generate first condition information; the control terminal loads the first condition information on the condition setting page and generates the target task package;The control terminal sends the updated target task package to the fire emergency robot, so that the fire emergency robot moves the primary object to the object-to-object location. 7. The method according to claim 3, characterized in that, if the task object is a valve target, the initial task package is a valve start / stop task package, and the control terminal, in response to the condition information input on the condition setting page, generates a target task package and sends it to the fire emergency robot, including: the control terminal, in response to a second input trigger, selects the valve position box corresponding to the valve target in the scene status information; the control terminal, in response to the second condition information input on the condition setting page, generates a target task package; the second condition information includes rotation direction, rotation speed, and rotation torque; the control terminal sends the target task package to the fire emergency robot, so that the fire emergency robot uses the rotation torque to perform valve start / stop actions along the rotation direction at the rotation speed. 8. The method according to claim 1, characterized in that the method further comprises: when the fire emergency robot detects that any of the task conditions are not within the preset condition range, feeding back fault status information to the control terminal and pausing the target action; the control terminal resetting the task conditions according to the fault status information and updating the target task package sent to the fire emergency robot. 9. The method according to claim 1, characterized in that the fire emergency robot includes a processing center component, a sensing component, and an action execution component, and the fire emergency robot executes target actions according to the target task package, including: the processing center component parsing the target task package to obtain multiple sorted target actions; the processing center component calling the action execution component to execute each target action sequentially; the processing center component calling the sensing component to obtain visual information after the target action interacts with the current environment; and the processing center component adjusting the action of the action execution component according to the visual information until all the target actions have been executed. 10. A remote semi-automatic control system for a fire emergency robot, characterized in that it includes a control terminal and a fire emergency robot connected by remote communication; the control terminal includes: a search module, used to respond to a search command and send a search task package to the fire emergency robot; a task sending module, used to select an initial task package and set task conditions when receiving a task package command for the task object, construct a target task package and send it to the fire emergency robot; the fire emergency robot includes: a scene search module, used to perform a search action according to the search task package and return scene status information to the control terminal for display; a task object marking module, used to mark any task object detected by the fire emergency robot in the scene. (Claims 2 / 3, page 3, CN 120886234)A. The status information marks the task object; Action execution module, used by the fire emergency robot to execute the target action according to the target task package. Claims 3 / 3 Page 4 CN 120886234 A A remote semi-automatic control method and system for a fire emergency robot

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 641,662, filed May 2, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to the field of robot control technology, and in particular to a remote semi-automatic control method and system for a fire emergency robot. Background Art

[0003] At fire scenes, there are often potential dangers such as the fire itself and the building being prone to collapse, and explosive materials being prone to explosion and injury. Relying on traditional manual close-range fire extinguishing can easily cause casualties. With the progress of science and the development of technology, robots are gradually being applied to emergency safety fields such as fire rescue. Fire extinguishing by remotely controlling robots can effectively ensure personnel safety.

[0004] Traditional and commonly used control methods are usually remote master-slave remote control operations. Operators need to use control handles to fully control the robot's walking and all movements, which has problems such as high operational complexity, poor operational accuracy, high training costs, and inconvenience for operators to observe when operating remotely, thus restricting the application of fire emergency robots in actual combat.

[0005] To this end, existing technologies have also begun to try to develop fully autonomous fire emergency robots. However, due to the complexity of the fire scene environment, its control strategy is highly dependent on human experience, making it difficult to unify strategies to execute different emergency rescue tasks, resulting in a relatively narrow application scenario and low control flexibility for fully autonomous robots. Summary of the Invention

[0006] The present invention provides a remote semi-automatic control method and system for fire emergency robots, which solves the technical problem that due to the complexity of the fire scene environment, its control strategy is highly dependent on human experience, making it difficult to unify strategies to execute different emergency rescue tasks, resulting in a relatively narrow application scenario and low control flexibility for fully autonomous robots.

[0007] The present invention provides a remote semi-automatic control method for a fire emergency robot, involving a control terminal with remote communication connection and a fire emergency robot. The method includes:

[0008] The control terminal responds to a search command and sends a search task package to the fire emergency robot;

[0009] The fire emergency robot performs a search action according to the search task package and returns scene status information to the control terminal for display;

[0010] When the fire emergency robot detects any task object, it marks the task object in the scene status information;

[0011] When the control terminal receives a task package command for the task object, it selects an initial task package and sets task conditions, constructs a target task package, and sends it to the fire emergency robot.

[0012] The fire emergency robot performs the target action according to the target task package.

[0013] Optionally, the method further includes:

[0014] The control terminal responds to the task package creation instruction and selects multiple task actions from a preset action library;

[0015] The control terminal responds to the requirement setting instruction and selects the pending action that matches the requirement setting instruction from the multiple task actions, and creates a condition setting page;

[0016] The control terminal responds to the sorting operation instruction and sorts all the task actions and the pending actions to generate a candidate task package.

[0017] Optionally, when the control terminal receives a task package instruction for the task object, it selects an initial task package and sets task conditions to construct a target task package and sends it to the fire emergency robot, including:

[0018] When the control terminal receives a task package instruction for the task object, it selects the initial task package specified by the task package instruction from a plurality of candidate task packages;

[0019] The control terminal displays the condition setting page corresponding to the initial task package in a predetermined display area of ​​the scene status information;

[0020] The control terminal responds to the condition information entered on the condition setting page, generates a target task package, and sends it to the fire emergency robot.

[0021] Optionally, if the task object is a fire source target, then the initial task package is a fire extinguishing task package. The control terminal responds to the condition information entered on the condition setting page, generates a target task package, and sends it to the fire emergency robot, including:

[0022] The control terminal responds to the first input trigger, selects the corresponding fire extinguishing agent limitation condition on the condition setting page, and generates a target task package;

[0023] The control terminal sends the target task package to the fire emergency robot, so that the fire emergency robot selects a fire extinguishing agent that meets the fire extinguishing agent limitation condition to perform a fire extinguishing action.

[0024] Optionally, the control terminal further includes an operating device, which further includes:

[0025] generating a jet parabolic trajectory corresponding to the fire emergency robot in the scene status information by responding to a first pointing action through the operating device;

[0026] updating the existing target task package by responding to a determination instruction for the jet parabolic trajectory through the operating device;

[0027] sending the updated target task package to the fire emergency robot so that the fire emergency robot selects the extinguishing agent and performs the extinguishing action according to the jet parabolic trajectory.

[0028] Optionally, the control terminal further includes an operating device. If the task object is a moving target, the initial task package is a moving task package. The control terminal generates a target task package in response to the condition information entered on the condition setting page.The task package is then sent to the fire emergency robot, including:

[0029] Responding to a second pointing action via the control device, selecting a primary moving target and a moving destination location at the optical axis specified in the scene status information to generate first condition information;

[0030] Loading the first condition information on the condition setting page and generating a target task package;

[0031] Sending the updated target task package to the fire emergency robot so that the fire emergency robot moves the primary moving target to the moving destination location.

[0032] Optionally, if the task object is a valve target, then the initial task package is a valve start / stop task package. The control terminal responds to the condition information input on the condition setting page, generates a target task package, and sends it to the fire emergency robot, including:

[0033] The control terminal responds to the second input trigger and selects the valve position box corresponding to the valve target in the scene status information;

[0034] The control terminal responds to the second condition information input on the condition setting page and generates a target task package; The second condition information includes rotation direction, rotation speed, and rotation torque;

[0035] The control terminal sends the target task package to the fire emergency robot so that the fire emergency robot uses the rotation torque to perform valve start / stop actions along the rotation direction at the rotation speed.

[0036] Optionally, the method further includes:

[0037] When the fire emergency robot detects that any of the task conditions are not within the preset condition range, it feeds back fault status information to the control terminal and suspends the target action;

[0038] The control terminal resets the task conditions according to the fault status information and updates the target task package sent to the fire emergency robot.

[0039] Optionally, the fire emergency robot includes a processing center component, a sensing component, and an action execution component. The fire emergency robot executes the target action according to the target task package, including:

[0040] The processing center component parses the target task package to obtain multiple sorted target actions;

[0041] The processing center component calls the action execution component to execute each of the target actions in sequence;

[0042] The processing center component calls the sensing component to obtain visual information after the target action interacts with the current environment;

[0043] The processing center component adjusts the action of the action execution component according to the visual information until all the target actions have been executed.

[0044] The present invention also provides a remote semi-automatic control system for a fire emergency robot, comprising a control terminal and a fire emergency robot connected by remote communication; the control terminal includes:

[0045] The search module is used to respond to search instructions and send search task packages to the fire emergency robot;

[0046] The task sending module is used to select an initial task package and set task conditions when receiving a task package instruction for the task object, and construct a target task package to send to the fire emergency robot;

[0047] The fire emergency robot includes:

[0048] A scene search module is used to perform search actions according to the search task package and return scene status information to the control terminal for display;

[0049] A task object marking module is used to mark the task object in the scene status information when the fire emergency robot detects any task object;

[0050] An action execution module is used for the fire emergency robot to perform target actions according to the target task package.

[0051] As can be seen from the above technical solutions, the present invention has the following advantages:

[0052] The present invention responds to search commands through the control terminal and sends search task packages to the fire emergency robot; the fire emergency robot performs search actions according to the search task packages and returns scene status information to the control terminal for display; when the fire emergency robot detects any task object, it marks the task object in the scene status information; when the control terminal receives a task package command for a task object, it selects an initial task package and sets task conditions, constructs a target task package, and sends it to the fire emergency robot; the fire emergency robot performs target actions according to the target task package. This achieves semi-automatic control of the fire emergency robot, effectively improving the robot's flexibility in emergency situations through a combination of user decision-making and robot execution.

[0053] Brief Description of the Drawings: In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention on page 3 / 13 of this specification, CN 120886234 A. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 is a flowchart of the steps of a remote semi-automatic control method for a fire emergency robot provided by an embodiment of the present invention;

[0055] Figure 2 is a schematic diagram of the action sequencing of a fire extinguishing task provided by an embodiment of the present invention;

[0056] Figure 3 is a schematic diagram of some condition settings provided by an embodiment of the present invention;

[0057] Figure 4 is a schematic diagram of the execution steps of a first fire extinguishing task provided by an embodiment of the present invention;

[0058] Figure 5 is a schematic diagram of the execution steps of a second fire extinguishing task provided by an embodiment of the present invention;

[0059] Figure 6 is a structural block diagram of a remote semi-automatic control system for a fire emergency robot provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0060] This invention provides a remote semi-automatic control method and system for a fire emergency robot, which addresses the technical problem that due to the complex environment at fire scenes, the control strategy highly relies on human experience, making it difficult to unify strategies for executing different emergency rescue tasks, resulting in a narrow application scenario and low control flexibility for fully autonomous robots.

[0061] To make the invention's objectives, features, and advantages more apparent and understandable, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] Please refer to Figure 1, which is a flowchart of the steps of a remote semi-automatic control method for a fire emergency robot provided by an embodiment of this invention.

[0063] The present invention provides a remote semi-automatic control method for a fire emergency robot, involving a control terminal with remote communication connection and a fire emergency robot. The method includes:

[0064] Step 101: The control terminal responds to a search command and sends a search task package to the fire emergency robot;

[0065] The control terminal refers to a device used for functions such as action editing, task package construction, and condition setting of the fire emergency robot. It may include only a touch-screen visual interface or a visual interface connected to peripheral devices. Its type may include, but is not limited to, visual interfaces such as tablet computers and VR devices and their connected external devices such as controllers.

[0066] The fire emergency robot refers to an intelligent robot used to deal with fire scenarios. It includes at least a processing center component, a sensing component, and an action execution component to match different scenario operation requirements in the fire environment, such as moving objects, extinguishing fires, valve opening and stopping, and material transportation.

[0067] The search command refers to the command generated by the control terminal when a function button is triggered by the outside or a request is received, which is used to trigger the control terminal to select and send relevant search task packages. The search task package refers to a set of instructions containing multiple sequentially executed actions, which may include at least the robot's search route, movement mode, perception of the task object to be detected, and related instruction conditions.

[0068] In this embodiment, the control terminal and the fire emergency robot can be remotely connected to facilitate remote operation by the operator, effectively avoiding the uncontrollable risks of personnel entering the emergency scene. After the fire emergency robot is placed at the entrance of the emergency scene, the operator can trigger it on the control terminal to start the fire emergency robot. Then, the operator can further trigger the control terminal, such as by clicking the corresponding search option, so that the control terminal responds to the search instructions generated by the click. (Instruction manual 4 / 13 pages 8 CN 120886234 A)The fire emergency robot selects the search task package corresponding to the search instruction and sends it to the fire emergency robot.

[0069] Step 102: The fire emergency robot performs the search action according to the search task package and returns the scene status information to the control terminal for display.

[0070] After receiving the search task package, the fire emergency robot parses the search task package to obtain multiple sorted search action instructions. The fire emergency robot executes the search action instructions in order to call different mechanisms on the fire emergency robot to perform search actions. At the same time, it calls the sensing components to perceive the environment and returns the scene status information of the fire emergency robot's location to the control terminal for display, so that the operator can make real-time decisions on the actions of the fire emergency robot according to the scene status information.

[0071] For example, after receiving the search task package, the fire emergency robot calls the moving mechanism to move the robot as a whole according to the search route. At the same time, it calls the sensing components on board to obtain real-time images of the robot's environment from multiple perspectives and feeds them back to the control terminal for display.

[0072] It should be noted that the scene status information refers to the images on the screen that indicate the scene and operating status of the fire emergency robot, respectively, through different areas. These images can be constructed from the real-world environment of the robot or by digitally mapping the real-world environment. This may include, but is not limited to, images of the robot's scene, a map synchronously built by the robot, its own position, selectable task objects, action previews in the task package, action effect previews, task package execution status, and robot status. For example, there is a leaking valve at an accident scene. The operator wants to close the valve. The scene is displayed on a circular screen. The circular screen simultaneously displays the scene map and the robot's position. After the robot arrives at the valve's location, the operator uses an external device to select the valve-closing task package. The valve on the display screen is selected as a candidate task object by a detection box. After the operator selects the target valve and command conditions (rotation direction, speed, target torque, etc.), the robot automatically executes the action of rotating to close the valve. The interface displays torque feedback, speed, number of rotations, and other information in real time. When the torque exceeds the robot motor torque limit, the task package stops executing and displays the fault information and progress on the interface. The operator resets the command conditions, such as deceleration and torque increase, and selects the target number of revolutions based on the completed progress. After the task is completed, the interface displays the status information of the task completion, and the robot status bar displays the robot's remaining battery power, estimated remaining working time, component temperature, and other status information.

[0073] Step 103: When the fire emergency robot detects any task object, it marks the task object in the scene status information;

[0074] The task object refers to the equipment, location, personnel, or fire source located in the emergency scene that needs to be handled by the fire emergency robot.

[0075] In this embodiment, the fire emergency robot can obtain the scene status information of the current environment through the sensing component.The system detects and identifies the scene status information using relevant detection algorithms. When the sensing component detects any task object, it marks the task object in the scene status information and feeds it back to the control terminal so that the control terminal can know the location of the task object operated by the fire emergency robot in a timely manner.

[0076] When a task object is detected, in order to avoid the fire emergency robot losing the task object due to its movement, the fire emergency robot can pause the execution of the search action and switch to standby mode to wait for further task packages to be issued by the control terminal.

[0077] In addition, when the robot is patrolling or performing a search task, the operator can choose to stop the robot from moving and enter standby mode at any time according to the situation. After the operator selects the task package and the preset conditions required for the task package, the robot automatically executes the task package. During the execution, the operator can stop the automatic mode and switch to remote control mode at any time. In addition to detecting task objects through visual perception, the sensing component can also perform robot positioning and map construction of the current emergency scene through radar perception assistance. Instruction manual 5 / 13 page 9 CN 120886234 A

[0078] In one example of the present invention, the method further includes the following steps S11-S13:

[0079] S11: The control terminal responds to the task package creation instruction and selects multiple task actions from the preset action library;

[0080] The task package creation instruction refers to the instruction generated by the control terminal in response to external triggers, used to select task actions.

[0081] In order to improve the operating efficiency of the fire emergency robot, candidate task packages can be created in advance according to multiple common requirements of emergency scenarios. Specifically, the control terminal responds to the task package creation instruction and selects multiple task actions from the preset action library. Each task action can be displayed on the visual interface of the control terminal in the form of icons. The operator can move each icon to the task package creation interface by dragging, clicking, etc., as the action basis of the candidate task package.

[0082] S12. The control terminal responds to the requirement setting command, selects the pending action that matches the requirement setting command from multiple task actions, and creates a condition setting page;

[0083] After completing the selection of task actions, if some of them require setting execution conditions, the control terminal can respond to the input requirement setting command and select the pending action that matches the requirement setting command from multiple task actions. For example, in a fire extinguishing task package, it is necessary to use fire extinguishing agents to extinguish fires at target locations. At this time, it is necessary to select the type of fire extinguishing agent and select different fire extinguishing equipment according to different target locations. For this purpose, corresponding condition setting pages can be created in "Fire Extinguishing Agent Selection Action" and "Fire Extinguishing Equipment Selection Action" respectively, and items or condition requirements related to the action can be added to the condition setting page. This is so that users can make adaptive adjustments according to the actual task objects later.

[0084] S13. The control terminal responds to the sorting operation command, sorts all task actions and pending actions, and generates a candidate task package.

[0085] In this embodiment, after the creation of the condition setting page for pending actions is completed, all task actions and pending actions are sorted to generate a candidate task package.

[0086] Specifically, the order of actions can be determined by emergency rescue and disposal task experts and robot control experts, or sorted according to historical processing data of emergency scenarios. This embodiment of the invention does not limit this.

[0087] For example, as shown in Figure 2, the action sorting for fire extinguishing tasks can be as follows:

[0088] (1) Use a depth camera to locate the three-dimensional position coordinates of the fire source relative to the fire cannon carried by the robot.

[0089] (2) Adjust the horizontal angle of the fire cannon so that the fire cannon aims at the fire source in the horizontal direction.

[0090] (3) Calculate a suitable jet parabolic trajectory according to the fire source position and the fire cannon outlet velocity, and adjust the fire cannon elevation angle.

[0091] (4) Adjust the fire monitor's direction dynamically according to the jet's landing point to accurately extinguish the fire source.

[0092] (5) Once the fire source is detected to be extinguished, the fire monitor stops spraying extinguishing agent and returns to standby zero position.

[0093] Specifically, the calculation of the fire monitor's elevation angle can be performed by using machine vision methods to identify the jet trajectory (e.g., after converting grayscale, frame alignment and subtraction are performed to filter out stationary objects and leave the jet images with differences between frames), find the jet landing point A', and then calculate the relative position of A' and the target landing point A. For horizontal deviation, the horizontal rotation of the fire monitor is used to compensate. For front-back deviation and height deviation, the angle that needs to be adjusted for elevation angle is calculated after substituting the deviation.

[0094] For the adjustment of the fire monitor's direction, it can be assumed that the initial velocity is v0 and the water cannon's elevation angle is α, then the horizontal initial velocity is vx0 and the vertical initial velocity is vz0. The vertical direction is affected by the gravitational acceleration g. The horizontal direction is affected by the air resistance acceleration a (a is related to the velocity, assuming the air is still, wind speed and direction are not considered, and the coefficient is a fixed value K). After establishing the mathematical model, it is ensured that at time t, the horizontal displacement is equal to the relative horizontal distance. The vertical displacement is equal to the relative vertical distance, so the value of the pitch angle α can be solved (the parabolic trajectory is also solved accordingly). In some cases, there may be two values, so consider directly taking the smaller pitch angle. Manual 6 / 13 pages 10 CN 120886234 A

[0095] Step 104, when the control end receives the task package instruction for the task object, it selects the initial task package and sets the task conditions, constructs the target task package and sends it to the fire emergency robot;

[0096] The task package instruction refers to the instruction generated by the operator after learning about the task object, triggered by clicking, sliding or dragging operations on the visual interface, used to select the target task package that matches the state of the task object, and at the same time, in displaying the targetAfter setting the conditions for the task package, input or load the various task conditions to construct a complete target task package.

[0097] The initial task package refers to a template-style task package pre-written by the operator based on various common tasks in emergency scenarios, such as fire extinguishing, handling, valve closing, or material transportation. It may include multiple task condition items that require parameter or action adjustment according to different situations. The target task package refers to the task package generated after setting the task conditions for each task condition item based on the scene status information of the task object, including multiple sorted robot action execution instructions.

[0098] In this embodiment, after the control terminal receives the scene status information marked with the task object and displays it on the visual interface, the operator inputs different task package instructions according to the different types of the task object, thereby selecting different initial task packages and setting the task conditions of each initial task package to generate a target task package, which is then sent to the fire emergency robot.

[0099] The execution order of the task packages is determined by the operator. Specifically, it will be carried out according to the on-site situation and under the decision of the on-site commander. For example, in a fire accident, a search task package is executed first, and after the fire source is found, multiple fire extinguishing task packages are executed. When obstacles are encountered while continuing to advance, a clearing / moving task package is executed. When trapped personnel are found, a task package for delivering protective equipment is executed, etc.

[0100] In an example of the present invention, step 104 may include the following sub-steps S21-S23:

[0101] S21: When the control terminal receives a task package instruction for the task object, it selects the initial task package specified by the task package instruction from multiple candidate task packages;

[0102] The candidate task package refers to a set of action instructions pre-built by the user, which can be adapted to various different emergency scenario types, and some action instructions have corresponding condition setting pages.

[0103] After the fire emergency robot detects and identifies the task object, since the situation in the emergency scenario is usually more complex, relying solely on automatic algorithms to determine the execution of the task package may not be flexible and timely in responding to the current situation. Therefore, after the control terminal receives the scene status information marked with the task object and displays it on the visual interface, it selects at least one initial task package that meets the current complex situation from multiple candidate task packages by responding to the task package command input by the user on the control terminal.

[0104] For example, in a fire accident, the search task package is executed first, multiple fire extinguishing task packages are executed after the fire source is found, the obstacle clearing / moving task package is executed when obstacles are encountered while continuing to advance, and the task package for delivering protective equipment is executed when trapped personnel are found, etc.

[0105] S22, the control terminal displays the condition setting page corresponding to the initial task package in the predetermined display area of ​​the scene status information;

[0106] In this embodiment, the size of the visual interface of the control terminal is usually large, and there are task objects and other...Additional irrelevant scene information. To enable users to set conditions for the task package in real time according to changes in the task object, a portion of the visual interface on the control terminal can be divided as a predetermined display area to display the condition setting page in the selected initial task package, waiting for the user to input condition information.

[0107] The condition setting page may include condition setting items corresponding to all actions that need to be set in the initial task package or condition setting items for a single specific action. If no conditions need to be set for any of the actions in the initial task package, the confirmation button can be displayed directly. Specification 7 / 13 pages 11 CN 120886234 A

[0108] As shown in Figure 3, Figure 3 shows a schematic diagram of some condition settings in an embodiment of the present invention. In a fire extinguishing task, if multiple fire sources are encountered, the operator needs to determine which fire source to extinguish first and which fire extinguishing agent to use based on the situation on site and experience. After the conditions are set, the robot will execute the fire extinguishing task package and use the selected fire extinguishing agent to extinguish the selected fire source. Another example: a task package for moving objects also needs to specify the moving object and the moving destination. After the conditions are set, the robot will move the specified object to the specified location. Another example: In the valve switching task, the operator needs to specify the target valve and the direction, torque or number of rotations required to switch the valve. After the conditions are set, the robot will open or close the specified valve.

[0109] S23, the control terminal responds to the condition information entered on the condition setting page, generates a target task package and sends it to the fire emergency robot.

[0110] In an example of the present invention, if the task object is a fire source target, the initial task package is a fire extinguishing task package. S23 may include the following sub-steps:

[0111] The control terminal responds to the first input trigger, selects the corresponding fire extinguishing agent limitation conditions on the condition setting page, and generates a target task package;

[0112] The control terminal sends the target task package to the fire emergency robot so that the fire emergency robot selects a fire extinguishing agent that meets the fire extinguishing agent limitation conditions to perform the fire extinguishing action.

[0113] The fire source target may include one or more. When the task object is detected to be a fire source target, the selected initial task package is a fire extinguishing task package and the corresponding condition setting page is displayed. The control terminal receives the first input trigger, such as a value, parameter, or click selection. On the condition setting page, the corresponding extinguishing agent limitation condition is selected, and information such as the type of extinguishing agent is selected. Then, in response to the click of the confirmation button, the target task package is generated.

[0114] It should be noted that the extinguishing agent limitation condition is used to limit the different types of extinguishing agents and the output amount of extinguishing agents corresponding to different fire source targets.

[0115] As shown in Figure 4, the operator uses a tablet computer as the control terminal to operate the fire emergency robot. After the robot executes the search task package and discovers a fire source, the camera footage on the robot is displayed on the interactive interface of the tablet computer.The screen displays a fire source target marked on the interactive interface. After the operator detects the fire source on the screen, they click a button to stop the robot from moving and put it into standby mode. After selecting a fire extinguishing task package, the corresponding condition setting page is opened. The operator clicks on the alternative fire source target identified by the target detection system, sets the extinguishing agent type and other instruction conditions, and then clicks execute. The robot performs the corresponding fire extinguishing action according to the fire extinguishing task package.

[0116] Further, the control terminal is also equipped with an operating device. S23 may also include the following sub-steps:

[0117] The operating device responds to the first pointing action to generate the jet parabolic trajectory corresponding to the fire emergency robot in the scene status information;

[0118] The operating device responds to the determination instruction for the jet parabolic trajectory to update the existing target task package;

[0119] The control terminal sends the updated target task package to the fire emergency robot so that the fire emergency robot selects the extinguishing agent and performs the fire extinguishing action according to the jet parabolic trajectory.

[0120] In this embodiment, the user can also perform a first pointing action by manipulating the device. The manipulating device is equipped with a gyroscope and other devices, which can generate a jet parabolic trajectory in the scene status information in response to the first pointing action for the user to preview. By responding to the command to determine the jet parabolic trajectory through the manipulating device, the existing target task package is updated and sent to the fire emergency robot to perform the fire extinguishing action.

[0121] The manipulating device can be a handle device, a click device, a VR device, or other types of operating devices that are connected to the visual interface of the control terminal. This embodiment of the invention does not limit the type of device. Specification 8 / 13 pages 12 CN 120886234 A

[0122] Taking a VR device as an example, as shown in Figure 5, the operator uses a VR device to guide the robot in the fire extinguishing task. After the robot performs the search task package and discovers the fire source, the screen captured by the camera on the robot is displayed on the interactive interface of the tablet computer. At this time, the fire source target is marked on the interactive interface. After the operator detects the fire source on the screen, they click a button to stop the robot and put it into standby mode. The robot's handle can generate a jet trajectory based on the outlet velocity of the fire cannon on the robot and preview it on the interactive interface. The operator can use the trajectory preview to control the orientation adjustment of the fire cannon in the fire extinguishing task package, so that the sprayed extinguishing agent directly reaches the landing point of the preview trajectory. After confirmation, the fire extinguishing task package is issued to the robot, and the robot performs the fire extinguishing action according to the trajectory.

[0123] In another example of the present invention, the control terminal is also equipped with a manipulation device. If the task object is a moving target, the initial task package is a moving task package. S23 may include the following sub-steps:

[0124] By responding to the second pointing action of the manipulation device at the optical axis specified position of the scene state information, the primary moving target and the moving destination position are selected to generate the first condition information;

[0125] The control terminal loads the first condition information on the condition setting page and generates a target task package;

[0126] The control terminal sends the updated target task package to the fire emergency robot so that the fire emergency robot moves the primary object to the object destination location.

[0127] In this embodiment, when the control terminal is equipped with a manipulation device, if the task object is an object to be moved, such as the transportation of a certain obstacle or material, the initial task package selected at this time is the object moving task package. The manipulation device can respond to the second pointing action to generate an optical axis in the scene status information and specify a certain position as the optical axis specified position. The primary object to be moved and the object destination location are selected according to the optical axis specified position to generate the first condition information. Among them, the task target specified by the first optical axis specified position is the primary object to be moved. If there is no task target at the optical axis specified position, it is determined that the position is the object destination location. If there is a primary object to be moved, the object moving relationship between the two is established as the first condition information. The control terminal loads the first condition information on the condition setting page and generates a target task package to send to the fire emergency robot.

[0128] For example, VR devices can display panoramic images of a scene captured by an airborne panoramic camera. VR devices use digital mapping to combine the real-world scene with a spatial point cloud model. The operator will issue commands within the virtual-real hybrid space constructed by the VR device. To move an obstacle to a designated location, the operator can use the controller to select a task package on the virtual control panel. Then, the operator uses the controller to aim and select the task target in the model space (the controller generates an optical axis; aligning the optical axis with the target object allows for selection). Finally, the operator uses the controller to aim at the ground in the space, select the destination, and confirm execution.

[0129] In another example of the present invention, if the task object is a valve target, the initial task package is a valve start / stop task package, and S23 may include the following sub-steps:

[0130] The control terminal responds to the second input trigger and selects the valve position box corresponding to the valve target in the scene status information;

[0131] The control terminal responds to the second condition information input on the condition setting page and generates a target task package; the second condition information includes rotation direction, rotation speed and rotation torque;

[0132] The control terminal sends the target task package to the fire emergency robot so that the fire emergency robot uses rotation torque to perform valve start / stop actions along the rotation direction at a rotation speed.

[0133] In this embodiment, for the valve target, the site map and the robot position can be displayed simultaneously on the circular screen. After the robot arrives at the valve location, the operator uses an external device to select the valve start / stop task package. The valve in the display screen is selected by the detection box as a candidate task target. After the operator selects the target valve and the command conditions (rotation direction, rotation speed, target torque, etc.), the robot automatically performs the rotation to close the valve.

[0134] Step 105: The fire emergency robot executes the target actions according to the target task package.

[0135] In one example of the present invention, step 105 may include the following sub-steps: Specification 9 / 13 Page 13 CN 120886234 A

[0136] The processing center component parses the target task package to obtain multiple sorted target actions;

[0137] The processing center component calls the action execution component to execute each target action in sequence;

[0138] The processing center component calls the perception component to obtain visual information after the target action interacts with the current environment;

[0139] The processing center component adjusts the action of the action execution component according to the visual information until all target actions have been executed.

[0140] The processing center component can be a high-performance chip capable of parsing task packages, issuing instructions, and executing algorithms. The perception component refers to the components used to perceive the robot's environment. These can include visual perception and radar perception, among others. Visual perception can be a combination of 3D object recognition and depth camera ranging. 3D object recognition can be implemented using open-source algorithms, such as using a binocular camera to acquire visual information. The binocular camera generates a point cloud map using parallax. Through 3D object recognition, the pixel coordinates of the vertices of the target object detection box (cubic prism) in the 3D state can be obtained in the normal image. Then, based on these pixel coordinates, a search is performed in the point cloud map to obtain the 3D coordinates of the target object detection box vertices relative to the camera. Finally, the motion execution component is invoked to grasp the object. The motion execution component refers to the specific actuators used to perform fire extinguishing actions, handling actions, valve rotation actions, and material transport actions. These can include, but are not limited to, robotic arms, fire extinguishing agent storage containers, fire monitors, and moving mechanisms.

[0141] In this embodiment, the processing center component parses the target task package to obtain multiple sorted target actions, and calls the action execution component to execute each target action in sequence. For example, if the robot executes a fire extinguishing task package, it will use the selected extinguishing agent to extinguish the selected fire source; if it is a moving task package, the robot will move the specified object to the specified location; if it is a valve start / stop task package, the robot will open or close the specified valve according to the direction, torque, or number of rotations required to open or close the valve. Specifically, there is a leaking valve at a certain location at the accident site, and the operator wants to close the valve. The scene at the scene is displayed on a circular screen. The circular screen simultaneously displays the scene map and the robot's position. After the robot arrives at the location of the valve, the operator uses an external device to select the valve start / stop task package. The valve in the display screen is selected by the detection box as a candidate task target. After the operator selects the target valve and the command conditions (rotation direction, speed, target torque, etc.), the robot automatically executes the action of rotating to close the valve. The interface displays information such as torque feedback, speed, and number of rotations in real time.

[0142] Meanwhile, some actions within the task package need to be completed automatically with the assistance of the perception system. To this end, the perception component can be invoked to obtain visual information after the target action interacts with the current environment, and the action execution component can be executed according to the visual information.The robot adjusts its actions until all target actions have been executed. For example, in a moving task, after obtaining the target destination location, it uses visual SLAM to locate and map itself, updates the calculated path in real time, and performs obstacle avoidance and detour actions when encountering obstacles.

[0143] In addition, the robot will also provide feedback on the execution status of the task package. Whether the task package has been completed, whether there is a deviation in the execution effect, the cause of the unexpected termination, and the progress will be fed back to the control terminal in real time. If an abnormal situation occurs during the execution of the task package and it cannot continue to execute, the robot will report the fault information and wait for the operator to handle it. The handling methods include modifying the task instruction condition settings, abandoning the task package, and switching to remote control mode.

[0144] In an example of the present invention, the method further includes the following steps S31-S32:

[0145] S31: When the fire emergency robot detects that any task condition is not within the preset condition range, it feeds back the fault status information to the control terminal and suspends the target action;

[0146] S32: The control terminal resets the task conditions according to the fault status information and updates the target task package and sends it to the fire emergency robot.

[0147] The preset condition range refers to the range value set according to the various action execution capabilities of the fire emergency robot, such as the range of executable torque, the range of movable size, the range of rotation speed, the range of range, and the range of extension. Specification 10 / 13 pages 14 CN 120886234 A

[0148] In this embodiment, after receiving each target task package, the fire emergency robot parses it in sequence and executes each target action in turn. At the same time, the fire emergency robot can also determine the condition type of the task condition before executing the target action, and further determine whether the task condition is within the preset condition range corresponding to that condition type. If the task condition is not within the preset condition range, it indicates that the fire emergency robot may malfunction when executing the target action. The fault status information is fed back to the control terminal, and the current target action execution is suspended to prevent the fault from worsening. After receiving the fault status information, the control terminal responds to the user instruction to reset the task conditions, updates the target task package and resends it to the fire emergency robot for parsing and continuing the execution of the target action.

[0149] For example, when the torque of the valve rotation exceeds the torque limit of the robot motor, the task package stops executing and displays the fault information and the progress on the interface. The operator resets the command conditions, such as deceleration and torque increase, and selects the target number of revolutions according to the progress. After the task is completed, the interface displays the status information of the task completion, and the robot status bar displays the robot's remaining battery power, estimated remaining working time, component temperature, and other status information.

[0150] In this embodiment of the invention, the control terminal responds to the search command and sends the search task package to the fire emergency machine.The fire emergency robot performs search actions according to the search task package and returns scene status information to the control terminal for display. When the fire emergency robot detects any task object, it marks the task object in the scene status information. When the control terminal receives the task package instruction for the task object, it selects the initial task package and sets the task conditions, constructs the target task package and sends it to the fire emergency robot. The fire emergency robot performs the target action according to the target task package. This realizes semi-automatic control of the fire emergency robot. By combining user decision-making with robot execution, the robot's response flexibility in emergency scenarios is effectively improved.

[0151] Please refer to Figure 6. Figure 6 shows a structural block diagram of a remote semi-automatic control system for a fire emergency robot in an embodiment of the present invention.

[0152] This embodiment of the invention provides a remote semi-automatic control system for a fire emergency robot, including a control terminal and a fire emergency robot connected by remote communication; the control terminal 61 includes:

[0153] a search module 611, used to respond to a search command and send a search task package to the fire emergency robot;

[0154] a task sending module 612, used to select an initial task package and set task conditions when receiving a task package command for a task object, and construct a target task package to send to the fire emergency robot;

[0155] The fire emergency robot 62 includes:

[0156] a scene search module 621, used to perform a search action according to the search task package and return scene status information to the control terminal for display;

[0157] a task object marking module 622, used to mark the task object in the scene status information when the fire emergency robot detects any task object;

[0158] an action execution module 623, used for the fire emergency robot to perform a target action according to the target task package.

[0159] Optionally, the control terminal 61 further includes:

[0160] an action selection module, used to respond to a task package creation instruction and select multiple task actions from a preset action library;

[0161] a page setting module, used to respond to a requirement setting instruction, select pending actions that match the requirement setting instruction from multiple task actions, and create a condition setting page;

[0162] an action sorting module, used to respond to a sorting operation instruction, sort all task actions and pending actions, and generate a candidate task package.

[0163] Optionally, the task distribution module 612 includes:

[0164] an initial task package selection submodule, used to select the initial task package specified by the task package instruction from multiple candidate task packages when a task package instruction for a task object is received;

[0165] a page display submodule, used to display the condition setting page corresponding to the initial task package in a predetermined display area of ​​the scene status information;

[0166] The target task package generation submodule is used to generate a target task package and send it to the fire emergency robot in response to the condition information entered on the condition setting page.

[0167] Optionally, if the task object is a fire source target, the initial task package is a fire extinguishing task package. The target task package generation submodule is specifically used for:

[0168] The control terminal responds to the first input trigger, selects the corresponding fire extinguishing agent limitation condition on the condition setting page, and generates a target task package;

[0169] The control terminal sends the target task package to the fire emergency robot so that the fire emergency robot selects a fire extinguishing agent that meets the fire extinguishing agent limitation condition to perform the fire extinguishing action.

[0170] Optionally, the control terminal is further provided with an manipulation device, and the target task package generation submodule is specifically used for:

[0171] responding to the first pointing action through the manipulation device to generate the jet parabolic trajectory corresponding to the fire emergency robot in the scene status information;

[0172] responding to the determination instruction for the jet parabolic trajectory through the manipulation device to update the existing target task package;

[0173] the control terminal sends the updated target task package to the fire emergency robot so that the fire emergency robot selects the extinguishing agent and performs the extinguishing action according to the jet parabolic trajectory.

[0174] Optionally, the control terminal is also equipped with a manipulation device. If the task object is a moving target, the initial task package is a moving task package. The target task package generation submodule is specifically used for:

[0175] Selecting the primary moving target and the moving destination position at the optical axis specified by the manipulation device in response to the second pointing action to generate first condition information;

[0176] Loading the first condition information on the condition setting page and generating the target task package;

[0177] Sending the updated target task package to the fire emergency robot so that the fire emergency robot moves the primary moving target to the moving destination position.

[0178] Optionally, if the task object is a valve target, the initial task package is a valve start / stop task package, and the target task package generation submodule is specifically used for:

[0179] The control terminal responds to the second input trigger and selects the valve position box corresponding to the valve target in the scene status information;

[0180] The control terminal responds to the second condition information input on the condition setting page and generates the target task package; the second condition information includes rotation direction, rotation speed and rotation torque;

[0181] The control terminal sends the target task package to the fire emergency robot so that the fire emergency robot uses rotation torque to perform valve start / stop actions along the rotation direction at a rotation speed.

[0182] Optionally, the fire emergency robot 62 further includes:

[0183] A fault feedback module, used to feed back fault status information to the control terminal and pause the target action when any task condition is detected to be outside the preset condition range;

[0184] The control terminal 61 further includes:

[0185] A condition reset module, used to reset the task conditions according to the fault status information and update the target task package.The data is distributed to the fire emergency robot.

[0186] Optionally, the fire emergency robot includes a processing center component, a sensing component, and an action execution component. Module 623, as described on pages 12 / 13 of the Action Execution Specification (CN 120886234 A), is specifically used for:

[0187] The processing center component parses the target task package to obtain multiple sorted target actions;

[0188] The processing center component calls the action execution component to execute each target action sequentially;

[0189] The processing center component calls the sensing component to obtain visual information after the target action interacts with the current environment;

[0190] The processing center component adjusts the action of the action execution component according to the visual information until all target actions have been executed.

[0191] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0192] In the several embodiments provided by this invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of modules is merely a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces. The indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0193] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of this embodiment.

[0194] In addition, the functional modules in the various embodiments of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0195] The above description and embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Specification 13 / 13 pages 17 CN 120886234 AFigure 1. Appendix 1 / 6 of the instruction manual, page 18, CN 120886234 A Figure 2. Appendix 2 / 6 of the instruction manual, page 19, CN 120886234 A Figure 3. Appendix 3 / 6 of the instruction manual, page 20, CN 120886234 A Figure 4. Appendix 4 / 6 of the instruction manual, page 21, CN 120886234 A Figure 5. Appendix 5 / 6 of the instruction manual, page 22, CN 120886234 A Figure 6. Appendix 6 / 6 of the instruction manual, page 23, CN 120886234 A Abstract: Abdominal ultrasound examination method, system, and device. REMOTE AND SEMI-AUTONOMOUS METHOD AND SYSTEM FOR CONTROLLING FIREFIGHTING AND EMERGENCY ROBOT. The present application provides a remote and semi-autonomous method and system for controlling a firefighting and emergency robot. A search task package is transmitted to the firefighting and emergency robot by a control terminal in response to a search command transmitted by an operator; a search... operation is performed by the firefighting and emergency robot based on the search task package, to return scene status information to the control terminal for display; in response to detecting a task object by the firefighting and emergency robot, the task object is labeled in the scene status information by thefirefighting and emergency robot for display to the operator; in response to receiving a task instruction from the operator regarding the task object, an initial task package is determined, a task condition is requested from the operator, and a target task package is constructed based on the task condition and is transmitted to the firefighting and emergency robot by the control terminal; and a target operation is autonomously planned based on the target task package and is performed by the firefighting and emergency robot, thereby achieving semi-autonomous control of the firefighting and emergency robot. By performing the method combining a user decision-making with an autonomous planning of the robot, response flexibility of the robot in emergency scenarios is effectively improved.

Claims

1. A remote semi-automatic control method for a fire emergency robot, characterized in that, The method involves a control terminal with remote communication connection and a fire emergency robot, and includes: The control terminal responds to the search command and sends a search task package to the fire emergency robot; The fire emergency robot performs a search action according to the search task package and returns scene status information to the control terminal for display. When the fire emergency robot detects any task object, it marks the task object in the scene status information; When the control terminal receives a task package instruction for the task object, it selects an initial task package and sets task conditions, constructs a target task package, and sends it to the fire emergency robot. The fire emergency robot performs the target actions according to the target task package.

2. The method according to claim 1, characterized in that, The method further includes: The control terminal responds to the task package creation instruction and selects multiple task actions from a preset action library; The control terminal responds to the requirement setting command, selects the pending action that matches the requirement setting command from multiple task actions, and creates a condition setting page; The control terminal responds to the sorting operation command, sorts all the task actions and the pending actions, and generates a candidate task package.

3. The method according to claim 1, characterized in that, When the control terminal receives a task package instruction for the task object, it selects an initial task package and sets task conditions, constructs a target task package, and sends it to the fire emergency robot, including: When the control terminal receives a task package instruction for the task object, it selects the initial task package specified by the task package instruction from a plurality of candidate task packages. The control terminal displays the condition setting page corresponding to the initial task package in the predetermined display area of ​​the scene status information; The control terminal responds to the condition information entered on the condition setting page, generates a target task package, and sends it to the fire emergency robot.

4. The method according to claim 3, characterized in that, If the target of the task is a fire source, then the initial task package is a fire extinguishing task package. The control terminal responds to the condition information entered on the condition setting page, generates a target task package, and sends it to the fire emergency robot, including: The control terminal responds to the first input trigger, selects the corresponding fire extinguishing agent limitation condition on the condition setting page, and generates the target task package. The control terminal sends the target task package to the fire emergency robot, so that the fire emergency robot can select a fire extinguishing agent that meets the fire extinguishing agent limitation conditions and perform fire extinguishing actions.

5. The method according to claim 4, characterized in that, The control terminal is also equipped with an operating device, and includes: The control device responds to the first pointing action and generates the jet parabolic trajectory corresponding to the fire emergency robot in the scene status information. The existing target mission package is updated by responding to the command to determine the trajectory of the jet parabola through the manipulation device; The control terminal sends the updated target task package to the fire emergency robot, so that the fire emergency robot selects the extinguishing agent and performs the fire extinguishing action according to the jet parabolic trajectory.

6. The method according to claim 3, characterized in that, The control terminal is also equipped with a manipulation device. If the task object is a moving target, the initial task package is a moving task package. The control terminal responds to the condition information entered on the condition setting page, generates a target task package, and sends it to the fire emergency robot, including: By responding to the second pointing action of the manipulation device at the optical axis specified in the scene state information, the primary object to be moved and the object to be moved are selected to generate the first condition information; The control terminal loads the first condition information on the condition setting page and generates a target task package; The control terminal sends the updated target task package to the fire emergency robot, so that the fire emergency robot moves the primary object to the object relocation destination.

7. The method according to claim 3, characterized in that, If the task target is a valve, then the initial task package is a valve start / stop task package. The control terminal responds to the condition information entered on the condition setting page, generates a target task package, and sends it to the fire emergency robot, including: The control terminal responds to the second input trigger and selects the valve position box corresponding to the valve target from the scene status information; The control terminal responds to the second condition information entered on the condition setting page and generates a target task package; the second condition information includes rotation direction, rotation speed, and rotational torque. The control terminal sends the target task package to the fire emergency robot, so that the fire emergency robot uses the rotational torque to perform valve opening and stopping actions along the rotational direction at the rotational speed.

8. The method according to claim 1, characterized in that, The method further includes: When the fire emergency robot detects that any of the task conditions are not within the preset range, it sends fault status information to the control terminal and suspends the target action. The control terminal resets the task conditions according to the fault status information and updates the target task package sent to the fire emergency robot.

9. The method according to claim 1, characterized in that, The fire emergency robot includes a processing center component, a sensing component, and an action execution component. The fire emergency robot executes target actions according to the target task package, including: The processing center component parses the target task package to obtain multiple sorted target actions; The processing center component calls the action execution component to execute each of the target actions in sequence; The processing center component calls the perception component to obtain visual information after the target action interacts with the current environment; The processing center component adjusts the actions of the action execution component according to the visual information until all the target actions have been executed.

10. A remote semi-automatic control system for a fire emergency robot, characterized in that, Includes a remote communication connection between a control terminal and a fire emergency robot; the control terminal includes: The search module is used to respond to search commands and send search task packages to the fire emergency robot. The task distribution module is used to select an initial task package and set task conditions when it receives a task package instruction for the task object, and then construct a target task package to distribute to the fire emergency robot. The fire emergency robot includes: The scene search module is used to perform search actions according to the search task package and return scene status information to the control terminal for display. The task object labeling module is used to label any task object in the scene status information when the fire emergency robot detects any task object. The action execution module is used for the fire emergency robot to perform target actions according to the target task package.

Images