A window-breaking method based on scene understanding
Through drone dynamic modeling, sliding mode momentum interactive force estimation and force/position hybrid control, combined with physical interaction and passive energy management, the problem of complex structure and relying on pilot control in drone window breaking technology is solved, and the problem of using the simulator to achieve Efficient and safe independent window breaking operation.
Patent Information
- Application Number
- CN202411104525.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-13
AI Technical Summary
The existing drone window breaking technology has complex mechanical structures, resulting in low reliability, and relying on pilot control, making it difficult to achieve high-difficulty autonomous window breaking operation, and lack of flight safety in dynamic environments.
Through dynamic modeling, sliding mode momentum interactive force estimator design and force/position hybrid controller, combined with physical interaction and passive energy management, the drone can achieve precise control and autonomous adaptation during the window breaking process.
It significantly improves the safety operation success rate and task quality of the drone in dynamic changing environments, ensuring the reliability and stability of autonomous window breaking.
Smart Images

Figure CN118662804B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a window breaking and crossing technology under scene understanding, which is applicable to an unmanned aerial vehicle system that needs to perform window breaking operations in tasks such as emergency rescue, fire fighting, and security patrol, and belongs to the field of special operations of flying robots. Background Art
[0002] With the continuous advancement of drone technology, drones have achieved remarkable results in open spaces and special operations, and are widely used in agriculture, logistics, security and other fields. However, there are relatively few studies on drone operations in closed spaces, especially breaking windows to enter closed areas. Drone window breaking operations to enter closed spaces have important practical significance and potential value. For example, in emergency rescue scenarios, such as underground pipe gallery fires, chemical leaks and other emergencies, drones can quickly enter the accident site through window breaking technology, conduct environmental monitoring, preliminary rescue and data collection, and provide timely and accurate information for subsequent rescue work, significantly improving rescue efficiency and effectiveness. In these scenarios, drone window breaking technology can enter closed spaces that are difficult to reach by traditional methods, conduct efficient patrols and monitoring, ensure regional safety and provide key on-site data support.
[0003] As an emerging field, drone window breaking technology has shown great potential in specific scenarios, but it still faces many challenges and problems in practical applications and research. The first is the design and optimization of window breaking tools. There are significant differences in the efficiency of window breaking between the commonly used window breaking hammers and window breakers. How to combine these tools with drones to design efficient, adaptable and reliable window breaking equipment has become a difficult problem in the research of drone window breaking technology. In addition, during the window breaking process, the operating environment of the drone will change, and these dynamic changes may cause the state of the drone to change. How to ensure the flight safety of the drone after the environment changes is also a major challenge.
[0004] Chinese patent application CN202311587863.9 (A window-breaking firefighting drone) discloses a window-breaking drone consisting of a drone body and support plates arranged on the outer walls around the drone body. The outer walls around the drone body are fixedly connected with clamping plates, and the support plates are fixedly connected between the top and bottom inner walls of the clamping plates. The top outer wall at one end of the support plate is rotatably connected with a rotating shaft, and the circumferential outer wall of the rotating shaft is fixedly connected with multiple blades distributed at equal distances, and the bottom outer wall of the support plate is fixedly connected with a driving motor. The gear rod will push out the mounting plate that fits the drone body. As the mounting plate is extended laterally, the impact head will be continuously pushed to the position where the window needs to be broken, thereby utilizing the speed of the drone itself and the speed at which the mounting plate is pushed out laterally to achieve the window-breaking effect.
[0005] Chinese patent application CN202210903825.9 (A high-speed drone emergency rescue window breaking method) discloses a high-speed drone emergency rescue window breaking method, including a drone, a balance bar, a push rod, a fixed rod, a collision block, and a fixed block; specifically including positioning, hitting the glass and breaking the glass, after the collision block hits the glass, it is sucked on the glass, and the glass is further blasted by the combustible material inside the collision block, causing the glass to break. In this invention, the collision is performed by the stretching and rebounding movement of the collision block, and the impact force of the collision block is increased by mechanical movement, so that the collision block is subjected to force at the moment of contact with the window, and the force is accepted and converted into impact force to hit the window, and then the collision block is adsorbed on the window, and the window is further blasted, so that emergency treatment can be carried out in time, safety is improved, and remote control is performed through a remote control, which is convenient to operate, prevents innocent people around from being affected, and has a better safety factor.
[0006] It can be seen that the above invention patents are mainly focused on the structural design of window-breaking drones, which have two main problems: first, the structure of window-breaking drones is relatively complex, and the complex mechanical structure will reduce the reliability of the window-breaking operation process; second, the flight of the drone during the window-breaking process mainly depends on the pilot's control, so the success rate of window breaking is heavily dependent on the pilot's experience, and the quality of the operation cannot be guaranteed. In summary, the above methods do not consider the problem of intelligent autonomous window breaking under the simple window-breaking drone structure composed of a window breaker and a drone, and it is difficult to complete the difficult autonomous window-breaking operation task. Summary of the invention
[0007] The technical problem solved by the present invention is: to overcome the defects of the existing methods and provide a window-breaking method under scene understanding for an autonomous window-breaking drone system. The framework can ensure the safety of the window-breaking drone during operation and improve the autonomous adaptability of the window-breaking drone in the process of external environment changes.
[0008] The technical solution of the present invention is: a broken window crossing method under scene understanding, and its implementation steps are as follows:
[0009] The first step is to model the dynamics of the window-breaking drone.
[0010] When the drone is breaking a window, the additional external force generated by its contact with the glass will affect the dynamics of the drone. According to the Newton-Lagrange theorem, considering the interactive external force during the contact process, the dynamic model of the drone during the window breaking process is established as follows: In the formula, Indicates the position of the drone. Represents the speed of the drone, is the total weight of the window-breaking drone, represents the drone control amount, represents the gravitational acceleration vector, Indicates that external force is encountered during the window breaking process.
[0011] The second step is to design the sliding mode momentum interaction force estimator.
[0012] When breaking windows, drones need to apply interaction forces accurately to effectively break the glass. In addition, the window breaking process is generally short, and estimating the interaction force in a short time is also a big challenge. Therefore, in order to achieve accurate interaction force estimation in a short time during the window breaking process, a sliding mode momentum observer is designed to meet the needs of external force estimation during the window breaking process. The interaction force observer estimator is designed as follows: In the formula represents the momentum of the drone, represents the estimated value of the UAV momentum, represents the momentum estimation error, represents the estimated contact force, Represents a symbolic function.
[0013] The third step is to design the force / position hybrid window breaking controller;
[0014] During the window-breaking process of a drone, on the one hand, it is necessary to ensure the precise interaction force application point, that is, to ensure accurate trajectory tracking control. The drone can accurately locate and adjust its own position through the onboard sensors and flight control system, so that it can accurately track the preset trajectory during flight. On the other hand, the drone needs to apply a window-breaking force forward to trigger the limit switch of the onboard window breaker. After the limit switch is activated, the metal probe in the window breaker pops out quickly, accurately hits the glass, and instantly shatters it, paving the way for the smooth progress of the drone's subsequent missions.
[0015] In order to meet the above requirements of precise position and force control, this section designs a position controller and a force tracking controller for the UAV, where the position tracking controller is in the form of the following: In the formula and are the proportional coefficient matrix and the differential coefficient matrix respectively, is the position tracking error, is the speed tracking error. Output of the position controller.
[0016] The contact force tracking controller works together with the position controller, and its specific form is as follows: In the formula and are the proportional coefficient and differential coefficient matrices respectively, is the force tracking error, It is the expected tracking external force, that is, the external force required to trigger the window breaker limit switch.
[0017] In summary, the total control amount of the UAV mission can be written as follows: The fourth step is to design a safety strategy for window-breaking drones.
[0018] When the drone breaks the glass, the state of the drone will change significantly due to the change in the contact surface. At this time, its behavior needs to be properly adjusted to ensure that the drone can adaptively adjust its state according to the changes in the environment. This adjustment helps the drone to remain stable and efficient in complex environments and successfully complete the mission.
[0019] By definition, this physical interaction involves a two-way exchange of energy between the drone and the environment. In this regard, passivity theory provides a powerful tool to analyze the stability properties of control systems without having to make assumptions about the structure of the environment. Intuitively, a system is passive if it does not generate energy itself.
[0020] First, analyze the passivity of the closed-loop system relative to the power port , the system energy function is constructed as follows: In the formula For quality The passivity indicates that the system does not generate internal energy, that is, it satisfies the following inequality: Since the mass of the drone does not change during flight, according to the above formula The derivative of can be obtained as follows: Using the UAV model and the designed force / position hybrid controller, the closed-loop equation of the system can be obtained as follows: Substituting the closed-loop equation of the system into the above formula, we can obtain: In the formula , we can see that there is an energy injection term in the above formula The entire system does not meet the passive nature, and the closed-loop system cannot guarantee stability during operation. In order to meet the passive nature of the system, the system can be connected to a virtual energy tank. As an energy storage device, the energy tank can store system energy by connecting to the system and manage the energy of the system during the window breaking process. The design of the energy tank is as follows: In the formula It is the state of the energy tank. is the input and output of the power port, and: , The energy stored in the system is compensated by the energy injected into the system (the control item that causes the system energy to increase in the controller) and the energy dissipated by the system (the control item that causes the system energy to decrease in the controller) is stored. The energy storage device (virtual energy tank) and the closed-loop system constitute the entire system, which can realize the passive control of the entire system, that is: In the formula Indicates whether to enable the energy storage process. It is used to control the speed of energy storage in the energy tank. From the above formula, we can get that the energy flow power of the energy tank is: At this point, the energy of the entire system consisting of the energy tank and the drone system is expressed as: The total power of the system is characterized by: It can be seen that the total power of the system is less than the power injected by the external force. As long as the energy storage tank is not completely exhausted, the passivity of the system can be guaranteed when interacting with the external environment.
[0021] In order to avoid causing unstable control output of the UAV in practice, a limited energy must be allocated for the window breaking task. To this end, it is necessary to set upper and lower energy values for the tank, which are and To ensure the execution of the task, we set , and the minimum energy tank value meets .
[0022] As can be seen from the above, as long as the energy storage tank is not exhausted to the lowest point, the passivity and stability of the system can be guaranteed. Therefore, the energy dissipation speed in the energy tank can be limited by constraining the power and minimum energy of the energy tank, so as to avoid the energy tank from being consumed to the minimum value, which can be achieved by using a power valve.
[0023] Considering the control input that affects the passivity , its input power can be achieved by adjusting the corresponding power flow, that is, by modifying the input of the controller. The modified controller input is: In the formula is the scale factor, whose value is ,in: is the maximum power value.
[0024] Finally, the energy and power limiting factors are multiplied by a general factor ,Right now , so that in This prevents energy from being drawn from the storage tank, thus ensuring the stability of the system.
[0025] In summary, considering the power and energy constraints, the control amount of the UAV is: After the above adjustment of the controller output, the passivity of the entire system consisting of the closed-loop system and the virtual energy tank can be ensured, thereby ensuring the stability of the UAV's flight during the entire window breaking process.
[0026] The advantages of the present invention compared with the prior art are:
[0027] Compared with the traditional impact-type window breaking method, this method can significantly improve the success rate of safe operation of window-breaking drones in dynamically changing environments, ensure the quality of the operation tasks, and has the advantages of good safety and high success rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is a design flow chart of a window-breaking method based on scene understanding of the present invention. DETAILED DESCRIPTION
[0029] A general window-breaking UAV platform is taken as an example to illustrate the specific implementation of the system and method. When performing window-breaking tasks, the UAV has very high requirements for its own safety.
[0030] like Figure 1 As shown, the specific implementation steps of the present invention are as follows:
[0031] The first step is to model the dynamics of the window-breaking drone.
[0032] When the drone is breaking a window, the additional external force generated by its contact with the glass will affect the dynamics of the drone. According to the Newton-Lagrange theorem, considering the interactive external force during the contact process, the dynamic model of the drone during the window breaking process is established as follows: In the formula, Indicates the position of the drone. Represents the speed of the drone, is the total weight of the window-breaking drone, represents the drone control amount, represents the gravitational acceleration vector, Indicates that external force is encountered during the window breaking process.
[0033] The second step is to design the sliding mode momentum interaction force estimator.
[0034] When breaking windows, drones need to apply interaction forces accurately to effectively break the glass. In addition, the window breaking process is generally short, and estimating the interaction force in a short time is also a big challenge. Therefore, in order to achieve accurate interaction force estimation in a short time during the window breaking process, a sliding mode momentum observer is designed to meet the needs of external force estimation during the window breaking process. The interaction force observer estimator is designed as follows: In the formula represents the momentum of the drone, represents the estimated value of the UAV momentum, represents the momentum estimation error, represents the estimated contact force, Represents a symbolic function.
[0035] The third step is to design the force / position hybrid window breaking controller;
[0036] During the window-breaking process of a drone, on the one hand, it is necessary to ensure the precise interaction force application point, that is, to ensure accurate trajectory tracking control. The drone can accurately locate and adjust its own position through the onboard sensors and flight control system, so that it can accurately track the preset trajectory during flight. On the other hand, the drone needs to apply a window-breaking force forward to trigger the limit switch of the onboard window breaker. After the limit switch is activated, the metal probe in the window breaker pops out quickly, accurately hits the glass, and instantly shatters it, paving the way for the smooth progress of the drone's subsequent missions.
[0037] In order to meet the above requirements for precise position and force control, this section designs a position controller and a force tracking controller for the UAV. The form of the position tracking controller is as follows: In the formula and are the proportional coefficient matrix and the differential coefficient matrix respectively, is the position tracking error, is the speed tracking error. Output of the position controller.
[0038] The contact force tracking controller works together with the position controller, and its specific form is as follows: In the formula and are the proportional coefficient and differential coefficient matrices respectively, is the force tracking error, It is the expected tracking external force, that is, the external force required to trigger the window breaker limit switch.
[0039] In summary, the total control amount of the UAV mission can be written as follows: The fourth step is to design a safety strategy for window-breaking drones.
[0040] When the drone breaks the glass, the state of the drone will change significantly due to the change in the contact surface. At this time, its behavior needs to be properly adjusted to ensure that the drone can adaptively adjust its state according to the changes in the environment. This adjustment helps the drone to remain stable and efficient in complex environments and successfully complete the mission.
[0041] By definition, this physical interaction involves a two-way exchange of energy between the drone and the environment. In this regard, passivity theory provides a powerful tool to analyze the stability properties of control systems without having to make assumptions about the structure of the environment. Intuitively, a system is passive if it does not generate energy itself.
[0042] First, analyze the passivity of the closed-loop system relative to the power port , the system energy function is constructed as follows: In the formula For quality The passivity indicates that the system does not generate internal energy, that is, it satisfies the following inequality: Since the mass of the drone does not change during flight, according to the above formula The derivative of can be obtained as follows: Using the UAV model and the designed force / position hybrid controller, the closed-loop equation of the system can be obtained as follows: Substituting the closed-loop equation of the system into the above formula, we can obtain: In the formula , we can see that there is an energy injection term in the above formula , the entire system does not meet the passive nature, and the closed-loop system cannot guarantee stability during operation. In order to meet the passive nature of the system, the system can be connected to a virtual energy tank. As an energy storage device, the energy tank can store system energy by connecting to the system and manage the energy of the system during the window breaking process. The design of the energy tank is as follows: In the formula It is the state of the energy tank. is the input and output of the power port, and: The energy stored in the system is compensated by the energy injected into the system (the control item that causes the system energy to increase in the controller) and the energy dissipated by the system (the control item that causes the system energy to decrease in the controller) is stored. The energy storage device (virtual energy tank) and the closed-loop system constitute the entire system, which can realize the passive control of the entire system, that is: In the formula Indicates whether to enable the energy storage process. It is used to control the speed of energy storage in the energy tank. From the above formula, we can get that the energy flow power of the energy tank is: At this point, the energy of the entire system consisting of the energy tank and the drone system is expressed as: The total power of the system is characterized by: From the above formula, we can see that the total power of the system is less than the power injected by the external force. As long as the energy storage tank is not completely exhausted, the passivity of the system can be guaranteed when interacting with the external environment.
[0043] In order to avoid causing unstable control output of the UAV in practice, a limited energy must be allocated for the window breaking task. To this end, it is necessary to set upper and lower energy values for the tank, which are and To ensure the execution of the task, we set , and the minimum energy tank value meets As can be seen from the above, as long as the energy storage tank is not exhausted to the lowest point, the passivity and stability of the system can be guaranteed. Therefore, the energy dissipation speed in the energy tank can be limited by constraining the power and minimum energy of the energy tank, so as to avoid the energy tank from being consumed to the minimum value, which can be achieved by using a power valve.
[0044] Considering the control input that affects the passivity , its input power can be achieved by adjusting the corresponding power flow, that is, by modifying the input of the controller. The modified controller input is: In the formula is the scale factor, whose value is ,in: is the maximum power value.
[0045] Finally, the energy and power limiting factors are multiplied by a general factor ,Right now , so that in This prevents energy from being drawn from the storage tank, thus ensuring the stability of the system.
[0046] In summary, considering the power and energy constraints, the control amount of the UAV is: After the above adjustment of the controller output, the passivity of the entire system consisting of the closed-loop system and the virtual energy tank can be ensured, thereby ensuring the stability of the UAV's flight during the entire window breaking process.
[0047] The contents not described in detail in the specification of the present invention belong to the prior art known to the professional and technical personnel in this field.
Claims
1. A window-breaking method based on scene understanding, characterized in that: The following steps are involved: Step 1: Dynamic modeling of window-breaking drone; Step 2: Design of sliding mode momentum interaction force estimator; Step 3: Design of force / position hybrid window breaking controller; Step 4: Design of safety strategy for window-breaking drones; The first step is as follows: According to the Newton-Lagrange theorem, considering the interactive external forces during the contact process, the dynamic model of the drone during the window breaking process is established as follows: In the formula, Indicates the position of the drone. Represents the speed of the drone, is the total weight of the window-breaking drone, represents the drone control amount, represents the gravitational acceleration vector, Indicates that external force is encountered during the window breaking process; The second step is as follows: In order to achieve accurate interaction force estimation in a short time during the window breaking process, a sliding mode momentum observer is designed to meet the requirements of external force estimation during the window breaking process. The interaction force observer estimator is designed as follows: In the formula represents the momentum of the drone, represents the estimated value of the UAV momentum, represents the momentum estimation error, represents the estimated contact force, represents a symbolic function; The third step is as follows: In order to meet the needs of precise position and force control, a position controller and a force tracking controller are designed for the drone. The form of the position controller is as follows: In the formula and are the proportional coefficient matrix and the differential coefficient matrix respectively, is the position tracking error, is the speed tracking error, It is the output of the position controller; Working together with the position controller is the force tracking controller, which is shown below: In the formula and are the proportional coefficient and differential coefficient matrices respectively, is the force tracking error, is the desired tracking external force; The total control amount of the UAV mission is written as follows: ; The fourth step is as follows: First, analyze the passivity of the closed-loop system relative to the power port , the system energy function is constructed as follows: In the formula For quality The diagonal matrix formed by ; the passivity indicates that the system does not generate internal energy, that is, it satisfies the following inequality: Since the mass of the drone does not change during flight, according to the above formula The derivative of , we get the following formula: Using the UAV dynamics model and the designed force / position hybrid window breaking controller, the closed-loop equation of the system is obtained as follows: Substituting the closed-loop equation of the system into the above formula, we get: In the formula , we can see that there is an energy injection term in the above formula , the whole system does not meet the passive nature, and the closed-loop system cannot guarantee stability during operation; in order to meet the passive nature of the system, the system is connected to a virtual energy tank; the energy tank acts as an energy storage device, which stores system energy by connecting to the system and manages the energy of the system during the window breaking process; The design of the energy tank is as follows: In the formula It is the state of the energy tank. is the input and output of the power port, and at the same time, The energy stored in the system is compensated by the energy injected into the system and the energy dissipated by the system is stored. The energy storage device and the closed-loop system constitute the entire system to realize the passive control of the entire system, that is: In the formula Indicates whether to enable the energy storage process. It is used to control the speed of energy storage in the energy tank. According to the above formula, the energy flow power of the energy tank is: At this point, the energy of the entire system consisting of the energy tank and the drone system is expressed as: The total power of the system is characterized by: It can be seen that the total power of the system is less than the power injected by the external force. As long as the energy tank is not completely exhausted, the passivity of the system can be guaranteed when interacting with the external environment; In order to avoid causing unstable control output of the UAV in practice, a limited energy must be allocated for the window breaking task. To this end, it is necessary to set upper and lower energy values for the energy tank, which are and ; To ensure the execution of the task, we set , and the minimum energy tank value meets ; It can be seen that as long as the energy tank is not exhausted to the lowest point, the passivity and stability of the system can be guaranteed; Therefore, the energy dissipation speed in the energy tank is limited by constraining the power and minimum energy of the energy tank, so as to avoid the energy tank energy consumption reaching the minimum value, which can be achieved by using a power valve. Considering the control input that affects the passivity , its input power is realized by adjusting the corresponding power flow, that is, by modifying the input of the controller. The modified controller input is: In the formula is the scale factor, whose value is ,in: is the maximum power value; Finally, the scale factors are multiplied by a common factor ,Right now , so that in When the energy is in the energy tank, it blocks the energy from being obtained, thus ensuring the stability of the system; In summary, considering the power and energy constraints, the control amount of the UAV is: After the above adjustment of the controller output, the passivity of the entire system consisting of the closed-loop system and the virtual energy tank can be ensured, thereby ensuring the stability of the UAV's flight during the entire window breaking process.
