Public safety agent perception planning control system and method
By combining a multimodal perception module, an intelligent planning module, and a precise control module, and utilizing reinforcement learning and imitation learning algorithms, a public safety intelligent agent perception, planning, and control system is constructed. This solves the problems of perception accuracy, path planning, and collaborative control in complex scenarios of existing systems, and achieves efficient public safety management.
Patent Information
- Application Number
- CN202511653328.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-12
- Publication Date
- 2026-02-17
AI Technical Summary
Existing public safety management systems suffer from insufficient real-time perception accuracy, suboptimal path planning, inefficient equipment coordination and control, and inadequate information fusion capabilities when facing complex scenarios and diverse threats, making it difficult to effectively address security threats and disaster relief needs in densely populated areas.
By employing a multimodal perception module, an intelligent planning module, a precise control module, and a model training and optimization module, combined with reinforcement learning and imitation learning algorithms, multimodal data fusion, intelligent planning, and collaborative control are achieved, thus constructing a public safety intelligent agent perception, planning, and control system.
It has improved the accuracy and scope of environmental perception, optimized the efficiency of planning and decision-making, enhanced collaborative control capabilities, improved system adaptability and robustness, reduced management costs, and enhanced the intelligence level and efficiency of public safety management.
Smart Images
Figure CN121544442A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of the intersection of artificial intelligence and public safety, and relates to a public safety intelligent agent perception planning control system and method. Background Technology
[0002] In current public safety management, traditional security and emergency response methods face numerous severe challenges. On the one hand, existing monitoring systems largely rely on manual monitoring or simple video analysis algorithms, making it difficult to achieve real-time, comprehensive, and accurate understanding of complex scenarios. For example, in densely populated places such as large shopping malls and train stations, the high-density flow of people, obstruction, and complex behavioral patterns make traditional vision-based target detection and tracking algorithms prone to misjudgments and missed detections, failing to promptly detect potential security threats such as abnormal gatherings and violent conflicts.
[0003] On the other hand, in emergency response scenarios, such as fires and earthquakes, rescue personnel and equipment lack efficient intelligent planning and control tools. Traditional path planning methods fail to fully consider dynamically changing environmental factors, such as the spread of fire and smoke at the fire scene, and road collapses and building debris distribution after an earthquake, resulting in suboptimal rescue routes and delays in rescue efforts. Simultaneously, the lack of effective coordinated control among various emergency equipment prevents the formation of an efficient emergency response system, reducing the overall efficiency of rescue operations.
[0004] Furthermore, faced with increasingly diverse and intelligent security threats, such as complex crimes combining cyberattacks and physical security threats, traditional security systems are struggling to keep up with information fusion and decision-making capabilities. Data from different types of sensors (such as cameras and sensor networks) are isolated and cannot be deeply integrated and analyzed, making it difficult to uncover potential security risk correlations and resulting in a lack of comprehensiveness and foresight in security decisions.
[0005] Therefore, developing a public safety intelligent agent model that can comprehensively perceive complex environmental information, accurately plan action strategies, and achieve efficient collaborative control has become an urgent need to improve public safety assurance capabilities and address new security challenges. Summary of the Invention
[0006] In order to overcome at least one deficiency of the prior art, the present invention provides a public safety intelligent agent perception planning control system and method.
[0007] To achieve the above objectives, the present invention adopts the following technical solution: a public safety intelligent agent perception planning and control system, comprising a multimodal perception module, an intelligent planning module, a precise control module, and a model training and optimization module. The multimodal perception module is connected to the intelligent planning module, and the intelligent planning module and the multimodal perception module are respectively connected to the precise control module. The model training and optimization module is respectively connected to the multimodal perception module, the intelligent planning module, and the precise control module.
[0008] Furthermore, the multimodal perception module is used to collect and process environmental data from various sensors, including visual images, audio signals, and environmental parameters. The multimodal perception module provides real-time environmental perception information to the intelligent planning module, while also receiving feedback information from the intelligent planning module and adjusting the perception strategy according to task requirements.
[0009] Furthermore, the intelligent planning module, based on the environmental perception information and task objectives provided by the multimodal perception module, uses reinforcement learning algorithms to generate the optimal action planning strategy and outputs the planning results to the precision control module.
[0010] Furthermore, the precision control module performs precise control of public safety equipment based on the instructions generated by the intelligent planning module, including equipment motion control, operation execution, and coordinated operation.
[0011] Furthermore, the precision control module receives and executes control commands from the intelligent planning module, while simultaneously transmitting the actual operating status of the equipment and environmental feedback information back to the intelligent planning module. The precision control module works in collaboration with the multimodal sensing module to adjust control parameters based on the sensing information, adapting to the dynamically changing environment.
[0012] Furthermore, the model training and optimization module provides the multimodal perception module, intelligent planning module, and precise control module with trained and optimized multimodal perception model, intelligent planning model, and precise control model, and updates and optimizes the model online based on feedback data from actual applications.
[0013] A public safety intelligent agent perception planning and control method includes the following steps:
[0014] Step 1: Construct a multimodal sensing fusion system: Collect and process multimodal data and fuse the multimodal data to obtain comprehensive sensing information;
[0015] Step 2: Construct an intelligent planning model based on reinforcement learning and imitation learning. The intelligent planning model generates action planning strategies based on the fused data and task objectives.
[0016] Step 3: Establish a precise control and collaborative execution mechanism to execute control commands and drive the actions of public safety equipment;
[0017] Step 4: Implement adaptive control for the agent, enabling it to adjust control parameters and strategies in real time based on environmental changes and task execution.
[0018] Furthermore, the multimodal perception fusion system integrates visual, auditory, olfactory, and various environmental sensor data through a multimodal perception fusion framework, and performs feature-level and decision-level fusion of different types of sensor data through a multimodal data fusion algorithm.
[0019] Furthermore, the action planning strategy in step 2 specifically includes:
[0020] Step 21: Construct an intelligent planning model using reinforcement learning algorithms, imitation learning algorithms, and hierarchical planning mechanisms;
[0021] Step 22: The intelligent planning model generates an initial strategy based on the fused data and task objectives;
[0022] Step 23: Based on the reinforcement learning algorithm, the intelligent agent interacts with the environment through continuous trial and error in the public safety environment, and continuously adjusts its strategy to learn the optimal action strategy based on the reward signals fed back by the environment.
[0023] Step 24: Based on the imitation learning algorithm, the agent optimizes its strategy by combining learning from expert experience with autonomous learning.
[0024] Step 25: Generate macro-strategic planning and micro-tactical planning based on the hierarchical planning mechanism of public safety scenarios, and output the optimal action planning strategy.
[0025] Furthermore, the hierarchical planning mechanism divides public safety scenarios into macro and micro levels. At the macro level, long-term strategic plans are formulated based on task objectives and global environmental information, while at the micro level, short-term tactical plans are made in conjunction with real-time perception data.
[0026] In summary, the advantages of this invention are:
[0027] 1) This invention integrates multiple disciplines such as computer vision, sensor fusion, reinforcement learning, and path planning, aiming to empower various public safety equipment and systems, such as security robots, intelligent surveillance cameras, and emergency response drones, to enable them to have accurate environmental perception, efficient task planning, and reliable action control capabilities. This will improve the level of intelligent management in the field of public safety, effectively address security challenges in complex scenarios such as crime prevention, disaster emergency response, and management of densely populated areas, and provide solid technical support for ensuring public safety.
[0028] 2) This invention effectively improves the accuracy and range of perception: By using a multimodal perception fusion system, information from multiple sensors is comprehensively utilized to effectively improve the accuracy and range of perception in complex public safety scenarios.
[0029] 3) This invention can optimize planning and decision-making efficiency: Based on the intelligent planning model of reinforcement learning and imitation learning, the intelligent agent can quickly generate the optimal action strategy in a complex and ever-changing environment.
[0030] 4) This invention effectively enhances collaborative control capabilities: The precise control and collaborative execution mechanism ensures efficient collaboration among multiple agents and devices. In collaborative patrol tasks of multiple security robots, the distributed collaborative control strategy makes the task allocation among robots more reasonable, greatly improving the patrol coverage rate. At the same time, it avoids mutual collisions and repeated patrols, greatly improving the patrol effect and resource utilization efficiency.
[0031] 5) This invention improves system adaptability and robustness. Through model training and optimization modules, the entire system can adjust strategies and parameters in real time according to environmental changes and task requirements, exhibiting strong adaptability and robustness. When faced with sudden disasters causing drastic environmental changes, such as in the ruins after an earthquake, the rescue equipment can quickly adapt to the new environment and continue to perform rescue missions, effectively ensuring the continuity and reliability of public safety operations.
[0032] 6) This invention effectively reduces the cost of public safety management: This patent can realize the intelligent and automated management of public safety, reduce the reliance on a large number of manual guards and interventions, improve the efficiency and quality of safety management, reduce the probability of safety accidents, and thus indirectly reduce the economic losses caused by accidents, with significant economic and social benefits. Attached Figure Description
[0033] Figure 1 This is a diagram of the architecture of the public safety intelligent agent perception planning and control system of the present invention.
[0034] Figure 2 This is a flowchart of the multimodal sensing fusion process of the present invention.
[0035] Figure 3 This is a flowchart of the intelligent planning model decision-making process of the present invention. Detailed Implementation
[0036] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.
[0037] Example:
[0038] like Figure 1 As shown, a public safety intelligent agent perception planning and control system includes a multimodal perception module, an intelligent planning module, a precise control module, and a model training and optimization module. The multimodal perception module is connected to the intelligent planning module, and the intelligent planning module and the multimodal perception module are respectively connected to the precise control module. The model training and optimization module is connected to the multimodal perception module, the intelligent planning module, and the precise control module respectively.
[0039] The multimodal perception module is used to collect and process environmental data from various sensors, including visual images, audio signals, and environmental parameters. Advanced perception algorithms are used to analyze and understand the data, extracting key information related to public safety, such as personnel identity, behavioral status, and danger signals. The preliminarily processed information is then transmitted to the intelligent planning module.
[0040] The multimodal perception module provides the intelligent planning module with real-time and comprehensive environmental perception information, and its accuracy and timeliness greatly improve the decision-making quality of the intelligent planning module. Simultaneously, it receives feedback information from the intelligent planning module and adjusts its perception strategy according to task requirements, such as improving the perception accuracy of specific areas or targets when potential security threats are detected.
[0041] The intelligent planning module, based on environmental perception information and task objectives provided by the multimodal perception module, uses reinforcement learning algorithms to generate optimal action planning strategies. These strategies include developing long-term task execution plans and short-term real-time decisions, such as planning security patrol routes, emergency response procedures, and resource allocation schemes. The planning results are then output to the precision control module.
[0042] The intelligent planning module generates action planning strategies based on perceived information and task objectives, and sends planning instructions to the precision control module to control the agent's actions. Furthermore, through feedback interaction with the precision control module, it obtains action execution results to evaluate and improve the planning strategies, forming a closed-loop optimization mechanism.
[0043] The precision control module, based on instructions generated by the intelligent planning module, precisely controls various public safety equipment (such as security robots, drones, and surveillance cameras) to achieve motion control, operational execution, and collaborative operation. During the control process, it monitors the equipment's operating status and environmental feedback information in real time to ensure the accuracy and reliability of the control, and feeds back the execution results to the intelligent planning module.
[0044] The precision control module receives and executes control commands from the intelligent planning module, while simultaneously transmitting the actual operating status of the equipment and environmental feedback information back to the intelligent planning module, providing a basis for its subsequent decision-making. Furthermore, the precision control module works in conjunction with the multimodal sensing module to adjust control parameters based on sensing information to adapt to dynamically changing environments.
[0045] The model training and optimization module provides trained and optimized multimodal perception, intelligent planning, and precise control models for the multimodal perception, intelligent planning, and precise control modules, ensuring efficient operation of each module. It utilizes extensive public safety scenario data (including historical case data and simulation data) to perform supervised and unsupervised training on the models, adjusting model parameters to improve performance and generalization ability. Simultaneously, based on feedback data from real-world applications, the model is updated and optimized online to continuously adapt to the ever-changing public safety environment. Data and feedback information from each module during actual operation are collected for further model training and optimization, promoting continuous improvement in the overall system performance.
[0046] like Figures 2-3 As shown, a public safety intelligent agent perception planning and control method includes the following steps:
[0047] Step 1: Construct a multimodal sensing fusion system: Collect and process multimodal data and fuse the multimodal data to obtain comprehensive sensing information;
[0048] The multimodal perception fusion system integrates visual, auditory, olfactory, and various environmental sensor data through a multimodal perception fusion framework. It also uses multimodal data fusion algorithms to fuse different types of sensor data at the feature level and decision level.
[0049] Advanced deep learning-based object detection and recognition algorithms are employed to process visual perception data, enabling accurate identification of people, objects, and abnormal behaviors in complex backgrounds and under occlusion conditions. For example, by analyzing pedestrian postures and movement sequences, it can determine whether abnormal behaviors such as running or pushing exist; deep learning object detection and recognition algorithms include object detection models based on the improved Transformer architecture.
[0050] Sound recognition technology is used to detect danger signals such as gunshots, explosions, and cries for help in auditory perception data, and combined with sound source localization algorithms to determine the approximate location of the danger.
[0051] By using environmental sensing devices such as gas sensors and smoke sensors, changes in environmental parameters related to disasters such as fires and chemical leaks can be monitored in real time.
[0052] In feature-level fusion, neural networks are used to extract and fuse features from different modal data to generate a comprehensive feature vector. In decision-level fusion, the independent decision results of each modal sensor are combined, and the final comprehensive perception information is obtained through strategies such as voting and weighting, thereby improving the accuracy and reliability of perception.
[0053] Step 2: Construct an intelligent planning model based on reinforcement learning and imitation learning. The intelligent planning model generates action planning strategies based on the fused data and task objectives.
[0054] The specific steps are as follows:
[0055] Step 21: Construct an intelligent planning model using reinforcement learning algorithms, imitation learning algorithms, and hierarchical planning mechanisms;
[0056] Step 22: The intelligent planning model generates an initial strategy based on the fused data and task objectives;
[0057] Step 23: Based on the reinforcement learning algorithm, the intelligent agent interacts with the environment through continuous trial and error in the public safety environment, and continuously adjusts its strategy to learn the optimal action strategy based on the reward signals fed back by the environment.
[0058] For example, in security patrol missions, an agent receives a positive reward for each security hazard it successfully discovers and addresses; if it fails to respond in a timely manner or makes an incorrect decision, it receives a negative reward, thereby driving the agent to continuously optimize patrol paths and decision-making strategies.
[0059] Step 24: Based on the imitation learning algorithm, the agent optimizes its strategy by combining learning from expert experience with autonomous learning.
[0060] Expert experience, including patrol route planning and emergency response procedures from seasoned security personnel, can accelerate the learning process of intelligent agents and improve the quality of initial strategies. By utilizing technologies such as Generative Adversarial Networks (GANs), expert strategies can be integrated with strategies learned autonomously by the intelligent agent. This allows the agent to both draw upon the mature experience of human experts and continuously explore and innovate in the real environment, thereby enhancing the adaptability and effectiveness of its planning strategies.
[0061] Step 25: Generate macro-strategic planning and micro-tactical planning based on the hierarchical planning mechanism of public safety scenarios, and output the optimal action planning strategy.
[0062] The hierarchical planning mechanism divides public safety scenarios into macro and micro levels. At the macro level, long-term strategic planning is formulated based on task objectives and global environmental information, such as identifying key patrol areas and pre-deployment plans for emergency resources within specific time periods. At the micro level, short-term tactical planning is conducted using real-time perception data, such as quickly planning action paths and selecting appropriate countermeasures based on currently identified security threats. Through hierarchical planning, the efficiency and flexibility of planning are improved, enabling intelligent agents to make reasonable decisions at different time scales.
[0063] Step 3: Establish a precise control and collaborative execution mechanism to execute control commands and drive the actions of public safety equipment;
[0064] Based on the decision-making instructions generated by the intelligent planning model, various public safety equipment is precisely controlled, and the equipment's operating status and environmental feedback information are fed back to the intelligent planning model. For security robots, motion control algorithms are used to precisely adjust their speed, direction, and posture, ensuring that they can efficiently perform patrol and monitoring tasks according to the planned path, and respond quickly to security incidents when needed, such as rapidly approaching targets and collecting evidence on-site.
[0065] For multi-agent collaborative scenarios, such as collaborative patrols by multiple security robots and collaborative emergency response by drones and ground rescue equipment, a distributed collaborative control strategy is designed. Utilizing a distributed consensus algorithm, each agent reaches a consensus on actions based on shared information, enabling collaborative operations such as task allocation, path coordination, and resource sharing. For example, in fire rescue, drones are responsible for aerial reconnaissance of the fire and the situation of trapped personnel, transmitting the information in real time to ground rescue robots. The ground rescue robots plan rescue routes based on the information provided by the drones, while simultaneously coordinating with other rescue equipment to carry out firefighting and rescue operations, avoiding mutual interference and improving collaborative execution efficiency.
[0066] Step 4: Implement adaptive control for the agent, enabling it to adjust control parameters and strategies in real time based on environmental changes and task execution.
[0067] In scenarios with significant environmental changes, such as in the ruins after an earthquake, rescue robots can adjust their movement patterns and control parameters in real time according to changes in the terrain to maintain stable mobility and ensure the continued progress of rescue missions.
[0068] This embodiment uses a large urban railway station as a public scenario. Railway stations have a large flow of people and a complex environment, posing various security risks such as theft, violence, and fire. Both a traditional single visual perception system and the public safety intelligent agent perception planning and control system of this application are used to test the detection accuracy and security patrol efficiency of public safety at railway stations.
[0069] A large number of security devices are deployed at the train station, including intelligent surveillance cameras, security patrol robots, and environmental sensors. The multimodal perception module uses intelligent surveillance cameras to capture high-definition video images and, through a Transformer-based target detection and recognition algorithm, monitors abnormal behavior in crowds in real time, such as running or fighting. Simultaneously, it combines audio sensors to detect abnormal sounds such as shouts and alarms, and uses sound source localization technology to determine the location of the sound source. Environmental sensors monitor parameters such as temperature and smoke concentration in real time for fire early warning. After preprocessing, the data from each sensor is used to generate comprehensive environmental perception information through a multimodal data fusion algorithm and then transmitted to the intelligent planning module.
[0070] After receiving perceived information, the intelligent planning module makes decisions using a model that combines reinforcement learning and imitation learning. For example, in security patrol missions, the initial patrol path of the patrol robot references the patrol experience of senior security personnel (imitation learning). During actual patrols, the robot dynamically adjusts its patrol path based on real-time perceived environmental information, such as the discovery of suspicious individuals or areas, using reinforcement learning algorithms to improve patrol efficiency and safety. In emergencies such as fires, the intelligent planning module quickly plans the optimal fire extinguishing and evacuation routes based on fire spread models, personnel distribution information, and the location of fire-fighting equipment, and sends instructions to the precision control module.
[0071] The precision control module, based on instructions from the intelligent planning module, precisely controls the actions of security patrol robots and other emergency equipment. Patrol robots rapidly move to designated locations along planned paths to track and monitor suspicious individuals. In fire rescue operations, firefighting robots and equipment quickly reach the fire scene according to planned paths, coordinating firefighting efforts while guiding the orderly evacuation of personnel. During execution, the precision control module monitors the equipment's operational status and environmental feedback information in real time, such as the robot's battery level, walking speed, and changes in fire intensity, and feeds this information back to the intelligent planning module so that it can adjust its strategies promptly.
[0072] The model training and optimization module regularly collects data generated during the actual operation of the railway station, including various safety incident cases and equipment operation data, to train and optimize the multimodal perception model, intelligent planning model, and precision control model. By continuously optimizing the model parameters, the model's adaptability to the complex environment of the railway station and the accuracy of its decision-making are improved.
[0073] After practical application testing, the railway station public safety management system using this technology achieved an accuracy rate of over 95% in detecting abnormal behavior, improved security patrol efficiency by 40%, and shortened response time for emergencies such as fires by 60%, effectively enhancing the public safety level of railway stations and verifying the effectiveness and practicality of this invention in the field of public safety.
[0074] The same method was used to test other public safety scenarios. The test results showed that in the monitoring of densely populated places, the accuracy of detecting abnormal human behavior was improved by more than 30% compared with traditional single visual perception systems. It can detect potential security threats earlier and more accurately, providing strong support for timely preventive measures. In emergency response scenarios, the rescue route planning time was shortened by more than 50%, and the planned route was more reasonable, effectively avoiding dangerous areas, improving rescue efficiency, and increasing the probability of successful rescue. In the field of security monitoring, it can reduce manpower costs by more than 35%.
[0075] Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.
Claims
1. A public safety agent perception planning control system, characterized in that: The system comprises a multi-modal perception module, an intelligent planning module, a precise control module, and a model training and optimization module.
2. The public safety agent-aware planning and control system of claim 1, wherein: The multi-modal perception module is used to collect and process environmental data from various sensors, including visual images, audio signals, and environmental parameters.
3. The public safety agent awareness planning control system of claim 1, wherein: The intelligent planning module generates optimal action planning strategies based on the environmental perception information provided by the multi-modal perception module and task objectives.
4. The public safety agent awareness planning control system of claim 1, wherein: The precise control module controls public safety equipment accurately based on the instructions generated by the intelligent planning module.
5. The public safety agent awareness planning control system of claim 4, wherein: The precise control module receives control instructions from the intelligent planning module and executes them while feeding back the actual operation of the equipment and environmental feedback information to the intelligent planning module.
6. The public safety agent awareness planning control system of claim 1, wherein: The model training and optimization module provides trained and optimized multi-modal perception models, intelligent planning models, and precise control models for the multi-modal perception module, intelligent planning module, and precise control module. 7.A public safety intelligent agent perception planning control method, characterized in that: The system comprises the following steps: Step 1: Construct a multi-modal perception fusion system to collect and process multi-modal data and fuse them to obtain comprehensive perception information. Step 2: Construct an intelligent planning model based on reinforcement learning and imitation learning to generate action planning strategies based on fused data and task objectives. Step 3: Construct a precise control and collaborative execution mechanism to execute control instructions and drive public safety equipment to act. Step 4: Perform adaptive control on the agent to enable it to adjust control parameters and strategies in real time based on environmental changes and task execution conditions.
8. The method of claim 7, wherein: The multi-modal perception fusion system integrates visual, auditory, olfactory, and various environmental sensor data through a multi-modal perception fusion framework.
9. The system and method for public safety agent awareness planning control according to claim 7, wherein: The action planning strategy of Step 2 specifically includes: Step 21: Construct an intelligent planning model using reinforcement learning algorithms, imitation learning algorithms, and hierarchical planning mechanisms. Step 22: The intelligent planning model generates an initial strategy based on fused data and task objectives. Step 23: Based on reinforcement learning algorithms, the agent interacts with the environment through continuous trial and error in the public safety environment, adjusts strategies based on environmental feedback rewards, and learns optimal action strategies. Step 24: Based on imitation learning algorithms, the agent optimizes strategies by learning expert experience and self-learning. Step 25: generate macro-strategic planning and micro-tactical planning based on the hierarchical planning mechanism of public safety scenarios, and output the optimal action planning strategy.
10. The method of claim 9, wherein: The hierarchical planning mechanism divides the public safety scenario into a macro level and a micro level. On the macro level, long-term strategic planning is made according to task objectives and global environmental information. On the micro level, short-term tactical planning is made in combination with real-time perception data.