Public space safety risk identification system based on mobile inspection robot
By introducing the allocation of local and edge computing resources into the mobile inspection robot system and optimizing the inspection route, the problems of insufficient intelligence and low inspection efficiency in existing technologies are solved, enabling real-time identification and rapid response to public space safety risks.
Patent Information
- Application Number
- CN202410985171.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-26
AI Technical Summary
Existing mobile inspection robots lack sufficient intelligence in identifying safety risks in public spaces, resulting in data transmission delays and human error in judgment, leading to low inspection efficiency and difficulty in quickly identifying and addressing safety risks.
The system employs a mobile inspection robot architecture that combines local data processing and analysis modules, edge data processing and analysis modules, and cloud servers. It assigns security risk identification tasks, utilizes local and edge computing resources to reduce data transmission latency, and optimizes inspection routes based on real-time identification results through a path planning module.
It enables the immediate identification and rapid handling of safety risks, reduces the workload of personnel, improves inspection efficiency, and meets the needs of public spaces for rapid identification and response to safety risks.
Smart Images

Figure CN119017374A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of security technology, and in particular to a public space safety risk identification system based on a mobile inspection robot. Background Art
[0002] Maintaining social security is the core task of social governance, which is related to the safety of people's lives and property and the stability of social order. Public spaces (such as transportation hubs, cultural facilities, and commercial places) have the characteristics of large spatial scale and complex functions, high population density, and highly complex composition and activities. Security management usually involves multiple departments and multiple factors, requiring a high level of cross-departmental cooperation and comprehensive governance capabilities. Public spaces, as places where people gather and carry out activities, are highly related to the public interest and welfare, and the potential harm of security risks is extremely great. Therefore, the security prevention standards for public spaces are extremely strict.
[0003] Traditional public space security measures mainly consist of monitoring equipment and patrol personnel. Monitoring equipment, including environmental sensors and image acquisition equipment, is usually deployed in fixed locations, with monitoring blind spots and relying on long-term manual monitoring and diagnosis in the background. Patrol personnel regularly patrol fixed routes in public spaces, mainly identifying security risks through visual means. The workload is heavy and it is difficult for personnel to carry multiple monitoring equipment for a long time, which may lead to the omission of security risk events.
[0004] With the development of mobile inspection robot technology, some public spaces have deployed security inspection robots as an important supplement to existing security measures. Inspection robots can move freely in space, making up for the defects of fixed monitoring equipment with blind spots and poor flexibility. Inspection robots can also carry a variety of monitoring equipment for a long time, support in-depth inspections of various security risk events, and effectively reduce the workload of inspection personnel. Thanks to the robot's own intelligent endowment, some inspection robots have also begun to be used to autonomously analyze and identify some environmental safety risks, effectively assisting the back-end manual workflow.
[0005] Although the introduction of mobile inspection robots has improved the security management level of public spaces, the existing technical solutions still have the following two disadvantages:
[0006] First, the intelligence level of existing security inspection robots still needs to be improved. Similar to fixed monitoring, inspection robots generally have difficulty analyzing monitoring data autonomously, and still need to transmit data to the background for manual judgment. There is a significant time delay in the remote transmission of data to the user terminal, and manual post-processing will further delay the time for security risk identification and warning, and will also be accompanied by problems such as omissions and misjudgments. Therefore, the existing technical solutions do not meet the requirements of quickly identifying and handling security risks in public spaces, reducing personnel workload and governance complexity.
[0007] Secondly, the inspection efficiency of existing security inspection robots still needs to be improved. Inspection robots usually mechanically complete "net-casting" patrols along a preset route, and each work cycle takes a long time. In this inspection mode, the environmental information obtained by the robot's perception module cannot guide the motion module in real time. For example, even if a high-risk area is found, it is difficult to reverse the preset route and stop nearby for detailed inspection. Under the requirements of rapid discovery and rapid intervention of security risk events in public spaces, it is necessary to conduct key inspections of potential high-risk areas to improve efficiency, rather than mechanically traversing the entire space containing a large number of low-risk areas. Summary of the invention
[0008] According to one aspect of the present disclosure, a public space safety risk identification system based on a mobile inspection robot is provided, the system comprising: a mobile inspection robot, a local data processing and analysis module, an edge data processing and analysis module, a path planning module, and a cloud server, wherein:
[0009] The mobile inspection robot is used to walk along a planned path in a preset area and collect real-time environmental parameters, video image data, audio data, and positioning data at each position on the planned path;
[0010] The local data processing and analysis module is used to perform a first type of security risk identification task according to the environmental parameters;
[0011] The edge data processing and analysis module is used to perform a second type of security risk identification task according to the video image data and the audio data, wherein the computational complexity of the second type of security risk identification task is higher than that of the first type of security risk identification task;
[0012] The path planning module is used to re-plan the path according to the positioning data, the execution result of the first type of security risk identification task, and the execution result of the second type of security risk identification task;
[0013] The cloud server is used to interact with the mobile inspection robot, the local data processing and analysis module, and the edge data processing and analysis module for data exchange;
[0014] The computing power of the edge data processing and analysis module is greater than the computing power of the local data processing and analysis module.
[0015] Among them, the local data processing and analysis module and the path planning module are arranged on the mobile inspection robot, and the edge data processing and analysis module is arranged in the preset area.
[0016] In a possible implementation manner, the first type of security risk identification task includes at least one of the following subtasks:
[0017] A subtask of detecting excessive temperature based on the environmental parameters;
[0018] A subtask of detecting excessive dangerous gas content based on the environmental parameters;
[0019] A subtask of air quality detection based on the environmental parameters;
[0020] The subtask of fire point judgment based on thermal imaging images;
[0021] The subtask of judging the door and window opening status based on thermal imaging images,
[0022] Wherein, the video image data includes the thermal imaging image.
[0023] In a possible implementation, performing the first type of security risk identification task according to the environmental parameter includes:
[0024] Performing data preprocessing on the environmental parameters to obtain preprocessed environmental parameters;
[0025] The preprocessed environmental parameters are subjected to a first type of security risk identification task using any one or more of a threshold detection method, a statistical analysis method, a time series analysis method or a machine learning classification model.
[0026] In a possible implementation manner, the second type of security risk identification task includes at least one of the following subtasks:
[0027] A subtask of performing dangerous goods identification based on the video image data;
[0028] A subtask of performing face recognition based on the video image data;
[0029] A subtask of performing abnormal behavior identification of personnel based on the video image data;
[0030] A subtask of performing smoke recognition based on the video image data;
[0031] A subtask of performing fire identification based on the video image data;
[0032] A subtask of performing left-behind object recognition based on the video image data;
[0033] A subtask of performing people gathering detection based on the video image data and / or the audio data;
[0034] A subtask of detecting destructive behavior based on the audio data;
[0035] A subtask of abnormal sound detection is performed based on the audio data.
[0036] In a possible implementation, performing the second type of security risk identification task according to the video image data and the audio data includes:
[0037] Preprocessing the video image data and the audio data to obtain the preprocessed video image data and the audio data;
[0038] The preprocessed video image data and / or the audio data are input into a pre-trained security risk identification model, and the execution results of each subtask of the second type of security risk identification task are determined according to the output of the security risk identification model.
[0039] In a possible implementation, the re-planning a path according to the positioning data, the execution result of the first type of security risk identification task, and the execution result of the second type of security risk identification task includes:
[0040] If the execution result of any subtask in the execution result of the first type of security risk identification task or the execution result of the second type of security risk identification task shows that a security risk event of a corresponding specified level is identified, or the confidence level in the execution result of any subtask is lower than a first preset value, then the path is replanned for the subtask, wherein if the confidence level in the execution result is higher than a second preset value, it is determined that a corresponding security risk event is identified, and the second preset value is higher than the first preset value;
[0041] The mobile inspection robot is controlled to move according to the re-planned path, and the safety risk event is re-judged for the subtask.
[0042] In a possible implementation, replanning a path for the subtask includes:
[0043] Determine a local area according to the positioning data corresponding to the subtask;
[0044] A collection step is set for the local area, and a detailed inspection path is planned, wherein when the mobile inspection robot moves along the detailed inspection path, real-time environmental parameters, video image data, audio data, and positioning data of each position on the detailed inspection path are collected at each collection step.
[0045] In a possible implementation, the path planning module is further used to:
[0046] Receive external path control information, and generate a new planned path according to the path control information.
[0047] In a possible implementation, the mobile inspection robot includes: a robot chassis, a speaker, an upper power supply, a robot housing, a camera assembly, an environmental parameter sensor, an omnidirectional microphone, a sensor bracket, an upper computer, a data transmission module, a warning light strip, and a positioning device.
[0048] The robot chassis is used to carry the upper device to move after receiving a motion instruction, wherein the motion instruction is obtained based on the planned path.
[0049] The loudspeaker is used to transmit sound alarm information to people around the scene when the execution result of any subtask in the execution result of the first type of security risk identification task and the execution result of the second type of security risk identification task shows that a corresponding security risk event has been identified.
[0050] The upper power supply is used to provide electrical energy.
[0051] The robot shell is used to support the upper sensors and cameras and to protect the upper power supply and upper computer.
[0052] The camera assembly is used to collect the video image data.
[0053] The environmental parameter sensor is used to collect the environmental parameter.
[0054] The omnidirectional microphone is used to collect the audio data.
[0055] The sensor bracket is used to load the environmental parameter sensor at a reasonable working height.
[0056] The upper computer includes the local data processing and analysis module and the path planning module.
[0057] The data transmission module is used to realize data interaction between the mobile inspection robot and the edge data processing and analysis module and the cloud server.
[0058] The warning light strip is used to transmit light alarm information to people around the scene when the execution result of any subtask in the execution result of the first type of safety risk identification task or the execution result of the second type of safety risk identification task shows that a corresponding safety risk event has been identified.
[0059] The positioning device is used to perform real-time positioning of the mobile inspection robot and obtain the positioning data.
[0060] In a possible implementation, the environmental parameter sensor includes at least one of a temperature and humidity sensor, a CO2 concentration sensor, a TVOC sensor, a PM2.5 sensor, a formaldehyde concentration sensor, and a metal oxide gas sensor;
[0061] In a possible embodiment, the camera assembly includes a front camera and a rear camera. The front camera is used to acquire the field of view image and infrared thermal imaging in the forward direction of the mobile inspection robot, and the rear camera is used to acquire the field of view image and infrared thermal imaging in the opposite direction of the forward direction of the mobile inspection robot.
[0062] In a possible implementation, the edge data processing and analysis module is deployed on an edge server in a data center, a server room, or a dedicated edge computing node in the preset area.
[0063] The disclosed embodiment allocates the security risk identification task to the local data processing and analysis module and the edge data processing and analysis module according to the task calculation amount, and uses the server to realize the interaction, so as to realize the comprehensive utilization of the computing resources and storage space of the local data processing and analysis module, the edge data processing and analysis module and the cloud server, reduce the data transmission delay, and ensure the instant identification of the public space security risk event. In addition, the path is re-planned according to the positioning data, the execution result of the first type of security risk identification task, and the execution result of the second type of security risk identification task, so as to realize the high flexibility of path planning, effectively shorten the inspection task cycle, and enhance the level of cognition of the risk event area. In addition, the disclosed embodiment sets the first type of security risk identification task to be executed in the local data processing and analysis module, and the second type of security risk identification task to be executed in the edge data processing and analysis module. On the one hand, compared with the related technology, the data transmission delay can be reduced and the real-time performance of risk identification can be improved. On the other hand, the calculation of the security risk identification task is allocated according to the computing power, so as to improve the task execution efficiency, and realize the balance between computing power scheduling and computing real-time performance, so that the disclosed embodiment can quickly identify and handle security risks, reduce the workload of personnel and the complexity of governance, and improve the inspection efficiency.
[0064] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.
[0066] Figure 1 A block diagram of a public space safety risk identification system based on a mobile inspection robot according to an embodiment of the present disclosure is shown.
[0067] Figure 2 A schematic diagram of a mobile inspection robot according to an embodiment of the present disclosure is shown.
[0068] Figure 3 A workflow diagram of a public space safety risk identification system based on a mobile inspection robot according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0069] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0070] In the description of the present disclosure, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure.
[0071] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0072] In the present disclosure, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.
[0073] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0074] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.
[0075] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.
[0076] See also Figure 1 , Figure 1 A block diagram of a public space safety risk identification system based on a mobile inspection robot according to an embodiment of the present disclosure is shown.
[0077] like Figure 1 As shown, the system includes: a mobile inspection robot 100, a local data processing and analysis module 200, an edge data processing and analysis module 300, a path planning module 400, and a cloud server 500, wherein:
[0078] The mobile inspection robot 100 is used to walk along a planned path in a preset area and collect real-time environmental parameters, video image data, audio data, and positioning data at each position on the planned path;
[0079] The local data processing and analysis module 200 is used to perform a first type of security risk identification task according to the environmental parameters;
[0080] The edge data processing and analysis module 300 is used to perform a second type of security risk identification task according to the video image data and the audio data, wherein the computational complexity of the second type of security risk identification task is higher than that of the first type of security risk identification task;
[0081] The path planning module 400 is used to re-plan the path according to the positioning data, the execution result of the first type of security risk identification task, and the execution result of the second type of security risk identification task;
[0082] The cloud server 500 is used to perform data interaction with the mobile inspection robot 100, the local data processing and analysis module 200, and the edge data processing and analysis module 300;
[0083] The computing power of the edge data processing and analysis module 300 is greater than the computing power of the local data processing and analysis module 200.
[0084] The local data processing and analysis module 200 and the path planning module 400 are arranged on the mobile inspection robot 100, and the edge data processing and analysis module 300 is arranged in the preset area.
[0085] The disclosed embodiment distributes the security risk identification tasks to the local data processing and analysis module 200 and the edge data processing and analysis module 300 according to the task computing amount, and uses the server to realize interaction, thereby realizing the comprehensive utilization of the computing resources and storage space of the local data processing and analysis module 200, the edge data processing and analysis module 300 and the cloud server 500, reducing the data transmission delay, ensuring the instant identification of public space security risk events, and replanning the path according to the positioning data, the execution results of the first type of security risk identification tasks, and the execution results of the second type of security risk identification tasks, thereby realizing high flexibility in path planning, effectively shortening the inspection task cycle, and enhancing the level of cognition of risk event areas. Therefore, the disclosed embodiment can quickly identify and handle security risks, reduce personnel workload and governance complexity, and improve inspection efficiency.
[0086] It is worth mentioning that the related technologies usually collect information and transmit it to a remote server for processing, which performs all or most of the operations and partially relies on manual analysis. The security risk identification time is significantly delayed, which does not meet the comprehensive management requirements of public space to quickly discover and intervene in security risks. In addition, the inspection route of the security robot in the related technology lacks feedback optimization, and the robot is prone to inefficient full-space traversal. After obtaining the security risk identification results, the robot fails to be guided to adjust the inspection route in time, and more security resources are invested in the review of high-risk areas. It can be seen that the public space security risk identification system based on the mobile inspection robot 100 in the embodiment of the present disclosure can reduce data transmission delays, quickly identify and handle security risks, reduce personnel workload and management complexity, and improve inspection efficiency compared to the related technology.
[0087] In summary, the embodiment of the present disclosure constructs a local and edge combined ladder computing network through the local data processing and analysis module 200 and the edge data processing and analysis module 300, which can achieve a balance between computing power scheduling and real-time computing. It should be understood that from the perspective of computing power configuration, computing power is usually cloud>edge>local; from the perspective of data transmission delay, delay is usually local<edge<cloud. Therefore, the embodiment of the present disclosure does not perform security risk identification in the cloud (cloud server), which can reduce the delay of remote data transmission. Secondly, for local and edge processing, the embodiment of the present disclosure constructs a trade-off between computing power and transmission delay through task allocation: for tasks that can be processed by the computing power of the local data processing and analysis module 200 (the first type of security risk identification tasks), the local data processing and analysis module 200 is used as much as possible to process them locally on the mobile inspection robot 100, so that no data transmission is required to reduce delays; tasks that cannot be executed by local computing power (the second type of security risk identification tasks) are handed over to the edge data processing and analysis module 300 for calculation and completion, so that computing power is improved through the edge data processing and analysis module 300, and delays are reduced compared to using cloud servers for execution.
[0088] It can be seen that the embodiment of the present disclosure sets the first type of security risk identification tasks to be executed in the local data processing and analysis module, and the second type of security risk identification tasks to be executed in the edge data processing and analysis module. On the one hand, compared with the related technologies, it can reduce data transmission delays and improve the real-time performance of risk identification. On the other hand, it distributes the calculations of security risk identification tasks according to computing power, thereby improving task execution efficiency and achieving a balance between computing power scheduling and computing real-time performance. This enables the embodiment of the present disclosure to quickly identify and handle security risks, reduce personnel workload and governance complexity, and improve inspection efficiency.
[0089] For example, in order to further reduce the delay, each security risk identification task can be set to be implemented in the local data processing and analysis module 200 as much as possible.
[0090] Illustratively, the disclosed embodiment uses a mobile inspection robot 100 to move autonomously in a preset area, collect security information of multiple dimensions (real-time environmental parameters, video image data, audio data, and positioning data at each position on the planned path), and transmit the data in real time to a multi-level computing network composed of a local data processing and analysis module 200, an edge data processing and analysis module 300, and a cloud server 500, to perform data storage, feature extraction, pattern recognition, and visualization on the environmental parameters, positioning data, video image data, and audio stream data, and obtain identification results of various security risk issues.
[0091] Exemplarily, the data analysis results returned by the local data processing and analysis module 200 and the edge data processing and analysis module 300 are also aggregated to the cloud server 500, and the data is centrally displayed and managed via the user terminal (not shown). In one example, the user terminal is also called user equipment (UE), mobile station (MS), mobile terminal (MT), etc., which is a device that provides voice and / or data connectivity to users, such as a handheld device with wireless connection function, a vehicle-mounted device, etc. At present, some examples of terminals are: mobile phones, tablet computers, laptops, PDAs, mobile Internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control (Industrial Control), wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids (Smart Grid), wireless terminals in transportation safety (Transportation Safety), wireless terminals in smart cities (Smart City), wireless terminals in smart homes (Smart Home), wireless terminals in Internet of Vehicles, etc.
[0092] Exemplarily, the path planning module 400 receives the positioning data, the execution results of the first type of security risk identification task, and the execution results of the second type of security risk identification task (of course, it can also receive information such as the time, location, type, occurrence, and confidence of the security risk event), and plans and updates the work route in real time based on this and feeds it back to the mobile inspection robot 100. In the embodiment of the present disclosure, the execution results of the local data processing and analysis module 200 and the edge data processing and analysis module 300 are directly uploaded to the path planning module 400 to reduce the time loss of the data transmission process, and perform path planning as soon as possible to update the inspection route.
[0093] For example, the back-end management personnel can also remotely send the inspection task route (planned path) or other parameters through the cloud server 500 by operating the user terminal. The disclosed embodiment constructs a motion control process for autonomously collecting and processing security data and finally feeding it back to the inspection robot, as well as a remote manual query and management process for security data and identification results. Through the hierarchical configuration of intelligent equipment and remote security management personnel, accurate and fast unmanned inspection of public space safety risks is achieved.
[0094] The disclosed embodiments do not limit the specific implementation methods of the mobile inspection robot 100, the local data processing and analysis module 200, the edge data processing and analysis module 300, the path planning module 400, and the cloud server 500. Those skilled in the art may adopt appropriate technical solutions according to actual conditions and needs.
[0095] For example, in some possible implementations, the local data processing and analysis module 200 can be implemented by a processing component. In one example, the processing component includes but is not limited to a separate processor, or a discrete component, or a combination of a processor and a discrete component. The processor may include a controller having an execution instruction function in an electronic device, and the processor may be implemented in any appropriate manner, for example, by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components. Inside the processor, the executable instructions may be executed by hardware circuits such as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers.
[0096] Exemplarily, the local data processing and analysis module 200 can be set on the mobile inspection robot 100. Since the local computing power of the mobile inspection robot 100 is limited and needs to be used by multiple tasks such as motion control, environmental perception, data analysis and processing, real-time positioning and navigation, etc., the local data processing and analysis module 200 in the embodiment of the present disclosure is mainly used for the identification tasks of some public space safety risk events with small data scale, simple data processing logic, and no need for complex machine learning model training. Such safety risk events specifically include but are not limited to tasks such as temperature over-limit detection, dangerous gas over-limit detection, and air quality over-limit detection based on environmental parameter sensors, as well as tasks such as fire point judgment and door and window opening status judgment based on thermal imagers.
[0097] Exemplarily, the edge data processing and analysis module 300 can be implemented by deploying edge devices through edge technology. The edge data processing and analysis module 300 can be in a preset area. The preset area can be, for example, a target building space, such as a railway station waiting hall, a shopping mall and other public spaces, or a factory, a community, a villa and other spaces. The edge data processing and analysis module 300 can be set in a computer room in the preset area, so that the mobile inspection robot 100 can communicate with the edge data processing and analysis module 300 through a communication network (such as a local area network).
[0098] Exemplarily, the disclosed embodiment can deploy the edge data processing and analysis module 300 on an edge server in a data center, a server room or a dedicated edge computing node inside a public space (preset area), which has more abundant computing resources and storage space, and supports the identification of some public space safety risk events that are difficult to complete with the local computing power of the mobile inspection robot 100, such as large data scale, complex data processing logic, and complex machine learning model training. Such safety risk events specifically include but are not limited to tasks such as dangerous goods identification, face recognition, abnormal behavior recognition of personnel, smoke recognition, fire recognition, and left-behind object recognition based on video data, and tasks such as destructive behavior detection, personnel gathering detection, and abnormal sound detection based on audio data. Compared with the cloud server 500, the edge data processing and analysis module 300 is deployed as close as possible to the task area of the mobile inspection robot 100, and communicates with the mobile inspection robot 100 in a two-way manner through a private network, a local area network or the Internet, reducing communication delays, improving real-time performance, and meeting the requirements of comprehensive management of public spaces for rapid identification of safety risks. With the scalability of the edge data processing and analysis module 300, if new security risk events are included in the monitoring scope or multiple robots are deployed to perform collaborative inspection tasks, new data processing and analysis algorithms are deployed on the existing servers, which will efficiently realize the application customization and expansion of the public space safety risk identification system.
[0099] Exemplarily, the path planning module 400 of the embodiment of the present disclosure may include the aforementioned processing components, and the path planning module 400 may also be set on the mobile inspection robot 100, so that the embodiment of the present disclosure can reduce communication delays and dependence on the network and external services, and improve the reliability and stability of the system. Exemplarily, the path planning module 400 can be used to save the pre-set inspection route and receive the real-time posture information of the chassis, as well as the security risk event identification status returned in real time by the local data processing and analysis module 200 and the edge data processing and analysis module 300. When a security risk event is identified or the security risk event identification confidence is low (uncertain whether it occurs), the path planning module 400 can terminate the pre-set inspection route, plan a new inspection route based on the space coverage algorithm or other algorithms around the real-time position, and control the mobile inspection robot 100 to conduct a local detailed inspection in the area around the security risk event, wherein the specific setting method of the local detailed inspection is not limited in the embodiment of the present disclosure, for example, the interval for collecting relevant data can be set to be shorter and the range can be smaller, that is, a more refined inspection. The path planning module 400 can also receive external feedback alarm information returned by the cloud server 500 or target location instructions issued by remote security management personnel, and update a new inspection route based on the path planning algorithm according to the alarm location or target location.
[0100] It is worth mentioning that, considering the constraints such as data transmission bandwidth and communication time delay, the cloud server 500 in the embodiment of the present disclosure does not undertake specific computing tasks such as the identification of security risk events, and is only used for data visualization and manual issuance of control instructions. Its main function is to receive, store and visualize part of the original measurement data and the security risk event identification status returned in real time by the local data processing and analysis module 200 and the edge data processing and analysis module 300, so as to facilitate remote security management personnel to view and review the on-site security risk status, and issue external feedback alarm information and manually set target position instructions when necessary.
[0101] The following is an exemplary introduction to possible implementations of the mobile inspection robot 100.
[0102] See also Figure 2 , Figure 2 A schematic diagram of a mobile inspection robot 100 according to an embodiment of the present disclosure is shown.
[0103] For example, in a possible implementation, Figure 2 As shown, the mobile inspection robot 100 may include: a robot chassis 1, a speaker 2, an upper power supply 3, a robot housing 4, a camera assembly, an environmental parameter sensor 6, an omnidirectional microphone 7, a sensor bracket 8, an upper computer 10, a data transmission module 11, a warning light strip 12, and a positioning device 13.
[0104] The robot chassis 1 is used to carry the upper device to move after receiving a motion instruction, wherein the motion instruction is obtained based on the planned path.
[0105] The loudspeaker 2 is used to transmit sound alarm information to people around the scene when the execution result of any subtask in the execution result of the first type of security risk identification task and the execution result of the second type of security risk identification task shows that a corresponding security risk event has been identified.
[0106] The upper power supply 3 is used to provide electrical energy.
[0107] The robot housing 4 is used to support the upper sensor and camera, and to protect the upper power supply 3 and the upper computer 10.
[0108] The camera assembly is used to collect the video image data.
[0109] The environmental parameter sensor 6 is used to collect the environmental parameters.
[0110] The omnidirectional microphone 7 is used to collect the audio data.
[0111] The sensor bracket 8 is used to load the environmental parameter sensor 6 at a reasonable working height.
[0112] The upper computer 10 includes the local data processing and analysis module 200 and the path planning module 400.
[0113] The data transmission module 11 is used to realize data interaction between the mobile inspection robot 100 and the edge data processing and analysis module 300 and the cloud server 500.
[0114] The warning light strip 12 is used to transmit light alarm information to people around the scene when the execution result of any subtask in the execution result of the first type of safety risk identification task or the execution result of the second type of safety risk identification task shows that a corresponding safety risk event has been identified.
[0115] The positioning device 13 is used to perform real-time positioning of the mobile inspection robot 100 and obtain the positioning data.
[0116] The disclosed embodiments do not limit the specific implementation methods of components such as the robot chassis 1, the speaker 2, the upper power supply 3, the robot housing 4, the camera assembly, the environmental parameter sensor 6, the omnidirectional microphone 7, the sensor bracket 8, the upper computer 10, the data transmission module 11, the warning light strip 12, and the positioning device 13. Those skilled in the art can adopt relevant technologies to implement them according to actual conditions and needs.
[0117] Exemplarily, the robot chassis 1 may include a chassis support structure, an outer shell, a wheel set, a motor and a drive, a motion controller, a chassis power supply and a charging interface, and after receiving a motion instruction, it carries the upper equipment to move.
[0118] Exemplarily, the upper power source 3 may include an energy storage component, which may include a battery pack, which may include a lithium-ion battery, a lithium polymer battery, a sodium-ion battery, a perovskite battery, etc.
[0119] Exemplarily, the data transmission module 11 can send the multi-dimensional security information (environmental parameters, video image data, audio data, positioning data) collected by the mobile inspection robot 100, and some security risk event identification results obtained by the local data processing and analysis module 200 in the upper computer 10 to the edge data processing and analysis module 300 and the cloud server 500 in real time, and receive some security risk event identification results and motion control instructions returned by the edge data processing and analysis module 300 and the cloud server 500. Among them, the signals used by the data transmission module 11 for data transmission specifically include but are not limited to WiFi, Bluetooth, cellular network, Zigbee, etc., and the protocols used for data transmission specifically include but are not limited to TCP / IP, UDP, Modbus, HTTP, ROS communication protocols, etc.
[0120] In a possible implementation, the environmental parameter sensor 6 includes at least one of a temperature and humidity sensor, a CO2 concentration sensor, a TVOC sensor, a PM2.5 sensor, a formaldehyde concentration sensor, a metal oxide gas sensor, etc.;
[0121] In a possible implementation, Figure 2 As shown, the camera assembly may include a front camera 5 and a rear camera 9, wherein the front camera 5 is used to realize the acquisition of the field image and infrared thermal imaging of the forward direction of the mobile inspection robot 100, and the rear camera 9 is used to realize the acquisition of the field image and infrared thermal imaging of the forward opposite direction of the mobile inspection robot 100. Exemplarily, the front camera 5 and the rear camera 9 may include but are not limited to a full-color camera, an infrared camera, a starlight camera, and a thermal imager, etc., and may be set separately or multiple units may be coaxially connected in parallel, and are responsible for the acquisition of the field image and infrared thermal imaging of the forward direction of the mobile inspection robot 100.
[0122] Exemplarily, the upper-level computer 10 includes but is not limited to an industrial computer, an embedded system, a single-chip microcomputer and an ordinary computer, etc. The local data processing and analysis module 200 and the path planning module 400 are arranged in the upper-level computer 10. The upper-level computer 10 is used to read and process the inspection robot chassis 1 and the data of various upper-level devices, so as to realize control functions such as positioning navigation, path planning, obstacle avoidance, inspection point selection, sound and light alarms and identification of some safety risk events.
[0123] Exemplarily, the positioning device 13 includes but is not limited to a global positioning system (GPS), a lidar positioning module, an ultrasonic positioning module, a Bluetooth positioning module, a WiFi positioning module, a visual positioning module (such as including a depth camera), etc., which are used to obtain real-time position information of the mobile inspection robot 100 in a public space.
[0124] The embodiments of the present disclosure do not limit the specific task types included in the first category of security risk identification tasks and the second category of security risk identification tasks, and do not limit the specific processing and operation methods of each task. Those skilled in the art can set them according to actual conditions and needs.
[0125] Exemplarily, in a possible implementation manner, the first type of security risk identification task may include at least one of the following subtasks:
[0126] A subtask of detecting excessive temperature based on the environmental parameters, wherein the environmental parameters in the subtask include temperature parameters;
[0127] A subtask of detecting excessive dangerous gas content based on the environmental parameters, wherein in this subtask, the environmental parameters include various dangerous gas content parameters, and the present disclosure does not limit the specific dangerous gas, and those skilled in the art can set it according to actual conditions and needs, for example, the dangerous gas may include formaldehyde, etc.;
[0128] A subtask of air quality detection based on the environmental parameters, wherein the environmental parameters in this subtask include various gas content parameters, such as carbon dioxide, PM2.5, etc.;
[0129] The subtask of fire point judgment based on thermal imaging images;
[0130] The subtask of judging the door and window opening status based on thermal imaging images,
[0131] Wherein, the video image data includes the thermal imaging image.
[0132] The embodiments of the present disclosure do not limit the specific implementation method of performing the first type of security risk identification task according to the environmental parameters. Those skilled in the art may set it according to actual conditions and needs. For example, in a possible implementation, performing the first type of security risk identification task according to the environmental parameters may include:
[0133] Performing data preprocessing on the environmental parameters to obtain preprocessed environmental parameters;
[0134] The preprocessed environmental parameters are subjected to a first type of security risk identification task using any one or more of a threshold detection method, a statistical analysis method, a time series analysis method or a machine learning classification model.
[0135] The specific method of data preprocessing in the embodiments of the present disclosure is not limited in the embodiments of the present disclosure. For example, it may include data cleaning, denoising, smoothing and other preprocessing methods to ensure the quality and reliability of the data.
[0136] The embodiments of the present disclosure do not limit the specific implementation methods of the threshold detection method, statistical analysis method, time series analysis method or machine learning classification model, and those skilled in the art can set them according to actual conditions and needs. For example, in anomaly detection based on threshold judgment (threshold detection method), the threshold can be set with reference to standard specifications. The statistical analysis method, time series analysis method or machine learning classification model can be implemented by inputting historical environmental data of the same preset area, such as analyzing historical environmental data using a statistical analysis model, analyzing historical environmental data using a time series analysis model, or analyzing historical environmental data using a machine learning classification model. Anomaly detection is performed based on the results of statistical analysis, time series analysis or machine learning classification model. After detecting anomalies, the system alarms and records the time, location, type and confidence of the security risk event. Of course, the embodiments of the present disclosure do not limit the specific implementation methods of the statistical analysis model, time series analysis model or machine learning classification model, and those skilled in the art can adopt relevant technologies to implement them according to actual conditions and needs.
[0137] The disclosed embodiments do not limit the specific implementation of the subtask of fire point judgment based on thermal imaging images and the subtask of door and window opening status judgment based on thermal imaging images. Those skilled in the art can implement them using relevant technologies according to actual conditions and needs.
[0138] For example, based on the fact that thermal images can intuitively display the temperature distribution of an object, the fire point can be determined. In thermal images, areas with higher temperatures will appear as brighter colors, while areas with lower temperatures will appear as darker colors. Therefore, by observing the brightness and darkness of the colors in the image, it is possible to preliminarily determine which areas may have higher temperatures and thus may be fire points. Of course, further judgments can be made based on the temperature distribution. For example, when a fire occurs, the flames and high-temperature smoke will form an obvious temperature area. This area will show specific patterns and shapes in the thermal images. By analyzing these patterns and shapes, the location of the fire point and the development of the fire can be further confirmed.
[0139] For example, for thermal imaging images, the depth of color reflects the temperature. The colors in the picture can be used to identify whether the external doors and windows are open (if open, the color of the gap must be significantly different from the surrounding area). In addition, other subtasks can also be performed. For example, based on the thermal imaging image, it can be determined whether the air-conditioning terminal equipment is running (if running, the temperature of the air outlet / radiation terminal must be significantly different from the indoor air temperature), the air outlet temperature / radiation panel surface temperature, etc.
[0140] In a possible implementation manner, the second type of security risk identification task may include at least one of the following subtasks:
[0141] A subtask of performing dangerous goods identification based on the video image data, wherein in this subtask, the dangerous goods can be defined according to actual conditions and needs, and this embodiment of the present disclosure does not limit this;
[0142] A subtask of performing face recognition based on the video image data;
[0143] A subtask of identifying abnormal behavior of personnel based on the video image data, wherein in this subtask, abnormal behavior of personnel can be defined according to actual conditions and needs, which is not limited in the embodiment of the present disclosure;
[0144] A subtask of performing smoke recognition based on the video image data;
[0145] A subtask of performing fire identification based on the video image data;
[0146] A subtask of performing left-behind object recognition based on the video image data;
[0147] A subtask of performing people gathering detection based on the video image data and / or the audio data;
[0148] A subtask of destructive behavior detection based on the audio data, wherein the destructive behavior in this subtask can be defined according to actual conditions and needs, which is not limited in the embodiment of the present disclosure;
[0149] A subtask of performing abnormal sound detection based on the audio data, wherein in this subtask, the abnormal sound can be defined according to actual conditions and needs, which is not limited in the embodiment of the present disclosure.
[0150] The embodiments of the present disclosure do not limit the specific implementation method of performing the second type of security risk identification task according to the video image data and the audio data. Those skilled in the art may set it according to actual conditions and needs. For example, in a possible implementation, performing the second type of security risk identification task according to the video image data and the audio data may include:
[0151] Preprocessing the video image data and the audio data to obtain the preprocessed video image data and the audio data;
[0152] The preprocessed video image data and / or the audio data are input into a pre-trained security risk identification model, and the execution results of each subtask of the second type of security risk identification task are determined according to the output of the security risk identification model.
[0153] The embodiments of the present disclosure do not limit the specific method of preprocessing the video image data and the audio data. Those skilled in the art can set it according to actual conditions and needs. For example, the preprocessing of the video image data may include image denoising, enhancement, and resizing, and the preprocessing of the audio data may include denoising, filtering, downsampling, and slicing.
[0154] The embodiments of the present disclosure do not limit the specific type and implementation of the security risk identification model. Those skilled in the art may adopt relevant technologies to implement it according to actual conditions and needs.
[0155] Exemplarily, the security risk identification model can adopt a pattern recognition model based on deep learning (such as YOLO, SSD, etc.), and input the video image data into the pattern recognition model based on deep learning to accurately locate and identify multiple required targets (such as faces, fire points, dangerous goods, etc.) in the image. Exemplarily, for prohibited objects or identification targets that will not appear in public spaces under safe operation conditions, completing target detection can identify whether a security risk event has occurred; Exemplarily, for identification targets that appear in public spaces under safe operation conditions but show significantly different features in security risk events, it is still necessary to extract the appearance, position, motion state and other features of the identified targets through image processing methods or deep learning methods, and perform anomaly detection based on threshold detection, statistical analysis, machine learning classification models, time series analysis and other methods. Exemplarily, after detecting an anomaly, the system alarms and records the time, location, type and confidence level of the security risk event. In this way, the disclosed embodiment utilizes a security risk identification model based on video image data (such as visible light or infrared video image data) to identify security risk events such as dangerous goods identification, face recognition, abnormal behavior of personnel, smoke recognition, fire recognition, and abandoned items identification. Of course, the aforementioned fire point judgment and door and window opening status judgment can also be used as the second type of security risk identification task, which is not limited to the disclosed embodiment.
[0156] Exemplarily, the security risk identification model can be implemented based on a machine learning method or a deep learning method to detect events on audio data. For example, the collected audio data can be input into the security risk identification model, and whether there is an abnormal sound can be determined based on the output result of the security risk identification model. For abnormal sound detection, completing target detection (such as matching an abnormal sound) can identify whether a security risk event has occurred; for voice data of people in public spaces, a deep learning-based speech recognition model (such as CTC, etc.) can be used to convert voice signals into text, and perform speech analysis and keyword extraction on the recognized text to check whether there is abnormal information related to security risks. After detecting an abnormality, the system alarms and records the time, location, type and confidence of the security risk event. In this way, the disclosed embodiment uses a security risk identification model based on audio data to identify security risk events such as destructive behavior detection, personnel gathering detection, and abnormal sound detection.
[0157] The security risk identification model can be trained based on the existing training method using the corresponding sample data and labels according to the reasoning task to be completed. This application does not limit the training method of the security risk identification model.
[0158] In a possible implementation, the re-planning of the path according to the positioning data, the execution result of the first type of security risk identification task, and the execution result of the second type of security risk identification task may include:
[0159] If the execution result of any subtask in the execution result of the first type of security risk identification task or the execution result of the second type of security risk identification task shows that a security risk event of a corresponding specified level is identified, or the confidence level in the execution result of any subtask is lower than a first preset value, then the path is replanned for the subtask, wherein if the confidence level in the execution result is higher than a second preset value, it is determined that a corresponding security risk event is identified, and the second preset value is higher than the first preset value;
[0160] The mobile inspection robot 100 is controlled to move according to the re-planned path, and the safety risk event is re-determined for the subtask.
[0161] The embodiments of the present disclosure do not limit the specific sizes of the first preset value and the second preset value, and those skilled in the art may set them according to actual conditions and needs.
[0162] Exemplarily, if the local data processing and analysis module 200 and the edge data processing and analysis module 300 judge that the confidence of the execution result is higher than the second preset value when executing the corresponding subtask, it is determined that the corresponding security risk event is identified. Of course, in the embodiment of the present disclosure, the security risk event can be further divided into low-risk security events, high-risk security events, or divided into more categories, which is not limited to the embodiment of the present disclosure. For security risk events of different degrees, the embodiment of the present disclosure can set corresponding response actions. For example, if a high-risk security event is identified (the confidence of the corresponding high-risk security event is higher than the set confidence threshold, i.e., the second preset value), the speaker 2 and the warning light strip 12 can be used for sound and light warning, and the alarm information is sent to the cloud server 500, instructing the remote security management personnel to handle it as soon as possible. It should be understood that for each classified security risk event, a corresponding first preset value and a second preset value can be set, which is not limited to the embodiment of the present disclosure.
[0163] Exemplarily, if a security risk event of a corresponding specified level is identified, or the confidence in the execution result of the subtask is lower than the first preset value (the result of risk identification is uncertain), the path planning module 400 can plan a path around the current position (the result of risk identification of the current position and the surrounding area is uncertain) to further distinguish the risk status of the current position. Of course, the security risk event of the specified level in this example can be a security risk event of each classification, and those skilled in the art can specify it according to actual conditions and needs. Preferably, the security risk event of the specified level can be set to a low-risk security event, that is, if it is determined that a low-risk security event is identified or the current event identification result is uncertain (the confidence is lower than the first preset value), the path planning is re-performed. In this case, the mobile inspection robot 100 will re-plan the path near the current inspection point and conduct an area review. The preset inspection task route will be terminated until the risk is eliminated. The path planning module 400 will demarcate the local area to be reviewed around the current position with a manually preset length and width, and the mobile inspection robot 100 will move within the local area for review.
[0164] Among them, high, medium and low levels of safety risk events can be divided according to needs. For example, safety risk events related to smoke and fire can be set to a high level, and safety risk events related to left behind items can be set to a low level, etc.
[0165] The embodiment of the present disclosure does not limit the specific implementation manner of the path planning module 400 for path planning, and those skilled in the art may configure it according to actual conditions and needs. For example, in a possible implementation manner, replanning the path for the subtask may include:
[0166] Determine a local area according to the positioning data corresponding to the subtask;
[0167] A collection step is set for the local area, and a detailed inspection path is planned, wherein when the mobile inspection robot 10 moves along the detailed inspection path, real-time environmental parameters, video image data, audio data, and positioning data of each position on the detailed inspection path are collected at each collection step.
[0168] The disclosed embodiments do not limit the implementation method of determining the local area based on the positioning data corresponding to the subtask. For example, the area within the preset range of the positioning data corresponding to the subtask can be the local area. The positioning data corresponding to the subtask can be the positioning data corresponding to the execution result of the subtask showing that the corresponding specified level of security risk event is identified. For example, the mobile inspection robot 100 passes through points A, B, and C during the movement, and collects environmental parameters, video image data, audio data, and positioning data corresponding to the points. Among them, for points A and B, no subtask with a security risk event is identified, and at point C, a subtask with a low-level security risk event is identified (for example, identification of items left behind), then the positioning data corresponding to the subtask is the positioning data of point C.
[0169] The embodiment of the present disclosure does not limit the setting method of the size of the collection step length. Technical personnel in this field can set it according to actual conditions and needs. To improve the accuracy of detailed inspection, the collection step length can be smaller than the preset collection step length. The preset collection step length can be a parameter commonly set for the mobile inspection robot 100.
[0170] The embodiments of the present disclosure do not limit the specific implementation method of planning the detailed inspection path. Those skilled in the art can use relevant technologies to implement it according to actual conditions and needs. For example, the path planning module 400 can re-plan the regional detailed inspection route based on grid search algorithm, heuristic algorithm, roaming method and other methods when planning the detailed inspection path, and iteratively explore the local area through rules. In each collection step, the mobile inspection robot 100 collects environmental physical parameters, positioning data, audio and video data, and multi-dimensional security information, and transmits it to the local data processing and analysis module 200 and the edge data processing and analysis module 300 in real time to identify all events in the current security risk event library.
[0171] In a possible implementation, the path planning module 400 may also be used for:
[0172] Receive external path control information, and generate a new planned path according to the path control information.
[0173] For example, a user may send path control information to the path planning module 400 through the cloud server 500 , and the path control information may include a new planned path.
[0174] The following is an exemplary introduction to the working process of the public space safety risk identification system based on the mobile inspection robot 100.
[0175] See also Figure 3 , Figure 3 A workflow diagram of a public space safety risk identification system based on a mobile inspection robot 100 according to an embodiment of the present disclosure is shown.
[0176] For example, Figure 3 As shown, the public space safety risk identification system based on the mobile inspection robot 100 of the embodiment of the present disclosure may include the following steps to identify risks in a preset area:
[0177] Step S1, equipment initialization: during an inspection task cycle of the mobile inspection robot 100, a preset planned path is loaded.
[0178] Step S2, external feedback alarm information judgment: the alarm information reported by people in the public space and judged by remote security management personnel is summarized on the cloud server 500 and sent to the mobile inspection robot 100. The mobile inspection robot 100 requests the cloud server 500 for external feedback alarm information. If it exists, it goes to step S8, otherwise it goes to step S3.
[0179] Step S3, global inspection mode: when there is no external feedback alarm information after initialization, and no security risk event has been identified in the current task cycle, the mobile inspection robot 100 moves in the target space along the preset inspection route. When encountering an obstacle on the inspection route, the mobile inspection robot 100 performs obstacle avoidance path planning to bypass the obstacle; when the obstacle is temporarily difficult to bypass, the mobile inspection robot 100 will keep waiting until the obstacle leaves. During the inspection process, the mobile inspection robot 100 stops at the preset inspection point, collects environmental physical parameters, positioning data, audio and video data, and multi-dimensional security information, and transmits it to the local data processing and analysis module 200 and the edge data processing and analysis module 300 in real time to identify all events in the current security risk event library. Exemplarily, security risk events are manually divided into two categories: high-risk security events and low-risk security events according to their degree of harm. The identification results are transmitted to the path planning module 400 in real time to enter step S4.
[0180] Step S4, high-risk security event judgment: The path planning module 400 makes a judgment based on the returned security risk event detection results. When the detection results indicate that there are high-risk security events in the security risk event library, in order to avoid frequent false alarms caused by model recognition errors, only when the confidence level of the event identified as high risk is greater than the manually set confidence threshold, enter step S5, otherwise enter step S6.
[0181] Step S5, equipment alarm: once a high-risk security incident is discovered, the mobile inspection robot 100 will give an on-site alarm by means of voice broadcasting through the speaker 2 and flashing of the light strip, and at the same time, the alarm data will be sent to the cloud server 500 to instruct the remote security management personnel to handle it as soon as possible.
[0182] Step S6, judgment of security risk events of specified level: A person skilled in the art can specify a security risk event of specified level according to actual conditions and needs. Preferably, a security risk event of specified level can be set to a low-risk security event. Therefore, in this step, the path planning module 400 makes a judgment based on the returned security risk event detection result. When the detection result shows that there is a low-risk security event in the security risk event library, or the confidence level of the event identification as any risk event is lower than the manually set confidence threshold (indicating that the result of risk identification is uncertain), step S8 is entered; otherwise, step S7 is entered.
[0183] Step S7, risk elimination: If no safety risk events are detected and no uncertain event identification results occur, it means that there is no risk at the current patrol point. The mobile patrol robot 100 will continue to perform patrol tasks along the preset route in the global patrol mode of step S3 until the next patrol task cycle or a risk event is identified.
[0184] Step S8, local detailed inspection mode: When step S2 determines that an external feedback alarm message appears or step S6 determines that a risk safety event of a specified level (such as a low-risk safety event) is identified or the current event identification result is uncertain, the mobile inspection robot 100 will re-plan the path near the current inspection point and conduct a regional review. The identification result is transmitted to the path planning module 400 in real time to enter step S9.
[0185] Step S9, risk event judgment: The path planning module 400 makes a judgment based on the returned security risk event detection result. When the detection result indicates that there is any risk event in the security risk event library, it proceeds to step S10, otherwise it proceeds to step S7. If the current event recognition result is still uncertain, the mobile inspection robot 100 will continue to iteratively explore the local area until the quality of the collected data can support the deterministic recognition of risk events.
[0186] Step S10, equipment alarm: Similar to step S5, once a high-risk security incident is discovered, the mobile inspection robot 100 will give an on-site alarm by means of voice broadcasting through the speaker 2 and flashing of the light strip. At the same time, the alarm data will be sent to the cloud server 500 to instruct the remote security management personnel to handle it as soon as possible.
[0187] The public space safety risk identification system based on the mobile inspection robot 100 of the embodiment of the present disclosure can be applied in a variety of scenarios, such as airport terminals, hospitals, large shopping malls, etc., which is not limited to the embodiment of the present disclosure.
[0188] Taking the indoor security risk identification of airport terminals as an example, maintaining the safe and orderly operation of airports and ensuring convenient and efficient travel for passengers are the primary requirements for building a four-type airport with "safety, green, intelligence, and humanity" as the core. Compared with other building spaces, airport terminals have a large population density, complex personnel composition, complex spatial functions, and intersecting spatial streamlines. Security management involves many departments and factors. Under the combined effect of the above factors, the security management of airport terminals is relatively difficult. In addition, as an important domestic and international transportation hub, airports have strict security prevention standards, and accidents caused by security risks are more harmful. Therefore, the public space security risk identification system based on the mobile inspection robot of the disclosed embodiment is used to identify security risks in airport terminals, which can quickly identify and deal with security risks, reduce personnel workload and management complexity, and improve inspection efficiency. Among them, potential safety risk events in airport terminals specifically include but are not limited to temperature excess detection, hazardous gas excess detection, air quality excess detection, ignition point judgment, door and window open status judgment, dangerous goods identification, face recognition, abnormal personnel behavior identification, smoke identification, fire identification, left-behind item identification, destructive behavior detection, personnel gathering detection, abnormal sound detection, etc.
[0189] Among them, based on the environmental physical parameter sensor data, safety risk events such as temperature over-limit detection, dangerous gas over-limit detection, and air quality over-limit detection are identified. The main steps are: first, the environmental data collected by the mobile inspection robot 100 is preprocessed, including data cleaning, denoising, smoothing, etc., to ensure the quality and reliability of the data; secondly, according to the pre-set threshold or data-driven method, the preprocessed data is detected for anomalies. In the anomaly detection based on threshold judgment, the threshold can be manually set with reference to the standard specification. In the data-driven method, by inputting the historical environmental data of the same terminal space, anomaly detection is performed based on statistical analysis, time series analysis, or machine learning classification model. After detecting the anomaly, the system alarms and records the time, location, type, and confidence level of the safety risk event.
[0190] Among them, based on visible light or infrared video image data, safety risk events such as fire point judgment, door and window opening status judgment, dangerous goods identification, face recognition, passenger abnormal behavior identification, smoke identification, fire identification, and left-behind item identification are judged. The main steps are: first, the images collected by the mobile inspection robot 100 are pre-processed, including image noise reduction, enhancement, size adjustment, etc.; secondly, a pattern recognition model based on deep learning (such as YOLO, SSD, etc.) is used to accurately locate and identify multiple required targets (such as faces, fire points, dangerous goods, etc.) in the image. For prohibited objects or identification targets that will not appear in the terminal under safe operation conditions, completing target detection can identify whether a safety risk event has occurred; for identification targets that appear in the terminal under safe operation conditions but present significantly different features in safety risk events, it is still necessary to extract the appearance, position, motion state and other features of the identified targets through image processing methods or deep learning methods, and perform anomaly detection based on threshold detection, statistical analysis, machine learning classification models, time series analysis and other methods. After detecting an anomaly, the system alarms and records the time, location, type and confidence level of the security risk event.
[0191] Among them, based on audio data, security risk events such as destructive behavior detection, crowd gathering detection, and abnormal sound detection are identified. The main steps are: first, the audio data collected by the mobile inspection robot 100 is preprocessed, including noise reduction, filtering, downsampling, and slicing; secondly, machine learning methods or deep learning methods are used to detect events on the audio data. For abnormal sound patterns, completing target detection can identify whether a security risk event has occurred; for the voice data of people in the terminal, a deep learning-based speech recognition model (such as CTC, etc.) is used to convert the voice signal into text, and the recognized text is subjected to speech analysis and keyword extraction to check whether there is any abnormal information related to security risks. After detecting an abnormality, the system alarms and records the time, location, type, and confidence level of the security risk event.
[0192] The advantages and effects of the present application compared with the prior art are explained below from the technical level and the application level respectively.
[0193] On the technical level: the transmission and analysis mode of security inspection information in the traditional method has been improved. Based on edge computing technology, the comprehensive utilization of local computing, edge server and cloud server computing resources and storage space has been realized, which reduces data transmission delays and ensures the immediate identification of public space security risk events. The working mode of mechanical inspection along a fixed path in the traditional security inspection method has been improved. By providing real-time feedback of security risk event results to the path planning module 400, an autonomous optimization mode of inspection routes driven by security interest points has been developed, which effectively shortens the inspection task cycle and enhances the level of awareness of risk event areas.
[0194] At the application level: an automated inspection solution for public space safety risk identification has been formed, which effectively solves the problems of high workload and omissions in the work of security management personnel, realizes efficient identification and real-time reporting of safety risk events, and provides a high level of protection for the daily operation and risk management of public spaces such as transportation hubs, cultural venues, and commercial facilities.
[0195] The public space safety risk identification system based on the mobile inspection robot 100 of the disclosed embodiment, on the one hand, realizes the effective identification of safety risks by rationally combining monitoring equipment in the system and developing a method for identifying common risk events in public spaces. On the other hand, by improving the intelligence level of the inspection robot in autonomously identifying safety risk events and autonomously planning inspection routes, the time delay in information collection and event detection in security inspection tasks is reduced, and the governance needs of rapid detection and rapid response to public space safety risks are guaranteed. Therefore, the public space safety risk identification system based on the mobile inspection robot 100 of the disclosed embodiment can solve three technical problems:
[0196] 1) Identification of safety risks in public spaces: Focus on safety risk events that occur frequently or require special precautions in public spaces, rationally configure the information collection equipment carried by patrol robots, and develop appropriate safety risk event feature recognition algorithms to achieve efficient and autonomous execution of public space safety governance tasks.
[0197] 2) Rapid and autonomous analysis of multi-dimensional inspection information: Based on edge computing technology, local computing, edge servers and cloud servers are used in a coordinated manner to achieve a trade-off between local computing resources and data transmission delays. The multi-dimensional inspection information collected by security robots can be automatically processed and analyzed nearby to meet the requirements of rapid perception and rapid response in public space security management, reduce personnel workload and reduce task complexity.
[0198] 3) Autonomous optimization of inspection routes driven by points of interest: By realizing information interaction between the motion module and the perception module, the barriers between traditional security robot path inspection and information collection are broken, and autonomous optimization of robot inspection routes based on safety risk events as points of interest is realized. Limited security inspection resources are allocated to high-risk areas first, shortening the effective task cycle, and further ensuring the needs of rapid perception and rapid response in public space safety management.
[0199] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A public space safety risk identification system based on a mobile inspection robot, characterized in that: The system includes: a mobile inspection robot, a local data processing and analysis module, an edge data processing and analysis module, a path planning module, and a cloud server, wherein: The mobile inspection robot is used to walk along a planned path in a preset area and collect real-time environmental parameters, video image data, audio data, and positioning data at each position on the planned path; The local data processing and analysis module is used to perform a first type of security risk identification task according to the environmental parameters; The edge data processing and analysis module is used to perform a second type of security risk identification task according to the video image data and the audio data, wherein the computational complexity of the second type of security risk identification task is higher than that of the first type of security risk identification task; The path planning module is used to re-plan the path according to the positioning data, the execution result of the first type of security risk identification task, and the execution result of the second type of security risk identification task; The cloud server is used to interact with the mobile inspection robot, the local data processing and analysis module, and the edge data processing and analysis module for data exchange; The computing power of the edge data processing and analysis module is greater than the computing power of the local data processing and analysis module. Among them, the local data processing and analysis module and the path planning module are arranged on the mobile inspection robot, and the edge data processing and analysis module is arranged in the preset area.
2. The system according to claim 1, characterized in that The first type of security risk identification task includes at least one of the following subtasks: A subtask of detecting excessive temperature based on the environmental parameters; A subtask of detecting excessive dangerous gas content based on the environmental parameters; A subtask of air quality detection based on the environmental parameters; The subtask of fire point judgment based on thermal imaging images; The subtask of judging the door and window opening status based on thermal imaging images, Wherein, the video image data includes the thermal imaging image.
3. The system according to claim 1 or 2, characterized in that: The performing of the first type of security risk identification task according to the environmental parameters includes: Performing data preprocessing on the environmental parameters to obtain preprocessed environmental parameters; The preprocessed environmental parameters are subjected to a first type of security risk identification task using any one or more of a threshold detection method, a statistical analysis method, a time series analysis method or a machine learning classification model.
4. The system according to claim 1 or 2, characterized in that: The second type of security risk identification task includes at least one of the following subtasks: A subtask of performing dangerous goods identification based on the video image data; A subtask of performing face recognition based on the video image data; A subtask of performing abnormal behavior identification of personnel based on the video image data; A subtask of performing smoke recognition based on the video image data; A subtask of performing fire identification based on the video image data; A subtask of performing left-behind object recognition based on the video image data; A subtask of performing people gathering detection based on the video image data and / or the audio data; A subtask of detecting destructive behavior based on the audio data; A subtask of abnormal sound detection is performed based on the audio data.
5. The system according to claim 1, characterized in that The performing of the second type of security risk identification task according to the video image data and the audio data includes: Preprocessing the video image data and the audio data to obtain the preprocessed video image data and the audio data; The preprocessed video image data and / or the audio data are input into a pre-trained security risk identification model, and the execution results of each subtask of the second type of security risk identification task are determined according to the output of the security risk identification model.
6. The system according to claim 1, characterized in that The re-planning of the path according to the positioning data, the execution result of the first type of security risk identification task, and the execution result of the second type of security risk identification task includes: If the execution result of any subtask in the execution result of the first type of security risk identification task or the execution result of the second type of security risk identification task shows that a security risk event of a corresponding specified level is identified, or the confidence level in the execution result of any subtask is lower than a first preset value, then the path is replanned for the subtask, wherein if the confidence level in the execution result is higher than a second preset value, it is determined that a corresponding security risk event is identified, and the second preset value is higher than the first preset value; The mobile inspection robot is controlled to move according to the re-planned path, and the safety risk event is re-judged for the subtask.
7. The system according to claim 6, characterized in that Replan the path for this subtask, including: Determine a local area according to the positioning data corresponding to the subtask; A collection step is set for the local area, and a detailed inspection path is planned, wherein when the mobile inspection robot moves along the detailed inspection path, real-time environmental parameters, video image data, audio data, and positioning data of each position on the detailed inspection path are collected at each collection step.
8. The system according to claim 6, characterized in that The path planning module is also used for: Receive external path control information, and generate a new planned path according to the path control information.
9. The system according to claim 1, characterized in that The mobile inspection robot includes: a robot chassis, a speaker, an upper power supply, a robot shell, a camera assembly, an environmental parameter sensor, an omnidirectional microphone, a sensor bracket, an upper computer, a data transmission module, a warning light strip, and a positioning device. The robot chassis is used to carry the upper device to move after receiving a motion instruction, wherein the motion instruction is obtained based on the planned path. The loudspeaker is used to transmit sound alarm information to people around the scene when the execution result of any subtask in the execution result of the first type of security risk identification task and the execution result of the second type of security risk identification task shows that a corresponding security risk event has been identified. The upper power supply is used to provide electrical energy. The robot shell is used to support the upper sensors and cameras and to protect the upper power supply and upper computer. The camera assembly is used to collect the video image data. The environmental parameter sensor is used to collect the environmental parameter. The omnidirectional microphone is used to collect the audio data. The sensor bracket is used to load the environmental parameter sensor at a reasonable working height. The upper computer includes the local data processing and analysis module and the path planning module. The data transmission module is used to realize data interaction between the mobile inspection robot and the edge data processing and analysis module and the cloud server. The warning light strip is used to transmit light alarm information to people around the scene when the execution result of any subtask in the execution result of the first type of safety risk identification task or the execution result of the second type of safety risk identification task shows that a corresponding safety risk event has been identified. The positioning device is used to perform real-time positioning of the mobile inspection robot and obtain the positioning data.
10. The system according to claim 1, characterized in that The edge data processing and analysis module is deployed on an edge server in a data center, a server room, or a dedicated edge computing node in the preset area.
Citation Information
Patent Citations
Field station intelligent robot inspection system and method
CN112213979A
Method and system for remotely and intelligently detecting goods in storage space
CN115147748A
Intelligent workshop control method and system based on edge calculation
CN116740821A
Intelligent identification method and system for electricity-related public safety hidden trouble based on edge AI
CN118247697A
Intelligent management method and system based on edge computing
WO2020143094A1
Cited By
Safety management risk dynamic monitoring system for safety production
CN119360589A
Safety management risk dynamic monitoring system for safe production
CN119360589B
Six-axis robot arm device based on terahertz detection and control method
CN120503177A
