Intelligent auxiliary system of fire-fighting robot

Abstract

The invention discloses a fire-fighting robot intelligent auxiliary system in the field of Internet of Things. The fire-fighting robot intelligent auxiliary system comprises a central control platform, an Internet of Things center, a mobile terminal, a robot and a peripheral system. The robot and peripheral system is used for collecting real-time information of a site for the central control platform to judge and position, and assisting the robot in navigation and obstacle avoidance; the central control platform is used for analyzing and processing equipment information and environment information of the robot and the peripheral system and sending decision control information to control the robot and the peripheral system to make a behavior corresponding to the decision control information, and the behavior at least comprises one of the following content items: autonomous cruise and automatic fire extinguishing; the Internet of Things center is used for receiving and backing up the field information processed by the central control platform; and the mobile terminal is used for receiving the field information processed by the central control platform and receiving a remote control command so as to realize man-machine interaction. The system has the environment monitoring function, and fire prevention and safety management under the unmanned environment are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Things, and in particular relates to an intelligent auxiliary system for a fire-fighting robot. Background Art

[0002] With the rapid development of social economy and the accelerated urbanization process, the density of use in urban core areas is increasing. In places such as libraries and warehouses, once a fire occurs, especially when unattended, it may lead to serious fire accidents. At present, fire prevention in such areas mainly relies on fire monitoring methods such as cameras and traditional fire extinguishers. However, the lack of intelligent alarm equipment and advanced prevention and management technology requires a lot of manpower and material resources.

[0003] With the improvement of production and manufacturing capabilities, more and more robots are being used in safety management. Using robots for unmanned inspections can effectively prevent the occurrence of fires. However, most of the firefighting robot systems on the market currently monitor only based on a single signal source and cannot automatically extinguish fires without human intervention. In addition, flame recognition is greatly affected by factors such as environmental conditions and sensor stability, which can easily lead to false alarms. Therefore, there is an urgent need for an intelligent firefighting robot system that can adapt to complex environments and has the ability to collect multiple information sources. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent auxiliary system for fire-fighting robots, which has environmental monitoring capabilities, improves the accuracy of fire source identification, and realizes preventive safety management of fire situations in an unmanned environment.

[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions: a fire-fighting robot intelligent assistance system, comprising: a central control platform, an Internet of Things center, a mobile terminal, a robot and a peripheral system; the robot and the peripheral system are used to collect real-time information on the scene for the central control platform to judge and locate, and assist the robot in navigation and obstacle avoidance; the central control platform is used to analyze and process the equipment information and environmental information of the robot and the peripheral system, and send decision control information to control the robot and the peripheral system to perform actions corresponding to the decision control information, and the actions include at least one of the following content items: autonomous cruising and automatic fire extinguishing; the Internet of Things center is used to receive and back up the on-site information processed by the central control platform; the mobile terminal is used to receive the on-site information processed by the central control platform and receive remote control commands to realize human-computer interaction.

[0006] As the information processing and decision-making center of the system, the central control platform receives equipment information and field data from the robot and peripheral systems, and sends decision-making control information to complete the control of the robot; synchronizes the collected environmental information and equipment information to the Internet of Things platform, issues early warning signals, realizes real-time monitoring, and stores data to summarize the laws of sudden fires; synchronizes equipment information, environmental information, issues early warning signals to mobile terminals, receives control commands from mobile terminals, and issues remote commands to control the robot and peripheral systems to stop cruising and perform tasks, helping the robot to perform firefighting work more flexibly.

[0007] As a further improvement of the technical solution of the present invention, the central control platform includes a fusion identification component, a positioning and navigation component, a communication component and a remote control component. The fusion identification component performs a fusion judgment on the fire source of the environmental information, calculates the fire source information of the open fire according to the deep learning fire source identification algorithm based on yolov8, and finally locates the fire source in combination with the robot position fed back by the laser radar, or comprehensively judges the probability of the existence of a dark fire igniting at the scene of the fire source according to the fire source identification algorithm based on multi-source fusion. When the probability of the fire source is not found, the positioning and navigation component controls the robot to cruise the scene on a fixed path. After the fire extinguishing conditions are met, the positioning and navigation component controls the robot to go to the fire extinguishing point and issues a fire extinguishing command according to the fire source position information calculated by the fusion identification component. The remote control component synchronizes the robot equipment information and environmental information to the mobile terminal, and receives the control command of the mobile terminal. The communication component reports the processed on-site information to the Internet of Things center and the mobile terminal, sends the control command of the mobile terminal to the robot and the peripheral system, and sends the decision control information of the central control platform to the robot and the peripheral system.

[0008] Through the above technical solution, the basic network data transmission of the system upper end is completed, and the control plan of the robot and peripheral system is formulated. The robot can cruise autonomously, and people can also actively interrupt the autonomous cruise through the mobile terminal and flexibly control the robot's work. By remotely controlling the robot, people can avoid directly facing danger and difficulties and improve work efficiency.

[0009] As a further improvement of the technical solution of the present invention, the robot and peripheral system include a main control board, a motion chassis, a laser radar, a binocular camera, an environmental perception module, a single-chip microcomputer module and a fire extinguishing module. The main control board sends the processed on-site information for the central control platform to make decisions, receives and executes the control instructions of the central control platform, and the main control board includes a data transmission component, an IAAP component and a SLAM component. The data transmission component is used to realize data transmission between the main control board and the central control platform, the laser radar, the binocular camera, the environmental perception module, the single-chip microcomputer module and the fire extinguishing module. The binocular camera captures the environmental image and forms a feature map through the IAAP component. The laser radar realizes 2D scanning of the environment and completes the establishment of a 2D map through the SLAM component. The environmental perception module collects on-site environmental gas information and temperature and humidity information. The single-chip microcomputer module processes the environmental data collected by the environmental perception module and the control instructions issued by the main control board, and converts the control instructions into PWM waves to control the fire extinguishing module.

[0010] Through the above technical solution, the binocular camera mounted on the robot can capture images of the environment and feed them back to the central control platform in real time. The benefits of real-time capture include early detection of fires, reduction of fire spread, and determination of environmental information. The map formed by the lidar and the feedback of radar ranging data can guide the robot's movement through the central control platform's algorithm control decision-making, perform obstacle avoidance navigation and automatic cruising, help improve the effectiveness of fire response and fire fighting and rescue, and protect people's lives and property safety.

[0011] As a further improvement of the technical solution of the present invention, the main control board also includes a control actuator, which is used to process the control information of the central control platform and convert it into basic control instructions to control the movement of the motion chassis and the fire extinguishing module.

[0012] As a further improvement of the technical solution of the present invention, the motion chassis includes four drive motors and four Mecanum wheels arranged in a one-to-one correspondence with the drive motors. The drive motors are provided with Hall encoders for speed measurement. The main control board calculates the robot's motion speed based on the acquired speed measurement data, and sends the processed data to the central control platform. The main control board converts the motion strategy issued by the central control platform into real-time motion angular velocity and linear velocity, and then decomposes the speed to complete PWM control to realize the movement of the robot. The motion strategy includes autonomous cruise obstacle avoidance, fire extinguishing point navigation and remote control.

[0013] As a further improvement of the technical solution of the present invention, the fire extinguishing module includes a servo, a connecting rod and a dry powder fire extinguisher. The servo is hardware-connected to the single-chip microcomputer module via a DuPont line. The servo is controlled by PWM processed by the single-chip microcomputer module, and the dry powder fire extinguisher is operated by the connecting rod.

[0014] Through the above technical solution, the fire extinguishing module can quickly extinguish the fire and control the spread of the fire after the fire is discovered. The dry powder fire extinguisher is easier to replace the fire extinguishing agent, which expands the application scope of fire extinguishing compared to water fire extinguishing.

[0015] As a further improvement of the technical solution of the present invention, the environmental perception module includes a temperature and humidity sensor, a methane sensor, a carbon monoxide sensor, a hydrogen sulfide sensor and an oxygen sensor fixed on the surface of the robot, and is connected to the single-chip microcomputer module hardware through a DuPont line. The single-chip microcomputer module samples analog quantities through ADC and obtains digital quantities of temperature and humidity, processes the output analog signal into real environmental information, and then transmits the processed data to the main control board.

[0016] Through the above technical solution, the surface materials of multiple sensors react with the gases in the air to cause changes in the output analog signals. The microcontroller module processes the data through the ADC channel to convert the analog quantity into the on-site environmental value. The fusion sensing component then performs probability estimation based on the multi-source fusion fire source identification algorithm to determine whether there is smoldering in the environment, form an early warning signal, and prevent fire before it occurs. These data are reported to the Internet of Things center through the central control platform. By monitoring the environmental data, the environmental changes can be grasped, which is of great significance to fire research.

[0017] As a further improvement of the technical solution of the present invention, the environmental perception module collects environmental information of temperature and humidity, carbon monoxide, methane, hydrogen sulfide and oxygen and sends it to the single-chip microcomputer module. The single-chip microcomputer module processes the environmental information after analog-to-digital conversion, and then sends the processed environmental information by the single-chip microcomputer module to the main control board, and finally sends it to the central control platform for processing and judgment.

[0018] Through the above technical solution, it is possible to monitor the environment in real time before a fire occurs, analyze data to detect potential risk factors such as gas leaks, and promptly curb hidden dangers of fire. It realizes the monitoring of environmental data, analyzes environmental data, studies the laws of sudden social disasters, and reduces the possibility of fire.

[0019] As a further improvement of the technical solution of the present invention, the IAAP component and the SLAM component are used to respectively drive the binocular camera and the lidar to collect on-site information and perform preprocessing. The SLAM component has a built-in cartographer for mapping, obtains the position and shape information of the target object by continuously scanning the laser beam, completes the establishment of the grid map, and estimates the posture of the radar body by comparing the map with the previous frame data and the current frame data of the radar. The main control board receives the data collected by the lidar and the binocular camera and sends the processed real-time image and map as well as the robot posture to the central control platform.

[0020] As a further improvement of the technical solution of the present invention, the central control platform identifies open flames and dark fires respectively based on the deep learning fire source recognition algorithm of yolov8 and the fire source recognition algorithm based on multi-source fusion, and then makes a comprehensive judgment; after identifying the open flame, the relative position of the fire source and the robot is determined according to the binocular ranging method, and the final fire source information is calculated in combination with the robot position information. After comprehensively judging that the probability of fire reaches a certain condition, the central control platform sends an early warning signal to the Internet of Things center and the mobile terminal.

[0021] The deep learning fire source recognition algorithm based on yolov8 includes fire source recognition and fire source positioning. The fire source recognition uses the recognition algorithm based on yolov8 to learn and process the image of the real-time image information received by the central control platform to determine whether a fire source has occurred. If the algorithm determines that a fire source exists, the positioning algorithm first performs binocular ranging to determine the relative position (distance and deflection angle) between the fire source and the robot, and calculates the fire source position in combination with the robot position information of the positioning and navigation component to determine the fire source point and the fire extinguishing point. The fire source recognition algorithm based on multi-source fusion is designed and implemented according to the on-site environmental values ​​received by the central control platform. A multi-source fire source recognition method that integrates BP neural network and DS evidence theory is designed to effectively identify smoldering fire and improve recognition accuracy, and complete early warning in smoldering conditions.

[0022] Through the above technical solution, the dual detection of open flames and hidden fires is effectively achieved, the error of sensors is reduced, and multi-source fusion overcomes the limitations of fire source detection in complex environments, reducing the possibility of fire.

[0023] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention can detect fire in time and extinguish fire in time through the system design of central control platform, mobile terminal, Internet of Things center, robot and peripheral system. At the same time, it can carry out inspection and fire fighting work around the clock, avoiding the expansion of sudden fire in unmanned environment. It has a high degree of intelligence and greatly improves the fire prevention and response capabilities.

[0024] (2) The adopted fusion recognition algorithm combines image-based deep learning and multi-source fusion fire source recognition algorithms, and has high-precision fire source recognition capabilities. It overcomes the problems of low detection accuracy caused by complex environments and avoids the misjudgment problem caused by the low accuracy of a single sensor. The recognition algorithm combined with multiple information sources effectively reduces the evidence conflicts in the case of multiple evidence conflicts and reduces the time delay of decision-making.

[0025] (3) The robot's binocular camera and environmental perception module can collect environmental images and information such as gas, temperature and humidity. The central control platform can intuitively send on-site data to people, realizing real-time detection around the clock. The data records of the IoT center will help to summarize the patterns of fire occurrence in the later stage, provide data support for post-event evaluation and improvement, and help to summarize experience and lessons.

[0026] (4) The design and implementation of the robot's automatic cruise and remote control modes completes real-time monitoring in an unmanned environment and also provides a channel for human intervention, improving the flexibility of the system's operation. The reference of human-computer interaction avoids inefficient decision-making of the system under special circumstances. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the overall structure of the system of the present invention.

[0028] Figure 2 It is a schematic diagram of the communication structure of the fire extinguishing module of the present invention.

[0029] Figure 3 This is a process diagram of fire source judgment based on yolov8 of the present invention.

[0030] Figure 4 This is a schematic diagram of the binocular ranging principle of the present invention.

[0031] Figure 5 Schematic diagram of the communication structure of the environment perception module of the present invention.

[0032] Figure 6 This is a structural diagram of fire source identification based on multi-source fusion in the present invention.

[0033] Figure 7 It is the overall control flow chart of the present invention. DETAILED DESCRIPTION

[0034] like Figure 1 As shown, a fire-fighting robot intelligent assistance system includes: a central control platform, an Internet of Things center, a mobile terminal, a robot and a peripheral system; the robot and the peripheral system are used to collect real-time information on the scene for the central control platform to judge and locate, and assist the robot in navigation and obstacle avoidance; the central control platform is used to analyze and process the equipment information and environmental information of the robot and the peripheral system, and send decision control information to control the robot and the peripheral system to perform actions corresponding to the decision control information, and the actions include at least one of the following content items: autonomous cruising and automatic fire extinguishing; the Internet of Things center is used to receive and back up the on-site information processed by the central control platform; the mobile terminal is used to receive the on-site information processed by the central control platform and receive remote control commands to realize human-computer interaction.

[0035] The IoT center is a service platform (in this invention, the Alibaba Cloud IoT platform), which forms an IoT system with the central control platform and mobile terminals (as clients, including APP and web pages). The central control platform receives information collected and processed by robots and peripheral systems. As the device end, the central control platform is responsible for providing the IoT system with real-time on-site data and robot and peripheral status information, and reporting the processed data to the IoT center in real time; downlink communication uses business applications to issue commands and push data to the client. As a data collection and monitoring platform, the IoT center does not control robots and peripheral systems, but only monitors the working status and on-site environment and collects data for background analysis to summarize the laws of sudden fires.

[0036] The mobile terminal combines embedded and Internet of Things technologies, and uses Java language to design and develop mobile phone software and web pages as mobile terminals. The central control platform synchronizes equipment and on-site information to the mobile terminal in real time through network communication. When the on-site trigger conditions exist, the central control platform immediately sends an early warning signal to the mobile terminal when a fire source or probability of fire occurs, realizing real-time monitoring and real-time early warning.

[0037] The central control platform includes fusion recognition components, positioning and navigation components, communication components and remote control components; the robot and peripheral systems include main control board, motion chassis, laser radar, binocular camera, environmental perception module, single-chip microcomputer module and fire extinguishing module.

[0038] The data transmission component is used for WiFi communication between the main control board and the central control platform for data transmission, and maintains the serial port communication between the main control board and the peripheral system; the IAAP component and the SLAM component are the driving parts of the binocular camera and the lidar respectively, and serve as the basic functional components for collecting environmental images and map coordinates; the lidar and binocular vision are installed on the robot as peripherals to assist the robot in working and collecting environmental information; the binocular camera collects images, the lidar scans environmental information, and the main control board obtains the environmental images collected by the binocular camera through the IAAP component via serial port communication, and pre-processes the environmental images, and finally sends the pre-processed images to the robot. It is transmitted to the central control platform via the WiFi network; the main control board obtains the point cloud information collected by the lidar through the SLAM component via serial communication, uses cartographer to build a map, obtains the position and shape information of the target object by continuously scanning the laser beam, completes the establishment of the grid map, and estimates the position and posture of the radar body by comparing the map with the previous frame data and the current frame data. The main control board finally transmits the obtained map and robot posture to the central control platform via the WiFi network; the control actuator is used to process the control information of the central control platform and convert it into basic control instructions, including the execution of automatic cruising, fire extinguishing and remote control instructions.

[0039] The motion chassis serves as the motion module of the robot, including four drive motors and four Mecanum wheels corresponding to the drive motors. The drive motors are equipped with Hall encoders for speed measurement. The main control board obtains the speed measurement data by controlling the actuator and obtains the robot's motion speed through the built-in closed-loop PID algorithm. The robot's trajectory is calculated based on the motion speed (angular velocity and linear velocity), and the processed data is sent to the central control platform. The control actuator converts the motion strategy issued by the central control platform into real-time motion angular velocity and linear velocity, and then decomposes the speed to complete PWM control to realize the robot's motion. The motion strategies include autonomous cruise obstacle avoidance, fire extinguishing point navigation and remote control.

[0040] The MCU module is based on STM32 MCU and is connected to the main control board via TTL-USB. The main control board sends the processed on-site information to the central control platform for decision-making, receives and executes the control instructions of the central control platform. The main control board, as the on-site control center of the robot, is equipped with Ubuntu22.04 and ROS2 humble system. Specifically, the main control board includes data transmission components, IAAP (Image Acquisition And Processing) components, SLAM (Simultaneous Localization And Mapping, real-time positioning and map building) components and control actuators; the environmental perception module includes the DHT11 digital temperature and humidity sensor, MQ-4 methane sensor, MQ-7 carbon monoxide sensor, MQ-136 hydrogen sulfide sensor and AO-03 oxygen sensor fixed on the surface of the robot. Each sensor is connected to the single-chip microcomputer module hardware through DuPont lines. The single-chip microcomputer module samples analog quantities through ADC and obtains digital quantities of temperature and humidity, processes the output analog signals into real environmental information, and then transmits the processed data to the main control board, which is finally sent to the central control platform for processing and judgment; the fire extinguishing module includes a servo, a connecting rod and a dry powder fire extinguisher. The servo is connected to the single-chip microcomputer module hardware through a DuPont line. The servo is controlled by PWM processed by the single-chip microcomputer module, and the dry powder fire extinguisher is operated by the connecting rod. After the fusion recognition component determines that there is an open fire, the central control platform sends the fire source point and the fire extinguishing point to the robot through the WiFi network. After the robot receives the fire source location information, it stops automatic cruising and approaches the fire extinguishing point. After the central control platform determines that the robot has reached the fire extinguishing point, it sends a fire extinguishing command to the robot. The main control board receives the fire extinguishing command and uses the single-chip microcomputer module to perform PWM control on the servo. The servo starts the dry powder fire extinguisher through the connecting rod to spray the fire extinguishing agent to complete the automatic fire extinguishing. Figure 2 shown.

[0041] The data transmission component is used for WiFi communication between the main control board and the central control platform for data transmission, and maintains the serial port communication between the main control board and the peripheral system; the IAAP component and the SLAM component are the driving parts of the binocular camera and the lidar respectively, and serve as the basic functional components for collecting environmental images and map coordinates; the lidar and binocular vision are installed on the robot as peripherals to assist the robot in working and collecting environmental information; the binocular camera collects images, the lidar scans environmental information, and the main control board obtains the environmental images collected by the binocular camera through the IAAP component via serial port communication, and pre-processes the environmental images, and finally sends the pre-processed images to the robot. It is transmitted to the central control platform via the WiFi network; the main control board obtains the point cloud information collected by the lidar through the SLAM component via serial communication, uses cartographer to build a map, obtains the position and shape information of the target object by continuously scanning the laser beam, completes the establishment of the grid map, and estimates the position and posture of the radar body by comparing the map with the previous frame data and the current frame data. The main control board finally transmits the obtained map and robot posture to the central control platform via the WiFi network; the control actuator is used to process the control information of the central control platform and convert it into basic control instructions, including the execution of automatic cruising, fire extinguishing and remote control instructions.

[0042] The motion chassis serves as the motion module of the robot, including four drive motors and four Mecanum wheels corresponding to the drive motors. The drive motors are equipped with Hall encoders for speed measurement. The main control board obtains the speed measurement data by controlling the actuator and obtains the robot's motion speed through the built-in closed-loop PID algorithm. The robot's trajectory is calculated based on the motion speed (angular velocity and linear velocity), and the processed data is sent to the central control platform. The control actuator converts the motion strategy issued by the central control platform into real-time motion angular velocity and linear velocity, and then decomposes the speed to complete PWM control to realize the robot's motion. The motion strategies include autonomous cruise obstacle avoidance, fire extinguishing point navigation and remote control.

[0043] The communication component uses WiFi network transmission, realizes WiFi Internet access through a wireless router, and connects to the robot network. It receives device information and field data of the robot and peripheral systems through WiFi, and then the fusion perception component, positioning navigation component and remote control component process and judge this information, and form decision-making control information to send to the main control board, control the motion chassis and fire extinguishing module to respond and complete the work and feedback the working status. After receiving the on-site situation, the remote control component publishes device information and field data to the Internet of Things center and mobile terminal in real time, and receives control commands from the mobile terminal, responds to the remote control command to terminate the robot's automatic cruise execution.

[0044] The positioning and navigation components include positioning and navigation. The Adaptive Monte Carlo Localization (AMCL) algorithm is used to complete 2D probabilistic positioning, and the adaptive Monte Carlo positioning method is implemented. The laser radar feedback data can be used to locate the absolute position of the robot in the existing fixed space in real time. The navigation adopts the dynamic window algorithm (Dynamic Window Approaches, DWA) to realize path planning, and combines the real-time data of the laser radar for real-time obstacle avoidance and navigation. In the fixed space, the navigation part first adopts the cruising mode to control the robot to patrol repeatedly along the specified path in a certain area. The fire-fighting robot equipped with a multi-functional peripheral system completes real-time monitoring of the environment along a fixed path. In the event of a sudden fire, the cruising is terminated, and the DWA navigates to the fire-fighting point fed back by the fusion recognition component to perform the fire-fighting task.

[0045] The fusion recognition component makes a fusion judgment on the fire source based on the environmental information, calculates the fire source information of the open flame according to the deep learning fire source recognition algorithm based on yolov8, and finally locates the fire source in combination with the robot position fed back by the lidar, or comprehensively judges the probability of the existence of a hidden fire igniting the fire source at the scene according to the fire source recognition algorithm based on multi-source fusion. When the probability of the fire source is not found, the positioning and navigation component controls the robot to cruise the scene on a fixed path. After the fire extinguishing conditions are met, the positioning and navigation component controls the robot to go to the fire extinguishing point and issues a fire extinguishing command according to the fire source location information calculated by the fusion recognition component. The remote control component synchronizes the robot equipment information and environmental information to the mobile terminal, and receives the control command from the mobile terminal.

[0046] The fusion recognition component combines the fire source identification prediction of open flames and smoldering. The open flame part uses visual recognition. The central control platform receives the pre-processed image information from the main control board and uses the fire source recognition algorithm based on yolov8 to determine whether there is a fire source in the environment. In the case of a fire source, the relative distance and angle between the fire source and the robot are determined by binocular vision pixel scale ranging, and combined with the robot's position in the map, the location of the fire source and the fire extinguishing point in the map is finally calculated.

[0047] Figure 3 The yolov8 used in the present invention is an end-to-end convolutional neural network, which uses a model trained with a large number of data sets to perform recognition. Overfitting is prevented by methods such as data enhancement and regularization. The algorithm first pre-processes the image collected by the binocular camera (including image enhancement, format conversion, etc.), and then recognizes it through the pre-trained model to obtain the probability of fire. When the probability is greater than 0.65, it is determined that there is an open flame in the environment.

[0048] Compared with other detection and recognition algorithms, the deep learning fire source recognition algorithm adopted by the present invention realizes the simultaneous detection of multiple objects in the image, the judgment results are highly accurate, and can realize real-time detection on equipment with limited resources. It also has strong robustness, assisting robots to extinguish fires in time.

[0049] Combine the following Figure 4 The binocular ranging method is described in detail. The specific process is as follows: In the figure, cameras 1 and 2 are the positions of two binocular cameras, and the distance between the two cameras is , and its field of view is , the distance to the target is , the captured image width is , the image pixels are The target fire source is , taken by camera 1 The pixel coordinates of the point are , taken by camera 2 The pixel coordinates of the point are , It's about the lens. Included Pixels, so The distance represented by each pixel.

[0050] The formula is as follows:

[0051] Find the distance Compared with other distance measurement methods, binocular distance measurement has no recognition rate limit. In principle, it does not need to identify and then measure obstacles. All obstacles can be directly measured. During the robot's movement, different real-time points are continuously calculated to ensure the accuracy of the fire source location.

[0052] like Figure 5 and Figure 6As shown, the present invention takes into account the limitations of visual-based fire source identification in complex environments, combines the fire source identification algorithm based on multi-source fusion, and designs and implements a multi-source fire detection method that integrates BP neural network and DS evidence theory. This method introduces neural network when constructing the basic probability distribution of evidence, which effectively reduces the influence of sensor error. Through the environmental perception module, temperature, humidity, carbon monoxide concentration, hydrogen sulfide concentration, methane concentration and oxygen concentration are selected as data sources to construct a fire-free and smoldering identification framework. The algorithm first uses BP neural network to construct the basic probability assignment function of carbon monoxide, methane and hydrogen sulfide concentrations; sets three pre-processed characteristic values ​​of carbon monoxide, methane and hydrogen sulfide concentrations (the difference between the observed value and the normal value, the average value and the rate of change) as the input of the BP neural network. We construct a three-layer neural network model, which has three neurons in the input layer, which are the average value, rate of change and the difference between the observed value and the normal value of CO concentration (or methane, hydrogen sulfide concentration). The input data is normalized, and the output layer has three neurons, which are the probability of no fire, the probability of smoldering fire, and the probability of fire source without open flame. The judgment probability of no open flame or smoldering is set as the output of the BP neural network; the basic probability distribution function of oxygen concentration, temperature, and humidity is constructed using fuzzy membership functions; the evidence weight is then modified by calculating the support of evidence and the support of the focal element to obtain the probability assignment after the fusion of each data source; the fusion probability assignment is fused multiple times using the combination rule to obtain the final judgment probability of the fire. When the sum of the probabilities of smoldering is greater than 0.8, we determine that there is a risk of fire, and the central control platform generates an early warning signal to send to the mobile terminal and the Internet of Things center.

[0053] like Figure 7As shown, the working principle of the present invention is as follows: for a fixed workspace, the system working process starts with map building. After the robot is started, it patrols the workspace multiple times, completes the grid map building through the SLAM component, synchronizes the map and the robot posture to the central control platform, the central control platform sets the autonomous cruise path, and the robot starts autonomous cruising. In a fixed space, the binocular camera and the environmental perception module collect environmental images and data in real time, which are then processed by the main control board and reported to the central control platform together with the working status information of the robot and peripheral systems. The central control platform synchronizes environmental information and device information to the mobile terminal and the Internet of Things platform, and calculates and identifies this information in real time through the fusion recognition component. If the system receives a remote control command, the autonomous cruise is terminated, and the remote control command is executed first. After completing the task, it returns to the autonomous cruise path and continues autonomous cruising. If the central control platform determines that there is a fire source, it will immediately send an early warning signal to the mobile terminal and the Internet of Things platform for early warning, and at the same time send the fire extinguishing point and the fire source point to the robot, stop the robot's autonomous cruising, and control the robot to go to the fire extinguishing point. When it is determined that the robot has reached the fire extinguishing point, it will immediately start the fire extinguishing module, spray the fire extinguishing agent, and complete the automatic fire extinguishing. The fire source location information is sent to the robot through the network. After the robot receives the fire source location information, it will stop the automatic cruising and approach the fire source location signal point. When the robot and the fire source are at a certain distance measured by the binocular ranging method, the fusion recognition component sends a trigger signal to the central control platform. After receiving the trigger signal, the central control platform immediately sends a fire extinguishing command to the fire extinguishing module through the network. The fire extinguishing module controls the servo to start the dry powder fire extinguisher to spray the fire extinguishing agent to complete the automatic fire extinguishing.

[0054] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A fire-fighting robot intelligent assistance system, characterized in that: include: Central control platform, Internet of Things center, mobile terminal, robot and peripheral system; The robot and peripheral system are used to collect real-time information on site for the central control platform to judge and locate, and assist the robot in navigation and obstacle avoidance; The central control platform is used to analyze and process the equipment information and environmental information of the robot and the peripheral system, and send decision control information to control the robot and the peripheral system to perform actions corresponding to the decision control information, and the actions include at least one of the following items: autonomous cruising and automatic fire extinguishing; The Internet of Things center is used to receive and back up the on-site information processed by the central control platform; The mobile terminal is used to receive the on-site information processed by the central control platform and receive the remote control command to realize human-computer interaction.

2. The fire-fighting robot intelligent assistance system according to claim 1, characterized in that: The central control platform includes a fusion identification component, a positioning and navigation component, a communication component and a remote control component. The fusion identification component performs a fusion judgment on the fire source of the environmental information, calculates the fire source information of the open fire according to the deep learning fire source identification algorithm based on yolov8, and finally locates the fire source in combination with the robot position fed back by the laser radar, or comprehensively judges the probability of the existence of a hidden fire igniting at the scene of the fire source according to the fire source identification algorithm based on multi-source fusion; when the probability of the fire source is not found, the positioning and navigation component controls the robot to cruise the scene on a fixed path. After the fire extinguishing conditions are met, the positioning and navigation component controls the robot to go to the fire extinguishing point according to the fire source position information calculated by the fusion identification component and issues a fire extinguishing command. The remote control component synchronizes the robot equipment information and environmental information to the mobile terminal, and receives the control command of the mobile terminal. The communication component reports the processed on-site information to the Internet of Things center and the mobile terminal, issues the control command of the mobile terminal to the robot and the peripheral system, and issues the decision-making control information of the central control platform to the robot and the peripheral system.

3. The fire-fighting robot intelligent assistance system according to claim 2, characterized in that: The robot and peripheral system include a main control board, a motion chassis, a laser radar, a binocular camera, an environment perception module, a single-chip microcomputer module and a fire extinguishing module. The main control board sends the processed on-site information to the central control platform for decision-making, receives and executes the control instructions of the central control platform, and includes a data transmission component, an IAAP component and a SLAM component. The data transmission component is used to realize data transmission between the main control board and the central control platform, the laser radar, the binocular camera, the environment perception module, the single-chip microcomputer module and the fire extinguishing module. The binocular camera realizes the capture of environmental images and forms a feature map through the IAAP component, and the laser radar realizes 2D scanning of the environment and completes the establishment of a 2D map through the SLAM component. The environment perception module collects on-site environmental gas information and temperature and humidity information. The single-chip microcomputer module processes the environmental data collected by the environment perception module and the control instructions issued by the main control board, and converts the control instructions into PWM waves to control the operation of the fire extinguishing module.

4. The fire-fighting robot intelligent assistance system according to claim 3, characterized in that: The main control board also includes a control actuator, which is used to process the control information of the central control platform and convert it into basic control instructions to control the movement of the motion chassis and the fire extinguishing module.

5. The fire-fighting robot intelligent assistance system according to claim 3, characterized in that: The motion chassis includes four drive motors and four Mecanum wheels corresponding to the drive motors. The drive motors are provided with Hall encoders for speed measurement. The main control board calculates the robot's motion speed based on the acquired speed measurement data and sends the processed data to the central control platform. The main control board converts the motion strategy issued by the central control platform into real-time motion angular velocity and linear velocity, and then decomposes the speed to complete PWM control to realize the movement of the robot. The motion strategy includes autonomous cruise obstacle avoidance, fire extinguishing point navigation and remote control.

6. The fire-fighting robot intelligent assistance system according to claim 3, characterized in that: The fire extinguishing module includes a steering gear, a connecting rod and a dry powder fire extinguisher. The steering gear is connected to the single-chip module hardware through a DuPont line. The steering gear is controlled by PWM processed by the single-chip module and the dry powder fire extinguisher is operated by the connecting rod.

7. The fire-fighting robot intelligent assistance system according to claim 3, characterized in that: The environmental perception module includes a temperature and humidity sensor, a methane sensor, a carbon monoxide sensor, a hydrogen sulfide sensor and an oxygen sensor fixed on the surface of the robot, and is connected to the single-chip computer module hardware through a DuPont line. The single-chip computer module samples analog quantities through ADC and obtains digital quantities of temperature and humidity, processes the output analog signals into real environmental information, and then transmits the processed data to the main control board.

8. The fire-fighting robot intelligent assistance system according to claim 3, characterized in that: The environmental sensing module collects environmental information of temperature and humidity, carbon monoxide, methane, hydrogen sulfide and oxygen and sends it to the single-chip microcomputer module. The single-chip microcomputer module processes the environmental information after analog-to-digital conversion, and then sends the processed environmental information from the single-chip microcomputer module to the main control board, which is finally sent by the main control board to the central control platform for processing and judgment.

9. The fire-fighting robot intelligent assistance system according to claim 3, characterized in that: The IAAP component and SLAM component are used to drive the binocular camera and lidar respectively to collect on-site information and perform preprocessing. The SLAM component has a built-in cartographer for mapping, obtains the position and shape information of the target object by continuously scanning the laser beam, completes the establishment of the grid map, and estimates the position and posture of the radar body by comparing the map with the previous frame data and the current frame data of the radar. The main control board receives the data collected by the lidar and binocular camera and sends the processed real-time image and map as well as the robot's position and posture to the central control platform.

10. A firefighting robot intelligent assistance system according to any one of claims 1 to 9, characterized in that: The central control platform identifies open flames and dark fires based on yolov8's deep learning fire source recognition algorithm and a fire source recognition algorithm based on multi-source fusion, and then makes a comprehensive judgment. After identifying an open flame, the relative position of the fire source and the robot is determined based on the binocular ranging method, and the final fire source information is calculated in combination with the robot's position information. After comprehensively judging that the probability of fire reaches a certain condition, the central control platform sends an early warning signal to the Internet of Things center and the mobile terminal.

Images