A front-end fire-fighting robot control method and system

Abstract

The present application discloses a front-end fire-fighting robot control method and system, which relates to the field of fire protection, including: setting multiple smoke sensors and infrared cameras in the target area, and connecting them with the fire-fighting robot to form a distributed fire detection network; when a fire is detected through a flame recognition algorithm, the positioning information of the fire area is sent to the fire-fighting robot; after the fire-fighting robot receives the positioning information of the fire area, it uses the laser radar sensor on board to collect environmental information in the target area, adopts the synchronous positioning and map construction SLAM algorithm, establishes a three-dimensional point cloud model of the buildings in the area, and uses the constructed three-dimensional point cloud model to perform self-positioning and navigation path planning. In view of the poor cross-regional operation and navigation capabilities of the fire-fighting robot in complex environments in the prior art, the present application improves the network collaboration capabilities of each component unit, the rapid identification and response capabilities of complex environments, and the navigation capabilities.

Description

Technical Field

[0001] The present application relates to the field of fire protection, and in particular to a front-end fire-fighting robot control method and system. Background Art

[0002] With the continuous advancement of my country's industrialization and urbanization, large industrial parks and urban construction sites are increasing. However, due to the complex environment and large number of combustibles in these areas, it is often difficult to detect and extinguish a fire in time, which can easily cause heavy casualties and property losses. The traditional firefighting mode mainly relies on fixed fire alarm facilities and manual firefighting, which has problems such as slow response speed, limited coverage, and low firefighting efficiency, and it is difficult to meet the current complex and changing fire safety needs. Therefore, it is urgent to develop an intelligent firefighting equipment that can realize early fire detection, rapid response, and autonomous fire extinguishing, so as to improve the fire prevention and control and disposal capabilities in complex environments.

[0003] In recent years, emerging technologies represented by robots, the Internet of Things, and artificial intelligence have flourished, providing new ideas for solving the above-mentioned fire safety problems. Some scholars have proposed using intelligent fire extinguisher robots to realize autonomous fire detection and fire extinguishing operations. Through environmental perception, autonomous navigation, intelligent decision-making and other technologies, robots have intelligent behavioral capabilities similar to those of humans, replacing firefighters to perform high-risk fire fighting tasks. At the same time, by deploying fire sensors on a large scale to form the Internet of Things, and using their perception data to command and dispatch fire extinguisher robots, the scope of fire monitoring can be effectively expanded, and the response speed and collaborative efficiency of fire extinguishing equipment can be improved. Combining intelligent fire extinguisher robots with Internet of Things technology to form a full-chain intelligent firefighting model of "sensing perception-intelligent analysis-autonomous execution" is expected to fundamentally improve the level of fire prevention and emergency response in complex environments.

[0004] At present, fire safety issues in construction sites and industrial parks are becoming increasingly prominent. Due to the complex and changeable environment in these areas, the deployment of traditional fixed fire detection and fire extinguishing facilities has many shortcomings: first, the buildings and facilities on the construction site need to change in real time with the progress of the construction, and it is difficult to flexibly deploy fixed fire fighting equipment; second, there are a large number of open areas in large parks, and the coverage cost of fixed fire fighting equipment is high, making it difficult to achieve full-area monitoring; third, once a fire occurs, the response range of fixed fire extinguishing equipment is limited, making it difficult to effectively extinguish a large-scale fire.

[0005] Although existing mobile firefighting equipment, such as remote-controlled tracked fire trucks, have overcome the shortcomings of fixed equipment to a certain extent, they still require on-site control by firefighters, resulting in low firefighting efficiency and the risk of personal injury. In response to the above problems, some scholars have proposed the use of intelligent fire extinguisher robots to achieve autonomous fire detection and mobile firefighting operations. However, most existing fire extinguisher robots use a single body sensor, which makes it difficult to fully perceive complex fire environments, has limited navigation capabilities, and cannot adapt to large-scale regional operations. In addition, due to the lack of information interaction with the environmental monitoring sensor network, the fire information obtained by the robot is incomplete, making it difficult to achieve early warning of fires and active firefighting.

[0006] Therefore, it is urgent to break through the following technical bottlenecks: first, to study the intelligent fusion of multi-source heterogeneous sensors and the precise positioning technology of fire areas in complex environments; second, to study the autonomous positioning, obstacle avoidance navigation and mobile operation technology of fire extinguisher robots in complex environments; third, to study the information interaction and collaborative operation scheduling technology between the fire-fighting Internet of Things and intelligent robots, so as to realize early warning of fires and active fire extinguishing. Summary of the invention

[0007] In response to the problem of poor navigation ability of fire-fighting robots in complex environments in the prior art, the present application provides a front-end fire-fighting robot control method and system, which improves the path navigation ability of the fire-fighting robot in complex environments by establishing a distributed fire detection network, using Bayesian reasoning to obtain the precise location of the fire, and mapping and positioning the target area and planning the path based on the acquired location, as well as performing closed-loop detection on the planned path.

[0008] The purpose of this application is achieved through the following technical solutions.

[0009] One aspect of the present application provides a front-end fire-fighting robot control method, including: setting a plurality of smoke sensors and infrared cameras in a target area, the smoke sensors and infrared cameras are connected to the fire-fighting robot through wireless communication to form a distributed fire detection network; when the distributed fire detection network detects a fire through a flame recognition algorithm, the positioning information of the fire area is sent to the fire-fighting robot; after the fire-fighting robot receives the positioning information of the fire area, it uses the onboard laser radar sensor to collect environmental information in the target area, adopts a synchronous positioning and map construction SLAM algorithm, establishes a three-dimensional point cloud model of the buildings in the area, and uses the constructed three-dimensional point cloud model to perform self-positioning and navigation path planning.

[0010] The fire-fighting robot navigates to the area where the fire occurs according to the planned navigation path and performs fire-fighting. This includes: the fire-fighting robot uses lidar and visual sensors to perceive the environment according to the planned navigation path, and autonomously navigates to the fire area according to the local environment information and the location of the target point through the local path planning algorithm DWA; after reaching the fire area, the visible light camera and infrared thermal imager on the robot are controlled to collect images of the fire scene, and the collected visible light images and infrared images are fused and registered, and the flame area is detected using the deep learning target detection algorithm YOLOv3; the detected flame area is located in the infrared image, and the flame pixel coordinates are converted into three-dimensional space coordinates according to the internal and external parameters of the infrared imager to obtain the fire source location information; according to the fire source location information, the fire-fighting device is controlled to aim at the fire source to extinguish the fire.

[0011] Furthermore, multiple smoke sensors and infrared cameras are set in the target area, and the smoke sensors and infrared cameras are connected to the fire-fighting robot through wireless communication to form a distributed fire detection network, including: multiple smoke sensors and infrared cameras are set at different locations in the target area; wherein the smoke sensors are used to detect smoke concentrations, and the infrared cameras are used to collect infrared images; a LoRa or WIFI wireless communication module is integrated on each smoke sensor and infrared camera, and the location information of each smoke sensor and infrared camera is bound to the system; the system platform receives data from each sensor node in real time and performs fire analysis and data preprocessing.

[0012] Furthermore, when the distributed fire detection network detects a fire through the flame recognition algorithm, the positioning information of the fire area is sent to the fire-fighting robot, including: the system platform fuses the pre-processed smoke concentration data and infrared image data of each sensor node for multi-source information, and uses the Bayesian reasoning method to establish a fire probability model; in the fire probability model, the smoke concentration data and infrared image data are used as observations, and the probability of fire occurrence is used as a hidden variable, and the posterior probability of fire occurrence under the condition of given observations is calculated through Bayesian reasoning; the area where the fire occurs is determined based on the sensor nodes whose posterior probabilities exceed the threshold, and the location information of the sensor nodes in the determined area is used as the preliminary positioning information of the fire area; the preliminary positioning information is transmitted to the fire-fighting robot, and the fire-fighting robot determines the starting point and end point of the walking and the navigation path based on its own position and environmental data. After arriving at the fire area, the fire-fighting robot will find the final fire point based on its own flame detector and perform local navigation.

[0013] Furthermore, the system platform fuses the preprocessed smoke concentration data and infrared image data of each sensor node for multi-source information, and uses the Bayesian reasoning method to establish a fire probability model, including: defining each sensor node as a node in a Markov random field, and the node's observations are the smoke concentration data and infrared image data collected by the corresponding node; determining the set of neighborhood nodes for each node based on the spatial distribution of the sensor nodes, associating the node's observations with the observations of the neighborhood nodes, and constructing the spatial correlation between the nodes; establishing a Markov random field model based on the spatial correlation between the nodes, defining the state of the fire as a hidden variable of the random field, and defining the observations of all nodes as observed variables of the random field; defining the energy function E(x) in the Markov random field; and solving the maximum a posteriori probability estimate of the fire occurrence in the Markov random field by minimizing the energy function E(x).

[0014] Furthermore, in the Markov random field, the energy function E(x) is defined as: Among them, x represents the state of fire occurrence; x i represents the fire status of the i-th node, y represents the observation of all nodes, y i represents the observation of the i-th node; φ i (x i ,y i ) represents the local evidence function of the i-th node, measuring the observation y i With fire status x i The compatibility of is, the larger the value is, the more the observation supports the fire state. The calculation formula is: Among them, μ(x i ) and Σ(x i ) represent the fire status x of the i-th node respectively. i The mean and covariance matrix of the observations under ψ ij (x i ,y i ) represents the spatial correlation function between the i-th node and the j-th node, which measures the correlation strength of the fire status of adjacent nodes. The larger the value, the more consistent the fire status of the two nodes. The calculation formula is: Among them, d(x i ,y j ) indicates the fire status x i and x j The distance measure between ij is the correlation strength parameter; Z(y) represents the partition function, which is used to normalize the probability distribution so that the cumulative probability of the posterior probability P(x|y) is equal to 1. The calculation formula is:

[0015] The energy function comprehensively considers the influence of local observation evidence of nodes and spatial correlation between nodes on fire status estimation, and uses the local evidence function φ i (x i ,y i ) measures the compatibility of the observed value with the fire state, through the spatial correlation function ψ ij (x i ,y i ) measures the correlation strength of the fire status of adjacent nodes, and uses the distribution function Z(y) to normalize the probability distribution. When minimizing the energy function to solve the fire status estimation, the local evidence term To make the estimation results consistent with the observed data, the spatial correlation term ∑ ij ψ ij (x i , x j ) makes the estimation results of adjacent nodes consistent. The two items constrain each other and work together to enable the Markov random field model to strike a balance between observational evidence and spatial correlation, and more accurately infer the location of the fire. This energy function makes full use of the spatial correlation characteristics of sensor data and improves the robustness and reliability of fire location in complex environments.

[0016] Further, the fire-fighting robot verifies and optimizes the preliminary positioning information to obtain the final positioning information of the fire area, including: optimizing the preliminary positioning information using a particle filter algorithm, initializing a set of particle samples according to the position distribution of sensor nodes in the fire area, each particle containing a fire position, a particle priori weight and node observation data; estimating the smoke concentration distribution and infrared radiation distribution at the corresponding particle according to the fire position contained in each particle, comparing the estimated distribution at the particle with the observation data collected by the node, and calculating the likelihood probability of the corresponding particle; multiplying the likelihood probability of the particle by the corresponding priori weight to obtain the posterior weight of the particle; resampling the particle sample according to the posterior weight of each particle in the particle sample, retaining particles with posterior weights greater than a threshold, and obtaining a resampled particle sample set; iteratively calculating the particle posterior weight and resampling the particles on the resampled particle sample set, and updating the particle sample set until the distribution of the particle sample set converges or the number of iterations reaches a threshold; weighted averaging the fire positions of each particle in the converged particle sample set to obtain the maximum posterior probability estimate of the fire position as the final positioning information of the fire area.

[0017] Furthermore, after receiving the positioning information of the fire area, the fire-fighting robot uses the laser radar sensor on board to collect environmental information in the target area, adopts the synchronous positioning and map construction SLAM algorithm, establishes a three-dimensional point cloud model of the buildings in the area, and uses the constructed three-dimensional point cloud model to perform self-positioning and navigation path planning, including: after receiving the positioning information of the fire area, the fire-fighting robot uses the laser radar sensor on board to scan the target area 360° to obtain the original point cloud data of the buildings and obstacles in the area; downsamples the original point cloud data to reduce the amount of data, and uses the random sampling consistency algorithm RANSAC to remove ground points to obtain pre-processed point cloud data; inputs the pre-processed point cloud data into the synchronous positioning and map construction algorithm to obtain the real-time posture of the fire-fighting robot in the target area, and establishes a three-dimensional point cloud map of the buildings in the area according to the real-time posture; projects the three-dimensional point cloud map onto a two-dimensional plane, represents it in the form of a grid map, takes the current position of the fire-fighting robot as the starting point, and the positioning information of the fire area as the target point, and constructs a cost map from the starting point to the target point; on the cost map, uses the A* search algorithm to plan the navigation path of the fire-fighting robot from the starting point to the target point. The fire-fighting robot determines the starting and ending points and navigation path of its walking according to its own position and environmental data. After reaching the fire area, the fire-fighting robot will find the final fire point according to its own flame detector and perform local navigation.

[0018] Furthermore, the preprocessed point cloud data is input into the simultaneous positioning and mapping algorithm to obtain the real-time position and posture of the fire-fighting robot in the target area, and a three-dimensional point cloud map of the buildings in the area is established according to the real-time position and posture, including: taking the initial position of the fire-fighting robot as the origin, and transforming the coordinates of the preprocessed point cloud data according to the real-time position and posture of the fire-fighting robot to obtain a local point cloud map with the current position and posture of the robot as the reference system; inputting the local point cloud map and the robot's position and posture into the simultaneous positioning and mapping algorithm, and constructing a factor graph model using a graph optimization-based method; according to the lidar data and the robot motion model, the extended Kalman filter algorithm is used to predict the robot's position and posture, and the prior estimate of the robot's position and posture is obtained, and the predicted position and posture are added as the lidar odometer factor. into the factor graph model; in the factor graph model, the laser radar odometer factor, loop detection factor and local map factor are used as edges in the graph to construct the geometric constraint relationship between the nodes in the graph; by extracting the feature points including corner points and plane points in the local point cloud map, a closed-loop detection algorithm based on appearance and spatial geometric information is used to determine whether the current position of the robot has reached the previous position; if a closed loop is detected, a closed-loop factor is added to the factor graph model; based on the constructed factor graph model, a nonlinear least squares optimization problem is constructed to solve the optimal pose and feature point position of each node in the factor graph model; the optimized robot pose is used as the real-time positioning result of the robot, and according to the optimized robot pose, the local point cloud map is converted into coordinates to obtain a three-dimensional point cloud map of the buildings in the area.

[0019] Furthermore, by extracting feature points containing corner points and plane points in the local point cloud map, a closed-loop detection algorithm based on appearance and spatial geometric information is used to determine whether the current position of the robot has reached the previous position; if a closed loop is detected, a closed-loop factor is added to the factor graph model, corresponding to the current pose node and the closed-loop matching pose node, including: dividing the local point cloud map into cubic voxels of the same size, and obtaining a feature point set of the local point cloud map by extracting the points containing the most corner points or plane points in each voxel as the feature points of the corresponding voxels; calculating the geometric descriptor and visual descriptor of each feature point in the feature point set, and The descriptor constructs a bag-of-words model to obtain a bag-of-words representation vector of a local point cloud map; the bag-of-words representation vector of the local point cloud map at the current posture node of the robot is matched with the bag-of-words representation vector of the local point cloud map at the historical posture node, and the similarity between the bag-of-words representation vectors is calculated; if the similarity is greater than a threshold, it is determined that the current position of the robot has reached the previous position and a closed loop is detected; according to the matching relationship of the local point cloud map, the relative posture transformation between the current posture node and the closed-loop matching posture node is calculated, and a closed-loop factor is added to the factor graph model, and the closed-loop factor connects the current posture node and the closed-loop matching posture node to represent the relative posture constraint between the two nodes.

[0020] Another aspect of the present application also provides a front-end fire-fighting robot control system for executing a front-end fire-fighting robot control method of the present application.

[0021] Compared with the prior art, the advantages of this application are:

[0022] This application utilizes smoke sensors and infrared cameras deployed in the target environment to form a distributed fire detection network. Through multi-source fusion of smoke concentration data and infrared image data, a fire probability model based on Markov random fields is established. In the probability graph model, the local observation evidence of sensor nodes and the spatial correlation between nodes are fully considered. The Markov random field is solved by Bayesian reasoning to obtain the maximum a posteriori probability estimate of the fire area location, which overcomes the defect that a single sensor is susceptible to interference in a complex environment and improves the accuracy and reliability of fire positioning.

[0023] This application uses the laser radar sensor carried by the fire-fighting robot to achieve simultaneous positioning and map construction through factor graph optimization. The pre-processed point cloud data is input into the SLAM algorithm, with the robot posture and local point cloud map as nodes, and the laser radar odometer, loop detection and feature matching as edges, to build a factor graph model and solve the graph optimization problem, build a three-dimensional geometric map of the target area online, and track the robot posture in real time, so that the robot can achieve autonomous positioning and map construction in an unknown environment at the same time. Based on the constructed map, by projecting the three-dimensional point cloud to a two-dimensional plane and constructing a cost map, and then using the A* algorithm for path search, the robot can autonomously plan a collision-free navigation path and achieve obstacle avoidance navigation in complex environments.

[0024] This application proposes a closed-loop detection algorithm based on appearance and geometric information. By extracting feature points from the local point cloud map and constructing a bag-of-words representation, the similarity of the bag-of-words vectors of different pose nodes is compared to determine whether the robot has reached the previous position. A closed-loop factor is added to the factor graph to impose a closed-loop constraint on the graph model, and the odometer drift is effectively suppressed through global optimization. At the same time, loop detection provides a position prior for the factor graph, and the optimized pose and map are more accurate, enabling the fire-fighting robot to maintain stable and accurate navigation for a long time in complex environments, improving navigation accuracy.

[0025] This application uses distributed multi-sensor fusion to achieve fire detection, fuses multi-source heterogeneous sensor data such as vision, infrared, radar, etc., and enhances the robot's ability to perceive the environment. At the same time, the robot's autonomous positioning and mapping are realized by combining the SLAM algorithm, so that the robot can build a three-dimensional geometric map of a complex environment online. Combining environmental perception with autonomous positioning and navigation, the fire-fighting robot can fully perceive the complex fire scene environment and use the on-site perception information to achieve accurate autonomous positioning and navigation. Multi-sensor fusion SLAM improves the environmental adaptability of the fire-fighting robot and enhances its ability to operate at complex disaster sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present application will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0027] Figure 1 is an exemplary flow chart of a front-end fire-fighting robot control method according to some embodiments of the present application;

[0028] Figure 2 It is a schematic diagram of an application scenario of a front-end fire-fighting robot control method according to some embodiments of the present application;

[0029] Figure 3 is a structural diagram of a fire extinguisher robot according to some embodiments of the present application;

[0030] Figure 4 This is a structural diagram of another fire extinguisher robot according to some embodiments of the present application. DETAILED DESCRIPTION

[0031] Example 1

[0032] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0033] Figure 1 is an exemplary flow chart of a front-end fire-fighting robot control method according to some embodiments of the present application. Figure 2This is a schematic diagram of an application scenario of a front-end fire-fighting robot control method shown in some embodiments of the present application, including: multiple smoke sensors and infrared cameras are set in the target area, and the smoke sensors and infrared cameras are connected to the fire-fighting robot through wireless communication to form a distributed fire detection network; when the distributed fire detection network detects a fire through a flame recognition algorithm, the positioning information of the fire area is sent to the fire-fighting robot; after the fire-fighting robot receives the positioning information of the fire area, it uses the onboard laser radar sensor to collect environmental information in the target area, adopts a synchronous positioning and map construction SLAM algorithm, establishes a three-dimensional point cloud model of the buildings in the area, and uses the constructed three-dimensional point cloud model to perform self-positioning and navigation path planning; the fire-fighting robot navigates to the area where the fire occurs according to the planned navigation path to extinguish the fire.

[0034] Figure 3 is a structural diagram of a fire extinguisher robot according to some embodiments of the present application, Figure 4 is another structure diagram of a fire extinguisher robot according to some embodiments of the present application, Figure 3 and Figure 4It can be seen that the fire extinguisher robot of the present application adopts a wheeled or tracked mobile chassis, which has good passability and maneuverability; the fuselage adopts a modular design, mainly including a sensor module, a fire extinguishing operation module, a control communication module, etc.; the sensor module integrates a camera, an infrared thermal imager, a laser radar, etc., which are used for fire monitoring and environmental perception; the fire extinguishing operation module includes a fire extinguishing spray gun and a fire cannon, which can spray fine water mist or foam for fire extinguishing; the control and communication module includes a main control unit, a display screen, and a communication antenna, which is responsible for information processing, display and remote communication; the power management module provides the power required for the robot to operate, and distributes and manages the power supply; the structure of the whole machine is compact and reasonable, and the sensors, actuators and control units are properly arranged, which is convenient for fire scene operations. The camera collects visible light images in real time, monitors the fire environment, and identifies fire characteristics such as flames and smoke; the infrared thermal imager collects infrared images, detects high-temperature areas based on temperature distribution, and determines the location of the fire source; the image recognition algorithm based on deep learning is used to extract and classify fire characteristics; the visual detection results are integrated with the infrared detection results to improve the accuracy of fire judgment; the fire information is uploaded to the monitoring center in real time, or alarms are issued through voice, display screen, etc. The laser radar scans the surrounding environment, obtains the point cloud data around the robot, and constructs an environmental map; the SLAM algorithm is used to combine the odometer data and the laser point cloud to achieve autonomous positioning and mapping of the robot; path planning is performed on the environmental map to generate a navigation path from the current position to the target fire source; the path tracking control uses the model predictive control (MPC) algorithm to generate speed, steering and other control instructions in real time based on the path curvature and the robot dynamics model; the control instructions are sent to the chassis motor to control the robot movement, and the trajectory is corrected according to the laser radar feedback; the robot can autonomously navigate to the vicinity of the fire source and adjust the fire extinguishing strategy according to the fire status. The fire-fighting spray gun can spray high-pressure fine water mist, which can cool down and extinguish fires through the gasification of water mist, absorbing heat and isolating oxygen; the fire-fighting cannon can fire fire-extinguishing bombs, which explode at the source of the fire, releasing a large amount of fire extinguishing agent to quickly extinguish the fire; visual servo control is used to adjust the pitch and azimuth angles of the spray gun and gun barrel according to camera feedback to aim at the fire source; the fire-extinguishing agent is supplied through a water pump and a drug supply system, and can be sprayed continuously for 20 minutes; the size of the fire is judged by infrared imaging, and the injection pressure and flow of the fire-extinguishing agent are automatically adjusted; when the fire is extinguished, the robot automatically stops spraying and returns to the standby position.The communication antenna uses a 5G module, which supports high-bandwidth, low-latency wireless communication, and realizes real-time data transmission between the robot and the monitoring center; in the absence of 5G network coverage, the robot can maintain communication with the monitoring center through satellite communication or self-organizing networking; the main control unit uses a high-performance industrial computer, running the ROS robot operating system, to schedule and control the various functional modules of the robot; the display screen displays the robot status information, fire parameters and fire-fighting operation screen in real time, which is convenient for firefighters to observe; the robot supports remote control function, and firefighters can manually control the robot movement and fire-fighting through the remote control to improve flexibility; the communication control module also has data recording and storage functions, which can collect data and log the operation process for subsequent analysis.

[0035] Specifically, the layout of sensor nodes: choose appropriate locations in the target area to deploy smoke sensors and infrared cameras. The smoke sensor uses a photoelectric smoke sensor, which determines the smoke concentration by measuring the light intensity attenuation caused by smoke, with a range of 0.1% / m10% / m. The infrared camera uses an uncooled focal plane infrared thermal imager with a detection band of 814μm and a spatial resolution of 320×240. When deploying sensors, it is necessary to fully consider environmental characteristics and monitoring blind spots, and achieve full coverage monitoring of the area by reasonably setting the sensor position.

[0036] Integration of wireless communication module: A Loar wireless communication module is integrated on each sensor, with an operating frequency of 2.4GHz and a transmission power of 4.5dBm. The communication module is connected to the sensor through a UART interface and is responsible for framing and wirelessly transmitting the data collected by the sensor. It is responsible for establishing and maintaining the entire network. The sensor node data is transmitted to the robot through a repeater.

[0037] Fire data preprocessing: The robot's processor reads the response data of the sensor node from the receiving buffer and determines whether the frame is complete based on the length field in the frame header. Then, the data frame is CRC-checked to determine whether the data has bit errors. If the check passes, the frame payload is unpacked according to the frame type and data type to obtain smoke concentration data or infrared image data. For smoke concentration data, the limited average filtering algorithm is used for filtering. A sliding window is set to save the latest n sampling values. If the new sampling value exceeds the set upper and lower thresholds, it is determined to be an abnormal value and is removed; otherwise, it is added to the sliding window and the average value of the data in the window is updated as the filtering result. For infrared image data, the median filtering algorithm is used to remove speckle noise and salt and pepper noise. The image is traversed pixel by pixel, and the median of the pixel values ​​in the 5×5 neighborhood around the current pixel is used to replace the current pixel value, which can effectively suppress isolated noise points. The filtered smoke concentration data is normalized according to the sensor range to obtain the concentration value in the range of [0, 1]. The statistical features of the concentration data, such as the mean, variance, and peak value, are extracted as the basis for subsequent fire judgment. The infrared image data is grayscale normalized and the original image data is mapped to the range of [0, 255]. Then the image contrast is enhanced through the histogram equalization algorithm to highlight the high temperature area. The characteristic parameters such as the hot spot center position, high temperature area, and average temperature of the image are extracted to prepare for the subsequent fire location and temperature estimation.

[0038] Establish a fire probability model and define a Markov random field: map each sensor node to a node in the Markov random field to form a node set V = {1, 2, ..., n}, where n is the total number of sensor nodes. Each node i and its neighboring node set N i The undirected graph G = (V, E) that constitutes the Markov random field, where E is the edge set of the graph, representing the spatial correlation between nodes. The observation y of each node i i It consists of two parts: smoke concentration s i and infrared image average temperature t i , represented as a two-dimensional feature vector. Fire state variable x i is the hidden variable of node i, which takes the value of 0 or 1, indicating whether a fire occurs at node i. The fire status of all nodes constitutes the hidden variable sequence x of the random field.

[0039] Specifically, in a specific embodiment of the present application, the Markov random field is constructed as follows: there are 100 sensor nodes, uniformly deployed in a 10×10 grid, with a node spacing of 5m; the node set V = {1, 2, ..., 100}, and each node i corresponds to a unique grid coordinate (x i ,y i ); Each node i and its surrounding 8 nodes form a neighborhood set N i , that is: Ni ={j||x i -x j |≤1 and |y i -y j |≤1,j≠i}. The edge set E between nodes contains all adjacent node pairs, with a total of 360 undirected edges, representing the spatial correlation between nodes. The neighborhood of node 1 is N1={2,11,12}.

[0040] The edges connected to node 1 are {(1, 2), (1, 11), (1, 12)}; the undirected graph G = (V, E) of the entire random field contains 100 nodes and 360 edges, describing the spatial topology of the sensor network. Observational features and hidden variables: The observed feature vector y of each node i i =(s i ,t i ), where s i is the smoke concentration, t i is the average temperature of infrared image; s i The value range of is [0, 1], which represents the normalized value of smoke density, 0 represents no smoke, and 1 represents dense smoke; t i The value range is [20, 100], which means the average temperature of the infrared image in degrees Celsius; the hidden variable x i ∈{0,1} indicates whether a fire occurs at node i, 0 indicates normal, and 1 indicates fire; the hidden variable of the entire random field is X=(x1,x2,......,x 100 ), represents the fire status of all nodes. Probabilistic graphical model: According to the Markov hypothesis, the fire status of each node is only related to the status of its neighboring nodes. Given the observed features of all nodes Y = {y i}, fire status X = {x i The joint probability distribution of} can be decomposed into: Among them, Z is the normalization factor; φ(x i ,y i ) The local evidence function of node i, and the observed feature y i Correlation; ψ(x i ,x j ) is the spatial correlation function between nodes i and j, reflecting the state correlation of neighboring nodes. The local evidence function can be defined as: Among them, w1 and w2 are the weight coefficients of smoke concentration and temperature characteristics, which can be set according to prior knowledge. The spatial correlation function can be defined as the Ising model: Among them, β is the spatial correlation strength parameter. When β>0, it means that the neighboring nodes tend to take the same state.

[0041] Model inference: Given the observed feature Y, the posterior probability distribution P(X|Y) of the latent variable X can be inferred by using the Bayesian method. The prior probability P(X) can be assumed to be uniformly distributed, and the likelihood function P(Y|X) can be obtained by the local evidence function φ(x i ,y i ) construction. Due to the complex structure of the Markov random field, the posterior probability P(X|Y) is difficult to solve explicitly, and an approximate inference method is needed. Commonly used inference algorithms include meanfield approximation, loopybeliefpropagation, etc., which realize probability estimation through iterative message passing. Taking the loopyBP algorithm as an example, each node i maintains an edge distribution b i (x i ), indicating that node i is in state x i The probability that, in each iteration, node i receives message m from neighbor node j j→i (x i ), and update its marginal distribution: After several iterations, the marginal distribution b i (x i ) converges, the fire probability of each node i can be obtained. Decision and warning: According to the edge distribution b i (x i ), we can get the posterior probability P(x) of fire occurring at each node i. i =1|Y), set the fire warning threshold δ = 0.7, if P(x i =1|Y)>δ, then it is determined that a fire occurs at node i, otherwise it is considered that the probability of a fire at a certain moment is as follows: node 1: P(x1=1|Y)=0.82, node 2: P(x2=1|Y)=0.35, node 3: P(x3=1|Y)=0.91, ..., and the fire node set A={1,3} can be obtained. The node positions in A are reported to the monitoring center for fire warning.

[0042] Divide the node neighborhood: According to the spatial coordinates (x i ,y i ,z i ), the Delaunay triangulation algorithm is used to triangulate the nodes. After triangulation, the three vertices of each triangle are neighboring nodes, forming the neighborhood set N of node i i . Create an adjacency matrix If node j is a neighbor node of i, then a ij =1, otherwise a ij = 0. The adjacency matrix represents the connection relationship of the nodes in the graph G.

[0043] Define the conditional distribution of observations: For each node i, its observation Comply with fire status x i Gaussian distribution under the condition, the probability density function is: f(y i |x i )=Ν(y i |μ(x i ),Σ(x i )); where μ(x i ) and Σ(x i ) are respectively states x i The mean vector and covariance matrix under can be obtained through the statistics of historical data: in, are the mean and variance of smoke density and temperature when there is no fire; is the corresponding parameter when fire occurs; ρ is the correlation coefficient between smoke concentration and temperature.

[0044] Define the prior distribution of fire status: Assume that the prior probability of fire status of all nodes is the same and obeys the Bernoulli distribution with parameter π: P(x i =1) =π,P(x i =0)=1-π; where π can be set according to the fire risk level of the environment, such as π=0.01 means the probability of fire is 1%. Define the joint distribution of Markov random field: According to the Hammersley-Clifford theorem, the joint distribution of Markov random field P(x, y) can be decomposed into the product form of potential function: Among them, Z is the partition function, φ i (x i ,y i ) is the local potential function of node i, ψ ij (x i ,x j ) is the interaction potential function between nodes i and j. Solve the probability model and transform the local potential function φ i (x i ,y i ) and the interaction potential function ψ ij (x i ,x j ) into the expression of energy function E(x), we get: Where Z is the partition function, which is a normalization constant that makes the sum of the probability distribution P(x) equal to 1. Since Z is independent of x, it can be ignored when solving for the optimal state x*. Therefore, minimizing E(x) is equivalent to maximizing

[0045] Gibbs sampling algorithm: Gibbs sampling generates a sample sequence of Markov random fields by updating the state of each node one by one, thereby approximating the posterior distribution P(x|y). The specific sampling steps are as follows: Randomly initialize the hidden variable state Let the number of iterations t = 0. For the tth iteration, update the state of each node i in turn Compute node i in two states x i =0,x i =1 local energy: Calculate the state transition probability of node i according to the energy value: in, Indicates the state of other nodes except node i at time t. From the Bernoulli distribution Medium sampling generation Update the state of node i. Repeat until the states of all nodes are updated and a complete state sample is obtained. Repeat the preset number of iterations (such as 1000 times) to obtain a set of state samples {X 1 ,X 2 ,......,X T Estimate the edge probability of node i based on the sampled state sequence: Where I(·) is the indicative function and T is the number of iterations. i =0|y)=1-P(x i =1|y).

[0046] Fire area location: According to the posterior probability P(x i =1|y), set the threshold λ∈(0,1), and determine whether a fire occurs at node i: if P(x i =1|y)>λ, then node i is determined as a fire node, otherwise it is determined as a non-fire node. i ,y i ,z i ) is added to the set S, where S is the estimated fire area. By querying the location-address mapping table, the Zigbee short address of each node in the fire node set S is obtained. A fire confirmation frame is sent to the fire node, requiring it to reply with a confirmation response frame. Nodes that do not receive a response are removed from S. Update the node list of the poller, delete the node addresses in S from the regular polling list, and add them to the fire polling list. According to the size of the fire area, the query frequency of the nodes in the fire polling list is dynamically adjusted to achieve key monitoring.

[0047] Particle filter algorithm: Particle initialization: According to the size and shape of the fire area, N particles are randomly scattered in the area. Each particle contains the following information: particle number i: 1≤i≤N; fire location coordinates: P i =[x i ,y i ,z i ]. Prior weights: Node observation data: D i ={d 1i ,d 2i ,......,d ni},in is the smoke concentration and infrared temperature observation value of node j at particle i. Particle weight update: For each particle i, according to its fire position p i , estimate the smoke density s at each node ji and infrared temperature t ji The Gaussian smoke diffusion model and radiation transmission model can be used to estimate the estimated observation data D at particle i. i '. The observed data D will be estimated i ' and the actual observed data D i Compare and calculate the likelihood probability: in, and is the observed noise variance of smoke concentration and temperature. Multiply the likelihood probability by the prior weight to obtain the unnormalized posterior weight of particle i: w i '=P i × i ; Normalize the posterior weights of all particles:

[0048] Particle resampling: According to the normalized posterior weight w i , use the roulette wheel method to resample N new particles from N particles. Particles with larger posterior weights are sampled multiple times, while particles with smaller weights may be eliminated, so that the particle samples are concentrated in areas with higher posterior probabilities. After resampling, set the weights of all particles to 1 / N to obtain the resampled particle set. Iterative update: Repeatedly update the particle set until one of the following stopping conditions is met: the distribution of the particle set converges, that is, the weight of the particle with the largest posterior weight exceeds the threshold (such as 0.9). The number of iterations reaches the preset maximum value (such as 100 times). Fire location: For the converged particle set, the fire location coordinates are weighted averaged according to the posterior weights of the particles to obtain the maximum posterior probability estimate of the fire location: Take p* as the final positioning coordinate of the fire area and pass it to the navigation module of the fire-fighting robot.

[0049] Environmental perception and autonomous positioning and navigation of the fire-fighting robot, laser radar data acquisition and preprocessing: After the fire-fighting robot receives the positioning coordinates [x*, y*, z*] of the fire area, it sets it as the target point p*. Control the robot chassis to rotate so that the direction of the onboard laser radar is aligned with the direction of p*. Start the laser radar to perform a 360° rotation scan with a scanning frequency of f s (e.g. 10Hz), the scanning angle resolution is Δθ (e.g. 0.1°). Each time the laser radar completes a scan, the acquired scanning point set D i ={p1,p2,......,p n} is sent to the point cloud processing module. The point cloud processing module receives the scan point set D i , extract each scan point p j The three-dimensional coordinates [x j ,y j ,z j ] and the reflection intensity value i j , construct the original point cloud data P i :P i ={[x1,y1,z1,i1],[x2,y2,z2,i2],......,[x n ,y n ,z n ,i n ]}; The original point cloud data P obtained from each scan i Accumulate and get the complete environment point cloud data P: Where m is the number of scans, m = 360° / Δθ. Point cloud downsampling: Perform voxel grid downsampling on the original point cloud P to reduce the amount of point cloud data. Divide the three-dimensional space into a cubic grid of size d×d×d, where d is the voxel side length (e.g. 0.1m). Traverse each point p in the point cloud P j , calculate the voxel number [i, j, k] to which it belongs: For each non-empty voxel, the coordinate average of all points in the voxel is calculated as the representative point of the voxel.

[0050] All voxel representative points constitute the downsampled point cloud data Q d :Q d ={q1,q2,......,q l}; where l is the number of non-empty voxels, l≤n. After downsampling, the point cloud density is greatly reduced and the amount of data is reduced. Ground point removal: The RANSAC algorithm is used to remove the downsampled point cloud Q d Perform plane fitting and extract ground point cloud Q g . Randomly select Q d Calculate the plane equation determined by these three points: ax+by+cz+d=0; calculate Q dThe distance from all points to the plane, and the number of points n whose distance is less than the threshold t (such as 0.05m) i Repeat several times (e.g. 100 times) to find the plane with the largest n_i as the ground plane. d Points whose distance from the ground plane is less than t are marked as ground points, forming the ground point cloud Q g From Q d Remove the ground points and get the non-ground point cloud Q: Q = Q d -Q g ; At this point, the collection and preprocessing of LiDAR data are completed, and the non-ground point cloud Q representing environmental obstacles is obtained, which is ready for subsequent positioning and mapping. The preprocessing stage uses a combination of voxel rasterization and RANSAC plane fitting to effectively filter out a large number of redundant ground points while reducing the point cloud density, which not only reduces the amount of data but also highlights the geometric features of obstacles.

[0051] Synchronous positioning and map construction: Initialize the fire-fighting robot's pose (position and orientation) to x0 = [0,0,0,0,0,0] and set it as the origin of the coordinate system. Based on the robot's odometer data, predict the robot's pose x at time t. t The prior estimate x t ', and establish the odometer factor based on the motion model: Among them, u t is the robot control quantity from time t-1 to time t, ⊕ is the operation symbol on the Lie group SE(3), and ||·||Σ represents the Mahalanobis distance. The preprocessed point cloud Q is transformed into the coordinate system. t is the local point cloud map L of the reference system: L t =T(x t )Q, where T(·) is the coordinate transformation matrix on SE(3). Extract L t The corner point and plane point features in the image are used to determine the current position x using a closed-loop detection algorithm based on appearance and geometric information. t Whether the previous position x has been reached j .

[0052] If a closed loop is detected, a closed loop factor is added to the factor graph: Where ΔT is the relative pose change when the loop is closed. Based on the local map L t , extract stable corner point and plane point features, and construct local map factors: Among them, p k For L t The characteristic points in k is the corresponding global map point. Combining the odometer factor, loop closure factor and local map factor, we construct a factor graph optimization problem: Solve the nonlinear least squares problem and obtain the optimal estimate of the robot posture and feature points. According to the optimized robot posture, the local point cloud map L t Transform it to the global coordinate system and add it to the global map to realize online map construction.

[0053] Plan the path according to the A* search algorithm and rasterize the map: convert the global point cloud map into a two-dimensional grid map to facilitate path search. Divide the xy plane into a square grid of size d×d, where d is the grid side length (e.g. 0.2m). Traverse each point p in the global point cloud map i , calculate the grid number [m, n] to which it belongs: Count the number of points in each grid and count the points greater than the threshold t o The grid (such as 5) is marked as occupied grid, and the number of points is less than t f (As shown in 1) are marked as free grids, and the rest are unknown grids. Set the value of the occupied grid to 1, the free grid to 0, and the unknown grid to 0.5 to construct a two-dimensional occupied grid map M.

[0054] A* algorithm search: Take the robot's current position x t The grids corresponding to the target point p are the starting point s and the ending point g. Perform A search on the occupied grid map M. Add the starting point s to the open list OPEN and initialize the cost of s to 0. Loop through the following steps until OPEN is empty or the ending point g is found: Take the node n with the smallest cost from OPEN and add it to the closed list CLOSED. If n is the ending point g, exit the loop and the search is successful. Expand the 8 adjacent nodes of node n {n i}, i = 1, 2, ..., 8. For each adjacent node n i , calculate its cost value f(n i ): f(n i )=g(n i )+h(n i ); where g(n i ) is from the starting point s to n i The actual cost, h(n i ) is from n i The estimated cost to the end point g, here using the Euclidean distance. i In CLOSED, skip; otherwise, if n i If it is not in OPEN, add it to OPEN and set n as its parent node; if n iIn OPEN, compare the original cost value with the newly calculated cost value, take the smaller one, and update its parent node. If the search is successful, start from the end point g, backtrack along the parent node pointer, and generate the optimal path from s to g {p0,p1,......,p k}, where p0=s,p k =g.

[0055] Path optimization and motion control: The global path {p i} is locally optimized to satisfy the robot's kinematic and dynamic constraints. A cubic spline is used to optimize the path nodes {p i} performs parametric interpolation to generate a smooth reference trajectory r(t). Based on the reference trajectory r(t), the trajectory tracking control algorithm is used to calculate the robot's velocity v(t) and acceleration a(t) in real time: Among them, v max and a max is the maximum speed and acceleration of the robot, k v and k a The real-time position of the robot is the tracking error attenuation coefficient, p(t). The calculated speed and acceleration control instructions are sent to the robot chassis to control the robot to move along the reference trajectory. During the movement, the laser radar data collection and positioning map are continuously executed to update the robot posture x t and the global point cloud map. When the robot reaches the target point p* (such as within 0.5m), it stops moving and sends the robot's current position x t Sent to the fire extinguishing control module to request to start the fire extinguishing operation.

[0056] Synchronous localization and map construction algorithm, factor graph model construction: the robot's position at time t is recorded as x t , the local point cloud map is denoted as L t According to the robot's motion model and the position x at time t-1 t-1 , using the extended Kalman filter (EKF) algorithm to t Make predictions and get a priori estimates Will As the LiDAR odometry factor f0(x t-1 ,x t ) is added to the factor graph to represent the robot motion constraints from time t-1 to time t: Among them, u t is the robot control quantity from time t-1 to time t, ⊕ is the operator symbol on the SE(3) group, Denotes the Mahalanobis distance. Extract L t Corner point and plane point features in the local map factor Among them, p k For L t The characteristic points in k is the corresponding feature point in the global map, and T(·) is the coordinate transformation on SE(3). l (x t ,L t ) is added to the factor graph to represent the constraint relationship between the robot pose and the local map.

[0057] Closed loop detection: L t Divide it into cubic voxels of the same size, extract the point with the most feature points in each voxel as the representative point of the voxel, and get L t The feature point set P t . t Each feature point p in i , calculate its geometric descriptor g i and the visual descriptor v i :g i =[x i ,y i ,z i ,α i ,β i ,γ i ];v i =ORB(p i ), where x i ,y i ,z i For p i The three-dimensional coordinates of i ,β i ,γ i For p i The normal vector direction angle, ORB(·) is the ORB feature extraction algorithm. The descriptors of all feature points in P_t are used to construct a bag-of-words model to obtain L t The bag-of-words representation vector w t Calculate w t The bag-of-words representation vector {w j Similarity of}: Select the historical pose node x with the highest similarity greater than the threshold τ m , judging that the robot's position at time t has reached the position at time m, that is, a closed loop is detected. Matching L t and L m The feature points in x are calculated t and x m The relative pose transformation ΔT between mt . Add closed-loop factors to the factor graph Among them, T(x t ) -1is the inverse transformation on SE(3), Σ c is the covariance matrix of the closed-loop error.

[0058] The nonlinear least squares method is used to solve the factor graph optimization problem and construct the least squares problem: all factors are represented as residual blocks r i (x t ,p k ): r l (x t ,p k )=T(x t ) k -q k ; r c (x t ,x m )=T(x t ) -1 T(x m )-ΔT mt ; Take the square norm of the residual block as the objective function and construct the least squares problem: Among them, Σ i is the information matrix of the i-th residual block, which is used to represent the uncertainty of the residual block.

[0059] LM algorithm solution: Initialize the robot pose x t and feature point p k The estimated value of and Calculate the residual vector under the current estimate and the Jacobian matrix Iterate the following steps until the maximum number of iterations is reached or convergence conditions are met: Calculate the incremental equation: (J T Σ -1 J+μI)Δx=-J T Σ -1 r; among them, is the increment to be calculated, μ is the damping factor of the LM algorithm, and I is the unit matrix. Solve the increment equation to obtain Δx. Update the estimated value: in, is the exponential mapping operation on the Lie algebra. Compute the residual under the new estimate According to the change of residual, adjust the size of damping factor μ: if ||r i+1 ||<||r i ||, then reduce μ to make the LM algorithm closer to the Gauss-Newton method; otherwise, increase μ to make the LM algorithm closer to the gradient descent method. i+1||<ε, the algorithm is considered to have converged and the iteration is exited; otherwise, it continues to the next iteration.

[0060] Map update: Update the robot posture obtained in the last iteration As the best estimate. Calculate the robot's position in the global coordinate system Where R(·) and t(·) represent the rotation matrix and translation vector in SE(3) respectively. t ={p i}Transform to the global coordinate system: P t ' is integrated into the global map, and the octree and voxel occupancy of the map are updated. Output the optimized robot pose and the updated global map. By constructing a factor graph optimization problem, integrating multiple constraints into a least squares problem and solving it iteratively with the LM algorithm, the optimization efficiency can be improved while ensuring convergence. The LM algorithm introduces a damping factor and adaptively adjusts between the Gauss-Newton method and the gradient descent method, overcoming the shortcomings of the two methods. It is an effective nonlinear optimization algorithm. In each iteration, the local map is transformed and fused according to the optimized robot posture, realizing real-time update of the map.

[0061] Real-time positioning: Solve the optimal pose estimate of the robot at time t through factor graph optimization Will As the real-time position and posture of the robot in the global coordinate system, including position coordinates (x, y, z) and attitude angles (roll, pitch, yaw). The global position and posture of the robot is published to other modules in real time, such as path planning and control, for their use. Global point cloud map construction: Based on the optimized robot position and posture Construct the transformation matrix T:

[0062]

[0063] For the local point cloud map L t For each point p i =(x',y',z') to transform the coordinates and get the point p in the global coordinate system i '=(x”,y”,z”):p i '=T×p i ; The transformed point cloud {p i'} is stitched into the global point cloud map G. G is downsampled by voxel, and the voxel side length is set to d (such as 0.1m). The point cloud in each d×d×d voxel is merged into one point to reduce the amount of point cloud data. The downsampled point cloud is smoothed using the moving least squares method to remove noise and outliers: for each point p i , select the neighboring point set {q j}. The least squares method is used to fit {q j} local tangent plane, we get p i The normal vector n at i Along the n i Direction P i Project and get the smoothed point p i '. The smoothed global point cloud map G is used as the three-dimensional model of the building.

[0064] Path planning: Project the global point cloud map G onto the xy plane to obtain a two-dimensional occupancy grid map M. Take the robot's current position as the starting point s and the fire area positioning information as the target point g to construct a cost map C: C has the same size and resolution as M, and the initial value is 0. For each grid c in C i :If c i To occupy the grid, then c i =∞; if c i is the starting point s or the target point g, then c i =0; otherwise, C ij =d(c i ,s)+d(c i ,g), where d(·) is the Euclidean distance. On the cost map C, use the A* search algorithm to plan the optimal path from s to g: Initialize the open list OPEN and the closed list CLOSED to be empty. Add s to OPEN and set f(s) = 0. Loop through the following steps until OPEN is empty or g is reached: Take the grid n with the smallest f value from OPEN and add it to CLOSED. If n is g, the planning is successful and the search is terminated. Expand n's 8-neighborhood grid {n_i} and calculate g(n i )=g(n)+d(n,n i ), and h(n i )=d(n i ,g). For each n i :If n i In CLOSED, skip; if n i If it is not in OPEN, add it to OPEN and set f(n i )=g(n i )+h(n i ); if n i In OPEN, and the new g(ni ) is less than the original value, then update f(n i ) and the parent node pointer. If the planning is successful, then start from g and trace back to the parent node pointer to obtain the optimal path from s to g {p0,p1,.......,p k}. i} is converted into the path points in the robot's odometer coordinate system as the navigation target path.

[0065] In the fire-fighting operation in the fire area, the robot controls the movement of the chassis motor according to the navigation path obtained by the path planning module, and moves autonomously to the fire area along the path. During the movement, the robot is positioned in real time through sensors such as odometers and lidar, and compared with the navigation path to calculate the heading deviation and lateral deviation. According to the deviation size, the robot's linear speed and angular velocity are adjusted by PID control or model predictive control to achieve path tracking control. After reaching the fire area, the robot stops moving and enters the fire-fighting operation mode.

[0066] Fire assessment and fire extinguishing strategy selection: Collect multi-source data of the fire area through the onboard flame sensors, infrared thermal imagers, visible light cameras, etc. Fusion process the collected data to extract fire characteristics, such as flame area, temperature distribution, smoke concentration, etc. According to the fire characteristics, assess the severity of the fire and the type of fire (such as Class A, Class B, Class C fire). Combined with environmental conditions (such as space size, ventilation conditions, distribution of flammable materials, etc.), select appropriate fire extinguishing media and fire extinguishing strategies. According to different fire types and severity, preset the type of fire extinguishing medium, injection parameters (such as pressure, flow, injection time, etc.) and fire extinguishing path.

[0067] Autonomous firefighting operation: According to the selected firefighting strategy, the firefighting device is controlled to spray the firefighting medium to the fire point and the fire spread area. The variable frequency speed regulation technology is used to automatically adjust the pressure and flow of the firefighting pump to achieve the best effect of the spraying of the firefighting medium. According to the fire distribution and environmental characteristics, the robot autonomously plans the firefighting path, such as spiral shape, Z shape, etc., to achieve full coverage of the fire scene. During the firefighting process, the flame sensor monitors the residual fire in real time and evaluates the firefighting effect. If residual fire is found, the robot automatically adjusts the firefighting path and spray parameters, and performs additional spraying on the residual fire point until the fire is completely extinguished.

[0068] Collaborative firefighting by multiple robots: At large fire scenes, multiple firefighting robots can be deployed to work together to improve firefighting efficiency. Each robot forms a distributed sensor network through wireless communication modules (such as 5G, Wi-Fi, Loar, etc.) to exchange fire and environmental information. A master control node is set up in the network to summarize the detection data of all robots, generate a global fire map, and distribute it to each robot. According to the global fire map, each robot autonomously allocates firefighting areas and tasks, coordinates actions, and avoids repeated spraying and blind area omissions. During the firefighting process, the robots share the firefighting progress and residual fire information in real time, dynamically optimize the firefighting strategy, and achieve efficient collaboration.

[0069] Selection and use of fire extinguishing media: Choose the appropriate fire extinguishing media for different types of fires: a Class A fire (ordinary solid material fire): mainly use water or water-based fire extinguishing agents. b Class B fire (flammable liquid fire): use foam fire extinguishing agent, dry powder fire extinguishing agent or halogenated fire extinguishing agent (such as heptafluoropropane). c Class C fire (electrical fire): first cut off the power supply, and then use carbon dioxide, halogenated or dry powder fire extinguishing agent. According to the spatial size and structural characteristics of the fire area, choose the appropriate fire extinguishing medium storage and spraying method. In a small space, it is advisable to use fine water mist or aerosol fire extinguishing agents with small volume and short spraying distance. In open areas, large-flow water cannons or foam cannons can be used to achieve long-distance and large-scale spraying.

[0070] Example 2

[0071] A fire detection network and a fire extinguishing robot are deployed in a warehouse area of ​​100m×100m. The specific parameters are as follows: Distributed fire detection network parameters; Number of smoke sensors: 25, distribution spacing: 20m, sampling frequency: 1Hz, range: 0-10% / ft; Number of infrared cameras: 25, distribution spacing: 20m, sampling frequency: 0.5Hz, wavelength: 8-14μm; Zigbee communication frequency: 2.4GHz, transmission power: 20dBm, communication distance: 150m. Example of sensor node location mapping table: Node 1: (-40, -40) <-> Address: 0x8e5f; Node 2: (-20, -40) <-> Address: 0x3a2c; ...; Robot polling period: 10s.

[0072] Fire location is based on multi-source information fusion. The initial fire probability detected at the position (20, 30) is 0.95, which exceeds the Bayesian inference threshold of 0.8, triggering a fire confirmation request. Confirmation requests are sent to the three adjacent nodes (-20, 20), (0, 20), and (20, 20). The smoke concentration observation data returned by the nodes are as follows: Node 1 (-20, 20): 1.6%; Node 2 (0, 20): 2.1%; Node 3 (20, 20): 3.5%. Based on the observation data, the parameters in the Markov random field energy function are estimated: local evidence terms: μ1 = 1.6%, μ2 = 2.1%, μ3 = 3.5%; Σ1 = 0.64, Σ2 = 0.81, Σ3 = 1.44. Spatial correlation term: β = 1.5; d12 = 20, d13 = 40, d23 = 20; construct the following Markov random field energy function: E(x) = Σ{-log[N(f(xi)|μi,Σi)]}-β·Σ{exp[-||xi-xj|| / d]}; where x is the latent variable fire probability, f(·) is the observation equation, and ||·|| is the Euclidean distance. The energy function E(x) is minimized using the particle filter algorithm to estimate the posterior distribution of the fire probability: 500 particles are initialized and randomly distributed in a 40m×40m area around the three nodes; the iteration is repeated 50 times: particle prediction: {x~(i)t} is obtained based on the particle set {x(i)t-1} at the previous moment; particle update: calculate the weight w(i)t=p(yt|x~(i)t) of each particle x~(i)t; resampling: if Neff=1 / Σ(w(i)t)^2<250 , then resample from {x~(i)t} according to the weight w(i)t; output estimate: x(i)t=Σ{w(i)t·x~(i)t}; after iterative solution, the converged particle set distribution is as follows: around node 1 (±5m): 65 → probability 0.13; around node 2 (±5m): 175 → probability 0.35; around node 3 (±5m): 260 → probability 0.52; take the regional center (22.5, 27.5) corresponding to the particle subset with the largest probability as the final fire location estimate.

[0073] SLAM positioning and navigation, lidar data acquisition: The lidar model is Velodyne VLP-16, with 16 lines in the vertical direction and 360° horizontal scanning; the scanning frequency is 40 Hz, that is, 40 frames of point cloud data are generated per second, and each frame of point cloud contains 57,600 measurement points; the angular resolution is 0.25°, then the horizontal scanning angle of each frame is 360° / 0.25°=1440 points; the robot movement speed is 0.5 m / s, and the data acquisition frequency is 2 Hz, that is, 1 frame of data is recorded every 0.5 m; the total length of the motion trajectory is 45 m, then a total of 45 m / (0.5 m / frame)=90 frames of laser data are recorded.

[0074] Point cloud data preprocessing: voxel grid filtering is performed on each frame of point cloud, and the voxel grid leaf size is set to 5 cm; after filtering, the size of each frame of point cloud is reduced to 1 / 8 of the original point cloud, about 7200 points; plane fitting is performed using the RANSAC algorithm, and the point-to-plane distance threshold is set to 10 cm; assuming that 25% of the points in each frame of point cloud are ground points, the ground can be well fitted after 50 iterations; after removing the ground points, each frame of point cloud only retains vertical features, such as walls, pillars, etc., about 5400 points.

[0075] Factor graph construction: add a keyframe node every 5 frames, i.e., 2.5m, and generate 18 keyframes for a 45m trajectory; calculate the relative pose transformation between keyframes through the laser odometer as the observation value of the odometer factor; assume that the measurement noise of the laser odometer is: translation 5cm, rotation 0.02rad, used to construct the covariance matrix; find the closed loop through loop detection in the 5th, 10th, and 15th frames, and form the closed loop factor with the frames 3 moments ago (1, 4, and 7 frames); the observation value of the closed loop factor is calculated by ICP registration, and the distance threshold during registration is set to 20cm, and the maximum number of iterations is 100 times; after ICP registration, the average pose transformation error at the closed loop is about: translation 8cm, rotation 0.05rad. Data example: point cloud (x, y, z) of the first frame: (-2.1, 0.3, 1.2), (-1.8, 0.5, 0.9), ..., (3.7, -2.4, -0.6). Point cloud (x, y, z) of the 5th frame: (1.2, -0.8, 1.5), (0.9, -1.1, 1.8), ..., (4.2, 3.6, -1.1). Odometry factors (dx, dy, dθ) of the 1st to 5th frames: (2.3, 0.1, 0.05), (2.4, -0.2, -0.02), (2.5, 0.3, 0.11), (2.2, 0.0, -0.07). ICP loop closure factor (dx, dy, dθ) of the 5th frame and the 1st frame: (9.8, -1.4, -0.12).

[0076] Loop detection: For the 18 keyframe point clouds, 200 SHOT feature points are extracted from each frame, and each feature point contains a 352-dimensional SHOT descriptor; for example, the SHOT feature point set of the first frame is: {s1_1, s1_2, ..., s1_200}, with a dimension of 200 × 352; K-means clustering is performed on the SHOT feature points of all keyframes to generate 100 visual words; the feature points of each keyframe are quantized into a 100-dimensional bag-of-words vector, which is expressed as the number of feature points belonging to each visual word; for example, the bag-of-words vector of the first frame is The quantity is: {15, 8, 3, ..., 24}, indicating that there are 15 feature points belonging to the first visual word, 8 belonging to the second word, and so on; the cosine similarity of the bag-of-words vectors of the current key frame (such as the 18th frame) and all previous key frames is calculated, and the results are as follows: similarity with the 3rd frame: 0.12; similarity with the 6th frame: 0.35; ...; similarity with the 15th frame: 0.92; It can be found that the current frame has the highest similarity with the 15th frame 3 moments ago, which exceeds the threshold of 0.85, so it is judged that the two form a closed loop.

[0077] Factor graph optimization: Construct a factor graph, which includes 18 keyframe pose nodes, 17 odometry factors, and 3 closed-loop factors (5-1, 10-4, 15-7); the observation value of the odometry factor is the relative pose between frames, and the measurement covariance is the diagonal matrix diag(0.05^2, 0.05^2, 0.02^2); the observation value of the closed-loop factor is obtained by ICP alignment, and the measurement covariance is set to diag(0.08^2, 0.08^2, 0.05^2); taking the first closed-loop factor as an example, the observation value is (-9.85, 1.38, 0.14), which represents the pose transformation of the 5th frame relative to the 1st frame; use g2o for graph optimization, and the optimization variables are 18 keyframe poses, with a total of 6×18=108 degrees of freedom.

[0078] Minimize the error function: E(x) = Σ{(z_odom-h_odom(x))^T·Σ_odom^(-1)·(z_odom-h_odom(x))}+Σ{(z_loop-h_loop(x))^T·Σ_loop^(-1)·(z_loop-h_loop(x))}; where x is the optimization variable, z is the observation value, h is the observation equation, and Σ is the measurement covariance; after iterative optimization, the maximum a posteriori estimate of the pose of each key frame is obtained, and the pose changes before and after optimization are as follows: pose change of the first frame: (-0.08, 0.12, -0.03); pose change of the second frame: (-0.15, 0.21, -0.06); ...; pose change of the 18th frame: (0.32, -0.45, 0.12).

[0079] Global map construction: The laser point clouds of 18 key frames are transformed into the global coordinate system according to the optimized pose; they are spliced ​​into a unified dense point cloud map, with a point cloud level of about 1 million points; the map point cloud is downsampled and voxel grid filtered to obtain a sparse point cloud map with a resolution of 10 cm; the sparse point cloud is projected onto the horizontal plane to generate a two-dimensional occupancy grid map, with an occupancy grid resolution of 10 cm. Each grid in the occupancy grid map is assigned an occupancy probability, for example: grid (25, 18): 0.92; grid (25, 19): 0.45; grid (26, 18): 0.68; grids with an occupancy probability greater than 0.5 are considered obstacles and need to be avoided when the robot navigates.

[0080] Path planning: Assume that the current position of the robot is (1.5, 1.5), the target fire position is (22.5, 27.5); the size of the grid map is 50×50, the resolution is 10cm, and the actual environment size is 5m×5m. The obstacles in the grid map are mainly distributed in the following areas: (10, 20)-(15, 40): rectangular obstacles; (30, 5)-(35, 15): rectangular obstacles; (20, 25)-(25, 40): L-shaped obstacles.

[0081] The A algorithm is used to search the path. The search process is as follows: OpenSet: {(1, 1)}, CloseSet: {}, current node: (1, 1), f=40.79, g=0, h=40.79; expand(1, 1)→{(1, 2), (2, 1), (2, 2)}, add to OpenSet, ..., current node: (22, 27), f=41.09, g=41.09, h=0, find the target point, the search ends, and the total time is 35ms; the optimal path length is 41 steps, corresponding to the actual path length of 41×0.1=4.1m; the optimal path obtained by A search is: (1, 1)-(2, 1)-(2, 2)-...-(22, 26)-(22, 27).

[0082] Path optimization: Connect the 41 path nodes obtained by A* search to form a closed path; multiply the path node coordinates by 0.1 and convert them into a coordinate sequence in the actual environment; use cubic Bezier curves to smooth the path, and set the control points as follows: P0: (0.1, 0.1); P1: (1.2, 0.2); P2: (2.0, 1.8); ..., P40: (2.2, 2.6); P41: (2.2, 2.7). Generate 41 Bezier curves and splice them into a smooth navigation path with a total length of about 45m.

[0083] The smoothed path nodes are: (0.1, 0.1), (0.15, 0.12), (0.22, 0.18), ..., (2.18, 2.68), (2.20, 2.70). Path execution: The smoothed path is discretized into 100 local target points, and the distance between each local target point is about 0.45m; the robot tracks and executes the local target points according to the wheel odometer and PID controller. Assuming that the robot linear velocity is 0.5m / s, the angular velocity is 0.8rad / s, and the PID parameters are as follows: Kp = 1.5, Ki = 0.2, Kd = 0.3; the error statistics of local path tracking are as follows: maximum lateral error: 0.15m; maximum longitudinal error: 0.22m; average lateral error: 0.08m; average longitudinal error: 0.12m. The robot successfully avoids the intermediate obstacles and safely arrives at the target fire area (2.25, 2.75) within 90s.

Claims

1. A front-end fire-fighting robot control method, comprising: Multiple smoke sensors and infrared cameras are set up in the target area, and the smoke sensors and infrared cameras are connected to the fire-fighting robot through wireless communication to form a distributed fire detection network; When the distributed fire detection network detects a fire through the flame recognition algorithm, the location information of the fire area is sent to the fire-fighting robot; After receiving the positioning information of the fire area, the fire-fighting robot uses the laser radar sensor on board to collect environmental information in the target area, adopts the simultaneous positioning and map construction SLAM algorithm, builds a three-dimensional point cloud model of the buildings in the area, and uses the constructed three-dimensional point cloud model for its own positioning and navigation path planning; The fire-fighting robot navigates to the area where the fire occurs according to the planned navigation path and extinguishes the fire; The distributed fire detection network consists of: A plurality of smoke sensors and infrared cameras are arranged at different locations in the target area; wherein the smoke sensors are used to detect smoke concentration, and the infrared cameras are used to collect infrared images; Integrate LoRa or WIFI wireless communication modules on each smoke sensor and infrared camera to bind the location information of each smoke sensor and infrared camera in the system; The system platform receives data from each sensor node and performs fire analysis; When the distributed fire detection network detects a fire through the flame recognition algorithm, it sends the location information of the fire area to the fire-fighting robot, including: The system platform fuses the pre-processed smoke concentration data and infrared image data of each sensor node into multi-source information and uses the Bayesian reasoning method to establish a fire probability model. In the fire probability model, smoke concentration data and infrared image data are used as observations, and the probability of fire occurrence is used as a latent variable. The posterior probability of fire occurrence under given observations is calculated through Bayesian inference. According to the sensor nodes whose posterior probability exceeds the threshold, the area where the fire occurs is determined, and the location information of the sensor nodes in the determined area is used as the preliminary positioning information of the fire area; The preliminary positioning information is transmitted to the fire-fighting robot, which determines the starting and ending points of the walk and the navigation path based on its own position and environmental data; The Bayesian reasoning method is used to establish a fire probability model, including: Each sensor node is defined as a node in the Markov random field, and the observations of the node are the smoke concentration data and infrared image data collected by the corresponding node; According to the spatial distribution of sensor nodes, the set of neighboring nodes of each node is determined, the observation of the node is associated with the observation of the neighboring nodes, and the spatial correlation between the nodes is constructed; Based on the spatial correlation between nodes, a Markov random field model is established, the state of fire occurrence is defined as the hidden variable of the random field, and the observations of all nodes are defined as the observed variables of the random field. In Markov random fields, the energy function is defined as ; Using the minimized energy function , solve the maximum a posteriori probability estimate of fire occurrence in the Markov random field; In Markov random fields, the energy function is defined as : ； Among them, x represents the state of fire occurrence; represents the fire status of the i-th node, y represents the observation value of all nodes, represents the observation value of the i-th node; Represents the local evidence function of the i-th node, measuring the observation Fire status The compatibility is calculated as follows: ； in, and They represent the mean and covariance matrix of the observations of the ith node under the fire state xᵢ; It represents the spatial correlation function between the i-th node and the j-th node, and measures the correlation strength of the fire status of adjacent nodes. The calculation formula is: ； in, Indicates fire status and The distance measure between is the correlation strength parameter; represents the partition function, and the calculation formula is: 。 2. The front-end fire-fighting robot control method according to claim 1, characterized in that: Verify and optimize the preliminary positioning information to obtain the final positioning information of the fire area, including: The particle filter algorithm is used to optimize the preliminary positioning information. According to the location distribution of sensor nodes in the fire area, a group of particle samples are initialized. Each particle contains the fire location, particle prior weight and node observation data. According to the fire location contained in each particle, the smoke concentration distribution and infrared radiation distribution at the corresponding particle are estimated, the estimated distribution at the particle is compared with the observation data collected by the node, and the likelihood probability of the corresponding particle is calculated; the likelihood probability of the particle is multiplied by the corresponding prior weight to obtain the posterior weight of the particle; According to the posterior weight of each particle in the particle sample, the particle sample is resampled, and the particles whose posterior weight is greater than the threshold are retained to obtain the resampled particle sample set; Iteratively perform particle posterior weight calculation and particle resampling on the resampled particle sample set, and update the particle sample set until the distribution of the particle sample set converges or the number of iterations reaches a threshold; The fire positions of each particle in the converged particle sample set are weighted averaged to obtain the maximum posterior probability estimate of the fire position as the final positioning information of the fire area.

3. The front-end fire-fighting robot control method according to claim 2, characterized in that: The SLAM algorithm is used to build a 3D point cloud model of the buildings in the area. The built 3D point cloud model is used for self-positioning and navigation path planning, including: After receiving the positioning information of the fire area, the fire-fighting robot uses the laser radar sensor on board to perform a 360-degree scan of the target area and obtain the original point cloud data of buildings and obstacles in the area; The original point cloud data is downsampled to reduce the data volume, and the random sampling consensus algorithm RANSAC is used to remove ground points to obtain the preprocessed point cloud data; The pre-processed point cloud data is input into the simultaneous positioning and mapping algorithm to obtain the real-time position and posture of the fire-fighting robot in the target area, and a three-dimensional point cloud map of the buildings in the area is established based on the real-time position and posture; The 3D point cloud map is projected onto a 2D plane and represented in the form of a grid map. The current position of the fire-fighting robot is taken as the starting point and the fire area positioning information is taken as the target point. A cost map from the starting point to the target point is constructed. On the cost map, the A* search algorithm is used to plan the navigation path of the fire-fighting robot from the starting point to the target point.

4. The front-end fire-fighting robot control method according to claim 3 is characterized in that: Obtain the real-time position and posture of the fire-fighting robot in the target area, and build a three-dimensional point cloud map of the buildings in the area based on the real-time position and posture, including: Taking the initial position of the fire-fighting robot as the origin, the pre-processed point cloud data is transformed into coordinates according to the real-time posture of the fire-fighting robot to obtain a local point cloud map with the current posture of the robot as the reference system; The local point cloud map and robot pose are input into the simultaneous localization and mapping algorithm, and the factor graph model is constructed using a graph optimization-based method. According to the lidar data and the robot motion model, the extended Kalman filter algorithm is used to predict the robot's posture, obtain a priori estimation of the robot's posture, and add the predicted posture as the lidar odometer factor to the factor graph model; In the factor graph model, the lidar odometer factor, loop detection factor, and local map factor are used as edges in the graph to construct the geometric constraint relationship between the nodes in the graph; By extracting feature points including corner points and plane points from the local point cloud map, a closed-loop detection algorithm based on appearance and spatial geometric information is used to determine whether the robot's current position has reached the previous position; if a closed loop is detected, a closed-loop factor is added to the factor graph model; Based on the constructed factor graph model, a nonlinear least squares optimization problem is constructed to solve the optimal pose and feature point position of each node in the factor graph model; The optimized robot posture is used as the real-time positioning result of the robot. According to the optimized robot posture, the local point cloud map is transformed into coordinates to obtain a three-dimensional point cloud map of the buildings in the area.

5. The front-end fire-fighting robot control method according to claim 4, characterized in that: A closed-loop detection algorithm based on appearance and spatial geometry information is used to determine whether the robot's current position has reached the previous position; If a closed loop is detected, a closed loop factor is added to the factor graph model, corresponding to the current pose node and the closed loop matching pose node, including: The local point cloud map is divided into cubic voxels of the same size, and the feature point set of the local point cloud map is obtained by extracting the points containing the most corner points or plane points in each voxel as the feature points of the corresponding voxel; Calculate the geometric descriptor and visual descriptor of each feature point in the feature point set, build a bag-of-words model based on the geometric descriptor and visual descriptor of the feature point, and obtain the bag-of-words representation vector of the local point cloud map; Match the bag-of-words representation vector of the local point cloud map at the robot's current posture node with the bag-of-words representation vector of the local point cloud map at the historical posture node, and calculate the similarity between the bag-of-words representation vectors; If the similarity is greater than the threshold, it is judged that the current position of the robot has reached the previous position and a closed loop is detected; According to the matching relationship of the local point cloud map, the relative pose transformation between the current pose node and the closed-loop matching pose node is calculated, and a closed-loop factor is added to the factor graph model. The closed-loop factor connects the current pose node and the closed-loop matching pose node to represent the relative pose constraint between the two nodes.

6. A front-end fire-fighting robot control system, characterized in that: include: At least one processing unit PE; used to execute instructions to implement the front-end fire-fighting robot control method according to any one of claims 1 to 5.

Images