A fire-fighting training simulation method and system based on AR glasses

Abstract

The present invention relates to a fire extinguishing training simulation method and system, including: obtaining a target image, the target image is collected by cameras on both sides of AR glasses; inputting the target image into a combustion object recognition model, determining the distance from the simulated combustion object in the target image to the AR glasses; inputting the water level information of the simulated combustion object and the fire extinguishing time corresponding to the water level information into a fire extinguishing model, predicting the fire intensity value of the simulated combustion object; obtaining the fire intensity change and the fire extinguishing progress value according to the fire intensity value and the corresponding fire extinguishing time; generating whether to increase the amount of water sprayed to the simulated combustion object according to the fire intensity value, and generating an instruction to move forward or backward according to the distance; and displaying the fire image and the fire extinguishing guidance plan on the AR glasses. The present invention displays the fire image and the fire extinguishing guidance plan through AR glasses to avoid the occurrence of safety hazards in real fire extinguishing training.

Description

Technical Field

[0001] The present invention relates to the technical field of firefighting training, and in particular to a firefighting training simulation method and system based on AR glasses. Background Art

[0002] At present, the common fire-fighting training method usually uses real fire to simulate fire for training fire-fighting operations. Specifically, combustible materials are placed in gasoline barrels and ignited for fire-fighting training. However, such training fire-fighting operation methods have certain safety hazards. Therefore, a fire-fighting training method that can ensure the personal safety of trainees is needed. Summary of the invention

[0003] The purpose of the present invention is to provide a fire extinguishing training simulation method and system based on AR glasses to solve the potential safety hazards of real fire extinguishing training.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A fire-fighting training simulation method based on AR glasses, comprising:

[0006] Acquire a target image, where the target image is acquired by cameras on both sides of the AR glasses;

[0007] Inputting the target image into a combustion object recognition model, and when the combustion object recognition model recognizes a simulated combustion object, determining a distance between the simulated combustion object in the target image and the AR glasses;

[0008] Acquire water level information of the simulated combustion object and a fire extinguishing time corresponding to the water level information; the water level information represents the amount of water sprayed toward the simulated combustion object, and the fire extinguishing time corresponding to the water level information represents the water spraying time corresponding to the water level information;

[0009] Inputting the water level information and the fire extinguishing time corresponding to the water level information into a fire extinguishing fire intensity model to predict the fire intensity value of the simulated combustion object;

[0010] Generate a simulated burning object fire intensity image according to the fire intensity value; the simulated burning object fire intensity image is displayed on the simulated burning object in the AR glasses to show the virtual fire intensity of the simulated burning object;

[0011] Generate a simulated combustion fire extinguishing guidance plan: generate whether to increase the amount of water sprayed to the simulated combustion according to the fire intensity value, and generate an advance or retreat instruction according to the distance;

[0012] The simulated combustion fire extinguishing guidance program is displayed on the AR glasses.

[0013] Optionally, the combustible object recognition model is an MTCNN model, comprising a SURF feature extractor, a backbone network and a regressor; the SURF feature extractor is used to extract SURF features from the input target image; the backbone network is used to perform global feature extraction on the SURF features; and the regressor is used to identify the simulated combustible object based on the global features.

[0014] Optionally, determining the distance between the simulated combustion object in the target image and the AR glasses specifically includes:

[0015] Extracting the outline of the simulated combustion object in the target image;

[0016] According to the binocular ranging principle and the outline of the simulated burning object, the distance between the simulated burning object and the AR glasses is calculated using the principle of similar triangles.

[0017] Optionally, the extracting the contour of the simulated combustion object in the target image specifically includes: extracting the contour of the simulated combustion object in the target image by using adaptive saliency.

[0018] Optionally, the fire extinguishing model includes a data generator and an LSTM backbone network;

[0019] The data generator is used to calculate the water level change rate according to the water level information and the corresponding fire extinguishing time;

[0020] The LSTM backbone network is used to predict the fire intensity value based on the water level information and the water level change rate.

[0021] Optionally, before inputting the water level information and the time corresponding to the water level information into the fire extinguishing model, the method further includes: training the fire extinguishing model, and the training method is as follows:

[0022] Acquire sample data when extinguishing a burning object; the sample data includes sample water level information and sample extinguishing time corresponding to the sample water level information;

[0023] Obtain the temperature value when extinguishing the burning object;

[0024] Determine a sample fire intensity value according to the temperature value, and use the sample fire intensity value as a label corresponding to the sample data;

[0025] Inputting the sample data into the fire intensity model to obtain a predicted fire intensity value;

[0026] Determine a loss function value according to the predicted fire intensity value and the label corresponding to the sample data;

[0027] Determine whether the loss function value meets the preset requirements. If not, optimize the network parameters in the fire extinguishing model according to the loss function value, and return to the step of "inputting the sample data into the fire extinguishing model to obtain the predicted fire intensity value". If so, stop the iteration to obtain the trained fire extinguishing model.

[0028] Optionally, the loss function value is a mean square error loss function value.

[0029] Optionally, it also includes: displaying a fire extinguishing progress value on the AR glasses; the fire extinguishing progress value is determined by using a weighted average algorithm based on the fire intensity value and the corresponding fire extinguishing time.

[0030] A fire-fighting training simulation system based on AR glasses, including: a gasoline barrel, a water gun, a sensor, a processor, AR glasses, a transmission line and a camera device;

[0031] The gasoline barrel is used to simulate combustion;

[0032] The water gun is used to simulate a fire extinguisher;

[0033] The sensor is located in the gasoline barrel and is used to collect water level information in the gasoline barrel and transmit the water level information to the processor; the water level information represents the amount of water sprayed toward the simulated combustion object, and the fire extinguishing time corresponding to the water level information represents the water spraying time corresponding to the water level information;

[0034] The camera device is located on the left and right sides of the AR glasses, and is used to capture images in real time and send the images to the processor;

[0035] The processor is used to:

[0036] Inputting the received image into the combustion object recognition model, wherein the received image is recorded as a target image;

[0037] When the combustion object recognition model recognizes the simulated combustion object, determining the distance between the simulated combustion object and the AR glasses in the target image;

[0038] Inputting the received water level information and the fire extinguishing time corresponding to the water level information into the fire extinguishing fire intensity model to predict the fire intensity value of the simulated combustion object;

[0039] Generate a simulated burning object fire intensity image according to the fire intensity value; the simulated burning object fire intensity image is set on the simulated burning object in the AR glasses to display the virtual fire intensity of the simulated burning object;

[0040] Generate a simulated combustion fire extinguishing guidance program: generate whether to increase the amount of water sprayed to the simulated combustion according to the fire intensity value, and generate a forward or backward instruction according to the distance; the simulated combustion fire extinguishing guidance program is displayed on the AR glasses;

[0041] The transmission line is used to connect the processor and the AR glasses, and transmit the simulated burning object fire image and the simulated burning object fire extinguishing guidance plan to the AR glasses;

[0042] The AR glasses are used to display the simulated burning object fire image and the simulated burning object fire extinguishing guidance plan.

[0043] According to the specific embodiments provided by the present invention, the following technical effects are disclosed: the fire extinguishing training simulation method and system provided by the present invention include: a target image, the target image is acquired by cameras on both sides of AR glasses; the target image is input into a combustion object recognition model, when the combustion object recognition model recognizes a simulated combustion object, the distance between the simulated combustion object in the target image and the AR glasses is determined; water level information of the simulated combustion object and the fire extinguishing time corresponding to the water level information are obtained; the water level information represents the amount of water sprayed to the simulated combustion object, and the fire extinguishing time corresponding to the water level information represents the water spraying time corresponding to the water level information; the water level information and the fire extinguishing time corresponding to the water level information are input into a fire extinguishing fire intensity model to predict the fire intensity value of the simulated combustion object; a fire intensity image of the simulated combustion object is generated according to the fire intensity value; the fire intensity image of the simulated combustion object is displayed on the simulated combustion object in the AR glasses to show the virtual fire intensity of the simulated combustion object; a fire extinguishing guidance plan for the simulated combustion object is generated: whether to increase the amount of water sprayed to the simulated combustion object is generated according to the fire intensity value, and an instruction to move forward or backward is generated according to the distance; and the fire extinguishing guidance plan for the simulated combustion object is displayed on the AR glasses. The present invention performs fire extinguishing by simulating burning objects instead of fire extinguishing objects, obtains a simulated burning object fire image and a fire extinguishing guidance plan, and displays the simulated burning object fire image and the fire extinguishing guidance plan through AR glasses. It is suitable for fire extinguishing training scenarios without real fire, avoids the occurrence of safety hazards in real fire extinguishing training, and ensures the personal safety of trainees. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flow chart of a fire extinguishing training simulation method based on AR glasses provided in Example 1 of the present invention;

[0046] Figure 2 This is a structural diagram of a combustion object identification model in Example 1 of the present invention;

[0047] Figure 3 This is a structural diagram of a fire extinguishing fire intensity model in Example 1 of the present invention;

[0048] Figure 4 Schematic diagram of binocular ranging principle in Embodiment 1 of the present invention;

[0049] Figure 5 This is a schematic diagram of fire change in Example 1 of the present invention;

[0050] Figure 6 This is a structural diagram of the fire-fighting training simulation system based on AR glasses provided in Example 2 of the present invention.

[0051] Figure numerals: 1-gasoline barrel; 2-water gun; 3-sensor; 4-processor; 5-AR glasses; 6-transmission line; 7-camera equipment. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The purpose of the present invention is to provide a fire extinguishing training simulation method and system based on AR glasses to solve the potential safety hazards of real fire extinguishing training.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1

[0056] This embodiment provides a fire extinguishing training simulation method based on AR glasses, see Figure 1 , the method comprising:

[0057] Step S1: Acquire a target image, where the target image is acquired by cameras on both sides of the AR glasses;

[0058] Step S2: inputting the target image into a combustion object recognition model, and when the combustion object recognition model recognizes a simulated combustion object, determining the distance between the simulated combustion object in the target image and the AR glasses.

[0059] Among them, Figure 2As shown, the combustion object recognition model is an MTCNN model, including a SURF (Speeded Up Robust Feature) feature extractor, a backbone network and a regressor; the SURF feature extractor is used to extract SURF features from the input target image; the backbone network is used to perform global feature extraction on the SURF features; the regressor is used to identify the simulated combustion object based on the global features.

[0060] The determining of the distance between the simulated combustion object in the target image and the AR glasses specifically includes:

[0061] Extracting the outline of the simulated combustion object in the target image;

[0062] According to the binocular ranging principle, coordinates are generated for the outline of the simulated combustion object, and based on the outline of the simulated combustion object, the distance between the simulated combustion object and the AR glasses is calculated using the principle of similar triangles.

[0063] Wherein, adaptive saliency is used to extract the contour of the simulated combustion object in the target image.

[0064] Step S3: obtaining water level information of the simulated combustion object and the fire extinguishing time corresponding to the water level information; the water level information represents the amount of water sprayed to the simulated combustion object, and the fire extinguishing time corresponding to the water level information represents the water spraying time corresponding to the water level information;

[0065] Step S4: inputting the water level information and the fire extinguishing time corresponding to the water level information into the fire extinguishing fire intensity model to predict the fire intensity value of the simulated combustion object.

[0066] Among them, Figure 3 As shown, the fire extinguishing model includes a data generator and an LSTM backbone network; the data generator is used to calculate the water level change rate according to the water level information and the corresponding fire extinguishing time; the LSTM backbone network is used to predict the fire intensity value according to the water level information and the water level change rate.

[0067] Step S5: Generate a simulated combustion object fire intensity image according to the fire intensity value; the simulated combustion object fire intensity image is displayed on the simulated combustion object in the AR glasses to show the virtual fire intensity of the simulated combustion object. In this embodiment, multiple simulated combustion object fire intensity images are generated according to the multiple predicted fire intensity values, and the multiple simulated combustion object fire intensity images are combined into a simulated combustion object fire intensity video and set on the simulated combustion object in the AR glasses to show the fire intensity of the simulated combustion object.

[0068] Step S6: Generate a simulated combustion object fire extinguishing guidance plan: generate whether to increase the amount of water sprayed to the simulated combustion object according to the fire intensity value, and generate an advance or retreat instruction according to the distance.

[0069] Step S7: Displaying the simulated combustion fire extinguishing guidance plan on the AR glasses.

[0070] In this embodiment, before inputting the target image into the combustion object recognition model in step S2, the method further includes: training the combustion object recognition model, and the training method is as follows:

[0071] Obtain a burning object image, where the burning object image is acquired by cameras on both sides of the AR glasses. Annotate the burning object position in the acquired burning object image to obtain a label corresponding to the burning object image. Divide the burning object image and the corresponding label into a training sample set and a test sample set.

[0072] An MTCNN model training framework is established, which includes a data enhancer, a SURF feature extractor, a backbone network and a regressor, wherein the data enhancer is used to enhance and clean image data, and the data enhancer performs data enhancement operations such as denoising, rotation, cropping and channel transformation on the combustion image input into the MTCNN model training framework, and performs data cleaning on the enhanced combustion image; the SURF feature extractor is used to perform SURF transformation of the RGB channels of the cleaned combustion image to extract SURF features and save them as SURF feature images, wherein the SURF feature extractor uses a kernel with a size of 7*7 and a step size of 5*5 to extract SURF features; the backbone network is used to perform deep learning calculation processing on the input SURF feature image to output image global features; and the regressor is used for intersection-over-union calculation.

[0073] The burning object images of the training sample set are sequentially input into the MTCNN model training framework for training. The data enhancer of the MTCNN model training framework performs data enhancement operations such as noise addition, rotation, cropping and channel transformation on the burning object images of the training sample set and performs data cleaning on the enhanced data to obtain a data enhanced image. The data enhanced image is then input into the SURF feature extractor to obtain the image SURF feature, and the image SURF feature is input into the backbone network to obtain the output image global feature, and the global feature is input into the regressor to calculate the gasoline barrel position in the image.

[0074] When all the combustion maps of the training sample set are trained, the data enhancer is discarded, and the SURF feature extractor, backbone network and regressor are saved as the MTCNN model.

[0075] The burning object images of the test set are input into the MTCNN model in sequence. The MTCNN model detects and identifies the positions of the burning objects in the burning object images of the test set and outputs a predicted burning object image with the positions of the burning objects marked with position boxes.

[0076] The predicted position box of the burning object position in the burning object image predicted by the MTCNN model is calculated with the label position box in the corresponding label to determine whether the MTCNN model detects and recognizes the burning object position in the burning object image correctly. The calculation formula of the intersection and union (IOU) is: A represents the predicted location box, A area represents the area of ​​the predicted location box A, B represents the label location box, and B area Represents the area of ​​the label position box B. When the intersection-over-union (IOU) calculation result is greater than 0.5, it is determined that the MTCNN model correctly identifies the location of the combustion object in the combustion object image, otherwise it is determined that the MTCNN model recognizes the location incorrectly.

[0077] When all the burning object images in the test set are input into the MTCNN model and the intersection-over-union ratio (IOU) is calculated and is greater than 0.5, the number of burning object images correctly recognized by the MTCNN model for the test set is obtained.

[0078] The accuracy of the MTCNN model is obtained by dividing the number of burning object images in the test set that are correctly recognized by the MTCNN model by the total number of burning object images in the test set. The MTCNN model with an accuracy greater than or equal to 98% is selected and saved as the burning object recognition model.

[0079] The gasoline barrel image is intercepted according to the position of the burning object in the burning object image, and the gasoline barrel contour in the gasoline barrel image is extracted using adaptive saliency. Figure 4 As shown, according to the binocular ranging principle, coordinates are generated for the outline of the simulated burning object, and based on the outline of the simulated burning object, the distance between the simulated burning object and the AR glasses is calculated using the principle of similar triangles. Among them, the midpoint coordinates of the gasoline barrel image captured by the camera on the left side of the AR glasses are (X1, Y1), and the midpoint coordinates of the gasoline barrel image captured by the camera on the right side of the AR glasses are (X2, Y2). The formula for calculating the distance between the gasoline barrel and the AR glasses using the principle of similar triangles is: Right now Z represents the distance between the gasoline barrel and the AR glasses, f represents the focal length of the camera device, and T represents the center distance between the two cameras.

[0080] Among them, the adaptive saliency is used to extract the gasoline barrel contour in the gasoline barrel image, specifically including:

[0081] The gasoline barrel image is subjected to Gaussian blur processing and a super-pixel image is generated from the processed blurred gasoline barrel image; a saliency image is extracted from the super-pixel image; a binarization threshold is calculated on the saliency image to obtain a binary image; a morphological opening and closing operation is performed on the binary image to remove noise points, thereby obtaining the gasoline barrel outline.

[0082] In this embodiment, before inputting the water level information and the time corresponding to the water level information into the fire extinguishing model in step S4, the method further includes: training the fire extinguishing model, and the training method is as follows:

[0083] Acquire sample data when extinguishing a burning object; the sample data includes sample water level information and sample extinguishing time corresponding to the sample water level information;

[0084] Obtain the temperature value when extinguishing the burning object;

[0085] Determine a sample fire intensity value according to the temperature value, and use the sample fire intensity value as a label corresponding to the sample data;

[0086] Inputting the sample data into the fire intensity model to obtain a predicted fire intensity value;

[0087] Obtaining a loss function value according to the predicted fire intensity value and a label corresponding to the predicted fire intensity value;

[0088] Determine whether the loss function value meets the preset requirements. If not, optimize the network parameters in the fire extinguishing model according to the loss function value, and return to the step of "inputting the sample data into the fire extinguishing model to obtain the predicted fire intensity value". If so, stop the iteration to obtain the trained fire extinguishing model.

[0089] In this embodiment, dry wood is placed in a gasoline barrel as a combustible material to perform a real combustion and fire extinguishing operation. A temperature sensor and a water level sensor are installed in the gasoline barrel. The temperature sensor and the water level sensor collect the temperature value and water level information in the gasoline barrel in real time, and transmit the collected temperature value and water level information to the processor. The processor of this embodiment uses a mobile processor device such as a mobile phone or an edge computing device, and records the fire extinguishing time corresponding to the temperature value and the water level information. Table 1 shows the collected temperature values ​​and the corresponding fire extinguishing time.

[0090] Table 1

[0091] Time (min) 1 2 3 4 5 6 7 8 9 Temperature value (℃) 220 300 350 445 600 980 1050 990 975 Time (min) 10 11 12 13 14 15 16 17 18 Temperature value (℃) 950 610 635 545 460 280 230 385 300 Time (min) 19 20 21 22 23 24 25 26 27 Temperature value (℃) 300 200 295 250 245 250 205 205 245

[0092] A fire extinguishing fire intensity model training framework is established, and the fire extinguishing fire intensity model training framework includes: a data generator, an LSTM backbone network, a linear regressor, and an iterator, wherein the data generator includes a fire intensity change model, and the fire intensity change model is used to generate fire intensity changes.

[0093] The sample fire intensity value is obtained according to the fact that the temperature value of the burning object during the combustion or fire extinguishing process is proportional to the fire intensity value of the burning object. The temperature change is obtained by collecting the temperature value of the burning object through the temperature sensor, that is, the temperature change is obtained by fitting N temperature values ​​using the least squares method. The expression of the temperature change is T(t)=a n t n +a n-1 t n-1 +…+a0, where a represents the polynomial coefficient, n is the polynomial order, and t is time, that is, the fire intensity change S(t)=T(t).

[0094] According to the data in Table 1, we can get Figure 5 The fire change model is as follows:

[0095] T(t)=-4.09×10 -6 t 8 +4.48×10 -4 t 7 -1.93×10 -2 t 6 +0.405t 5 -4.01t 4 +11.18t 3 +68.25t 2 -271.34t+426.16.

[0096] The sample data input into the fire extinguishing model includes water level information and the corresponding fire extinguishing time, and the sample fire intensity value is used as the label corresponding to the sample data. The sample data and the corresponding label are divided into a training set and a test set at a ratio of 3:1.

[0097] The N groups of water level information and corresponding fire extinguishing time of the training set are input into the LSTM fire extinguishing fire model training framework, and the data generator generates the water level change rate and the predicted fire intensity change, wherein the fire intensity change model in the data generator obtains the predicted fire intensity change according to the time, and the data generator calculates the water level change rate according to the water level value and the corresponding fire extinguishing time, and the water level change rate is calculated by dividing the difference of the water level values ​​by the time.

[0098] The water level information, water level change rate, and predicted fire intensity change form a 3*N matrix and are input into the LSTM backbone network, which outputs the predicted fire intensity value.

[0099] The predicted fire intensity value and the corresponding label are input into the linear regressor to calculate the mean square error (MSE) to obtain the mean square error loss value loss.

[0100] The calculation formula of mean square error loss (MSE) is: Where loss is the mean square error loss value, n is the number of training set samples, S' i is the ith predicted fire intensity value, S i is the label corresponding to the i-th predicted fire intensity value;

[0101] The mean square error loss value loss is input into the iterator, and the iterator iteratively trains the LSTM backbone network of the LSTM fire extinguishing fire intensity model to update the network parameters of the LSTM fire extinguishing fire intensity model.

[0102] Input the water level information and corresponding time of all test sets into the LSTM fire extinguishing model training framework after updating the network parameters to predict the fire value and output the predicted fire value. Calculate the mean square error (MSE) between the predicted fire value and the label in the test set. When MSE < 0.1, the error between the predicted fire value output by the LSTM fire extinguishing model training framework and the expected fire value is less than 0.1, and the model training is completed. The linear regressor and iterator are discarded, and the data generator and LSTM backbone network are retained as the LSTM fire extinguishing model; otherwise, return to the step of "inputting the N groups of water level information and corresponding fire extinguishing time of the training set into the LSTM fire extinguishing model". At this point, the trained fire extinguishing model is obtained.

[0103] Through the above steps, a simulated combustion fire image and a simulated combustion fire extinguishing guidance program are obtained, and the trainees wearing AR glasses perform fire extinguishing training operations according to the simulated combustion fire image and simulated combustion fire extinguishing guidance program displayed by the AR glasses. For example, when receiving that the distance between the trainee and the gasoline barrel (the distance between the AR glasses and the simulated combustion object) is reduced, the fire change, fire value and fire extinguishing progress value information of the gasoline barrel are combined in real time to generate a corresponding fire image of the gasoline barrel and generate a fire extinguishing guidance screen, prompting the trainee to perform retreat and corresponding operations, and judging whether the trainee is prompted to retreat according to the distance between the AR glasses and the gasoline barrel before and after the prompt. If the trainee does not retreat, the trainee is repeatedly prompted to perform the retreat operation and an alarm to protect the trainee is sent; if the trainee retreats, the fire change, fire value and fire extinguishing progress value of the gasoline barrel are combined in real time to generate a corresponding fire image of the gasoline barrel and generate a fire extinguishing guidance screen to prompt the trainee to perform the corresponding fire extinguishing training operation until the end of the training.

[0104] The fire-fighting training simulation method based on AR glasses provided by the present invention is suitable for fire-fighting training scenes without real fire, avoiding the safety hazards existing in real fire-fighting training and ensuring the personal safety of trainees. AR glasses are used to display virtual fire images and fire-fighting guidance pictures, which are suitable for trainees with different levels of fire-fighting operation capabilities, and the pre-set fire images can be adjusted according to the trainees with different levels of fire-fighting operation capabilities, effectively helping trainees to consolidate and improve their actual fire-fighting capabilities. The fire-fighting training method provided by the present invention collects the water level information and water level change rate of the gasoline barrel during the fire-fighting training through the water level sensor, which can reflect the hit rate of the trainees operating the water gun on the fire point, thereby knowing the proficiency of the fire-fighting training operation.

[0105] Example 2

[0106] This embodiment provides a fire fighting training simulation system based on AR glasses, see Figure 6 , including: a gasoline barrel 1, a water gun 2, a sensor 3, a processor 4, AR glasses 5, a transmission line 6 and a camera device 7.

[0107] The gasoline barrel 1 is used to simulate combustion;

[0108] The water gun 2 is used to simulate a fire extinguisher;

[0109] The sensor 3 is located in the gasoline barrel 1 and is used to collect water level information in the gasoline barrel 1 and transmit the water level information to the processor 4;

[0110] The camera device 7 is located on the left and right sides of the AR glasses 5 and is used to capture images in real time and send the images to the processor 4;

[0111] The processor 4 is used for:

[0112] Inputting the received image into the combustion object recognition model, wherein the received image is recorded as a target image;

[0113] When the combustion object recognition model recognizes the simulated combustion object, determining the distance between the simulated combustion object and the AR glasses in the target image;

[0114] Inputting the received water level information and the fire extinguishing time corresponding to the water level information into the fire extinguishing fire intensity model to predict the fire intensity value of the simulated combustion object;

[0115] Generate a simulated burning object fire intensity image according to the fire intensity value; the simulated burning object fire intensity image is set on the simulated burning object in the AR glasses to display the virtual fire intensity of the simulated burning object;

[0116] Generate a simulated combustion fire extinguishing guidance program: generate whether to increase the amount of water sprayed to the simulated combustion according to the fire intensity value, and generate a forward or backward instruction according to the distance; the simulated combustion fire extinguishing guidance program is displayed on the AR glasses;

[0117] The transmission line 6 is used to connect the processor 4 and the AR glasses 5, and transmit the simulated burning object fire image and the simulated burning object fire extinguishing guidance plan to the AR glasses;

[0118] The AR glasses 5 are used to display the simulated burning object fire image and the simulated burning object fire extinguishing guidance plan.

[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0120] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A fire-fighting training simulation method based on AR glasses, characterized in that: include: Acquire a target image, where the target image is acquired by cameras on both sides of the AR glasses; Inputting the target image into a combustion object recognition model, and when the combustion object recognition model recognizes a simulated combustion object, determining a distance between the simulated combustion object in the target image and the AR glasses; Acquire water level information of the simulated combustion object and a fire extinguishing time corresponding to the water level information; the water level information represents the amount of water sprayed toward the simulated combustion object, and the fire extinguishing time corresponding to the water level information represents the water spraying time corresponding to the water level information; Inputting the water level information and the fire extinguishing time corresponding to the water level information into a fire extinguishing fire intensity model to predict the fire intensity value of the simulated combustion object; Generate a simulated burning object fire intensity image according to the fire intensity value; the simulated burning object fire intensity image is displayed on the simulated burning object in the AR glasses to show the virtual fire intensity of the simulated burning object; Generate a simulated combustion fire extinguishing guidance plan: generate whether to increase the amount of water sprayed to the simulated combustion according to the fire intensity value, and generate an advance or retreat instruction according to the distance; The simulated combustion fire extinguishing guidance program is displayed on the AR glasses.

2. The method according to claim 1, characterized in that The combustion object recognition model is an MTCNN model, which includes a SURF feature extractor, a backbone network and a regressor; the SURF feature extractor is used to extract SURF features from the input target image; the backbone network is used to perform global feature extraction on the SURF features; and the regressor is used to identify the simulated combustion object based on the global features.

3. The method according to claim 1, characterized in that The determining of the distance between the simulated combustion object in the target image and the AR glasses specifically includes: Extracting the outline of the simulated combustion object in the target image; According to the binocular ranging principle and the outline of the simulated burning object, the distance between the simulated burning object and the AR glasses is calculated using the principle of similar triangles.

4. The method according to claim 3, characterized in that The extracting the contour of the simulated combustion object in the target image specifically includes: Adaptive saliency is used to extract the contour of the simulated combustion object in the target image.

5. The method according to claim 1, characterized in that The fire extinguishing model includes a data generator and an LSTM backbone network; The data generator is used to calculate the water level change rate according to the water level information and the corresponding fire extinguishing time; The LSTM backbone network is used to predict the fire intensity value based on the water level information and the water level change rate.

6. The method according to claim 1 or 5, characterized in that: Before inputting the water level information and the fire extinguishing time corresponding to the water level information into the fire extinguishing fire intensity model, the method further includes: training the fire extinguishing fire intensity model, and the training method is as follows: Acquire sample data when extinguishing a burning object; the sample data includes sample water level information and sample extinguishing time corresponding to the sample water level information; Obtain the temperature value when extinguishing the burning object; Determine a sample fire intensity value according to the temperature value, and use the sample fire intensity value as a label corresponding to the sample data; Inputting the sample data into the fire intensity model to obtain a predicted fire intensity value; Determine a loss function value according to the predicted fire intensity value and the label corresponding to the sample data; Determine whether the loss function value meets the preset requirements. If not, optimize the network parameters in the fire extinguishing model according to the loss function value, and return to the step of "inputting the sample data into the fire extinguishing model to obtain the predicted fire intensity value". If so, stop the iteration to obtain the trained fire extinguishing model.

7. The method according to claim 6, characterized in that The loss function value is a mean square error loss function value.

8. The method according to claim 1, characterized in that Also includes: The fire extinguishing progress value is displayed on the AR glasses; the fire extinguishing progress value is determined by using a weighted average algorithm according to the fire intensity value and the corresponding fire extinguishing time.

9. A fire-fighting training simulation system based on AR glasses, characterized in that: include: Gasoline barrels, water guns, sensors, processors, AR glasses, transmission lines and camera equipment; The gasoline barrel is used to simulate combustion; The water gun is used to simulate a fire extinguisher; The sensor is located in the gasoline barrel and is used to collect water level information in the gasoline barrel and transmit the water level information to the processor; the water level information represents the amount of water sprayed toward the simulated combustion object, and the fire extinguishing time corresponding to the water level information represents the water spraying time corresponding to the water level information; The camera device is located on the left and right sides of the AR glasses, and is used to capture images in real time and send the images to the processor; The processor is used to: Inputting the received image into the combustion object recognition model, wherein the received image is recorded as a target image; When the combustion object recognition model recognizes the simulated combustion object, determining the distance between the simulated combustion object and the AR glasses in the target image; Inputting the received water level information and the fire extinguishing time corresponding to the water level information into the fire extinguishing fire intensity model to predict the fire intensity value of the simulated combustion object; Generate a simulated burning object fire intensity image according to the fire intensity value; the simulated burning object fire intensity image is set on the simulated burning object in the AR glasses to display the virtual fire intensity of the simulated burning object; Generate a simulated combustion fire extinguishing guidance program: generate whether to increase the amount of water sprayed to the simulated combustion according to the fire intensity value, and generate a forward or backward instruction according to the distance; the simulated combustion fire extinguishing guidance program is displayed on the AR glasses; The transmission line is used to connect the processor and the AR glasses, and transmit the simulated burning object fire image and the simulated burning object fire extinguishing guidance plan to the AR glasses; The AR glasses are used to display the simulated burning object fire image and the simulated burning object fire extinguishing guidance plan.

Images