AR fire helmet image recognition and display method and system

Abstract

The present invention relates to the field of image data processing, specifically to an AR fire helmet image recognition and display method and system, which analyzes the environmental state data of the real scene where the AR fire helmet is located, obtains the visible light imaging interference characteristics of the real scene, and generates a visible light invalid imaging area distribution map; determines all untrustworthy image areas of the real scene visible light image, thereby obtaining infrared light shooting navigation information of the real scene, and performs local directional infrared light shooting in the real scene; based on the infrared light image, obtains the object state information of the corresponding spatial range, thereby generating an object morphology image as a basis for correcting the visible light image; fuses the visible light image and the object morphology image to generate a global navigation map of the real scene, and adjusts the display state of the global navigation map based on the wearer's motion characteristics. The present invention can use infrared light images to perform local image correction on visible light images, and display and navigate clear and accurate virtual images of fire scenes.

Description

Technical Field

[0001] The present invention relates to the field of image data processing, and in particular to an AR fire helmet image recognition and display method and system. Background Art

[0002] The environment at the fire scene is complex, and harsh environmental conditions such as thick smoke and high temperature increase the difficulty of rescue for firefighters. Specifically, in order to protect the personal safety of firefighters at the fire scene, firefighters will wear protective clothing and protective helmets. In order to improve the clarity of the firefighters' viewing field at the fire scene, the existing protective helmets will be equipped with goggles with AR display function. The goggles can capture and process the real scene images of the fire scene, thereby providing firefighters with virtual scene images with dynamic instructions and guidance information. Firefighters only need to watch the virtual scene images from the goggles to obtain real-time rescue and walking guidance information, accurately avoid obstacles at the fire scene, and quickly find and locate the rescue object.

[0003] However, the large smoke diffusion range and high temperature conditions at the fire scene will cause the real scene image to be distorted during shooting, resulting in problems such as low visibility and severe distortion of objects in the generated virtual scene image, reducing the global visibility and accuracy of the virtual scene image generated by the goggles. It can be seen that how to use visible light and infrared light shooting technology to perform partition shooting and image data processing of the fire scene according to the environmental conditions of the fire scene, and how to use infrared light images to perform local directional correction of the blurred area of ​​the visible light image, is of great significance to improving the image display clarity and accuracy of AR fire helmets. Summary of the invention

[0004] In order to reduce the impact of environmental factors such as thick smoke and high temperature at the fire scene on the image quality of visible light images of real scenes, the spatial areas in the real scene where visible light imaging is invalid are identified, and infrared light imaging is performed on the spatial areas where visible light imaging is invalid, thereby eliminating the visible light imaging interference caused by environmental factors such as thick smoke and high temperature; the infrared light image is then used to perform local image correction on the visible light image, so as to achieve clear and accurate virtual image display and navigation of the fire scene by AR fire helmets. The present invention provides an AR fire helmet image recognition and display method, which includes the following steps:

[0005] Collect environmental status data of a real scene where an AR fire helmet is located, perform spatial analysis on the environmental status data, and obtain visible light imaging interference characteristics of the real scene; based on the visible light imaging interference characteristics, determine a visible light invalid imaging area distribution map of the real scene;

[0006] Acquire a visible light image of the real scene, and determine all untrustworthy image areas of the visible light image based on the visible light invalid imaging area distribution map; determine infrared light shooting navigation information of the real scene based on the untrustworthy image areas;

[0007] Based on the infrared light shooting navigation information, an infrared light image of a corresponding spatial range in the real scene is acquired; the infrared light image is recognized to obtain object state information of the spatial range; based on the object state information, an object morphology image corresponding to the spatial range is generated;

[0008] Based on the position information of the untrustworthy picture area in the visible light image, the visible light image and the object morphology image are fused to generate a global navigation map of the real scene; based on the movement characteristics of the wearer of the AR fire helmet, the display state of the global navigation map is adjusted.

[0009] Preferably, the environmental status data of the real scene where the AR fire helmet is located is collected, and the spatial domain analysis is performed on the environmental status data to obtain the visible light imaging interference characteristics of the real scene; based on the visible light imaging interference characteristics, a visible light invalid imaging area distribution map of the real scene is determined, specifically:

[0010] Perform air environment particle and temperature detection on the real scene where the AR fire helmet is located to obtain air environment particle concentration data and air environment temperature data of the real scene; perform three-dimensional spatial domain analysis on the air environment particle concentration data and the air environment temperature data on the real scene to obtain visible light transmittance spatial attenuation characteristics and refractive index spatial change gradient characteristics of the air environment of the real scene, which are used as the visible light imaging interference characteristics;

[0011] Based on the spatial attenuation characteristics of the visible light transmittance and the spatial variation gradient characteristics of the refractive index, a visible light imaging quality parameter analysis is performed on the real scene to obtain contrast and resolution information of the visible light imaging picture of the real scene; based on the contrast and resolution information, all visible light invalid imaging areas of the real scene are located, thereby generating a visible light invalid imaging area distribution map of the real scene; wherein the visible light invalid imaging area distribution map represents the spatial distribution positions of all visible light invalid imaging areas of the real scene

[0012] Preferably, the obtaining of the visible light image of the real scene, determining all untrustworthy image areas of the visible light image based on the visible light invalid imaging area distribution map; and determining the infrared light shooting navigation information of the real scene based on the untrustworthy image areas, specifically comprises:

[0013] Perform binocular visible light photography on the real scene to obtain a visible light binocular image of the real scene; generate a visible light three-dimensional image of the real scene based on the binocular parallax of the visible light binocular image; map the visible light invalid imaging area distribution map to the spatial coordinate system where the visible light three-dimensional image is located, determine the mapping screen area of ​​each of all visible light invalid imaging areas represented by the visible light invalid imaging area distribution map in the visible light three-dimensional image, and thereby determine all untrustworthy screen areas of the visible light three-dimensional image;

[0014] Based on the spatial coordinate transformation relationship between the visible light three-dimensional image and the real scene, the mapping space areas of all untrustworthy image areas in the real scene are determined; based on the boundary positions of the mapping space areas, navigation information for infrared light shooting of all mapping space areas in the real scene is determined; wherein the navigation information includes navigation information on changes in shooting azimuth angles during the process of shooting the mapping space areas by the infrared camera of the AR fire helmet.

[0015] Preferably, based on the infrared light shooting navigation information, an infrared light image of a corresponding spatial range in the real scene is obtained; the infrared light image is recognized to obtain object state information of the spatial range; and based on the object state information, an object morphology image corresponding to the spatial range is generated, specifically:

[0016] Based on the infrared light shooting navigation information, the shooting azimuth angle of the infrared light camera of the AR fire helmet is controlled to shoot the corresponding spatial range in the real scene to obtain an infrared light image of the spatial range; pixel contour recognition is performed on the infrared light image to obtain the outer surface contour distribution position information of the real object in the spatial range, which is used as the object state information of the spatial range;

[0017] Based on the distribution position information of the outer surface contour of the real object, the infrared light image is subjected to pixel sharpening processing of the outer surface contour of the real object and background environment blurring processing to generate an object morphology image corresponding to the spatial range.

[0018] Preferably, based on the position information of the untrustworthy screen area in the visible light image, the visible light image and the object morphology image are fused to generate a global navigation map of the real scene; based on the motion characteristics of the wearer of the AR fire helmet, the display state of the global navigation map is adjusted, specifically:

[0019] Based on the regional boundary information of the untrustworthy image area in the global image of the visible light image, determine the position information of the untrustworthy image area in the visible light image; based on the position information, directionally fuse the object morphology image to the corresponding image area in the visible light image; perform resolution adjustment processing on all image areas of the fused object morphology image in the visible light image to generate a global navigation map of the real scene;

[0020] Collecting head movement posture data and movement speed data of the wearer of the AR fire helmet; determining the wearer's field of view azimuth change information based on the head movement posture data; determining the wearer's average movement speed information in the real scene based on the movement speed data;

[0021] Based on the field of view azimuth change information, the display azimuth of the global navigation map in the virtual display space of the AR fire helmet is adjusted; based on the average movement speed information, the dynamic display refresh frequency of the virtual guide graphics in the global navigation map is adjusted.

[0022] On the other hand, the present invention provides an AR fire helmet image recognition and display system, the system comprising the following modules:

[0023] An environmental state detection and analysis module is used to collect environmental state data of the real scene where the AR fire helmet is located, perform spatial analysis on the environmental state data, and obtain visible light imaging interference characteristics of the real scene;

[0024] A visible light invalid imaging area determination module, used to determine a visible light invalid imaging area distribution map of the real scene based on the visible light imaging interference feature;

[0025] An untrustworthy picture area determination module, used to obtain a visible light image of the real scene, and determine all untrustworthy picture areas of the visible light image based on the visible light invalid imaging area distribution map;

[0026] An infrared light photography navigation information determination module, used to determine the infrared light photography navigation information of the real scene based on the untrustworthy image area;

[0027] An infrared light shooting and recognition module, used to obtain an infrared light image of a corresponding spatial range in the real scene based on the infrared light shooting navigation information; and recognize the infrared light image to obtain object state information in the spatial range;

[0028] An object morphology image generation module, used to generate an object morphology image corresponding to the spatial range based on the object state information;

[0029] A global navigation map generation module, configured to generate a global navigation map of the real scene by fusing the visible light image and the object morphology image based on the position information of the untrustworthy picture area in the visible light image;

[0030] The navigation map display adjustment module is used to adjust the display status of the global navigation map based on the movement characteristics of the wearer of the AR fire helmet.

[0031] Preferably, the environmental state detection and analysis module collects environmental state data of the real scene where the AR fire helmet is located, performs spatial analysis on the environmental state data, and obtains the visible light imaging interference characteristics of the real scene, specifically:

[0032] Perform air environment particle and temperature detection on the real scene where the AR fire helmet is located to obtain air environment particle concentration data and air environment temperature data of the real scene; perform three-dimensional spatial domain analysis on the air environment particle concentration data and the air environment temperature data on the real scene to obtain visible light transmittance spatial attenuation characteristics and refractive index spatial change gradient characteristics of the air environment of the real scene, which are used as the visible light imaging interference characteristics;

[0033] The visible light invalid imaging area determination module determines a visible light invalid imaging area distribution map of the real scene based on the visible light imaging interference feature, specifically:

[0034] Based on the spatial attenuation characteristics of the visible light transmittance and the spatial variation gradient characteristics of the refractive index, a visible light imaging quality parameter analysis is performed on the real scene to obtain contrast and resolution information of the visible light imaging picture of the real scene; based on the contrast and resolution information, all visible light invalid imaging areas of the real scene are located, thereby generating a visible light invalid imaging area distribution map of the real scene; wherein the visible light invalid imaging area distribution map represents the spatial distribution positions of all visible light invalid imaging areas of the real scene.

[0035] Preferably, the untrustworthy image area determination module obtains the visible light image of the real scene, and determines all untrustworthy image areas of the visible light image based on the visible light invalid imaging area distribution map, specifically:

[0036] Perform binocular visible light photography on the real scene to obtain a visible light binocular image of the real scene; generate a visible light three-dimensional image of the real scene based on the binocular parallax of the visible light binocular image; map the visible light invalid imaging area distribution map to the spatial coordinate system where the visible light three-dimensional image is located, determine the mapping screen area of ​​each of all visible light invalid imaging areas represented by the visible light invalid imaging area distribution map in the visible light three-dimensional image, and thereby determine all untrustworthy screen areas of the visible light three-dimensional image;

[0037] The infrared light photography navigation information determination module determines the infrared light photography navigation information of the real scene based on the untrustworthy image area, specifically:

[0038] Based on the spatial coordinate transformation relationship between the visible light three-dimensional image and the real scene, the mapping space areas of all untrustworthy image areas in the real scene are determined; based on the boundary positions of the mapping space areas, navigation information for infrared light shooting of all mapping space areas in the real scene is determined; wherein the navigation information includes navigation information on changes in shooting azimuth angles during the process of shooting the mapping space areas by the infrared camera of the AR fire helmet.

[0039] Preferably, the infrared light shooting and recognition module acquires an infrared light image of a corresponding spatial range in the real scene based on the infrared light shooting navigation information; recognizes the infrared light image to obtain object state information in the spatial range, specifically:

[0040] Based on the infrared light shooting navigation information, the shooting azimuth angle of the infrared light camera of the AR fire helmet is controlled to shoot the corresponding spatial range in the real scene to obtain an infrared light image of the spatial range; pixel contour recognition is performed on the infrared light image to obtain the outer surface contour distribution position information of the real object in the spatial range, which is used as the object state information of the spatial range;

[0041] The object morphology image generation module generates an object morphology image corresponding to the spatial range based on the object state information, specifically:

[0042] Based on the distribution position information of the outer surface contour of the real object, the infrared light image is subjected to pixel sharpening processing of the outer surface contour of the real object and background environment blurring processing to generate an object morphology image corresponding to the spatial range.

[0043] Preferably, the global navigation map generation module generates a global navigation map of the real scene by fusing the visible light image and the object morphology image based on the position information of the untrustworthy picture area in the visible light image, specifically:

[0044] Based on the regional boundary information of the untrustworthy image area in the global image of the visible light image, determine the position information of the untrustworthy image area in the visible light image; based on the position information, directionally fuse the object morphology image to the corresponding image area in the visible light image; perform resolution adjustment processing on all image areas of the fused object morphology image in the visible light image to generate a global navigation map of the real scene;

[0045] The navigation map display adjustment module adjusts the display state of the global navigation map based on the motion characteristics of the wearer of the AR fire helmet, specifically:

[0046] Collecting head movement posture data and movement speed data of the wearer of the AR fire helmet; determining the wearer's field of view azimuth change information based on the head movement posture data; determining the wearer's average movement speed information in the real scene based on the movement speed data;

[0047] Based on the field of view azimuth change information, the display azimuth of the global navigation map in the virtual display space of the AR fire helmet is adjusted; based on the average movement speed information, the dynamic display refresh frequency of the virtual guide graphics in the global navigation map is adjusted.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The environmental status data of the real scene where the AR fire helmet is located is collected, and the environmental status data is analyzed in the airspace to obtain the visible light imaging interference characteristics of the real scene; based on the visible light imaging interference characteristics, the visible light invalid imaging area distribution map of the real scene is determined. The AR fire helmet is worn by firefighters during the rescue process at the fire scene. The visible light camera of the AR fire helmet will shoot the real scene of the fire scene, obtain the real scene image and perform AR conversion, so as to provide firefighters with a virtual scene image of the fire scene. Considering the complexity of the environment inside the fire scene, for example, the smoke diffused at the fire scene will block the visible light, causing the camera to be unable to clearly capture the image of the corresponding area, and the high temperature environment at the fire scene will cause the air refractive index to be uneven, resulting in the distortion of the real scene image captured. In this way, the area of ​​uneven air refractive index formed by smoke and high temperature in the fire scene becomes the area where the visible light camera cannot clearly and accurately capture the distribution of objects (i.e., the visible light invalid imaging area), while the area of ​​the fire scene that is not blocked by smoke and has uniform air refractive index can be clearly and accurately captured by the visible light camera. The area of ​​object distribution (i.e., the visible light effective imaging area) In order to accurately locate the distribution of invalid visible light imaging areas in real scenes, the visible light imaging interference caused by environmental factors such as smoke and high temperature at the fire scene is partitioned and identified, and the spatial range that requires additional infrared light shooting at the fire scene is accurately limited.

[0050] Obtain a visible light image of the real scene, and determine all untrusted image areas of the visible light image based on the visible light invalid imaging area distribution map; determine the infrared light shooting navigation information of the real scene based on the untrusted image area. Generally speaking, the virtual image display navigation of the AR fire helmet is based on the visible light image of the fire scene captured by the visible light camera. According to the above analysis, it can be seen that due to environmental factors such as smoke and high temperature in the fire scene, the local picture of the visible light image captured at the fire scene has defects such as blur or distortion. In order to accurately perform infrared image fusion correction on the corresponding local picture in the visible light image, it is necessary to first obtain the spatial area corresponding to the corresponding local picture in the visible light image at the real scene of the fire scene. To this end, the visible light image of the real scene is compared with the aforementioned distribution map of the visible light invalid imaging area to determine all untrusted picture areas in the visible light image (that is, the blurred or distorted picture areas in the visible light image). Then, based on the spatial range of the unreliable image area in the real scene, the infrared light shooting navigation information of the real scene is determined. In this way, the infrared camera of the AR fire helmet only needs to perform infrared shooting of part of the spatial range in the real scene according to the infrared light shooting navigation information, without the need to perform global shooting of the real scene, which effectively reduces the shooting control difficulty of the infrared camera and the workload of subsequent image data processing.

[0051] Based on the infrared light shooting navigation information, the infrared light image of the corresponding spatial range in the real scene is obtained; the infrared light image is identified to obtain the object state information of the spatial range; based on the object state information, the object morphology image corresponding to the spatial range is generated. Infrared light shooting is not affected by environmental factors such as smoke obstruction and high temperature at the fire scene, and can assist in supplementary shooting of the spatial range that cannot be clearly photographed with visible light. According to the infrared light shooting navigation information, the infrared light camera of the AR fire helmet is controlled to shoot some spatial ranges in the fire scene, and the infrared light images of these spatial ranges are identified to obtain the object state information of these spatial ranges, thereby making up for the above-mentioned insufficient shooting defects of visible light shooting. Based on the object state information, the object morphology images corresponding to these spatial ranges are generated, so that the object morphology images fully reflect the information such as the location of the objects in these spatial ranges.

[0052] Based on the position information of the untrusted screen area in the visible light image, the visible light image and the object morphology image are fused to generate a global navigation map of the real scene; based on the motion characteristics of the wearer of the AR fire helmet, the display state of the global navigation map is adjusted. The untrusted screen area in the visible light image is a screen area that needs to be corrected with the help of infrared light shooting technology. In order to completely and accurately fuse and correct the untrusted screen area, the position information of the untrusted screen area in the visible light image is used as a reference, and the visible light image and the object morphology image are fused to generate a global navigation map of the real scene, and the real scene is represented by global image navigation. The global navigation map is converted into a virtual image data stream and displayed through the AR fire helmet, thereby providing global virtual image navigation guidance information for firefighters. Based on the head movement characteristics and body movement speed characteristics of the firefighter wearing the AR fire helmet, the display azimuth of the global navigation map in the virtual display space and the dynamic display state of the virtual guide image in the global navigation map are adjusted, so that the AR fire helmet can match the running state of the firefighter, ensuring that the firefighter can obtain accurate and reliable virtual navigation guidance, and improve the efficiency of fire rescue. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0054] Figure 1 It is a flow chart of an AR fire helmet image recognition and display method provided by the present invention.

[0055] Figure 2It is a structural diagram of an AR fire helmet image recognition and display system provided by the present invention. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention rather than all structures are shown in the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] The terms "including" and "having" and any variations thereof in the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0058] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] See also Figure 1 As shown, the present invention provides an AR fire helmet image recognition and display method, the method comprising the following steps:

[0060] S100, collecting environmental status data of the real scene where the AR fire helmet is located, performing spatial analysis on the environmental status data, and obtaining visible light imaging interference characteristics of the real scene; based on the visible light imaging interference characteristics, determining a visible light invalid imaging area distribution map of the real scene.

[0061] Furthermore, the environmental status data of the real scene where the AR fire helmet is located is collected, and the spatial domain analysis of the environmental status data is performed to obtain the visible light imaging interference characteristics of the real scene; based on the visible light imaging interference characteristics, the visible light invalid imaging area distribution map of the real scene is determined, specifically:

[0062] The air environment particles and temperature of the real scene where the AR fire helmet is located are detected to obtain the air environment particle concentration data and air environment temperature data of the real scene; the air environment particle concentration data and air environment temperature data are analyzed in three-dimensional space about the real scene to obtain the spatial attenuation characteristics of the visible light transmittance and the spatial change gradient characteristics of the refractive index of the real scene air environment, which are used as the visible light imaging interference characteristics;

[0063] Based on the spatial attenuation characteristics of visible light transmittance and the spatial variation gradient characteristics of refractive index, visible light imaging quality parameter analysis is performed on the real scene to obtain the contrast and resolution information of the visible light imaging picture of the real scene; based on the contrast and resolution information, all visible light invalid imaging areas of the real scene are located, so as to generate a visible light invalid imaging area distribution map of the real scene; wherein, the visible light invalid imaging area distribution map represents the spatial distribution positions of all visible light invalid imaging areas of the real scene.

[0064] AR fire helmets are used at fire scenes, and mainly include a helmet body, a camera (including a visible light camera and an infrared camera), an AR display device, and a controller; wherein the helmet body is used to be worn on the head of a firefighter to provide head safety protection and air filtration protection, etc.; the camera, AR display device, and controller are all set on the helmet body; the camera is used to capture real scene image data (visible light image data and infrared light image data) of the fire scene; the controller is used to process the real scene image data and generate virtual image data; the AR display device is used to perform AR display based on the virtual image data, thereby providing firefighters with real-time virtual display images of the fire scene and providing navigation guidance for firefighters' rescue activities at the fire scene. The helmet body, camera, and AR display device in the AR fire helmet are all conventional technical means in this field, and will not be described in detail here.

[0065] The environmental conditions of the real scene of the fire scene are complex and harsh. The real scene space may be full of smoke and in a high temperature state, which seriously affects the visible light camera of the AR fire helmet to shoot the real scene image clearly and faithfully. Specifically, the smoke generated at the fire scene will affect the transmission of visible light, blocking the visible light and causing the visible light camera to be unable to clearly capture the image of the corresponding area in the real scene. The greater the smoke concentration in the corresponding area of ​​the real scene, the greater the blocking effect on the visible light, and the greater the light intensity attenuation rate after the visible light is transmitted through the corresponding area; the high temperature environment at the fire scene will cause the air environment at the fire scene to be heated unevenly, with the temperature of one part of the air environment area being lower; the temperature of another part of the air environment area being lower, making the refractive index of the overall air environment at the fire scene uneven. After the visible light is transmitted through the air environment with uneven refractive index and received by the visible light camera for imaging, the real scene image captured by the visible light camera will be distorted. It can be seen that environmental factors such as air particles (i.e. smoke) and temperature at the fire scene will affect the picture clarity quality of the real scene images taken by the visible light camera, making the areas in the fire scene obscured by smoke and with uneven air refractive index due to high temperature become areas where the visible light camera cannot clearly and accurately capture the distribution of objects (i.e., the visible light invalid imaging area). However, the areas in the fire scene that are not obscured by smoke and have uniform air refractive index can be clearly and accurately captured by the visible light camera.

[0066] In order to accurately locate the distribution of the visible light invalid imaging area in the real scene, the visible light imaging interference caused by environmental factors such as smoke and high temperature at the fire scene is partitioned and identified, and the spatial range of the fire scene that requires additional infrared light shooting is accurately limited. Specifically, in order to accurately distinguish and identify the visible light invalid imaging area and the visible light effective imaging area at the fire scene, the distributed particle sensing equipment and distributed temperature sensing equipment pre-set at the fire scene can be used to detect the air environment particle concentration and temperature value of the fire scene, and the air environment particle concentration data (i.e., the air environment particle concentration distribution data of the entire three-dimensional space at the fire scene) and the air environment temperature data (i.e., the air environment temperature distribution data of the entire three-dimensional space at the fire scene) at the fire scene are obtained, and the air environment particle concentration data and the air environment temperature data will be wirelessly transmitted to the AR fire helmet for further analysis and processing. Then, the air environment particle concentration data and air environment temperature data are analyzed for the three-dimensional spatial spatial changes of the fire scene, and the visible light transmittance spatial attenuation characteristics and refractive index spatial variation gradient characteristics of the air environment at the fire scene are obtained; among which, the visible light transmittance spatial attenuation characteristics refer to the distribution of the light intensity attenuation rate of visible light in the air environment at the fire scene when it is transmitted along different directions; the refractive index spatial variation gradient characteristics refer to the change rate of the refractive index of the air environment at the fire scene in different directions for the corresponding wavelength of visible light (such as 550nm visible light) corresponding to the unit length distance. The visible light transmittance spatial attenuation characteristics and refractive index spatial variation gradient characteristics of the air environment at the fire scene are used as visible light imaging interference characteristics, which is convenient for subsequent judgment of the visible light imaging contrast and resolution of different areas in the fire scene from the two aspects of smoke concentration and air temperature.

[0067] In addition, based on the spatial attenuation characteristics of visible light transmittance and the spatial variation gradient characteristics of refractive index, the global imaging quality parameter analysis of the fire scene is performed to obtain the contrast and resolution information of the global imaging picture of the fire scene. Generally speaking, when the spatial attenuation rate of visible light transmittance is higher, the contrast of the corresponding picture area in the visible light imaging picture is smaller; when the spatial variation gradient of the refractive index is higher, the resolution of the corresponding picture area in the visible light imaging picture is smaller. Based on the contrast and resolution information, the fire scene is divided into a visible light effective imaging area and a visible light invalid imaging area; wherein, if the contrast of a certain area in the fire scene corresponding to the picture area in the visible light imaging picture is less than a preset contrast threshold or the resolution is less than a preset resolution threshold, then a certain area in the fire scene is determined as a visible light invalid imaging area; otherwise, a certain area in the fire scene is determined as a visible light effective imaging area. Then, according to the spatial distribution positions of all visible light invalid imaging areas of the real scene, a visible light invalid imaging area distribution map of the real scene is generated. According to the environmental conditions such as smoke and high temperature at the fire scene, the visible light imaging interference caused by environmental factors such as smoke and high temperature is identified by region, and the spatial range of the fire scene that requires additional infrared light shooting is accurately limited.

[0068] S200, obtaining a visible light image of a real scene, determining all untrustworthy image regions of the visible light image based on a visible light invalid imaging region distribution map; and determining infrared light shooting navigation information of the real scene based on the untrustworthy image regions.

[0069] Further, a visible light image of a real scene is obtained, and based on a visible light invalid imaging area distribution map, all untrustworthy image areas of the visible light image are determined; based on the untrustworthy image areas, infrared light shooting navigation information of the real scene is determined, specifically:

[0070] A real scene is photographed with binocular visible light to obtain a visible light binocular image of the real scene; a visible light three-dimensional image of the real scene is generated based on the binocular parallax of the visible light binocular image; a visible light invalid imaging area distribution map is mapped to a spatial coordinate system where the visible light three-dimensional image is located, and the mapping screen areas of all visible light invalid imaging areas represented by the visible light invalid imaging area distribution map in the visible light three-dimensional image are determined, thereby determining all unreliable screen areas of the visible light three-dimensional image;

[0071] Based on the spatial coordinate transformation relationship between the visible light three-dimensional image and the real scene, the mapping space areas of all unreliable picture areas in the real scene are determined; based on the boundary position of the mapping space area, the navigation information for infrared light shooting of all mapping space areas in the real scene is determined; wherein the navigation information includes the navigation information of the shooting azimuth angle change during the process of the infrared camera of the AR fire helmet shooting the mapping space area.

[0072] Generally speaking, visible light images are more compatible with human vision, so the virtual image display navigation of the AR fire helmet is based on the visible light image of the fire scene captured by the visible light camera. According to the above analysis, due to environmental factors such as smoke and high temperature in the fire scene, the local picture of the visible light image captured at the fire scene has defects such as blur or distortion. In order to accurately perform infrared image fusion correction on the corresponding local picture in the visible light image, it is necessary to first obtain the spatial area corresponding to the real scene of the fire scene of the corresponding local picture in the visible light image. To this end, the fire scene is first photographed with binocular visible light using a visible light camera to obtain a visible light binocular image of the fire scene, and the visible light three-dimensional image of the fire scene is generated by calculating the binocular parallax. Since the visible light invalid imaging area distribution map corresponds to the real space where the fire scene is located, and the visible light three-dimensional image corresponds to the image shooting space corresponding to the visible light camera, and the real space and the image shooting space correspond to different spatial coordinate systems, and the local picture area with defects such as blur or distortion in the visible light image and the visible light invalid imaging area of ​​the fire scene are mutually corresponding and related. The visible light invalid imaging area distribution map is mapped to the spatial coordinate system of the visible light three-dimensional image, and the mapping screen areas of all visible light invalid imaging areas represented by the visible light invalid imaging area distribution map in the visible light three-dimensional image are determined. In this way, the mapped screen area is identified as the most unreliable screen area, providing a basis for subsequent targeted directional infrared light shooting of the fire scene.

[0073] In addition, based on the spatial coordinate transformation relationship between the visible light three-dimensional image and the real scene, all unreliable image areas are inverted to the fire scene respectively, and the mapping space areas of all unreliable image areas corresponding to the real scene at the fire scene are obtained. In this way, the mapping space area belongs to the area where the visible light camera cannot clearly capture the details of the objects, and it is necessary to use the infrared camera to take additional supplementary photos. Then, based on the boundary position of the mapping space area, the navigation information for infrared light shooting of all mapping space areas in the fire scene is determined. In this way, the infrared camera of the AR fire helmet only needs to change its own shooting azimuth according to the infrared light shooting navigation information, and can perform infrared shooting of part of the spatial range in the fire scene, without the need to take a global shot of the real scene fire scene, which effectively reduces the shooting control difficulty of the infrared camera and the workload of subsequent image data processing.

[0074] S300, based on the infrared light shooting navigation information, obtaining an infrared light image of a corresponding spatial range in the real scene; recognizing the infrared light image to obtain object state information of the spatial range; and generating an object morphology image corresponding to the spatial range based on the object state information.

[0075] Further, based on the infrared light shooting navigation information, an infrared light image of a corresponding spatial range in the real scene is obtained; the infrared light image is recognized to obtain object state information of the spatial range; based on the object state information, an object morphology image corresponding to the spatial range is generated, specifically:

[0076] Based on the infrared light shooting navigation information, the shooting azimuth angle of the infrared light camera of the AR fire helmet is controlled to shoot the corresponding spatial range in the real scene to obtain the infrared light image of the spatial range; the infrared light image is subjected to pixel contour recognition to obtain the outer surface contour distribution position information of the real object in the spatial range, which is used as the object state information in the spatial range;

[0077] Based on the distribution position information of the outer surface contour of the real object, the infrared image is subjected to the pixel sharpening processing of the outer surface contour of the real object and the blurring processing of the background environment to generate an object morphology image corresponding to the spatial range.

[0078] Infrared light shooting is not affected by environmental factors such as smoke and high temperature at the fire scene, and can assist in supplementary shooting of the spatial range that cannot be clearly photographed with visible light. According to the infrared light shooting navigation information, the infrared light camera of the AR fire helmet is controlled to change the shooting azimuth to shoot some spatial ranges in the fire scene, and the infrared light images of these spatial ranges are pixel-recognized to obtain the distribution position information of the outer surface contours of the real objects in these spatial ranges, which is used as the object state information of the spatial range to make up for the insufficient shooting defects of visible light shooting. Based on the distribution position information of the outer surface contour of the real object, the infrared light image is subjected to the pixel sharpening processing of the outer surface contour of the real object and the background environment blurring processing to generate the object morphology image corresponding to the spatial range. Among them, the pixel sharpening processing of the outer surface contour of the real object and the background environment blurring processing belong to the conventional technical means in this field, which will not be introduced in detail here; after the object morphology image is subjected to the contour pixel sharpening processing and the background environment blurring processing, it can visually highlight the outer surface contour of the real object, and improve the visual discernibility of the corresponding real object in the virtual image display space of the AR fire helmet.

[0079] S400, based on the position information of the unreliable picture area in the visible light image, fuses the visible light image and the object morphology image to generate a global navigation map of the real scene; based on the movement characteristics of the wearer of the AR fire helmet, adjusts the display status of the global navigation map.

[0080] Furthermore, based on the position information of the untrustworthy picture area in the visible light image, the visible light image and the object morphology image are fused to generate a global navigation map of the real scene; based on the motion characteristics of the wearer of the AR fire helmet, the display state of the global navigation map is adjusted, specifically:

[0081] Based on the regional boundary information of the untrusted image area in the global image of the visible light image, the position information of the untrusted image area in the visible light image is determined; based on the position information, the object morphology image is directionally fused to the corresponding image area in the visible light image; the resolution of all the image areas of the fused object morphology images in the visible light image are adjusted to generate a global navigation map of the real scene;

[0082] Collect the head movement posture data and movement speed data of the wearer of the AR fire helmet; determine the wearer's field of view azimuth change information based on the head movement posture data; determine the wearer's average movement speed information in the real scene based on the movement speed data;

[0083] Based on the field of view azimuth change information, the display azimuth of the global navigation map in the virtual display space of the AR fire helmet is adjusted; based on the average motion speed information, the dynamic display refresh frequency of the virtual guide graphics in the global navigation map is adjusted.

[0084] The untrusted image area in the visible light image is the image area that needs to be corrected with the help of infrared light shooting technology. In order to completely and accurately fuse and correct the untrusted image area, the object shape image is accurately fused to the corresponding untrusted image area based on the position information of the untrusted image area in the visible light image. Then, the resolution of all the image areas in the visible light image that have been fused with the object shape image is adjusted, so that the image area in the visible light image that has been fused with the object shape image has the same resolution as other adjacent image areas, and a global navigation map of the real scene is generated. The real scene is represented by global image navigation, and the global navigation map is converted into a virtual image data stream through an AR fire helmet and displayed, thereby providing firefighters with global virtual image navigation guidance information.

[0085] In addition, firefighters wearing AR fire helmets will tilt and swing their heads during the rescue process. At this time, the viewing field of the firefighters' eyes in the virtual display space of the AR fire helmet will also change accordingly. In order to prevent the global navigation map from being out of the viewing field of the firefighters during the display of the virtual display space of the AR fire helmet, the head movement posture data of the firefighters is collected and analyzed to determine the change information of the wearer's field of view azimuth, so as to adjust the display azimuth of the global navigation map in the virtual display space of the AR fire helmet, and ensure that the virtual display space of the AR fire helmet can always be aligned with the viewing field of the firefighters. In addition, firefighters need to walk along different paths when rescuing at the fire scene. In order to guide firefighters to accurately avoid obstacles and walk in the right direction, the virtual reality image displayed by the AR fire helmet to the firefighters is to add corresponding dynamic virtual guide graphics (such as guide arrow graphics) on the basis of the global navigation map. In order to make the dynamic display state of the virtual guide graphics consistent with the movement state of the firefighters, the speed sensor installed on the AR fire helmet is used to detect the movement speed data of the firefighters at the fire scene, and based on the movement speed data, the average movement speed information of the firefighters at the fire scene is determined to adjust the dynamic display refresh frequency of the virtual guide graphics in the global navigation map; generally speaking, the greater the average movement speed, the higher the dynamic display refresh frequency of the virtual guide graphics in the global navigation map, ensuring that firefighters can obtain accurate and reliable virtual navigation guidance.

[0086] See also Figure 2 As shown, the present invention provides an AR fire helmet image recognition and display system, which includes the following modules:

[0087] The environmental state detection and analysis module is used to collect the environmental state data of the real scene where the AR fire helmet is located, perform spatial analysis on the environmental state data, and obtain the visible light imaging interference characteristics of the real scene;

[0088] A visible light invalid imaging area determination module, used to determine a visible light invalid imaging area distribution map of a real scene based on visible light imaging interference characteristics;

[0089] An untrustworthy image region determination module is used to obtain a visible light image of a real scene and determine all untrustworthy image regions of the visible light image based on a visible light invalid imaging region distribution map;

[0090] An infrared light photography navigation information determination module, used to determine infrared light photography navigation information of a real scene based on an untrustworthy image area;

[0091] The infrared light shooting and recognition module is used to obtain the infrared light image of the corresponding spatial range in the real scene based on the infrared light shooting navigation information; recognize the infrared light image to obtain the state information of the object in the spatial range;

[0092] An object morphology image generation module is used to generate an object morphology image corresponding to a spatial range based on object state information;

[0093] A global navigation map generation module is used to generate a global navigation map of a real scene based on the position information of the untrusted picture area in the visible light image and the fusion of the visible light image and the object morphology image;

[0094] The navigation map display adjustment module is used to adjust the display status of the global navigation map based on the motion characteristics of the wearer of the AR fire helmet.

[0095] Furthermore, the environmental state detection and analysis module collects the environmental state data of the real scene where the AR fire helmet is located, performs spatial analysis on the environmental state data, and obtains the visible light imaging interference characteristics of the real scene, specifically:

[0096] The air environment particles and temperature of the real scene where the AR fire helmet is located are detected to obtain the air environment particle concentration data and air environment temperature data of the real scene; the air environment particle concentration data and air environment temperature data are analyzed in three-dimensional space about the real scene to obtain the spatial attenuation characteristics of the visible light transmittance and the spatial change gradient characteristics of the refractive index of the real scene air environment, which are used as the visible light imaging interference characteristics;

[0097] The visible light invalid imaging area determination module determines the visible light invalid imaging area distribution map of the real scene based on the visible light imaging interference characteristics, specifically:

[0098] Based on the spatial attenuation characteristics of visible light transmittance and the spatial variation gradient characteristics of refractive index, visible light imaging quality parameter analysis is performed on the real scene to obtain the contrast and resolution information of the visible light imaging picture of the real scene; based on the contrast and resolution information, all visible light invalid imaging areas of the real scene are located, so as to generate a visible light invalid imaging area distribution map of the real scene; wherein, the visible light invalid imaging area distribution map represents the spatial distribution positions of all visible light invalid imaging areas of the real scene.

[0099] Furthermore, the untrusted image area determination module obtains a visible light image of a real scene, and determines all untrusted image areas of the visible light image based on a visible light invalid imaging area distribution map, specifically:

[0100] A real scene is photographed with binocular visible light to obtain a visible light binocular image of the real scene; a visible light three-dimensional image of the real scene is generated based on the binocular parallax of the visible light binocular image; a visible light invalid imaging area distribution map is mapped to a spatial coordinate system where the visible light three-dimensional image is located, and the mapping screen areas of all visible light invalid imaging areas represented by the visible light invalid imaging area distribution map in the visible light three-dimensional image are determined, thereby determining all unreliable screen areas of the visible light three-dimensional image;

[0101] The infrared light photography navigation information determination module determines the infrared light photography navigation information of the real scene based on the untrusted image area, specifically:

[0102] Based on the spatial coordinate transformation relationship between the visible light three-dimensional image and the real scene, the mapping space areas of all unreliable picture areas in the real scene are determined; based on the boundary position of the mapping space area, the navigation information for infrared light shooting of all mapping space areas in the real scene is determined; wherein the navigation information includes the navigation information of the shooting azimuth angle change during the process of the infrared camera of the AR fire helmet shooting the mapping space area.

[0103] Furthermore, the infrared light shooting and recognition module obtains the infrared light image of the corresponding spatial range in the real scene based on the infrared light shooting navigation information; recognizes the infrared light image to obtain the object state information of the spatial range, specifically:

[0104] Based on the infrared light shooting navigation information, the shooting azimuth angle of the infrared light camera of the AR fire helmet is controlled to shoot the corresponding spatial range in the real scene to obtain the infrared light image of the spatial range; the infrared light image is subjected to pixel contour recognition to obtain the outer surface contour distribution position information of the real object in the spatial range, which is used as the object state information in the spatial range;

[0105] The object morphology image generation module generates the object morphology image corresponding to the spatial range based on the object state information, specifically:

[0106] Based on the distribution position information of the outer surface contour of the real object, the infrared image is subjected to the pixel sharpening processing of the outer surface contour of the real object and the blurring processing of the background environment to generate an object morphology image corresponding to the spatial range.

[0107] Furthermore, the global navigation map generation module generates a global navigation map of the real scene based on the position information of the untrustworthy picture area in the visible light image by fusing the visible light image and the object morphology image, specifically:

[0108] Based on the regional boundary information of the untrusted image area in the global image of the visible light image, the position information of the untrusted image area in the visible light image is determined; based on the position information, the object morphology image is directionally fused to the corresponding image area in the visible light image; the resolution of all the image areas of the fused object morphology images in the visible light image are adjusted to generate a global navigation map of the real scene;

[0109] The navigation map display adjustment module adjusts the display status of the global navigation map based on the motion characteristics of the wearer of the AR fire helmet, specifically:

[0110] Collect the head movement posture data and movement speed data of the wearer of the AR fire helmet; determine the wearer's field of view azimuth change information based on the head movement posture data; determine the wearer's average movement speed information in the real scene based on the movement speed data;

[0111] Based on the field of view azimuth change information, the display azimuth of the global navigation map in the virtual display space of the AR fire helmet is adjusted; based on the average motion speed information, the dynamic display refresh frequency of the virtual guide graphics in the global navigation map is adjusted.

[0112] The operation and effect of the AR fire helmet image recognition and display system of the present invention are corresponding to and consistent with the above-mentioned AR fire helmet image recognition and display method, and the AR fire helmet image recognition and display system will not be repeated here.

[0113] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AR fire helmet image recognition and display method, characterized in that: The method comprises the following steps: Collect environmental status data of the real scene where the AR fire helmet is located, perform spatial analysis on the environmental status data, and obtain visible light imaging interference characteristics of the real scene; based on the visible light imaging interference characteristics, determine a visible light invalid imaging area distribution map of the real scene, specifically: Perform air environment particle and temperature detection on the real scene where the AR fire helmet is located to obtain air environment particle concentration data and air environment temperature data of the real scene; perform three-dimensional spatial domain analysis on the air environment particle concentration data and the air environment temperature data on the real scene to obtain visible light transmittance spatial attenuation characteristics and refractive index spatial change gradient characteristics of the air environment of the real scene, which are used as the visible light imaging interference characteristics; Based on the spatial attenuation characteristics of the visible light transmittance and the spatial variation gradient characteristics of the refractive index, a visible light imaging quality parameter analysis is performed on the real scene to obtain contrast and resolution information of the visible light imaging picture of the real scene; based on the contrast and resolution information, all visible light invalid imaging areas of the real scene are located, thereby generating a visible light invalid imaging area distribution map of the real scene; wherein the visible light invalid imaging area distribution map represents the spatial distribution positions of all visible light invalid imaging areas of the real scene; Acquire a visible light image of the real scene, and determine all untrustworthy image areas of the visible light image based on the visible light invalid imaging area distribution map; determine infrared light shooting navigation information of the real scene based on the untrustworthy image areas; Based on the infrared light shooting navigation information, an infrared light image of a corresponding spatial range in the real scene is acquired; the infrared light image is recognized to obtain object state information of the spatial range; based on the object state information, an object morphology image corresponding to the spatial range is generated; Based on the position information of the untrustworthy picture area in the visible light image, the visible light image and the object morphology image are fused to generate a global navigation map of the real scene; based on the movement characteristics of the wearer of the AR fire helmet, the display state of the global navigation map is adjusted.

2. The method according to claim 1, characterized in that: The obtaining of the visible light image of the real scene, determining all untrustworthy image areas of the visible light image based on the visible light invalid imaging area distribution map; and determining the infrared light shooting navigation information of the real scene based on the untrustworthy image areas, specifically comprises: Perform binocular visible light photography on the real scene to obtain a visible light binocular image of the real scene; generate a visible light three-dimensional image of the real scene based on the binocular parallax of the visible light binocular image; map the visible light invalid imaging area distribution map to the spatial coordinate system where the visible light three-dimensional image is located, determine the mapping screen area of ​​each of all visible light invalid imaging areas represented by the visible light invalid imaging area distribution map in the visible light three-dimensional image, and thereby determine all untrustworthy screen areas of the visible light three-dimensional image; Based on the spatial coordinate transformation relationship between the visible light three-dimensional image and the real scene, determining the mapping spatial regions of all untrustworthy image regions in the real scene; Based on the boundary position of the mapping space area, navigation information for infrared light shooting of all the mapping space areas in the real scene is determined; wherein the navigation information includes navigation information of changes in shooting azimuth angle during the process of the infrared camera of the AR fire helmet shooting the mapping space area.

3. The method according to claim 1, characterized in that The method of acquiring an infrared light image of a corresponding spatial range in the real scene based on the infrared light shooting navigation information; recognizing the infrared light image to obtain object state information of the spatial range; and generating an object morphology image corresponding to the spatial range based on the object state information is specifically as follows: Based on the infrared light shooting navigation information, the shooting azimuth angle of the infrared light camera of the AR fire helmet is controlled to shoot the corresponding spatial range in the real scene to obtain an infrared light image of the spatial range; pixel contour recognition is performed on the infrared light image to obtain the outer surface contour distribution position information of the real object in the spatial range, which is used as the object state information of the spatial range; Based on the distribution position information of the outer surface contour of the real object, the infrared light image is subjected to pixel sharpening processing of the outer surface contour of the real object and background environment blurring processing to generate an object morphology image corresponding to the spatial range.

4. The method according to claim 1, characterized in that The method comprises fusing the visible light image and the object morphology image based on the position information of the untrustworthy picture area in the visible light image to generate a global navigation map of the real scene; Based on the motion characteristics of the wearer of the AR fire helmet, the display state of the global navigation map is adjusted, specifically: Determining position information of the untrustworthy picture area in the visible light image based on area boundary information of the untrustworthy picture area in the global picture of the visible light image; Based on the position information, directionally fuse the object morphology image to a corresponding picture area in the visible light image; Performing resolution adjustment processing on the picture areas of all fused object morphology images in the visible light image to generate a global navigation map of the real scene; Collecting head movement posture data and movement speed data of the wearer of the AR fire helmet; and determining the wearer's field of view azimuth change information based on the head movement posture data; Based on the motion speed data, determining average motion speed information of the wearer in the real scene; Based on the field of view azimuth change information, adjusting the display azimuth of the global navigation map in the virtual display space of the AR fire helmet; Based on the average motion speed information, the dynamic display refresh frequency of the virtual guide graphic in the global navigation map is adjusted.

5. An AR fire helmet image recognition and display system, characterized in that: The system includes the following modules: The environmental state detection and analysis module is used to collect environmental state data of the real scene where the AR fire helmet is located, perform spatial analysis on the environmental state data, and obtain the visible light imaging interference characteristics of the real scene, specifically: Perform air environment particle and temperature detection on the real scene where the AR fire helmet is located to obtain air environment particle concentration data and air environment temperature data of the real scene; perform three-dimensional spatial domain analysis on the air environment particle concentration data and the air environment temperature data on the real scene to obtain visible light transmittance spatial attenuation characteristics and refractive index spatial change gradient characteristics of the air environment of the real scene, which are used as the visible light imaging interference characteristics; The visible light invalid imaging area determination module is used to determine the visible light invalid imaging area distribution map of the real scene based on the visible light imaging interference characteristics, specifically: Based on the spatial attenuation characteristics of the visible light transmittance and the spatial variation gradient characteristics of the refractive index, a visible light imaging quality parameter analysis is performed on the real scene to obtain contrast and resolution information of the visible light imaging picture of the real scene; based on the contrast and resolution information, all visible light invalid imaging areas of the real scene are located, thereby generating a visible light invalid imaging area distribution map of the real scene; wherein the visible light invalid imaging area distribution map represents the spatial distribution positions of all visible light invalid imaging areas of the real scene; An untrustworthy picture area determination module, used to obtain a visible light image of the real scene, and determine all untrustworthy picture areas of the visible light image based on the visible light invalid imaging area distribution map; An infrared light photography navigation information determination module, used to determine the infrared light photography navigation information of the real scene based on the untrustworthy image area; An infrared light shooting and recognition module, used to obtain an infrared light image of a corresponding spatial range in the real scene based on the infrared light shooting navigation information; and recognize the infrared light image to obtain state information of objects in the spatial range; An object morphology image generation module, used to generate an object morphology image corresponding to the spatial range based on the object state information; A global navigation map generation module, configured to generate a global navigation map of the real scene by fusing the visible light image and the object morphology image based on the position information of the untrustworthy picture area in the visible light image; The navigation map display adjustment module is used to adjust the display status of the global navigation map based on the movement characteristics of the wearer of the AR fire helmet.

6. The system according to claim 5, characterized in that The untrustworthy image area determination module obtains the visible light image of the real scene, and determines all untrustworthy image areas of the visible light image based on the visible light invalid imaging area distribution map, specifically: Perform binocular visible light photography on the real scene to obtain a visible light binocular image of the real scene; generate a visible light three-dimensional image of the real scene based on the binocular parallax of the visible light binocular image; map the visible light invalid imaging area distribution map to the spatial coordinate system where the visible light three-dimensional image is located, determine the mapping screen area of ​​each of all visible light invalid imaging areas represented by the visible light invalid imaging area distribution map in the visible light three-dimensional image, and thereby determine all untrustworthy screen areas of the visible light three-dimensional image; The infrared light photography navigation information determination module determines the infrared light photography navigation information of the real scene based on the untrustworthy image area, specifically: Based on the spatial coordinate transformation relationship between the visible light three-dimensional image and the real scene, the mapping space areas of all untrustworthy image areas in the real scene are determined; based on the boundary positions of the mapping space areas, navigation information for infrared light shooting of all mapping space areas in the real scene is determined; wherein the navigation information includes navigation information on changes in shooting azimuth angles during the process of shooting the mapping space areas by the infrared camera of the AR fire helmet.

7. The system according to claim 5, characterized in that The infrared light shooting and recognition module obtains an infrared light image of a corresponding spatial range in the real scene based on the infrared light shooting navigation information; recognizes the infrared light image to obtain object state information in the spatial range, specifically: Based on the infrared light shooting navigation information, the shooting azimuth angle of the infrared light camera of the AR fire helmet is controlled to shoot the corresponding spatial range in the real scene to obtain an infrared light image of the spatial range; pixel contour recognition is performed on the infrared light image to obtain the outer surface contour distribution position information of the real object in the spatial range, which is used as the object state information of the spatial range; The object morphology image generation module generates an object morphology image corresponding to the spatial range based on the object state information, specifically: Based on the distribution position information of the outer surface contour of the real object, the infrared light image is subjected to pixel sharpening processing of the outer surface contour of the real object and background environment blurring processing to generate an object morphology image corresponding to the spatial range.

8. The system according to claim 5, characterized in that The global navigation map generation module generates a global navigation map of the real scene by fusing the visible light image and the object morphology image based on the position information of the untrustworthy picture area in the visible light image, specifically: Determining position information of the untrustworthy picture area in the visible light image based on area boundary information of the untrustworthy picture area in the global picture of the visible light image; Based on the position information, directionally fuse the object morphology image to a corresponding picture area in the visible light image; Performing resolution adjustment processing on the picture areas of all fused object morphology images in the visible light image to generate a global navigation map of the real scene; The navigation map display adjustment module adjusts the display state of the global navigation map based on the motion characteristics of the wearer of the AR fire helmet, specifically: Collecting head movement posture data and movement speed data of the wearer of the AR fire helmet; and determining the wearer's field of view azimuth change information based on the head movement posture data; Based on the motion speed data, determining average motion speed information of the wearer in the real scene; Based on the field of view azimuth change information, adjusting the display azimuth of the global navigation map in the virtual display space of the AR fire helmet; Based on the average motion speed information, the dynamic display refresh frequency of the virtual guide graphic in the global navigation map is adjusted.

Images