Thermal imaging monitoring control system

Abstract

A thermal imaging monitoring control system. At least one monitoring node is deployed in a monitoring area. The at least one monitoring node comprises at least one first monitoring node dispersed within the monitoring area. Each first monitoring node comprises a first thermal imaging camera. Each first thermal imaging camera is arranged at a first predetermined height from the ground, an optical axis thereof being substantially perpendicular to the ground, so as to perform thermal imaging of a corresponding imaging region on the ground, and each pixel region of a thermal imaging map obtained therefrom corresponds to each spatial region in the imaging region. A control system analyzes the thermal imaging map from the at least one first monitoring node to identify temperature-related information therein. This allows for precise identification, analysis, and pinpointing of on-site temperature-related information. In a fire-fighting application scenario, for example, this not only reduces the occurrence of missed and false fire hazard alarms, but it also pinpoints fire hazards within a monitoring area, thereby more effectively facilitating evacuation and search and rescue efforts.

Description

Thermal imaging monitoring and control system

[0001] This disclosure claims priority to a Chinese patent application filed on March 7, 2024, with application number 202410263200.X and invention name “Thermal Imaging Monitoring and Control System”. Technical Field

[0002] The present disclosure relates to monitoring technology, and in particular to a thermal imaging monitoring and control system. Background Art

[0003] Currently, fire monitoring, particularly in indoor environments, typically relies on smoke or heat detectors to monitor fire risks or incidents. However, these detectors often experience false alarms or missed alarms, often requiring on-site manual verification, which slows down fire response times and creates significant challenges for both users and fire departments.

[0004] In some fire monitoring scenarios, such as forest fire prevention and portable fire rescue, a single high-resolution thermal imaging camera is used to monitor the entire monitoring area using thermal imaging technology. However, the thermal imaging camera used in such scenarios requires high resolution to accurately monitor a relatively large monitoring area. Consequently, the required thermal imaging camera is expensive, making it unsuitable for widespread use and unsuitable for fire monitoring in, for example, indoor areas of buildings. Summary of the Invention

[0005] A technical problem to be solved by the present disclosure is to provide a thermal imaging monitoring and control system that can accurately identify, analyze and locate temperature-related information on site.

[0006] According to a first aspect of the present disclosure, a thermal imaging monitoring and control system is provided, comprising: at least one monitoring node, the at least one monitoring node comprising at least one first monitoring node, the at least one first monitoring node being distributed within a monitoring area, each first monitoring node comprising a first thermal imaging camera, the first thermal imaging cameras being respectively arranged at a first predetermined height from the ground, with an optical axis substantially perpendicular to the ground, so as to thermally image a corresponding imaging area on the ground, wherein each pixel area of ​​the obtained thermal imaging image corresponds to each spatial area in the imaging area; and a control system for acquiring a thermal imaging image from the at least one first monitoring node, and analyzing the acquired thermal imaging image to identify temperature-related information therein.

[0007] Optionally, there are multiple first monitoring nodes, and the first thermal imaging camera imaging areas of adjacent first monitoring nodes are adjacent to or overlap each other; the first thermal imaging camera imaging areas of multiple first monitoring nodes basically cover the ground of the monitoring area.

[0008] Optionally, the first thermal imaging camera is a thermal imaging camera with a wide field of view or a larger field of view to expand the monitoring range of a single first monitoring node; and / or the first thermal imaging camera is a high-resolution thermal imaging camera that can identify the category of the imaging object but not the individual; and / or the first thermal imaging camera includes at least one of a LWIR wavelength range thermal imaging sensor, a MWIR wavelength range thermal imaging sensor, a SWIR wavelength range thermal imaging sensor, a NIR wavelength range thermal imaging sensor, and a FIR wavelength range thermal imaging sensor.

[0009] Optionally, the control system performs at least one of the following functions: identifying a hot spot area in a thermal imaging image; identifying the type of imaging object in the hot spot area in the thermal imaging image, the type of imaging object in the hot spot area including at least one of fire, living organisms, vehicles, robots, and instruments; locating the spatial position corresponding to the hot spot area based on the position of the first monitoring node corresponding to the thermal imaging image in the monitoring area and the relative position of the hot spot area in the thermal imaging image.

[0010] Optionally, the system may also include: an artificial intelligence model system or a machine vision processing system for analyzing thermal images of a single area and the entire building.

[0011] Optionally, the training of the artificial intelligence model system is performed in the cloud; and / or the artificial intelligence model system is trained separately for different buildings and / or different cities or countries; and / or the artificial intelligence model system is trained in a classified manner based on building categories and / or human activity categories; and / or the artificial intelligence model system is trained based on the building categories and / or human activity categories and / or city or country information corresponding to the monitoring area targeted by the control system, and the corresponding training results are sent to the corresponding control system.

[0012] Optionally, the control system is a closed-loop automatic control system based on thermal imaging feedback and artificial intelligence or machine vision processing and control.

[0013] Optionally, the artificial intelligence model system or the machine vision processing system sends a linkage control command to the corresponding actuator based on the results of analyzing the thermal imaging image, and uses the linkage control results of the actuator to the linkage control command as training samples to train the artificial intelligence model system.

[0014] Optionally, the first monitoring node also includes a fire monitoring device, which includes a smoke detector and / or a temperature probe, wherein, through an artificial intelligence model system or a machine vision processing system or manual analysis, and combined with the detection results of the smoke detector and / or the temperature probe, a fire warning is issued to the monitored area to achieve full-time and full-feature fire alarms in the very early, early, mid-term and late stages, and / or at least part of the training samples used to train the artificial intelligence model system are obtained by combining the thermal imaging images from the first thermal imaging camera of each first monitoring node and the detection results of the fire monitoring equipment.

[0015] Optionally, the multiple monitoring nodes also include a second monitoring node, including a second thermal imaging camera, which is arranged at a second predetermined height from the ground, with an optical axis basically parallel to the ground, so as to perform thermal imaging of the corresponding spatial area within the monitoring area. The second predetermined height is lower than the first predetermined height, and the control system combines the thermal imaging images from the first thermal imaging camera and the second thermal imaging camera for analysis.

[0016] Optionally, calculation is performed in combination with the thermal imaging image acquired by the first thermal imaging camera, the first predetermined height, and the size information of the spatial area corresponding to the corresponding imaging area to obtain accurate position information of the imaging object in the thermal imaging image in the spatial area.

[0017] Optionally, the multiple monitoring nodes also include a third monitoring node, including a third thermal imaging camera, which is arranged in a designated monitoring area to perform thermal imaging of the designated monitoring area, and / or the optical axis of the third thermal imaging camera points to a designated monitoring target to perform thermal imaging of the designated monitoring target.

[0018] Optionally, the monitoring node also includes a fire emergency equipment control device, which controls the corresponding fire emergency equipment to perform emergency functions in response to the control system identifying the fire risk based on at least one of the fire risk point, the location of personnel at the fire scene, and the spatial layout of the monitoring area.

[0019] Optionally, the fire emergency equipment includes fire emergency signs and / or fire broadcasts. In response to identifying a fire risk, the fire emergency equipment control device controls the fire emergency equipment to dynamically change the content of the local emergency signs and the local broadcast content, and optimizes the design from an architectural design perspective to better guide the evacuation of on-site personnel by artificial intelligence or manual labor; and / or the fire emergency equipment includes fire sprinklers. In response to identifying a fire risk, the fire emergency equipment control device controls the fire sprinklers to perform sprinkler fire extinguishing operations by artificial intelligence or manual remote commands.

[0020] Optionally, the control system provides on-site feedback information to the central air-conditioning system based on the analysis results of the acquired thermal images so that the central air-conditioning system can perform dynamic adjustments; and / or the control system provides on-site feedback information to the lighting system based on the analysis results of the acquired thermal images so that the lighting system can perform lighting control; and / or the control system provides positioning tracking information to support active positioning and navigation of living organisms, vehicles or robots; and / or the control system provides on-site personnel body temperature information and positioning information to the epidemic prevention system based on the analysis results of the acquired thermal images.

[0021] Optionally, the thermal imaging camera calibrates the camera itself for temperature accuracy and precision based on the temperature constancy of the human body, based on the hot spot areas corresponding to the human body identified in the thermal image.

[0022] Optionally, the control system may include: at least one regional control unit, each connected to at least one monitoring node among multiple monitoring nodes via a serial bus; a main control unit, the regional control units are connected to the main control unit via Ethernet or a wireless network, and the main control unit uploads information to the fire alarm information demander or exchanges information with it through the fire linkage controller.

[0023] Optionally, the first monitoring node also includes: a communication device for communicating with the control system; a local control unit for adjusting the thermal imaging frame rate of the thermal imaging camera and / or the frame rate of the thermal imaging image sent to the communication system in response to instructions from the control system; and an image processing device for performing image processing on the thermal imaging image from the thermal imaging camera in the first monitoring node, the image processing including at least one of the following: image compression processing; identifying hot spots; cropping the thermal imaging image to retain only the image of the identified hot spots, so as to reduce the communication volume required for the communication system to transmit the thermal imaging image; filtering the thermal imaging image to filter out pixels whose corresponding temperatures are lower than a set temperature threshold; performing artificial intelligence processing on the thermal imaging image to send only the processed information to the control system, so as to reduce the communication volume required for the communication system to transmit the thermal imaging image and / or improve compatibility with existing fire protection systems; performing artificial intelligence processing on the thermal imaging image to further generalize the identified humans so as to replace the human image with a static graphic (e.g., a cartoon-shaped static graphic) and send only the processed information to the control system, thereby addressing privacy concerns in specific areas.

[0024] Optionally, the monitoring area includes multiple monitoring subareas, each of which includes multiple monitoring units, and each monitoring unit is equipped with at least one first monitoring node. The control system includes a master control unit corresponding to the monitoring area, multiple regional control units corresponding to the multiple monitoring subareas, and multiple unit controllers corresponding to the multiple monitoring units. The thermal imaging monitoring control system also includes an execution system comprising a master actuator corresponding to the monitoring area, multiple regional actuators corresponding to the multiple monitoring subareas, and multiple unit actuators corresponding to the multiple monitoring units. The unit actuators are configured to execute their functions in their corresponding monitoring units. The unit controller receives a thermal image from the first monitoring node provided in the corresponding monitoring unit and, based on the primary analysis and processing results of the thermal image, sends a unit control instruction to the unit actuator corresponding to the monitoring unit, so that the unit actuator executes its function in the corresponding monitoring unit in accordance with the unit control instruction. The unit controller also sends the received thermal image and / or the primary analysis and processing results to the regional actuator corresponding to the monitoring subarea to which the corresponding monitoring unit belongs. The regional control unit obtains a secondary analysis and processing result based on the received thermal image and / or primary analysis and processing result, and sends a regional control instruction based on the secondary analysis and processing result to the corresponding regional actuator, so that the regional actuator can distribute the control instruction to the corresponding multiple unit actuators. The regional control unit also sends the received thermal image and / or the received primary analysis and processing result and / or the received secondary analysis and processing result to the overall control unit. The overall control unit obtains a tertiary analysis and processing result based on the received thermal image and / or the received primary analysis and processing result and / or the received secondary analysis and processing result, and sends a general control instruction based on the tertiary analysis and processing result to the overall actuator, so that the overall actuator can distribute the control instruction to each unit actuator via the regional actuator.

[0025] Optionally, the control system and / or monitoring node can dynamically adjust the working, sleeping and shutdown time and / or time ratio of the control system and / or monitoring node based on manual or artificial intelligence processing analysis results to evaluate the busy and idle periods and / or frame rate requirements, so as to balance the system resource allocation and improve the service life of the system equipment.

[0026] Therefore, by distributing at least one (such as multiple) thermal imaging cameras with optical axes perpendicular to the ground in the monitoring area, the present disclosure provides a monitoring system based on thermal imaging (i.e., a thermal imaging monitoring and control system), which can accurately identify, analyze and locate information related to temperature on the scene, so that in fire-fighting application scenarios, for example, it can not only reduce the occurrence of missed reports and false alarms of fire hazards, but also accurately locate fire hazards in the monitoring area, provide more accurate information for fire-fighting work, and facilitate the smooth development of various tasks such as fire-fighting, for example, it is more convenient to carry out effective personnel evacuation and search and rescue. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0028] FIG1 is a schematic block diagram of a monitoring system according to an embodiment of the present disclosure.

[0029] FIG2 shows an arrangement of multiple monitoring nodes according to an embodiment of the present disclosure.

[0030] Figure 3 schematically illustrates an exemplary arrangement of a first monitoring node in a monitoring area in a top view. Figure 4 illustrates an arrangement of multiple monitoring nodes according to another embodiment of the present disclosure.

[0031] FIG5 schematically shows a deployment scheme of monitoring nodes and various fire emergency equipment in another embodiment of the present disclosure.

[0032] FIG6 is a schematic block diagram of a fire alarm monitoring system according to another embodiment of the present disclosure.

[0033] FIG7 is a schematic block diagram of a monitoring node according to an embodiment of the present disclosure.

[0034] FIG8 is a schematic architecture diagram of an existing conventional fire control system.

[0035] FIG9 is a schematic architecture diagram of a fire monitoring system based on thermal imaging that can be used according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0037] FIG1 is a schematic block diagram of a monitoring system according to an embodiment of the present disclosure.

[0038] As shown in FIG1 , the monitoring system according to an embodiment of the present disclosure includes a control system and at least one (eg, multiple) monitoring nodes.

[0039] Although FIG1 illustrates a control system located outside a monitoring node, it should be understood that in some embodiments, the control system may be located within the monitoring node. For example, if the monitoring system includes a monitoring node, the control system may be located within the monitoring node and considered a component of the monitoring node.

[0040] A plurality of monitoring nodes are arranged in the monitoring area, and monitor the corresponding part of the monitoring area respectively.

[0041] The fire alarm monitoring system disclosed herein can be applied to various scenarios. In some embodiments, the monitoring area can be a space within an industrial or residential building.

[0042] Each monitoring node can include a thermal imaging camera (also known as an "infrared camera") to thermally image the corresponding imaging area. The control system can obtain thermal images from each monitoring node and analyze the acquired thermal images to identify temperature-related information. Thus, the present disclosure provides a monitoring system based on thermal imaging.

[0043] In the context of the present disclosure, “temperature-related information” may refer to various temperature-related information that can be obtained from a thermal image.

[0044] As an example, when the monitoring system is applied to a fire alarm monitoring system in a fire protection application scenario, the temperature-related information may include, for example, fire risk.

[0045] For another example, the temperature-related information may include temperature information of each region in the thermal image.

[0046] For another example, the temperature-related information may include information about regions in a thermal image that are within a predetermined temperature range, such as information about regions that are within a constant temperature range of a human body.

[0047] In various firefighting and non-firefighting application scenarios, various temperature-related information can be obtained from thermal images according to application needs.

[0048] Multiple monitoring nodes can be arranged in the monitoring area according to certain specifications.

[0049] FIG2 shows an arrangement of multiple monitoring nodes according to an embodiment of the present disclosure.

[0050] As shown in Figure 2, a plurality of monitoring nodes T1 may be arranged at a first predetermined height above the ground within the monitoring area. The monitoring nodes T1 arranged in this manner may be referred to as "first monitoring nodes" hereinafter.

[0051] Accordingly, the first thermal imaging camera in the first monitoring node T1 can also be positioned at a first predetermined height above the ground within the monitoring area. The optical axis of the first thermal imaging camera is positioned substantially perpendicular to the ground, so as to thermally image the corresponding imaging area on the ground. In this way, each pixel region of the thermal image obtained by the first thermal imaging camera can correspond to a spatial region within the imaging area.

[0052] Calculation can also be performed in combination with the thermal image acquired by the first thermal imaging camera, the first predetermined height, and the size information of the spatial area corresponding to the corresponding imaging area to obtain more accurate position information of the imaging object in the thermal image in the spatial area.

[0053] For imaging objects within the imaging area of ​​the first thermal imaging camera, such as people or vehicles, the first thermal imaging camera will thermally image them, forming corresponding hot spots on the thermal imaging image formed on the thermal imaging sensor of the thermal imaging camera.

[0054] The first predetermined height can be set accordingly according to the on-site situation of the monitoring area and, if necessary, in combination with the imaging parameters of the first thermal imaging camera, so that the first thermal imaging camera can perform appropriate thermal imaging of the ground in its corresponding imaging area.

[0055] For example, the first predetermined height may correspond to the height of a building floor, i.e., the first monitoring node T1 and the first thermal imaging camera may be installed on the ceiling of a building floor. Alternatively, the first predetermined height may correspond to the height of a suspended ceiling within a building. Alternatively, the first predetermined height may be set based on the size of the ground imaging area to be covered (combined with the viewing angle of the first thermal imaging camera), subject to the constraints of the floor height or the suspended ceiling height.

[0056] Multiple first monitoring nodes T1 may be distributed throughout the monitoring area. These multiple first monitoring nodes T1 may form a monitoring network. The combined imaging areas of the first thermal imaging cameras in the multiple first monitoring nodes T1 may substantially cover the entire surface of the monitoring area. For example, the imaging areas of the first thermal imaging cameras of adjacent first monitoring nodes may be adjacent to or overlap.

[0057] FIG3 schematically shows an exemplary arrangement of the first monitoring nodes in the monitoring area in the form of a top view.

[0058] In FIG3 , small rectangular grids are used to represent spatial areas on the ground of the monitoring area, and four first monitoring nodes T1 are arranged in the monitoring area. The dotted lines respectively show the imaging areas (thermal imaging ranges) of the first thermal imaging cameras of the four first monitoring nodes T1.

[0059] As shown in Figure 3, the imaging areas of the first thermal imaging cameras of adjacent first monitoring nodes T1 can be set to overlap each other. Therefore, by setting the imaging areas of the first thermal imaging cameras of multiple first monitoring nodes T1, the ground of the entire monitoring area can be basically covered.

[0060] In some embodiments, the first thermal imaging camera can be a thermal imaging camera with a wide field of view or larger to expand the monitoring range of a single first monitoring node. FIG2 shows the field of view of the first thermal imaging camera of each first monitoring node T1 with a dotted line.

[0061] For example, the field of view angle of the first thermal imaging camera may be greater than 90 degrees.

[0062] It should be understood that the wide field of view in the present disclosure is not limited to being greater than 90 degrees, but also includes being less than 90 degrees. For example, in some embodiments or implementation scenarios, a field of view greater than 64 degrees can be considered a wide field of view.

[0063] In some embodiments, the first thermal imaging camera can be a thermal imaging camera designed, based on hardware and / or software, that can identify the category of an imaged object, rather than the individual. For example, the resolution or sensitivity of the thermal imaging camera can be such that it can identify the category of an imaged object, rather than the individual. By using, for example, a thermal imaging camera with a medium-low resolution (e.g., less than or equal to QVGA resolution, 320x240) or medium-low sensitivity, or a thermal imaging camera that undergoes post-processing such as software obfuscation or encryption, privacy concerns can be mitigated, thereby expanding the monitoring area. Furthermore, for monitoring areas and / or objects with specific requirements, the use of a high-resolution thermal imaging camera can be considered, for example, to provide privacy alerts when notifying the user of the presence of a thermal imaging monitoring device. It should be understood that the medium-low resolution in this disclosure is not limited to "less than or equal to QVGA resolution, 320x240" and can also include other resolutions. For example, in some embodiments, a resolution less than or equal to 640x480 can be considered medium-low resolution.

[0064] For monitoring and rescue targets in the field of fire monitoring, it is only necessary to determine the type and location of the target.

[0065] Privacy concerns in public places also make it impossible for surveillance systems to identify individuals in principle.

[0066] To this end, the present disclosure proposes that the first thermal imaging camera be capable of identifying the category of the imaged object but not the individual. To enable the first thermal imaging camera to identify categories but not individuals, the spatial resolution of the first thermal imaging camera needs to be configured. In some embodiments, the first thermal imaging camera is a wide-angle camera, its optical axis is substantially perpendicular to the ground, and it is at a predetermined height above the ground. With these three conditions in place, spatial resolution is solely related to the camera's resolution. Therefore, configuring the spatial resolution of the first thermal imaging camera translates to configuring the resolution of the first thermal imaging camera. In other words, the resolution of the first thermal imaging camera only needs to be set to a resolution that can identify the category of the imaged object but not the individual. It should be noted that the resolution here can refer to any resolution within the valid resolution range. By configuring the first thermal imaging camera so that the resulting thermal images cannot identify individuals, the possibility of identifying individuals can be eliminated at the source of the data, completely eliminating privacy concerns.

[0067] That is to say, for the first thermal imaging camera, especially when the first thermal imaging camera is a wide-angle camera, only when the condition of "a first predetermined height from the ground + the optical axis is perpendicular to the ground" is met, a monitoring space that is unrelated to the on-site building information is determined, and only then can the resolution of the first thermal imaging camera be set so that the first thermal imaging camera can only identify the category of the imaging object but not the individual.

[0068] Furthermore, since the resolution of the first thermal imaging camera is limited to a resolution sufficient to identify the category of the imaged object but not the individual, redundant resolution need not be considered. In other words, the resolution of the first thermal imaging camera is essentially non-redundant. Therefore, while meeting demand, the cost of thermal imaging cameras (whose cost is extremely sensitive to resolution) can be significantly reduced, facilitating their widespread adoption. Furthermore, this resolution reduces the amount of data the algorithm must process when performing positioning according to the positioning method described below.

[0069] In addition, the first thermal imaging camera may include at least one of a LWIR (long wave infrared) wavelength range thermal imaging sensor, a MWIR (mid wave infrared) wavelength range thermal imaging sensor, a SWIR (short wave infrared) wavelength range thermal imaging sensor, a NIR (near infrared) wavelength range thermal imaging sensor, and a FIR (far infrared) wavelength range thermal imaging sensor.

[0070] Based on the thermal image from the first monitoring node T1, the control system can perform analysis to identify the fire risk.

[0071] For example, the control system can identify hot spots in the thermal image. Hot spots in the thermal image correspond to areas with higher temperatures in the monitored area.

[0072] For example, the control system can identify the type of imaging object in the hot spot area of ​​the thermal image based on the thermal image. The type of imaging object in the hot spot area can include at least one of fire, living organisms (such as humans, animals), vehicles, robots, and instruments.

[0073] For example, the control system may also locate the spatial position corresponding to the hotspot area based on the position of the first monitoring node T1 corresponding to the thermal imaging image in the monitoring area and the relative position of the hotspot area in the thermal imaging image.

[0074] The position of the first monitoring node T1 in the monitoring area can be determined based on, for example, its address code. In other words, the position of the first monitoring node T1 in the monitoring area (such as an industrial or civil building) can be determined based on the address code assigned to the first monitoring node T1. Thereby, the imaging area of ​​the ground in the monitoring area corresponding to the first monitoring node T1 can be further determined. In addition, since each pixel area of ​​the thermal imaging image sensed by the first thermal imaging camera corresponds to each spatial area in the imaging area, the relative position of the imaging object corresponding to the hot spot area on the thermal imaging image can be determined within the imaging area (sensing range) of the first thermal imaging camera. Combined with the position of the first monitoring node T1 in the monitoring area, the specific position of the imaging object corresponding to the hot spot area in the monitoring area can be accurately obtained, thereby achieving precise positioning of the hot spot area.

[0075] The positioning information can be sent to the corresponding personnel / departments / units in real time or non-real time for fire-fighting purposes such as personnel evacuation, dispersal and search and rescue command in fire conditions, as well as other social functions and / or commercial purposes in non-fire conditions.

[0076] In order to reduce monitoring blind spots and expand the monitoring range, the use of wide-angle cameras is a necessary choice.

[0077] Especially when the camera's optical axis is installed essentially perpendicular to the ground, a wide-angle camera is a must. This is because the vertical position of the optical axis essentially reduces the monitoring range. Therefore, a wide-angle camera is needed to compensate for the blind spots and reduced monitoring range caused by the vertical position of the optical axis.

[0078] However, under current technological conditions, a direct consequence of wide-angle cameras is severe distortion. Regardless of whether the imaging principle of a wide-angle camera is linear or nonlinear mapping, severe distortion will result in a nonlinear mapping relationship between the final pixel area and the monitoring area. Furthermore, the distortion based on nonlinear mapping will make the final mapping relationship even more nonlinear.

[0079] Existing methods mainly reduce severe distortion by compensating for the severe distortion of wide-angle cameras. Compensation methods are divided into optical compensation and digital processing.

[0080] Optical compensation involves adding multiple lenses to reduce severe distortion at the source. However, these additional lenses can technically introduce negative effects, such as flare and ghosting caused by multiple reflections. Furthermore, thermal imaging camera lenses (typically germanium lenses made of the rare metal) are relatively expensive, and optical compensation requires additional lenses, making the cost extremely high.

[0081] Digital processing compensation generally reduces severe distortion through linearization. This means that a general approximate model is developed for the specific distortion correction target (often a linear mapping). General models are generally complex and lack precision due to their approximate nature. They can only compensate for global distortion, not local distortion, and certainly not distortion caused by individual errors. Furthermore, this model compensation process must be performed online, making it difficult to calibrate and inaccurate. Furthermore, due to the complex calculation formula, high processor performance is required, especially at high resolutions. Furthermore, in the case of wide-angle nonlinear mapping, information loss due to severe distortion makes it impossible to linearize the nonlinearity based on the mapping principle (see the world map example). This results in generally low positioning accuracy and even false positioning due to distortion.

[0082] In view of this, the present disclosure proposes a positioning solution for wide-angle cameras that can achieve truly accurate positioning without the need for the aforementioned optical or digital processing compensation.

[0083] The present disclosure sets the optical axis of the wide-angle camera (ie, the first thermal imaging camera) substantially perpendicular to the ground and arranged at a first predetermined height from the ground.

[0084] The first predetermined height is a predetermined height from the ground (rather than an absolute horizontal height or other heights), and the optical axis of the wide-angle camera is substantially perpendicular to the ground.

[0085] The "first predetermined height from the ground + optical axis perpendicular to the ground" not only ensures that each pixel area in the thermal image corresponds to each spatial area in the imaging area, but also allows this correspondence to be determined in advance through various methods. For example, this correspondence can be determined in advance using calibration objects or calculated based on the nonlinear mapping relationship of the wide-angle camera.

[0086] Based on this correspondence, the present disclosure determines the relative position of the imaging object corresponding to the hot spot area in the imaging area of ​​the first thermal imaging camera based on the relative position of the hot spot area in the thermal imaging image, and determines the specific position of the imaging object corresponding to the hot spot area in the monitoring area in combination with the position of the first monitoring node corresponding to the thermal imaging image in the monitoring area.

[0087] Different from the above-mentioned optical or digital processing compensation methods, the present invention targets the severe distortion in wide-angle cameras by positioning through "the first predetermined height from the ground + the optical axis perpendicular to the ground + the corresponding relationship", thereby achieving precise positioning without the need to use conventional optical or digital processing compensation methods.

[0088] For the purpose of this disclosure, "the first predetermined height from the ground + the optical axis perpendicular to the ground" is a predetermined fixed parameter for all cameras, which makes the correspondence between each pixel area of ​​the thermal imaging image and each spatial area in the imaging area equivalent to being determined when the camera leaves the factory.

[0089] Therefore, "the first predetermined height from the ground + the optical axis perpendicular to the ground" (including the correspondence between each pixel area of ​​the thermal imaging image and each spatial area in the imaging area) has essentially defined a complete positioning system, and this definition does not depend on the specific site conditions or information of the monitored space, so it can be used as an independent product for offline positioning.

[0090] Offline positioning means that no information needs to be collected on site, and positioning can be achieved when the camera leaves the factory.

[0091] While the present disclosure can also be implemented as online positioning, one advantage of offline positioning is that the offline positioning relationship (the relative positioning relationship between the hotspot area in the thermal image and the imaging object within the imaging area) can greatly simplify online positioning calculations. This can be achieved using a simple table lookup method, which simplifies online calculations and allows for offline positioning using processors with medium or low processing power. A major issue with online positioning using nonlinear mapping is that the calculation formula is complex and sensitive to internal and external parameters, requiring high processing power and resulting in low positioning accuracy.

[0092] In addition, offline positioning has the advantage that it can be calibrated offline to achieve precise positioning. The positioning relationship after calibration is stable and can therefore be saved for all monitoring spaces that meet the conditions (that is, it can be used for all first thermal imaging cameras). This is especially important for nonlinear mapping, because online calibration is complex and expensive, and is not practical for large monitoring arrays.

[0093] In some embodiments, the present disclosure can also obtain location information for one or more specific parts of the imaged object based on the type of the imaged object. Specifically, first, based on the type of the imaged object, hotspots corresponding to the specific parts of the imaged object can be identified in the thermal image, as well as pixel regions in the thermal image belonging to the specific parts of the imaged object. Next, based on the correspondence between each pixel region in the thermal image and each spatial region in the imaging area, the relative position of the specific part within the imaging area of ​​the first thermal imaging camera can be determined. Combined with the position of the first monitoring node corresponding to the thermal image within the monitoring area, the specific location of the specific part of the imaged object within the monitoring area can be determined. For example, if the imaged object is a person, hotspots corresponding to the person's feet can be identified in the thermal image, and the location information of the feet can be determined accordingly. For another example, if the imaged object is a vehicle, hotspots corresponding to specific parts of the vehicle, such as the front hood, front wheels, rear wheels, and trunk, can be identified in the thermal image, and the location information of these specific parts of the vehicle can be determined accordingly. In this way, by determining the location information of one or more specific parts of the imaged object based on the type of the imaged object, rather than the location information of the entire imaged object, the accuracy of the final location result can be improved. In some further embodiments, the present disclosure may also identify the posture of the imaging object based on the determined position information of one or more specific parts of the imaging object to determine the posture information of the imaging object.

[0094] In some embodiments, the present disclosure may also calculate the size information such as the height or length of the imaging object based on the type of the imaging object. Specifically, based on the type of the imaging object, the size relationship between the height in the vertical direction and the length in the horizontal direction of the imaging object is analyzed, and based on this, the pixel size within the boundary range of the hot spot area corresponding to the imaging object in the thermal imaging image is determined to determine whether it corresponds to the height or length of the imaging object, and the height or length of the imaging object is calculated accordingly. In some further embodiments, the recognition result of the type of the imaging object may be verified based on the calculated height or length of the imaging object to determine whether the recognition result of the type of the imaging object is accurate. For example, in a case where the calculated height or length of the imaging object obviously does not match the type of the imaging object, it may be determined that the recognition result of the type of the imaging object is incorrect, and the type of the imaging object may be re-identified.

[0095] In some embodiments, for an imaging object (such as fire) located at the edge of the monitoring area (such as a wall), the present disclosure can also calculate the height information of the imaging object, thereby providing more auxiliary information for thermal imaging monitoring.

[0096] The thermal imaging camera (first thermal imaging camera / second thermal imaging camera) in the present disclosure may include at least one thermal imaging sensor and at least one thermal imaging lens. For example, a thermal imaging camera may consist of one thermal imaging sensor and one thermal imaging lens, i.e., the thermal imaging camera may be a single camera. For another example, a thermal imaging camera may be composed of multiple thermal imaging sensors and / or multiple thermal imaging lenses, i.e., a thermal imaging camera may be composed of multiple cameras. When a thermal imaging camera is composed of multiple thermal imaging sensors and / or multiple thermal imaging lenses, the field of view angles of adjacent thermal imaging lenses may overlap or abut at their boundaries.

[0097] FIG4 shows an arrangement of multiple monitoring nodes according to another embodiment of the present disclosure.

[0098] As shown in FIG4 , the plurality of monitoring nodes may further include at least one second monitoring node T2. The second monitoring node T2 includes a second thermal imaging camera. The second thermal imaging camera (and the corresponding second monitoring node T2) may be arranged at a second predetermined height above the ground, with an optical axis substantially parallel to the ground, to thermally image a corresponding spatial area within the monitoring area.

[0099] Generally, the second predetermined height may be lower than the first predetermined height. For example, the second predetermined height may correspond to the height of a monitored object that may appear in the monitored area. For example, the second predetermined height may be approximately 1 meter.

[0100] The control system may combine the thermal imaging images from the first thermal imaging camera and the second thermal imaging camera for analysis.

[0101] FIG4 shows the field of view angle range of the second thermal imaging camera of each second monitoring node T2 by a dotted line.

[0102] For example, the field of view angle of the second thermal imaging camera may be less than 90 degrees.

[0103] By using a thermal imaging camera with a relatively smaller field of view (compared to the first thermal imaging camera), when using a thermal imaging sensor with the same resolution, the horizontally arranged second thermal imaging camera can provide a relatively higher angular resolution, thereby improving the thermal imaging clarity of imaging objects in the space. When the imaging range is relatively narrow (limited floor height inside the building), clearer thermal imaging can be achieved for imaging objects that are farther away in the space.

[0104] In addition, in some embodiments, the second monitoring node T2 may also cover corner areas that are not covered by the first monitoring node T1, such as the ceiling of a building.

[0105] In addition, in some embodiments, the monitoring node may further include a third monitoring node (not shown in the figure), and the third monitoring node may include a third thermal imaging camera.

[0106] For example, the third thermal imaging camera (third monitoring node) can be arranged in a designated monitoring area, such as an equipment room, a pipe shaft, inside a suspended ceiling, an elevator room, an elevator shaft, and other special monitoring areas to perform thermal imaging monitoring of the designated monitoring area.

[0107] For another example, the optical axis of the third thermal imaging camera may be directed toward a designated monitoring target, such as a designated device, a designated pipeline, or other special monitoring target, so as to perform thermal imaging monitoring on the designated monitoring target.

[0108] In this way, targeted fire monitoring can be carried out on designated monitoring areas or designated monitoring objects with special significance or higher fire risks.

[0109] Furthermore, in some embodiments, the monitoring system may also include an artificial intelligence model system or a machine vision processing system for analyzing thermal images of a single area and the entire building.

[0110] For example, training of artificial intelligence model systems can be performed in the cloud.

[0111] For example, the AI ​​model system can be trained separately for different buildings and / or different cities or countries.

[0112] For another example, the artificial intelligence model system can also be trained in a classified manner based on building categories and / or human activity categories.

[0113] In this way, the artificial intelligence model system can be trained based on the building category and / or human activity category and / or city or country information corresponding to the monitoring area targeted by the control system (or a monitoring system deployed somewhere), and the corresponding training results can be sent to the corresponding (monitoring system) control system.

[0114] In some embodiments, the control system may be a closed-loop automatic control system based on thermal imaging feedback (thermogram) and artificial intelligence or machine vision processing and control.

[0115] For example, an AI model system or machine vision processing system can send linkage control commands to corresponding actuators based on the results of thermal image analysis. These actuators can include those used for firefighting purposes, such as fire emergency signs, fire broadcasts, and fire sprinklers, as well as actuators used for non-firefighting purposes, such as central air conditioning systems, lighting systems, and epidemic prevention systems.

[0116] On the other hand, the linkage control results of the linkage control commands using these actuators can also be used as training samples to train the artificial intelligence model system.

[0117] Furthermore, each monitoring node (the first monitoring node T1, the second monitoring node T2, and the third monitoring node), especially the first monitoring node T1 distributed in the monitoring area, may also include other types of fire monitoring equipment. Other types of fire monitoring equipment may include smoke detectors and / or temperature sensors.

[0118] In this way, through artificial intelligence model systems or machine vision processing systems or manual analysis, and combined with the detection results of other fire monitoring equipment such as smoke detectors and / or temperature probes, fire warnings can be issued to the monitored areas, realizing full-time and full-feature fire alarms in the very early, early, mid-term and late stages.

[0119] On the other hand, by combining the thermal imaging images from the first thermal imaging camera of each first monitoring node and the detection results of each fire monitoring equipment, training samples (at least part of the training sample set) can be obtained for training the artificial intelligence model system.

[0120] The control system can use the results of analyzing the thermal images obtained from the thermal imaging cameras of each monitoring node in various application scenarios. For example, it can be used for fire monitoring purposes in some scenarios and for non-firefighting purposes in other scenarios.

[0121] As an application scenario for fire protection purposes, the control system can provide feedback information to the fire emergency equipment control device so that corresponding emergency functions can be executed.

[0122] In some embodiments, each monitoring node (first monitoring node T1, second monitoring node T2 and third monitoring node), especially the first monitoring node T1 distributed in the monitoring area, may also include a fire emergency equipment control device. The fire emergency equipment control device responds to the control system identifying the fire risk, and controls the corresponding fire emergency equipment to perform emergency functions based on at least one of the fire risk point, the location of the personnel at the fire scene, and the spatial layout of the monitoring area.

[0123] FIG5 schematically shows a deployment scheme of monitoring nodes and various fire emergency equipment in another embodiment of the present disclosure.

[0124] Fire emergency equipment may include, for example, fire emergency signs and / or fire broadcasts. In response to identifying a fire risk, the fire emergency equipment control device controls these fire emergency equipment to dynamically change the content of local emergency signs and local broadcasts, and optimizes the design from a building design perspective to better guide on-site personnel evacuation via artificial intelligence or manual intervention.

[0125] As shown in Figure 5, the fire emergency signs on both sides of the fire point provide evacuation direction instructions in opposite directions, so as to instruct on-site personnel to evacuate to a safe area as quickly as possible in the direction away from the fire point.

[0126] Similarly, fire alarms placed in different locations can also broadcast optimized evacuation prompts based on their own locations and the location of the fire, and if necessary, combined with the building design structure, so that on-site personnel can promptly understand the correct evacuation direction and evacuate to a safe area as soon as possible.

[0127] For example, fire emergency equipment may also include fire sprinklers. Upon identifying a fire risk, AI or manual remote control can be used to control the fire emergency equipment control device to control the corresponding fire sprinklers for fire extinguishing. This allows for linkage with the fire sprinklers. In some embodiments, architectural design optimization can be implemented to enable earlier and more accurate linkage with fire sprinklers via AI or manual remote control.

[0128] As an application scenario for non-firefighting purposes, the control system can provide on-site feedback information to the central air-conditioning system, lighting system, positioning tracking or navigation system, epidemic prevention system, etc.

[0129] For example, the control system can provide on-site feedback information to the central air conditioning system based on the analysis results of the acquired thermal images, so that the central air conditioning system can make dynamic adjustments. For example, by analyzing the thermal images, hotter and / or colder areas can be identified, and the on-site temperature information can be provided to the central air conditioning system so that the central air conditioning system can dynamically adjust to bring each area into the appropriate temperature range.

[0130] For example, the control system can provide on-site feedback information to the lighting system based on the analysis results of the acquired thermal imaging images, so that the lighting system can perform lighting control.

[0131] For example, the control system can provide on-site personnel location feedback information to the fan system based on the analysis results of the acquired thermal imaging images, so that the fan system can dynamically adjust and track to provide comfortable services.

[0132] For example, the control system can provide positioning and tracking information to support active positioning and navigation of living organisms, vehicles, or robots.

[0133] For example, the control system can provide the epidemic prevention system with on-site personnel temperature information and location information based on the analysis results of the acquired thermal imaging images. Existing epidemic prevention systems can only monitor the body temperature of mobile personnel in areas such as entrances. According to the embodiment of the present disclosure, the monitoring nodes distributed in the monitoring area can monitor the body temperature of personnel in the entire monitoring area. When a person with a body temperature exceeding the normal body temperature range is found, the body temperature information and location information of the relevant person can be provided to the epidemic prevention system so that the epidemic prevention system can accurately detect the relevant person.

[0134] For another example, the control system may also provide information for social service functions such as finding lost children and / or pets based on the analysis results of the acquired thermal images.

[0135] For example, the control system can also provide information for security monitoring, patrol systems, etc.

[0136] In addition, considering that the human body temperature range is relatively narrow, generally between 36.5 and 37 degrees Celsius, and has a certain degree of constancy, in some embodiments, the thermal imaging camera of each monitoring node can calibrate the temperature accuracy and precision of the camera itself based on the hot spot area corresponding to the human body identified in the thermal imaging image and based on the constancy of the human body temperature.

[0137] FIG6 is a schematic block diagram of a monitoring system according to another embodiment of the present disclosure.

[0138] As shown in FIG6 , the control system may include a general control unit and at least one regional control unit.

[0139] The regional control units may be connected to at least one monitoring node among the plurality of monitoring nodes via a serial bus, respectively.

[0140] The monitoring node network formed by serial field bus connection can be connected to the regional control unit or the main control unit via the serial field bus. The serial bus includes but is not limited to field buses such as RS485 / RS422. Each serial bus can be connected in series with at least two monitoring nodes (thermal imaging cameras). In some embodiments, each serial bus can be connected in series with more than five monitoring nodes (thermal imaging cameras).

[0141] The regional control units can be connected to the main control unit via Ethernet or wireless network. The main control unit can upload fire alarm information to the fire alarm information demander or exchange information with it through the fire linkage controller.

[0142] For example, the main control unit can be connected to the corresponding department / unit / equipment / Internet of Things, etc. through the fire linkage controller to upload information to it or exchange information.

[0143] FIG7 is a schematic block diagram of a monitoring node according to an embodiment of the present disclosure.

[0144] The monitoring node shown in FIG7 can be any one of the aforementioned first monitoring node T1 , second monitoring node T2 and third monitoring node, and can particularly be the first monitoring node T1 distributed in the monitoring area.

[0145] As shown in FIG7 , the monitoring node may include a thermal imaging camera to perform thermal imaging of imaging objects within its field of view angle.

[0146] In addition, as mentioned above, in some embodiments, the monitoring node may also include a smoke detector and / or a temperature sensor. The corresponding uses and functions have been described above and will not be repeated here.

[0147] In addition, as shown in FIG7 , the monitoring node may further include a communication device, a local control unit, and an image processing device.

[0148] The fire emergency equipment control device mentioned above can also be installed within the monitoring node. For example, the fire emergency equipment control device can be part of the local control unit. Alternatively, the local control unit can serve as the fire emergency equipment control device to perform corresponding control operations on the fire emergency equipment.

[0149] The communication device is used to communicate with the control system. For example, the communication device can be connected to the regional control unit via a serial bus to communicate data and information with the regional control unit, and upload thermal images or information analyzed from the thermal images to the regional control unit.

[0150] The local control unit may adjust the thermal imaging frame rate of the thermal imaging camera and / or the frame rate of the thermal image sent to the communication system in response to instructions from the control system.

[0151] The image processing device can perform various image processing on the thermal image from the thermal imaging camera in the first monitoring node.

[0152] For example, the image processing device can perform image compression processing on the thermal image to reduce the data volume of the thermal image, thereby reducing the data communication volume between the communication device and the control system and improving the communication speed.

[0153] For another example, the image processing device may also identify a hotspot region from the thermal image, wherein the hotspot region may be a region having a temperature corresponding to or higher than a predetermined temperature threshold.

[0154] By identifying the hot spots in the thermal image, it can be analyzed and identified to obtain the corresponding information.

[0155] For another example, the image processing device may also perform cropping processing on the thermal image, retaining only the image of the identified hotspot area, so as to reduce the communication volume required for the communication system to transmit the thermal image.

[0156] For example, the image processing device can filter the thermal image to remove pixels whose corresponding temperatures are below a set temperature threshold. Similarly, this filtering process can reduce the communication traffic required to transmit the thermal image and improve the efficiency of analysis and processing by the control system, artificial intelligence model system, or machine vision processing system.

[0157] For another example, the image processing device itself may also have certain artificial intelligence or machine vision processing capabilities, perform artificial intelligence or machine vision processing on the thermal image, and only send the processed information to the control system to reduce the communication volume required for the communication system to transmit the thermal image and / or improve compatibility with existing fire protection systems.

[0158] For another example, the image processing device can also perform artificial intelligence processing on the thermal image, further generalize the identified humans, so as to replace the human image with a static graphic (such as a cartoon-shaped static graphic), and only send the processed information to the control system, thereby addressing privacy concerns in specific areas.

[0159] In addition, the control system and / or each monitoring node of the monitoring system can dynamically adjust the working, sleeping and shutdown time and / or time ratio of the control system and / or monitoring nodes based on manual operations of the management personnel, or through artificial intelligence processing and analysis results to evaluate the busy and idle work periods and / or frame rate requirements, so as to balance the system resource allocation and improve the service life of the system equipment.

[0160] Hereinafter, a control system applicable to a fire alarm of a monitoring system according to an embodiment of the present disclosure will be described with reference to FIG8 and FIG9 .

[0161] FIG8 is a schematic architecture diagram of an existing conventional fire control system.

[0162] As shown in FIG8 , the fire monitoring space is divided into multiple fire zones, such as fire zone 1 to fire zone M. Smoke detectors and / or temperature detectors are deployed in each fire zone.

[0163] The smoke detector and / or temperature detector outputs switching feedback information according to their respective detection results.

[0164] There are two ways and scenarios to implement alarm and control based on the switch feedback of smoke detectors and / or temperature detectors.

[0165] One is the automatic alarm scenario. Only when at least two probes alarm will the fire alarm be considered not a false alarm, an alarm message will be sent to the fire alarm controller, and the fire-fighting program will be started.

[0166] The other is a manual confirmation scenario. When only one probe alarms, the firefighting procedure will not be automatically started. Instead, staff are required to confirm at the probe site whether the fire alarm is a false alarm. If it is not a false alarm, for example, by pressing the alarm button, an alarm message is sent to the fire alarm controller to start the firefighting procedure.

[0167] The fire alarm controller sends information to the fire linkage controller. Under manual control by staff, the fire linkage controller sends control information to various actuators on-site (for example, in the fire compartment where the sensor that issued the alarm is located). Actuators include fire alarms, fire sprinklers, and other devices. Each actuator then executes the fire response accordingly.

[0168] In existing fire control systems, smoke and temperature sensors essentially lose their effectiveness once the fire program is activated, and linkage control relies primarily on pre-set linkage logic and manual control by staff. Furthermore, minimal linkage control is based on fire compartments.

[0169] It can be seen that the existing fire control system is essentially an open-loop non-automatic control system, which has at least the following disadvantages:

[0170] The alarm information sent by the temperature sensor and / or smoke sensor is a switch value with very limited information and low reliability;

[0171] Lack of real-time feedback capability. Whether it is the time delay required for on-site manual confirmation or waiting until multiple alarm points (multiple detectors) in the middle of a fire trigger the automatic alarm program, no real-time feedback can be provided. For fires that develop in seconds, this means that the fire is irreversible.

[0172] Coordination, evacuation and search and rescue can only be carried out based on preset linkage logic and the experience of staff. Without the assistance of real-time feedback information, it is impossible to respond correctly to the dynamically changing fire and on-site conditions at any time, and even wrong responses are often made, resulting in significant losses.

[0173] FIG9 is a schematic architecture diagram of a fire monitoring system based on thermal imaging that can be used according to an embodiment of the present disclosure.

[0174] As shown in Figure 9, the embodiment of the present disclosure is aimed at fire protection application scenarios. In addition to the multiple fire protection zones (fire protection zone 1 to fire protection zone M, more generally, they can also be called "monitoring zones") of the existing fire protection monitoring system, it further introduces the new concept of fire protection units (fire protection unit 1 to fire protection unit N, more generally, they can also be called "monitoring units").

[0175] The fire prevention unit may be associated with each of the aforementioned monitoring nodes. Furthermore, the fire prevention unit may correspond to the imaging area of ​​the first thermal imaging camera of one or more first monitoring nodes.

[0176] The monitored space can contain multiple (for example, M) fire zones, each of which contains multiple (for example, N) fire units, where M and N are both positive integers. The specific values ​​of M and N can be optimized based on architectural design. The number of fire units in each fire zone can be the same or different depending on the actual situation. This division scheme can also be used in non-firefighting applications to optimize the application.

[0177] In other words, the monitoring space can be divided into three layers from large to small: the entire monitoring space, fire protection partitions, and fire protection units.

[0178] Accordingly, executors can be divided into three tiers based on the scope of their management, from largest to smallest: master executor, regional executor, and unit executor. The master executor manages multiple regional executors (e.g., regional executor 1 through regional executor M), and each regional executor manages multiple unit executors (e.g., unit executor 1 through unit executor N).

[0179] Furthermore, the control system can also be divided into three layers: a main control unit, multiple area control units corresponding to multiple fire protection zones (area control unit 1 to area control unit M), and monitoring node control units corresponding to multiple fire protection units (monitoring node control unit 1 to monitoring node control unit N, also referred to as "node control unit" or "unit controller" or "local control unit").

[0180] In this way, the embodiments of the present disclosure can realize up to three (three-level) closed-loop control systems based on holographic full-time feedback: fire protection unit-monitoring node control unit-unit actuator, fire protection partition-area control unit-area actuator and monitoring space overall-total control unit-total actuator.

[0181] 1. First-level control loop

[0182] Each fire prevention unit can be equipped with at least one monitoring node. The monitoring node sends the collected thermal imaging information (and fused temperature sensor information / smoke sensor information) holographically and at all times to the local control unit within the monitoring node, namely monitoring node control unit 1 to monitoring node control unit N shown in Figure 9.

[0183] For example, local control units can be further divided into local fire control units and local non-fire control units. Fire control functions can include fire monitoring, linkage, closed-loop positioning, evacuation, and search and rescue. Non-fire control functions can include positioning, air conditioning system control, lighting system control, fan control, and broadcasting.

[0184] When a fire protection unit corresponds to only one first monitoring node, the local control unit (or unit controller or node control unit) can be set inside the first monitoring node.

[0185] The local control unit or unit controller (monitoring node control unit 1 to monitoring node control unit N) can perform artificial intelligence or machine vision processing based on the thermal image feedback information. Based on the processing results, the local fire control unit of the local control unit issues a control / interaction instruction (unit control instruction) to the corresponding fire function unit actuator, and the local non-fire control unit of the local control unit issues a control / interaction instruction to the corresponding non-fire function unit actuator. At the same time, the local control unit will also upload the thermal imaging holographic full-time information (or processed information) to the upper-level control unit (for example, a regional control unit or a master control unit).

[0186] The unit actuators of the fire protection function may include, for example, fire emergency indication, fire broadcast, fire sprinkler, etc. The unit actuators of the non-fire protection function may include, for example, air conditioning control, lighting control, fan control, broadcast, etc. that support non-fire protection functions.

[0187] The first-level closed-loop control circuit, formed by the fire protection unit, the local control unit (monitoring node control unit), and the unit actuator, is suitable for local control of the fire protection unit. This control circuit has the fastest response time and the smallest application space, providing high flexibility and greatly reducing the interference of the linkage control on other monitored spaces.

[0188] 2. Second level control loop

[0189] Regional control units can also be divided into regional fire control units and regional non-fire control units.

[0190] The regional control unit can receive the full-time holographic information of the thermal imaging images of each fire protection unit from the multiple monitoring node control units associated with it, summarize and process them accordingly, and send regional control instructions to the regional actuator when necessary based on the processing results to perform corresponding regional linkage control.

[0191] The regional actuator can also be divided into a regional fire protection function execution unit and a regional non-fire protection function execution unit.

[0192] The regional actuator can execute regional linkage control functions (such as regional-level fire door linkage, etc.) based on the regional control instructions from the regional controller (i.e., regional control unit), and distribute instructions to each unit actuator to execute the unit linkage control instructions.

[0193] As the actuator of the second-level closed-loop control loop, the regional actuator issues instructions with higher priority than the instructions issued by the unit actuator, so as to facilitate the optimal linkage control of the entire region.

[0194] At the same time, the regional control unit will also upload the full-time information of the thermal imaging hologram (or processed information) to the upper-level general control unit.

[0195] 3. Third-level control loop

[0196] The main control unit can also be divided into a central fire control unit and a central non-fire control unit.

[0197] The overall control unit may summarize and process the received full-time information of the thermal imaging hologram of each unit, and send the processing results to the linkage controller to execute linkage control, for example, after manual confirmation.

[0198] The linkage controller can also be divided into fire linkage control and non-fire linkage control unit.

[0199] The linkage controller can interact with the staff through the human-machine interface after processing the global information, and issue linkage control instructions to the main actuator based on the linkage control logic.

[0200] The main actuator is also divided into fire protection main actuator and non-fire protection main actuator.

[0201] The master actuator can execute the overall linkage control function based on the instructions, such as non-fire power supply / fire power supply control, elevator control, etc., and distribute the instructions to the actuators in each area to execute the linkage control instructions.

[0202] As the third-level closed-loop control circuit, the instructions issued by the master actuator have higher priority than those issued by the regional actuators, so as to facilitate overall optimal linkage control.

[0203] At the same time, the master control unit can also upload thermal imaging holographic real-time information (or processed information) to a higher-level control system. For example, it can be stored and distributed locally, stored, distributed and processed in the cloud, or sent to a municipal management center / command center / control center, etc.

[0204] In addition, manual intervention such as manual confirmation and manual control can also be used as training input for artificial intelligence algorithms.

[0205] In addition, virtual thermal imaging information can be generated and artificial intelligence algorithms can be trained for virtual control systems.

[0206] It should be understood that the closed-loop control levels of the above system can be adjusted in terms of the number of levels and / or connection methods according to the characteristics of the monitored space.

[0207] Compared with the existing monitoring system, the monitoring system of the embodiment of the present disclosure is based on thermal imaging holographic full-time information feedback and artificial intelligence / machine vision processing, and can truly realize an automatic control system based on control concepts that does not require human participation in the construction field.

[0208] The monitoring system according to the present invention has been described above in detail with reference to the accompanying drawings.

[0209] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the systems and methods according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0210] While various embodiments of the present invention have been described above, the foregoing description is intended to be illustrative, non-exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A thermal imaging monitoring and control system, comprising: At least one monitoring node, the at least one monitoring node including at least one first monitoring node, the at least one first monitoring node being distributed within the monitoring area, each first monitoring node including a first thermal imaging camera, the first thermal imaging camera being arranged at a first predetermined height above the ground, with an optical axis substantially perpendicular to the ground, so as to thermally image a corresponding imaging area on the ground, wherein each pixel area of ​​the obtained thermal image corresponds to each spatial area in the imaging area; The control system acquires a thermal image from the at least one first monitoring node and analyzes the acquired thermal image to identify temperature-related information therein.

2. The thermal imaging monitoring and control system according to claim 1, wherein: There are multiple first monitoring nodes. Imaging areas of the first thermal imaging cameras of adjacent first monitoring nodes are adjacent to or overlap each other; The imaging areas of the first thermal imaging cameras of the multiple first monitoring nodes basically cover the ground of the monitoring area.

3. The thermal imaging monitoring and control system according to claim 1, wherein: The first thermal imaging camera is a thermal imaging camera with a wide field of view or a larger field of view, so as to expand the monitoring range of a single first monitoring node; and / or The first thermal imaging camera is a thermal imaging camera capable of identifying the category of an imaging object rather than identifying an individual based on hardware and / or software design; and / or The first thermal imaging camera includes at least one of a LWIR wavelength range thermal imaging sensor, a MWIR wavelength range thermal imaging sensor, a SWIR wavelength range thermal imaging sensor, a NIR wavelength range thermal imaging sensor, and a FIR wavelength range thermal imaging sensor.

4. The thermal imaging monitoring and control system according to claim 1, wherein: The control system performs at least one of the following functions: Identifying hot spots in the thermal image; Identifying a type of an imaging object in a hot spot area in the thermal image, where the type of the imaging object in the hot spot area includes at least one of fire, a living being, a vehicle, a robot, and an instrument; Based on the position of the first monitoring node corresponding to the thermal imaging image in the monitoring area and the relative position of the hot spot area in the thermal imaging image, the spatial position corresponding to the hot spot area is located.

5. The thermal imaging monitoring and control system according to claim 1, further comprising: Artificial intelligence model systems or machine vision processing systems are used to analyze and process thermal images of individual areas and entire buildings.

6. The thermal imaging monitoring and control system according to claim 5, wherein: Perform training of the artificial intelligence model system in the cloud; and / or The artificial intelligence model system is trained for different buildings and / or different cities or countries; and / or The artificial intelligence model system is trained based on building categories and / or human activity categories; and / or The artificial intelligence model system is trained based on the building category and / or human activity category and / or city or country information corresponding to the monitoring area targeted by the control system, and the corresponding training results are sent to the corresponding control system.

7. The thermal imaging monitoring and control system according to claim 5, wherein: The control system is a closed-loop automatic control system based on thermal imaging feedback and artificial intelligence or machine vision processing and control.

8. The thermal imaging monitoring and control system according to claim 7, wherein: The artificial intelligence model system or the machine vision processing system sends a linkage control command to the corresponding actuator according to the result of analyzing the thermal image. The artificial intelligence model system is trained using the linkage control results of the actuator on the linkage control command as training samples.

9. The thermal imaging monitoring and control system according to claim 5, wherein: The first monitoring node also includes a fire monitoring device, which includes a smoke detector and / or a temperature sensor. Among them, through artificial intelligence model system or machine vision processing system or manual analysis, combined with the detection results of smoke detectors and / or temperature sensors, fire warning is issued to the monitored area, realizing very early, early, mid-term and late full-feature fire alarms at all times, and / or At least part of the training samples used to train the artificial intelligence model system are obtained by combining the thermal imaging images from the first thermal imaging camera of each first monitoring node and the detection results of the fire monitoring equipment.

10. The thermal imaging monitoring and control system according to claim 1, wherein: The plurality of monitoring nodes further include a second monitoring node, comprising a second thermal imaging camera, the second thermal imaging camera being arranged at a second predetermined height from the ground, with an optical axis substantially parallel to the ground, so as to thermally image a corresponding spatial area within the monitoring area, the second predetermined height being lower than the first predetermined height, The control system combines the thermal imaging images from the first thermal imaging camera and the second thermal imaging camera for analysis.

11. The thermal imaging monitoring and control system according to claim 1, wherein: Calculation is performed in combination with the thermal imaging image obtained by the first thermal imaging camera, the first predetermined height, and the size information of the spatial area corresponding to the corresponding imaging area to obtain accurate position information of the imaging object in the thermal imaging image in the spatial area.

12. The thermal imaging monitoring and control system according to claim 1, wherein: The plurality of monitoring nodes further includes a third monitoring node including a third thermal imaging camera, The third thermal imaging camera is arranged in a designated monitoring area to perform thermal imaging of the designated monitoring area, and / or the optical axis of the third thermal imaging camera points to a designated monitoring target to perform thermal imaging of the designated monitoring target.

13. The thermal imaging monitoring and control system according to any one of claims 1 to 12, wherein: The monitoring node also includes a fire emergency equipment control device, which controls the corresponding fire emergency equipment to perform emergency functions in response to the control system identifying a fire risk based on at least one of the fire risk point, the location of personnel at the fire scene, and the spatial layout of the monitoring area.

14. The thermal imaging monitoring and control system according to claim 13, wherein: The fire emergency equipment includes fire emergency signs and / or fire broadcasts. In response to identifying a fire risk, the fire emergency equipment control device controls the fire emergency equipment to dynamically change the content of the local emergency signs and local broadcasts, and optimizes the design from an architectural design perspective to better guide on-site personnel evacuation through artificial intelligence or manual guidance; and / or The fire emergency equipment includes a fire sprinkler. In response to identifying a fire risk, the fire emergency equipment control device controls the fire sprinkler to perform a sprinkler fire extinguishing operation through artificial intelligence or manual remote instructions.

15. The thermal imaging monitoring and control system according to any one of claims 1 to 12, wherein: The control system provides on-site feedback information to the central air-conditioning system based on the analysis results of the acquired thermal images so that the central air-conditioning system can make dynamic adjustments; and / or The control system provides on-site feedback information to the lighting system based on the analysis results of the acquired thermal imaging image, so that the lighting system can perform lighting control; and / or The control system provides positioning and tracking information to support active positioning and navigation of living organisms, vehicles or robots; and / or The control system provides the on-site personnel's body temperature information and positioning information to the epidemic prevention system based on the analysis results of the acquired thermal imaging images.

16. The thermal imaging monitoring and control system according to any one of claims 1 to 12, wherein: The thermal imaging camera calibrates the camera itself for temperature accuracy and precision based on the temperature constancy of the human body, based on the hot spot areas corresponding to the human body identified in the thermal imaging image.

17. The thermal imaging monitoring and control system according to any one of claims 1 to 12, comprising: at least one regional control unit, connected to at least one monitoring node among the plurality of monitoring nodes via a serial bus; The main control unit, the regional control units are connected to the main control unit via Ethernet or wireless network, and the main control unit uploads the fire alarm information to the fire alarm information demander or exchanges information with it through the fire linkage controller.

18. The thermal imaging monitoring and control system according to any one of claims 1 to 12, wherein: The first monitoring node also includes: a communication device for communicating with the control system; a local control unit, responsive to instructions from the control system, to adjust a thermal imaging frame rate of the thermal imaging camera and / or a frame rate of the thermal image sent to the communication system; and An image processing device is configured to perform image processing on a thermal image from a thermal imaging camera in the first monitoring node, wherein the image processing includes at least one of the following: Image compression processing; Identify hotspot areas; Crop the thermal image to retain only the image of the identified hotspot area, so as to reduce the communication volume required for the communication system to transmit the thermal image; Filter the thermal image to remove pixels whose corresponding temperature is lower than the set temperature threshold; Performing artificial intelligence processing on the thermal image and sending only the processed information to the control system, thereby reducing the communication volume required for transmitting the thermal image by the communication system and / or improving compatibility with existing fire protection systems; The thermal image is processed using artificial intelligence, and the identified humans are further generalized so that the human image is replaced with a static graphic, and only the processed information is sent to the control system, thereby addressing privacy concerns in specific areas.

19. The thermal imaging monitoring and control system according to any one of claims 1 to 12, wherein: The monitoring area includes a plurality of monitoring zones, each monitoring zone includes a plurality of monitoring units, and each monitoring unit is provided with at least one first monitoring node; The control system includes a general control unit corresponding to the monitoring area, a plurality of regional control units corresponding to the plurality of monitoring zones, and a plurality of unit controllers corresponding to the plurality of monitoring units. The thermal imaging monitoring and control system further includes an execution system, the execution system including a master actuator corresponding to the monitoring area, a plurality of area actuators corresponding to the plurality of monitoring zones, and a plurality of unit actuators corresponding to the plurality of monitoring units, the unit actuators being configured to execute their functions in the corresponding monitoring units; The unit controller receives a thermal image from a first monitoring node provided in a corresponding monitoring unit, and sends a unit control instruction to a unit actuator corresponding to the monitoring unit based on a primary analysis and processing result of the thermal image, so that the unit actuator performs its function in the corresponding monitoring unit according to the unit control instruction; The unit controller also sends the received thermal image and / or its primary analysis and processing results to the regional actuator corresponding to the monitoring zone to which the corresponding monitoring unit belongs; The regional control unit obtains a secondary analysis and processing result based on the received thermal image and / or the primary analysis and processing result, and sends a regional control instruction to the corresponding regional actuator based on the secondary processing result, so that the regional actuator distributes the control instruction to the corresponding multiple unit actuators; The regional control unit also sends the received thermal imaging image and / or the received primary analysis and / or secondary analysis results to the overall control unit; The general control unit obtains a tertiary analysis and processing result based on the thermal imaging image and / or the primary analysis and processing result and / or the secondary analysis and processing result it receives, and sends a general control instruction to the general executor based on the tertiary processing result, so that the general executor distributes the control instruction to each unit executor via the regional executor.

20. The thermal imaging monitoring and control system according to any one of claims 1 to 12, wherein: The control system and / or the monitoring node dynamically adjusts the working, sleeping and shutdown time and / or time ratio of the control system and / or the monitoring node based on manual or artificial intelligence processing analysis results to evaluate the work busy and idle periods and / or frame rate requirements.

Images