A fire detection method, device, terminal equipment and storage medium

Abstract

The present application is applicable to the field of fireworks detection technology, and provides a fireworks detection method, device, terminal equipment and storage medium, the method comprising: obtaining visible light images and infrared remote sensing data at multiple moments within a preset continuous time period; performing fireworks target detection on the visible light images at each moment respectively, and obtaining suspected fireworks areas in the visible light images at each moment; selecting a marked frame image whose temperature value of the suspected fireworks area is greater than a preset temperature threshold from the visible light images at multiple moments according to the temperature value of the suspected fireworks area in the visible light images at each moment; counting the cumulative number of marked frame images within a preset continuous time period; when the cumulative number of marked frame images reaches a preset number, determining that fireworks appear in the target detection area. By simultaneously obtaining visible light images and infrared remote sensing data, and detecting the suspected fireworks area obtained by visible light image detection and then performing temperature detection on it, the accuracy and efficiency of fireworks detection can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of fire and smoke detection, and in particular, relates to a fire and smoke detection method, device, terminal equipment and storage medium. Background Art

[0002] At present, smoke and fire detection technology can monitor and warn potential fire hazards in real time, so as to timely discover and take measures in the early stages of fire, effectively reducing the losses caused by fire. Traditional smoke and fire detection methods rely on smoke sensors and fire prevention devices such as temperature and light intensity sensors. These devices can only trigger alarms when the smoke concentration reaches a high level, which cannot meet the needs of outdoor, large-scale, and real-time monitoring.

[0003] Although the current fireworks detection method based on image recognition can achieve real-time monitoring of a large range and multiple scenes, due to the physical characteristics of the fireworks target itself, which are changeable in color and shape, this method cannot accurately detect the characteristics of the fireworks target, resulting in low detection accuracy. Moreover, in the fireworks detection method based on image recognition, a monitoring device is usually used to collect visible light images, and then an infrared data acquisition device is called to perform temperature detection, so that the visible light image and infrared data cannot be accurately registered, resulting in low fireworks detection accuracy. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, system and terminal device for detecting fireworks, which can solve the problem of low accuracy of fireworks detection in existing fireworks detection methods due to the inability to accurately align visible light images and infrared data.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting smoke and fire, the method comprising:

[0006] Obtaining visible light images and infrared remote sensing data at multiple moments within a preset continuous time period, wherein the visible light image at each moment is used to characterize the visible light information of the target detection area at each moment, and the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment, and the visible light image and the infrared remote sensing data are obtained simultaneously by the same terminal device;

[0007] Performing fireworks target detection on the visible light image at each moment respectively, to obtain a suspected fireworks region in the visible light image at each moment, wherein the suspected fireworks region carries suspected fireworks region coordinate information, and the suspected fireworks region coordinate information is used to indicate a position of the suspected fireworks region in the visible light image;

[0008] According to the temperature value of the suspected fireworks area in the visible light image at each moment, a marked frame image in which the temperature value of the suspected fireworks area is greater than a preset temperature threshold is selected from the visible light images at multiple moments, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area;

[0009] Counting the cumulative number of the marked frame images within the preset continuous time period;

[0010] When the accumulated number of the marked frame images reaches a preset number, it is determined that fireworks appear in the target detection area.

[0011] In a possible implementation manner of the first aspect, before determining the temperature value of the suspected fireworks area from the infrared remote sensing data at the moment according to the coordinate information of the suspected fireworks area, the method includes:

[0012] Performing image conversion on the infrared remote sensing data at each moment to obtain an infrared image corresponding to the infrared remote sensing data at each moment;

[0013] The infrared image and the visible light image at each moment are registered with each other by calculating registration parameters, and the dual-light registration parameters of the infrared image and the visible light image are determined, so that the infrared image and the visible light image at each moment are aligned.

[0014] In a possible implementation manner of the first aspect, calculating registration parameters of the infrared image and the visible light image at each moment, and determining bi-optical registration parameters of the infrared image and the visible light image so as to align the infrared image and the visible light image at each moment, includes:

[0015] According to preset extraction conditions, key feature points in the visible light image and the infrared image are respectively extracted to obtain first feature points of the visible light image and second feature points of the infrared image;

[0016] Matching the first feature point and the second feature point to obtain a corresponding relationship between the first feature point and the second feature point;

[0017] Calculating an affine transformation matrix according to the correspondence between the first feature point and the second feature point;

[0018] The bi-optical registration parameters of the infrared image and the visible light image are determined according to the affine transformation matrix to align the infrared image with the visible light image.

[0019] In a possible implementation manner of the first aspect, determining the temperature value of the suspected fireworks area from the infrared remote sensing data at the moment according to the coordinate information of the suspected fireworks area includes:

[0020] According to the coordinate information of the suspected fireworks area and the dual-light registration parameters, the suspected fireworks area in the visible light image is mapped to the infrared image to obtain a mapping area of ​​the suspected fireworks area in the infrared image;

[0021] According to a preset infrared temperature measurement algorithm, the infrared remote sensing data corresponding to the mapping area is temperature-converted to obtain a temperature value of the suspected fireworks area.

[0022] In a possible implementation manner of the first aspect, performing fireworks target detection on the visible light image at each moment to obtain a suspected fireworks area in the visible light image at each moment includes:

[0023] Performing fireworks target detection on the visible light image at each moment respectively by using a fireworks target detection model to obtain a plurality of initial suspected fireworks regions in the visible light image at each moment, wherein the initial suspected fireworks regions include confidence levels;

[0024] According to the confidence being greater than a preset confidence threshold, the suspected fireworks region in the visible light image is screened out from the plurality of initial suspected fireworks regions.

[0025] In a possible implementation manner of the first aspect, before performing fireworks target detection on the visible light image at each moment by using a fireworks target detection model to obtain a plurality of initial suspected fireworks regions in the visible light image at each moment, the method further includes:

[0026] Acquire a training set, wherein the training set includes a plurality of sample images and target annotation results of the plurality of sample images;

[0027] An initial fireworks target detection model is trained using the multiple sample images and the target annotation results of the multiple sample images to obtain the fireworks target detection model.

[0028] In a second aspect, an embodiment of the present application provides a smoke and fire detection device, the device comprising:

[0029] An acquisition module, used to acquire visible light images and infrared remote sensing data at multiple moments in a preset continuous time period, wherein the visible light image at each moment is used to characterize the visible light information of the target detection area at each moment; and the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment;

[0030] an identification module, configured to perform fireworks target detection on the visible light image at each moment, respectively, to obtain a suspected fireworks region in the visible light image at each moment, wherein the suspected fireworks region carries suspected fireworks region coordinate information, and the suspected fireworks region coordinate information is used to indicate a position of the suspected fireworks region in the visible light image;

[0031] a marking module, configured to select, from the visible light images at multiple moments, a marked frame image in which the temperature value of the suspected fireworks area is greater than a preset temperature threshold, according to the temperature value of the suspected fireworks area in the visible light image at each moment, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area;

[0032] A statistics module, used for counting the cumulative number of the marked frame images within the preset continuous time period;

[0033] The determination module is used to determine that fireworks appear in the target detection area when the accumulated number of the marked frame images reaches a preset number.

[0034] In a third aspect, an embodiment of the present application provides a smoke and fire detection terminal device, the terminal device comprising: a visible light acquisition device, an infrared data acquisition device, an edge processor and an alarm device;

[0035] Wherein, the visible light acquisition device, the infrared data acquisition device, and the alarm device are respectively connected to the edge processor; the visible light acquisition device and the infrared data acquisition device are double-light parallel registration through the registration tooling;

[0036] The visible light acquisition device is used to acquire visible light images at multiple moments in a preset continuous time period, and send the visible light images at multiple moments to the edge processor, wherein the visible light image at each moment is used to represent the visible light information of the target detection area at each moment;

[0037] The infrared data acquisition device is used to collect infrared remote sensing data at multiple moments in the preset continuous time period, and send the infrared remote sensing data at multiple moments to the edge processor, wherein the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment;

[0038] The edge processor is used to receive the visible light images and the infrared remote sensing data at multiple moments within the preset continuous time period; and, respectively, perform fireworks target detection on the visible light images at each moment to obtain suspected fireworks areas in the visible light images at each moment, wherein the suspected fireworks areas carry suspected fireworks area coordinate information, and the suspected fireworks area coordinate information is used to indicate the position of the suspected fireworks area in the visible light image; and, based on the temperature value of the suspected fireworks area in the visible light image at each moment, select a marked frame image whose temperature value of the suspected fireworks area is greater than a preset temperature threshold from the visible light images at multiple moments, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment based on the suspected fireworks area coordinate information; and, count the cumulative number of the marked frame images within the preset continuous time period; and, when the cumulative number of the marked frame images reaches a preset number, determine that fireworks appear in the target detection area;

[0039] The alarm device is used to issue an alarm prompt when the edge processor determines that fireworks appear in the target detection area.

[0040] In a possible implementation manner of the third aspect, the terminal device further includes: a display device; wherein the display device is used to display the smoke and fire detection status of the target detection area, and to receive the alarm prompt.

[0041] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the smoke and fire detection method described in any one of the above is implemented.

[0042] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any of the above-mentioned smoke and fire detection methods.

[0043] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0044] The fireworks detection method provided by the embodiment of the present application obtains visible light images and infrared remote sensing data at multiple moments in a preset continuous time period, wherein the visible light image at each moment is used to characterize the visible light information of the target detection area at each moment, and the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment, and the visible light image and the infrared remote sensing data are simultaneously obtained by the same terminal device; fireworks target detection is performed on the visible light image at each moment to obtain the suspected fireworks area in the visible light image at each moment, wherein the suspected fireworks area carries the suspected fireworks area coordinate information, and the suspected fireworks area coordinate information is used to indicate the position of the suspected fireworks area in the visible light image; then, according to the temperature value of the suspected fireworks area in the visible light image at each moment, a marked frame image whose temperature value of the suspected fireworks area is greater than a preset temperature threshold is selected from the visible light images at multiple moments, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at the moment according to the suspected fireworks area coordinate information; finally, the cumulative number of marked frame images in the preset continuous time period is counted; when the cumulative number of marked frame images reaches the preset number, it is determined that fireworks appear in the target detection area. The method simultaneously acquires visible light images and infrared remote sensing data through the same terminal device, and first detects suspected fireworks areas on the visible light image and then performs temperature detection on the suspected fireworks areas, so that the visible light image and infrared data can be accurately aligned and the interference of light and background can be reduced, thereby improving the accuracy and efficiency of fireworks detection, while reducing costs and improving real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 It is a flowchart of a method for detecting fireworks provided in one embodiment of the present application;

[0047] Figure 2 is a schematic diagram of detecting fireworks targets in visible light images provided by an embodiment of the present application;

[0048] Figure 3 It is a structural schematic diagram of a smoke and fire detection device provided in one embodiment of the present application;

[0049] Figure 4 It is a structural schematic diagram of a smoke and fire detection terminal device provided in one embodiment of the present application;

[0050] Figure 5 It is a flowchart of a smoke and fire detection method provided in another embodiment of the present application. DETAILED DESCRIPTION

[0051] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0052] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0053] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0054] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0055] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0057] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for detecting fireworks provided in an embodiment of the present application. The method for detecting fireworks includes:

[0058] S11, obtaining visible light images and infrared remote sensing data at multiple moments in a preset continuous time period, wherein the visible light image at each moment is used to represent the visible light information of the target detection area at each moment, and the infrared remote sensing data at each moment is used to represent the temperature information of the target detection area at each moment, and the visible light image and the infrared remote sensing data are obtained simultaneously through the same terminal device;

[0059] S12, performing fireworks target detection on the visible light image at each moment, respectively, to obtain a suspected fireworks region in the visible light image at each moment, wherein the suspected fireworks region carries suspected fireworks region coordinate information, and the suspected fireworks region coordinate information is used to indicate a position of the suspected fireworks region in the visible light image;

[0060] S13, selecting, from the visible light images at multiple moments, a marked frame image in which the temperature value of the suspected fireworks area is greater than a preset temperature threshold, according to the temperature value of the suspected fireworks area in the visible light image at each moment, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area;

[0061] S14, counting the cumulative number of marked frame images within a preset continuous time period;

[0062] S15. When the accumulated number of marked frame images reaches a preset number, it is determined that fireworks appear in the target detection area.

[0063] It should be noted that, in this embodiment, the execution subject may be a terminal device integrated with an infrared and visible light acquisition module and a smoke and fire detection module, and no specific limitation is made to this.

[0064] It should be noted that in step S11, the visible light image is an image of the target detection area captured by the visible light acquisition device. The visible light image mainly reflects the information of the target detection area under the visible light band. Since fireworks usually have obvious characteristics under visible light (such as color change, brightness increase, etc.), the visible light image can be used to detect fireworks targets. The infrared remote sensing data is the infrared data of the target detection area captured by the infrared data acquisition device. The infrared remote sensing data mainly reflects the temperature information of the target detection area at different times. Since fireworks are accompanied by a significant increase in temperature when they occur, the infrared remote sensing data can be used to determine the temperature value of the target detection area to determine whether fireworks exist. The target detection area is a scene area where fireworks monitoring is required. The preset continuous time period is a pre-set continuous time range. Among them, the visible light image and the infrared remote sensing data are obtained simultaneously through the same terminal device, and the terminal device is integrated with infrared and visible light acquisition modules and fireworks detection modules. By continuously monitoring the target detection area, the visible light image and infrared remote sensing data are collected for a sufficiently long time to accurately determine whether fireworks occur.

[0065] In step S12, fireworks target detection is to process the visible light image through an image processing algorithm to identify suspected fireworks areas where fireworks may exist. In this embodiment, fireworks target detection is performed on the visible light image through a fireworks target detection model. The suspected fireworks area is an area that may contain fireworks after fireworks target identification is performed through the fireworks target detection model. The suspected fireworks area carries the suspected fireworks area coordinate information. The suspected fireworks area coordinate information is the position information of the suspected fireworks area, that is, it indicates the position of the suspected fireworks area in the visible light image, which is usually expressed in pixel coordinates.

[0066] In step S13, the temperature value of the suspected fireworks area represents the temperature of the suspected fireworks area at that moment, and is extracted from the infrared remote sensing data at the corresponding moment according to the coordinate information of the suspected fireworks area. The preset temperature threshold is a pre-set temperature value for judging whether fireworks exist. When the temperature value of the suspected fireworks area is greater than the preset temperature threshold, it indicates that the temperature of the suspected fireworks area at that moment is high, and the possibility of fireworks is high. The marked frame image is a visible light image in which the temperature value of the suspected fireworks area is greater than the preset temperature threshold.

[0067] The preset number is a preset threshold value of the number of marked frame images. When the cumulative number of the marked frame images counted in the preset continuous time period reaches the preset number, it can be determined that fireworks appear in the target detection area, triggering an alarm prompt.

[0068] Specifically, Figure 1As shown, first, data is acquired, that is, within a preset continuous time period, visible light images and infrared remote sensing data at multiple times are acquired through a visible light acquisition device and an infrared data acquisition device respectively; secondly, preliminary target screening is performed, that is, fireworks target detection is performed on the visible light image to identify the suspected fireworks area in the target detection area; then, secondary temperature screening is performed, that is, according to the suspected fireworks area coordinate information carried in the identified suspected fireworks area, the temperature value corresponding to the suspected fireworks area is extracted from the infrared remote sensing data, and the temperature value of the suspected fireworks area is screened, and the visible light image with a temperature value greater than a preset temperature threshold is marked as a marked frame image; finally, the number of marked frame images within the preset continuous time period is counted, and when the cumulative number of marked frame images reaches the preset number, it is determined that fireworks occur in the target detection area. Through preliminary target screening and secondary temperature screening, light and background interference are greatly reduced; at the same time, the real-time performance of the system is improved, and it can be easily and quickly deployed in various scenarios.

[0069] It can be understood that the method for detecting fireworks provided in the embodiment of the present application obtains visible light images and infrared remote sensing data at multiple moments in a preset continuous time period, wherein the visible light image at each moment is used to characterize the visible light information of the target detection area at each moment, and the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment, and the visible light image and the infrared remote sensing data are obtained simultaneously through the same terminal device; fireworks target detection is performed on the visible light image at each moment respectively, and a suspected fireworks area in the visible light image at each moment is obtained, wherein the suspected fireworks area carries suspected The coordinate information of the fireworks area and the coordinate information of the suspected fireworks area are used to indicate the position of the suspected fireworks area in the visible light image; then, according to the temperature value of the suspected fireworks area in the visible light image at each moment, the marked frame images whose temperature value of the suspected fireworks area is greater than the preset temperature threshold are selected from the visible light images at multiple moments, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area; finally, the cumulative number of marked frame images in a preset continuous time period is counted; when the cumulative number of marked frame images reaches the preset number, it is determined that fireworks appear in the target detection area. The method simultaneously obtains visible light images and infrared remote sensing data through the same terminal device, and first detects the suspected fireworks area on the visible light image and then performs temperature detection on the suspected fireworks area, so that the visible light image and the infrared data can be accurately registered and the interference of light and background is reduced, thereby improving the accuracy and efficiency of fireworks detection, while reducing costs and improving real-time performance.

[0070] In a possible implementation, before determining the temperature value of the suspected fireworks area from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area, the fireworks detection method includes:

[0071] Perform image conversion on the infrared remote sensing data at each moment to obtain the infrared image corresponding to the infrared remote sensing data at each moment;

[0072] The infrared image and the visible light image at each moment are registered with each other by calculating the registration parameters, and the dual-light registration parameters of the infrared image and the visible light image are determined, so that the infrared image and the visible light image at each moment are aligned.

[0073] It should be noted that infrared remote sensing data is infrared data of the target detection area captured by an infrared data acquisition device. Infrared remote sensing data mainly reflects the temperature information of the target detection area at different times. Infrared image is an image obtained by image conversion of infrared remote sensing data. Infrared image mainly reflects the temperature distribution of the target detection area. In different infrared images, areas with different temperatures are usually represented by different colors or brightness. Among them, the process of converting infrared remote sensing data into infrared image can be: converting infrared radiation in infrared remote sensing data collected by infrared data acquisition device into electrical signal; amplifying and filtering the electrical signal to obtain analog signal; converting analog signal into digital signal and performing error correction; arranging the corrected data to obtain infrared image. It can be understood that the invisible infrared radiation is converted into visible infrared image to provide visual information for subsequent use.

[0074] The bi-optical registration parameters are used to align the infrared image with the visible light image, and usually include transformation relationships such as translation, rotation, and scaling. The bi-optical registration parameters are obtained by calculating the registration parameters of the infrared image and the visible light image at each moment, so that the infrared image and the visible light image are spatially aligned, that is, the same object or area in the two images is consistent in position.

[0075] It can be understood that by registering the visible light image and the infrared image so that the infrared image is aligned with the visible light image, the temperature value of the corresponding area can be accurately extracted from the infrared image based on the coordinate information of the suspected fireworks area in the visible light image, thereby providing help for more accurately judging whether there are actually fireworks in the suspected fireworks area.

[0076] In a possible implementation, the infrared image and the visible light image at each moment are registered with each other by calculating the registration parameters, and determining the dual-light registration parameters of the infrared image and the visible light image, so as to align the infrared image and the visible light image at each moment, including:

[0077] According to preset extraction conditions, key feature points in the visible light image and the infrared image are extracted respectively to obtain first feature points of the visible light image and second feature points of the infrared image;

[0078] Matching the first feature point and the second feature point to obtain a corresponding relationship between the first feature point and the second feature point;

[0079] According to the correspondence between the first feature point and the second feature point, an affine transformation matrix is ​​calculated;

[0080] According to the affine transformation matrix, bi-optical registration parameters of the infrared image and the visible light image are determined to align the infrared image with the visible light image.

[0081] It should be noted that the preset extraction conditions are a series of conditions and parameters pre-set according to the actual scene and image features when extracting feature points. Key feature points are points in the image that are unique and stable. Key feature points can be corner points, edge points or texture-rich areas, etc. The first feature point is a point with significant features extracted from the visible light image according to the preset extraction conditions; the second feature point is a point with significant features extracted from the infrared image according to the preset extraction conditions. The affine transformation matrix is ​​a two-dimensional coordinate transformation matrix used to describe the transformation relationship between images, such as translation, rotation, and scaling. The matrix can maintain the linearity and parallelism of the image. The dual-light registration parameters are parameters used to align the infrared image with the visible light image, usually including the transformation relationship such as translation, rotation, and scaling, which can be obtained through the affine transformation matrix.

[0082] Specifically, according to preset extraction conditions, key feature points in the visible light image and key feature points in the infrared image are extracted respectively to obtain first feature points and second feature points; the extracted first feature points and second feature points are matched to obtain a correspondence between the matched feature points, and then an affine transformation matrix is ​​calculated based on the correspondence; then, based on the affine transformation matrix, dual-light registration parameters of the infrared image and the visible light image are determined, and then the dual-light registration parameters are used to align the infrared image with the visible light image.

[0083] In a possible implementation, determining the temperature value of the suspected fireworks area from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area includes:

[0084] According to the coordinate information of the suspected fireworks area and the dual-light registration parameters, the suspected fireworks area in the visible light image is mapped to the infrared image to obtain the mapping area of ​​the suspected fireworks area in the infrared image;

[0085] According to the preset infrared temperature measurement algorithm, the infrared remote sensing data corresponding to the mapping area is converted into temperature to obtain the temperature value of the suspected fireworks area.

[0086] It should be noted that the coordinate information of the suspected fireworks area is the location information of the suspected fireworks area, that is, it indicates the location of the suspected fireworks area in the visible light image. The dual-light registration parameters are parameters used to align the infrared image with the visible light image, usually including transformation relationships such as translation, rotation, and scaling. The mapping area is the area in which the suspected fireworks area is mapped from the visible light image to the corresponding infrared image based on the coordinate information of the suspected fireworks area and the dual-light registration parameters. Infrared remote sensing data is data collected by an infrared data acquisition device. Infrared remote sensing data can represent the temperature information of the target detection area. The preset infrared temperature measurement algorithm is an algorithm that converts infrared remote sensing data into temperature values. Its core is the Stefan-Boltzmann law, which describes the relationship between the infrared radiation emitted by an object and its temperature.

[0087] Specifically, according to the coordinate information of the suspected fireworks area in the visible light image and the dual-light registration parameters, the suspected fireworks area is mapped to the infrared image, that is, the pixel coordinates in the visible light image are converted to the corresponding coordinates in the infrared image, so as to obtain the mapping area of ​​the suspected fireworks area in the infrared image. Then, through the preset infrared temperature measurement algorithm, the infrared radiation intensity in the infrared remote sensing data corresponding to the mapping area is converted into a temperature value to obtain the temperature value of the suspected fireworks area.

[0088] It should be noted that infrared remote sensing data represents the infrared radiation intensity of an object, which is related to its temperature. The infrared radiation intensity can be converted into temperature according to the Stefan-Boltzmann law; this temperature is an absolute temperature, which is then converted into Celsius or Fahrenheit to obtain the temperature value of the suspected fireworks area.

[0089] It can be understood that the above method can accurately obtain the temperature value of the suspected fireworks area from the infrared image, which is crucial for further determining whether there are really fireworks in the suspected fireworks area.

[0090] In a possible implementation, fireworks target detection is performed on the visible light image at each moment to obtain the suspected fireworks area in the visible light image at each moment, including:

[0091] Performing fireworks target detection on the visible light image at each moment respectively through the fireworks target detection model, and obtaining a plurality of initial suspected fireworks regions in the visible light image at each moment, wherein the initial suspected fireworks regions include confidence levels;

[0092] According to the confidence being greater than a preset confidence threshold, a suspected fireworks region in the visible light image is screened out from the multiple initial suspected fireworks regions.

[0093] It should be noted that the fireworks target detection model is a deep learning model based on a convolutional neural network, which is used to detect fireworks targets from visible light images. To ensure the accuracy of fireworks detection, the fireworks target detection model is trained by annotating fireworks targets with a large number of visible light image samples. Moreover, the model can be calculated in real time in an edge processor after lightweight design and compression. The initial suspected fireworks area is detected by the fireworks target detection model. The initial suspected fireworks area may contain fireworks targets, which are usually represented in the form of rectangular boxes, including confidence. The confidence represents the numerical value of the fireworks target detection model's degree of certainty that the initial suspected fireworks area contains fireworks targets. Generally, the higher the confidence, the greater the probability that the fireworks target detection model believes that the suspected fireworks area contains fireworks targets. The preset confidence threshold is a preset confidence value for screening suspected fireworks areas. When the confidence of the initial suspected fireworks area is greater than the preset confidence threshold, the area is determined to be a suspected fireworks area.

[0094] It should be noted that if Figure 2 As shown, Figure 2 FIG. 1 is a schematic diagram of a method for detecting fireworks targets in a visible light image provided by an embodiment of the present application. Figure 2 As shown in , the collected visible light image is generally scaled first, and then input into the convolution model in the fireworks target detection model after normalization processing; the convolution model performs forward reasoning, and the output layer obtains three output feature maps of different scales, and each pixel position of each output feature map corresponds to the detection box information with this point as the center point, and each detection box information includes category confidence, coordinate information, etc.; then, non-maximum suppression filtering, that is, NMS post-processing, is performed, that is, setting the initial confidence threshold and NMS threshold to screen all detection boxes, and obtain non-overlapping detection results that meet the initial confidence threshold, that is, the initial suspected fireworks area; finally, the initial suspected fireworks area is confidence screened again, that is, according to the confidence greater than the preset confidence threshold, the suspected fireworks area in the visible light image is screened out from multiple initial suspected fireworks areas.

[0095] It can be understood that by performing fireworks target detection on the visible light image through the fireworks target detection model, multiple initial suspected fireworks areas in the visible light image can be obtained. Then, the confidence of each initial suspected fireworks area is compared with the preset confidence threshold, and the initial suspected fireworks area with a confidence greater than the preset confidence threshold is taken as the suspected fireworks area. By screening the preliminary detection results according to the confidence, the false alarm rate of the detection can be reduced and the accuracy and reliability of the detection can be improved.

[0096] In a possible implementation, before performing fireworks target detection on the visible light image at each moment by using the fireworks target detection model to obtain a plurality of initial suspected fireworks regions in the visible light image at each moment, the fireworks detection method further includes:

[0097] Obtaining a training set, wherein the training set includes a plurality of sample images and target annotation results of the plurality of sample images;

[0098] An initial fireworks target detection model is trained using multiple sample images and target annotation results of the multiple sample images to obtain a fireworks target detection model.

[0099] It should be noted that, in this embodiment, before performing fireworks target detection on the visible light image at each moment respectively through the fireworks target detection model, it is necessary to perform model training on the initial fireworks target detection model. The initial fireworks target detection model is a newly constructed fireworks target detection model that has not been trained. The training set is a data set used to train the initial fireworks target detection model, which includes known input features and expected output labels, that is, multiple sample images and target annotation results of multiple sample images. The sample image is a visible light image that contains or does not contain fireworks targets; the target annotation results of the sample image are the results of annotating the fireworks target areas in these sample images, that is, the suspected fireworks areas. The fireworks target detection model is a model that can accurately detect fireworks targets in visible light images after model training.

[0100] It can be understood that by training the initial fireworks target detection model, the model can accurately predict or classify new data and can accurately detect and identify fireworks targets in visible light images.

[0101] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0102] Corresponding to a method for detecting fireworks in the above embodiment, Figure 3 A schematic structural diagram of a smoke and fire detection device provided in an embodiment of the present application is shown. For ease of description, only the parts related to the embodiment of the present application are shown.

[0103] Reference Figure 3 , the smoke and fire detection device 3 of this embodiment comprises:

[0104] An acquisition module 31 is used to acquire visible light images and infrared remote sensing data at multiple moments in a preset continuous time period, wherein the visible light image at each moment is used to represent the visible light information of the target detection area at each moment, and the infrared remote sensing data at each moment is used to represent the temperature information of the target detection area at each moment, and the visible light image and the infrared remote sensing data are simultaneously acquired by the same terminal device;

[0105] The identification module 32 is used to perform fireworks target detection on the visible light image at each moment, and obtain the suspected fireworks region in the visible light image at each moment, wherein the suspected fireworks region carries the suspected fireworks region coordinate information, and the suspected fireworks region coordinate information is used to indicate the position of the suspected fireworks region in the visible light image;

[0106] The marking module 33 is used to select a marked frame image whose temperature value of the suspected fireworks area is greater than a preset temperature threshold from the visible light images at multiple moments according to the temperature value of the suspected fireworks area in the visible light image at each moment, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area;

[0107] A statistics module 34, used to count the cumulative number of marked frame images within a preset continuous time period;

[0108] The determination module 35 is used to determine that fireworks appear in the target detection area when the accumulated number of marked frame images reaches a preset number.

[0109] It can be understood that in this embodiment, the fire and smoke detection device 3 provided in the embodiment of the present application, first, obtains the visible light images and infrared remote sensing data at multiple moments in a preset continuous time period through the acquisition module 31, wherein the visible light image at each moment is used to characterize the visible light information of the target detection area at each moment, and the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment, and the visible light image and the infrared remote sensing data are obtained simultaneously through the same terminal device; secondly, the recognition module 32 performs fire and smoke target detection on the visible light image at each moment, and obtains the suspected fire and smoke area in the visible light image at each moment, wherein the suspected fire and smoke area carries There is coordinate information of suspected fireworks area, and the coordinate information of suspected fireworks area is used to indicate the position of the suspected fireworks area in the visible light image; then, the marking module 33 selects the marked frame images whose temperature value of the suspected fireworks area is greater than the preset temperature threshold from the visible light images at multiple moments according to the temperature value of the suspected fireworks area in the visible light image at each moment, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area; finally, the statistical module 34 counts the cumulative number of marked frame images in a preset continuous time period; when the cumulative number of marked frame images reaches the preset number, the determination module 35 determines that fireworks appear in the target detection area. By simultaneously acquiring visible light images and infrared remote sensing data, and first detecting the suspected fireworks area on the visible light image and then performing temperature detection on the suspected fireworks area, the visible light image and the infrared data can be accurately registered and the interference of light and background can be reduced, thereby improving the accuracy and efficiency of fireworks detection, while reducing costs and improving real-time performance.

[0110] In a possible implementation, the smoke detection device 3 includes:

[0111] The infrared remote sensing data conversion module is used to convert the infrared remote sensing data at each moment into an image to obtain an infrared image corresponding to the infrared remote sensing data at each moment;

[0112] The dual-light registration parameter determination module is used to calculate the registration parameters of the infrared image and the visible light image at each moment, and determine the dual-light registration parameters of the infrared image and the visible light image to align the infrared image and the visible light image at each moment.

[0113] In a possible implementation, the bi-optical registration parameter determination module is specifically used to:

[0114] According to preset extraction conditions, key feature points in the visible light image and the infrared image are extracted respectively to obtain first feature points of the visible light image and second feature points of the infrared image;

[0115] Matching the first feature point and the second feature point to obtain a corresponding relationship between the first feature point and the second feature point;

[0116] According to the correspondence between the first feature point and the second feature point, an affine transformation matrix is ​​calculated;

[0117] According to the affine transformation matrix, bi-optical registration parameters of the infrared image and the visible light image are determined to align the infrared image with the visible light image.

[0118] In a possible implementation, the smoke and fire detection device 3 includes a temperature value determination module, which is specifically used to:

[0119] According to the coordinate information of the suspected fireworks area and the dual-light registration parameters, the suspected fireworks area in the visible light image is mapped to the infrared image to obtain the mapping area of ​​the suspected fireworks area in the infrared image;

[0120] According to the preset infrared temperature measurement algorithm, the infrared remote sensing data corresponding to the mapping area is converted into temperature to obtain the temperature value of the suspected fireworks area.

[0121] In a possible implementation, the identification module 32 includes:

[0122] An initial suspected fireworks area identification unit is used to perform fireworks target detection on the visible light image at each moment by using a fireworks target detection model to obtain a plurality of initial suspected fireworks areas in the visible light image at each moment, wherein the initial suspected fireworks areas include confidence levels;

[0123] The suspected fireworks area determining unit is used to screen out the suspected fireworks area in the visible light image from the multiple initial suspected fireworks areas according to the confidence being greater than a preset confidence threshold.

[0124] In a possible implementation, the smoke and fire detection device 3 further includes a smoke and fire target detection model optimization module, and the smoke and fire target detection model optimization module is specifically used to:

[0125] Obtaining a training set, wherein the training set includes a plurality of sample images and target annotation results of the plurality of sample images;

[0126] An initial fireworks target detection model is trained using multiple sample images and target annotation results of the multiple sample images to obtain a fireworks target detection model.

[0127] It should be noted that, since the information interaction, execution process and other contents between the modules in the above-mentioned smoke and fire detection device 3 are based on the same concept as the method embodiment of the present application, their specific functions and technical effects can be found in the method embodiment part, and will not be repeated here.

[0128] The present application also provides a smoke and fire detection system. Figure 4 As shown, Figure 4 Schematic diagram of a smoke and fire detection terminal device provided in one embodiment of the present application. Figure 4 , the smoke and fire detection terminal device 4 includes: a visible light acquisition device 41, an infrared data acquisition device 42, an edge processor 43 and an alarm device 44;

[0129] Among them, the visible light acquisition device 41, the infrared data acquisition device 42, and the alarm device 44 are respectively connected to the edge processor 43; the visible light acquisition device 41 and the infrared data acquisition device 42 are double-light parallel registration through the registration tooling;

[0130] The visible light acquisition device 41 is used to acquire visible light images at multiple moments within a preset continuous time period, and send the visible light images at multiple moments to the edge processor 43, wherein the visible light image at each moment is used to represent the visible light information of the target detection area at each moment;

[0131] The infrared data acquisition device 42 is used to collect infrared remote sensing data at multiple moments in a preset continuous time period, and send the infrared remote sensing data at multiple moments to the edge processor 43, wherein the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment;

[0132] The edge processor 43 is used to receive visible light images and infrared remote sensing data at multiple moments within a preset continuous time period; and, respectively, perform fireworks target detection on the visible light image at each moment to obtain a suspected fireworks area in the visible light image at each moment, wherein the suspected fireworks area carries suspected fireworks area coordinate information, and the suspected fireworks area coordinate information is used to indicate the position of the suspected fireworks area in the visible light image; and, according to the temperature value of the suspected fireworks area in the visible light image at each moment, select a marked frame image whose temperature value of the suspected fireworks area is greater than a preset temperature threshold from the visible light images at multiple moments, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the suspected fireworks area coordinate information; and, count the cumulative number of marked frame images within the preset continuous time period; and, when the cumulative number of marked frame images reaches a preset number, determine that fireworks appear in the target detection area;

[0133] The alarm device 44 is used to issue an alarm prompt when the edge processor 43 determines that fireworks appear in the target detection area.

[0134] It should be noted that the fire and smoke detection terminal device 4 includes a visible light acquisition device 41, an infrared data acquisition device 42, an edge processor 43 and an alarm device 44, and the visible light acquisition device 41, the infrared data acquisition device 42 and the alarm device 44 are respectively connected to the edge processor 43. The visible light acquisition device 41 is responsible for collecting visible light images at multiple times within a preset continuous time period. The visible light image is the visible light information of the target detection area at each time, which is used for subsequent detection of fire and smoke targets. The visible light acquisition device 41 can be a visible light camera with high resolution to ensure that the collected visible light image is clear. The infrared data acquisition device 42 is responsible for collecting infrared remote sensing data at multiple times within a preset continuous time period. The infrared remote sensing data is the temperature information of the target detection area at each time, providing temperature data for fire and smoke target detection. The infrared data acquisition device 42 can be an infrared camera or an infrared sensor, which can accurately detect the temperature of objects in the target detection area to accurately determine the source of the fire. The edge processor 43 is a high-performance edge computing device with sufficient computing power to run deep learning algorithms to achieve real-time fire and smoke detection. The edge processor 43 is used to receive the visible light image sent by the visible light acquisition device 41 and the infrared remote sensing data sent by the infrared data acquisition device 42, and process the visible light image and the infrared remote sensing data to determine whether fireworks appear in the target detection area. The alarm device 44 is a device that issues an alarm prompt when fireworks appear in the target detection area. The alarm device 44 can be a sound alarm, a light alarm, or a wireless signal transmitter.

[0135] It should be noted that in the fire and smoke detection terminal device 4, the visible light acquisition device 41 and the infrared data acquisition device 42 need to be aligned in parallel by an alignment tool to enable accurate coordinate conversion. Usually, the alignment operation is only required when the fire and smoke detection terminal device 4 is initially assembled or installed at a fixed point.

[0136] It should be noted that the edge processor 43 involves multiple modules such as a fireworks target detection model, an infrared temperature measurement module, and a comprehensive analysis module in the process of processing visible light images and infrared remote sensing data to determine whether fireworks appear in the target detection area. Among them, the fireworks target detection model is a deep learning model based on a convolutional neural network, which is used to detect fireworks targets from visible light images. To ensure the accuracy of fireworks detection, the fireworks target detection model is obtained by training a large number of visible light image samples for fireworks annotation. Moreover, the model can be calculated in real time in the edge processor after lightweight design and compression. The core of the infrared temperature measurement module is to use the Stefan-Boltzmann law to describe the relationship between the infrared radiation emitted by an object and its temperature. The infrared temperature measurement module can accurately identify the temperature of each object in the target detection area. The comprehensive analysis module comprehensively analyzes the suspected fireworks area detected by the fireworks target detection model and the temperature detected by the infrared temperature measurement module, and comprehensively judges whether there is fireworks within a preset time period. This module can handle uncertainty and false alarms and improve the accuracy of detection.

[0137] See also Figure 5 , Figure 5 FIG. 1 is a flow chart of a method for detecting fireworks provided by another embodiment of the present application. Figure 5 In the embodiment, the specific process of performing fireworks detection by the fireworks detection terminal device 4 is as follows: first, within a preset continuous time period, the visible light acquisition device 41 acquires visible light images at multiple moments, and the infrared remote sensing data at multiple moments are acquired by the infrared data acquisition device 42, and the visible light images and infrared remote sensing data are sent to the edge processor 43; then, the edge processor 43 performs fireworks target detection on the visible light image through the fireworks target detection model, and identifies the suspected fireworks area in the target detection area; the infrared remote sensing data is converted into an infrared image, and the infrared image at each moment is converted into an infrared image. Calculate the registration parameters with the visible light image, map the suspected fireworks area to obtain the mapping area; extract the temperature value corresponding to the suspected fireworks area from the infrared remote sensing data according to the mapping area, and perform temperature screening on the temperature value of the suspected fireworks area, and mark the visible light image with the temperature value of the suspected fireworks area greater than the preset temperature threshold as the marked frame image; finally, determine the continuity of the fireworks, that is, count the number of marked frame images within a preset continuous time period, and when the cumulative number of marked frame images reaches the preset number, it is determined that fireworks have occurred in the target detection area, and an alarm is issued through the alarm device 44.

[0138] It can be understood that in this embodiment, the fireworks detection terminal device first uses the fireworks target detection model to perform preliminary target detection on the visible light image, identifies the suspected fireworks area, and provides the coordinate information of the suspected fireworks target area, and then performs a secondary temperature detection on the suspected fireworks target area. The two detections greatly reduce the light and background interference; at the same time, the infrared data acquisition device, the visible light acquisition device and the edge processor are integrated and deployed in the fireworks detection terminal device, making the alignment operation of the infrared data acquisition device and the visible light acquisition device easier, which not only ensures the accuracy and efficiency of the system detection, but also can greatly reduce the system cost and improve the real-time performance of the system; moreover, the fireworks detection terminal device can be easily and quickly deployed in various scenarios.

[0139] In a possible implementation, the smoke and fire detection terminal device 4 further includes: a display device; wherein the display device is used to display the smoke and fire detection status of the target detection area and receive alarm prompts.

[0140] It should be noted that the display device is a device that can display the fireworks detection situation of the target detection area in real time. Through the display device, the operator can intuitively understand whether there are fireworks in the target detection area, as well as the distribution of these fireworks, etc. At the same time, when fireworks appear, the display device can also receive the system's alarm prompt and display it, so that the operator can find and deal with the fireworks in time. Moreover, the operator can also manually confirm or cancel the alarm prompt through the display device.

[0141] It should be noted that the smoke and fire detection terminal device of this embodiment can be applied to scenes such as factory fire prevention, forest fire prevention, indoor and outdoor fire prevention, etc. The smoke and fire detection terminal device can be installed at a position and angle according to the monitoring scene distance and monitoring range, and real-time smoke and fire monitoring can be performed by setting the real-time smoke and fire detection function, confidence level, temperature threshold and other parameters in the background. By using the smoke and fire detection terminal device, it can be identified promptly and accurately at the early stage of a fire, and a fire warning can be provided promptly and accurately, thereby playing a role in fire prevention.

[0142] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0143] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting fireworks, characterized in that: include: Obtaining visible light images and infrared remote sensing data at multiple moments within a preset continuous time period, wherein the visible light image at each moment is used to characterize the visible light information of the target detection area at each moment, and the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment, and the visible light image and the infrared remote sensing data are obtained simultaneously by the same terminal device; Performing fireworks target detection on the visible light image at each moment respectively, to obtain a suspected fireworks region in the visible light image at each moment, wherein the suspected fireworks region carries suspected fireworks region coordinate information, and the suspected fireworks region coordinate information is used to indicate a position of the suspected fireworks region in the visible light image; According to the temperature value of the suspected fireworks area in the visible light image at each moment, a marked frame image in which the temperature value of the suspected fireworks area is greater than a preset temperature threshold is selected from the visible light images at multiple moments, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area; Counting the cumulative number of the marked frame images within the preset continuous time period; When the accumulated number of the marked frame images reaches a preset number, it is determined that fireworks appear in the target detection area.

2. The method for detecting smoke and fire according to claim 1, wherein: Before determining the temperature value of the suspected fireworks area from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area, the method includes: Performing image conversion on the infrared remote sensing data at each moment to obtain an infrared image corresponding to the infrared remote sensing data at each moment; The infrared image and the visible light image at each moment are registered with each other by calculating registration parameters, and the dual-light registration parameters of the infrared image and the visible light image are determined, so that the infrared image and the visible light image at each moment are aligned.

3. The method for detecting smoke and fire according to claim 2, wherein: The step of calculating registration parameters of the infrared image and the visible light image at each moment and determining the dual-light registration parameters of the infrared image and the visible light image so as to align the infrared image and the visible light image at each moment includes: According to preset extraction conditions, key feature points in the visible light image and the infrared image are respectively extracted to obtain first feature points of the visible light image and second feature points of the infrared image; Matching the first feature point and the second feature point to obtain a corresponding relationship between the first feature point and the second feature point; Calculating an affine transformation matrix according to the correspondence between the first feature point and the second feature point; The bi-optical registration parameters of the infrared image and the visible light image are determined according to the affine transformation matrix to align the infrared image with the visible light image.

4. The method for detecting smoke and fire according to claim 3, characterized in that: Determining the temperature value of the suspected fireworks area from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area includes: According to the coordinate information of the suspected fireworks area and the dual-light registration parameters, the suspected fireworks area in the visible light image is mapped to the infrared image to obtain a mapping area of ​​the suspected fireworks area in the infrared image; According to a preset infrared temperature measurement algorithm, the infrared remote sensing data corresponding to the mapping area is temperature-converted to obtain a temperature value of the suspected fireworks area.

5. The method for detecting smoke and fire according to claim 1, wherein: The performing fireworks target detection on the visible light image at each moment to obtain a suspected fireworks area in the visible light image at each moment includes: Performing fireworks target detection on the visible light image at each moment respectively by using a fireworks target detection model to obtain a plurality of initial suspected fireworks regions in the visible light image at each moment, wherein the initial suspected fireworks regions include confidence levels; According to the confidence being greater than a preset confidence threshold, the suspected fireworks region in the visible light image is screened out from the plurality of initial suspected fireworks regions.

6. The method for detecting smoke and fire according to claim 5, characterized in that: Before performing fireworks target detection on the visible light image at each moment by using the fireworks target detection model to obtain a plurality of initial suspected fireworks regions in the visible light image at each moment, the method further includes: Acquire a training set, wherein the training set includes a plurality of sample images and target annotation results of the plurality of sample images; An initial fireworks target detection model is trained using the multiple sample images and the target annotation results of the multiple sample images to obtain the fireworks target detection model.

7. A smoke and fire detection device, characterized in that: include: An acquisition module, used to acquire visible light images and infrared remote sensing data at multiple moments in a preset continuous time period, wherein the visible light image at each moment is used to characterize the visible light information of the target detection area at each moment; and the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment; an identification module, configured to perform fireworks target detection on the visible light image at each moment, respectively, to obtain a suspected fireworks region in the visible light image at each moment, wherein the suspected fireworks region carries suspected fireworks region coordinate information, and the suspected fireworks region coordinate information is used to indicate a position of the suspected fireworks region in the visible light image; a marking module, configured to select, from the visible light images at multiple moments, a marked frame image in which the temperature value of the suspected fireworks area is greater than a preset temperature threshold, according to the temperature value of the suspected fireworks area in the visible light image at each moment, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment according to the coordinate information of the suspected fireworks area; A statistics module, used for counting the cumulative number of the marked frame images within the preset continuous time period; The determination module is used to determine that fireworks appear in the target detection area when the accumulated number of the marked frame images reaches a preset number.

8. A smoke and fire detection terminal device, characterized in that: The terminal device includes: a visible light acquisition device, an infrared data acquisition device, an edge processor and an alarm device; Wherein, the visible light acquisition device, the infrared data acquisition device, and the alarm device are respectively connected to the edge processor; the visible light acquisition device and the infrared data acquisition device are dual-light parallel registration through a registration tool; The visible light acquisition device is used to acquire visible light images at multiple moments within a preset continuous time period, and send the visible light images at multiple moments to the edge processor, wherein the visible light image at each moment is used to represent the visible light information of the target detection area at each moment; The infrared data acquisition device is used to collect infrared remote sensing data at multiple moments in the preset continuous time period, and send the infrared remote sensing data at multiple moments to the edge processor, wherein the infrared remote sensing data at each moment is used to characterize the temperature information of the target detection area at each moment; The edge processor is used to receive the visible light images and the infrared remote sensing data at multiple moments within the preset continuous time period; and, respectively, perform fireworks target detection on the visible light images at each moment to obtain suspected fireworks areas in the visible light images at each moment, wherein the suspected fireworks areas carry suspected fireworks area coordinate information, and the suspected fireworks area coordinate information is used to indicate the position of the suspected fireworks area in the visible light image; and, based on the temperature value of the suspected fireworks area in the visible light image at each moment, select a marked frame image whose temperature value of the suspected fireworks area is greater than a preset temperature threshold from the visible light images at multiple moments, wherein the temperature value of the suspected fireworks area is determined from the infrared remote sensing data at that moment based on the suspected fireworks area coordinate information; and, count the cumulative number of the marked frame images within the preset continuous time period; and, when the cumulative number of the marked frame images reaches a preset number, determine that fireworks appear in the target detection area; The alarm device is used to issue an alarm prompt when the edge processor determines that fireworks appear in the target detection area.

9. The smoke and fire detection terminal device according to claim 8, characterized in that: The terminal device further comprises: a display device; wherein the display device is used to display the smoke and fire detection situation of the target detection area and receive the alarm prompt.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images