Smoke and fire detection method, system and device and readable storage medium

Abstract

The embodiment of the invention discloses a smoke and fire detection method, system and device and a readable storage medium. According to the embodiment of the invention, the environment image shot by at least one camera device in a photographing mode can be obtained, and smoke and fire detection processing is carried out on the single environment image. Furthermore, when the smoke and fire target is detected, the priority corresponding to the target equipment can be improved, so that smoke and fire detection processing can be performed on the target image shot by the target equipment preferentially. The computing power consumption for processing a single environment image is obviously smaller than that of a video stream, and the computing power of the smoke and fire detection equipment can be concentrated on the target image shot by the target equipment at the initial stage of the smoke and fire event, so that the utilization rate of computing resources can be effectively improved, and the utilization rate of the environment image is improved. The problems of large calculation amount, data processing lag, high hardware cost and the like are reduced, and rapid and effective smoke and fire detection in any indoor scene or outdoor scene is realized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, system, device, and readable storage medium for detecting fireworks. Background Art

[0002] With the development of new energy technologies, the number of large-scale new energy devices (such as electric vehicles and motorcycles, etc.) continues to increase, and accordingly, the number of charging devices used to charge large-scale new energy devices also increases.

[0003] Due to the wide variety of charging equipment brands and varying quality, spontaneous combustion incidents caused by battery thermal runaway are common. To address this, technologies often employ sensors or video streams to detect fire and smoke.

[0004] However, sensor-based fire and smoke detection methods are not suitable for open outdoor scenes, while video stream-based fire and smoke detection methods have problems such as high computational complexity, certain lags, and high hardware costs. Therefore, how to simply and effectively detect fire and smoke is an urgent problem that needs to be solved. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a method, system, device, and readable storage medium for detecting fire and smoke, so as to achieve rapid and effective fire and smoke detection in any indoor or outdoor scene.

[0006] In a first aspect, a method for detecting smoke and fire is provided, the method comprising:

[0007] Acquire an environment image captured by at least one camera device.

[0008] Performing smoke and fire detection processing on the environment image to determine a smoke and fire detection result corresponding to the environment image.

[0009] In response to the smoke and fire detection result indicating that a smoke and fire target is detected, the smoke and fire detection priority corresponding to the target device is increased, wherein the target device is used to indicate the camera device that photographed the smoke and fire target.

[0010] At least one target image captured by at least one target device is acquired.

[0011] In response to the smoke and fire detection result corresponding to the target image meeting the smoke and fire alarm condition, a smoke and fire alarm action is executed.

[0012] In some embodiments, the environmental image is captured by the camera device at a first frequency.

[0013] Increasing the priority of the target device includes:

[0014] The target device is controlled to shoot at a second frequency, wherein the second frequency is greater than the first frequency.

[0015] In some embodiments, increasing the priority of the target device includes:

[0016] An image queue for storing each of the environment images is cleared.

[0017] In some embodiments, the smoke and fire detection process includes at least the following steps:

[0018] The environment image or target image is input into a pre-trained fireworks recognition model for target detection, and a target detection result corresponding to the environment image or target image is obtained. The target detection result includes at least one or more of the following combinations: the existence status of the target object, the detection frame of the target object, and the confidence level that the target object in the detection frame is determined to be a fireworks target.

[0019] In response to the presence of the target object in the target detection result, a detection frame and / or confidence level corresponding to the target object in the target detection result is detected to obtain a fireworks detection result corresponding to the environment image or the target image.

[0020] In some embodiments, detecting the detection frame and / or confidence level corresponding to the target object in the target detection result to obtain the fireworks detection result corresponding to the environment image or the target image includes:

[0021] The size detection is performed on the detection frame in the target detection result based on a preset size detection threshold to determine a size detection result, wherein the size detection includes a combination of one or more of the following: aspect ratio detection, area detection, and intersection-over-union detection.

[0022] In response to the size detection result indicating that the corresponding detection frame meets a predetermined size condition, a confidence detection is performed on the confidence in the target detection result based on a preset confidence threshold to determine a confidence detection result.

[0023] In response to the confidence detection result indicating that the corresponding confidence meets a predetermined confidence condition, a fireworks detection result indicating that a fireworks target has been detected is generated.

[0024] In some embodiments, the confidence threshold is generated by a reference threshold and an adjustment coefficient, and the adjustment coefficient has a preset corresponding relationship with the weather type.

[0025] In some embodiments, the fireworks recognition model is trained by the following steps:

[0026] An original training set is obtained, wherein the original training set includes at least original positive samples, original negative samples, and annotations corresponding to the original positive samples, the original positive samples are sample images including fireworks targets, and the original negative samples are sample images not including fireworks targets.

[0027] An initial model is trained using the original positive samples, the original negative samples, and the annotations corresponding to the original positive samples.

[0028] A predetermined number of target negative samples are extracted from a target negative sample pool and added to the original training set to generate a target training set, wherein the target negative sample pool includes sample images of the target scene where the camera device is installed without a pyrotechnic target.

[0029] The initial model is trained twice using the target training set to determine the fireworks recognition model.

[0030] In some embodiments, in response to the smoke and fire detection result corresponding to the target image meeting the smoke and fire alarm condition, executing the smoke and fire alarm action includes:

[0031] Performing fireworks detection processing on the target image to determine a fireworks detection result corresponding to the target image.

[0032] In response to detecting that a first number of target images with fireworks targets exist in target images captured by a single target device within a first predetermined period of time, a fireworks alarm action is executed.

[0033] In response to detecting that a second number of target images with fireworks targets exist in the target images captured by the plurality of target devices within a second predetermined period of time, a fireworks alarm action is executed.

[0034] In some embodiments, the method further comprises:

[0035] A plurality of target images continuously captured by the target device are determined.

[0036] In response to the target device continuously capturing a third number of target images that pass the size detection but fail the confidence detection, the target device is restored to an initial priority level.

[0037] In response to the target device continuously capturing a fourth number of target images that fail size detection, the target device is restored to an initial priority level, wherein the fourth number is greater than the third number.

[0038] In a second aspect, a smoke and fire detection system is provided, the system comprising a smoke and fire detection device and at least one camera device.

[0039] Wherein, the camera device is used to capture environmental images.

[0040] The smoke and fire detection device is configured to execute the smoke and fire detection method as described in the first aspect above.

[0041] In a third aspect, a smoke and fire detection device is provided, the device comprising:

[0042] The environment image acquisition module is configured to acquire an environment image captured by at least one camera device.

[0043] The environment image processing module is configured to perform smoke and fire detection processing on the environment image and determine a smoke and fire detection result corresponding to the environment image.

[0044] The priority adjustment module is configured to increase the smoke detection priority corresponding to the target device in response to the smoke detection result indicating that a smoke target is detected, wherein the target device is used to indicate the camera device that photographed the smoke target.

[0045] The target image acquisition module is configured to acquire at least one target image captured by at least one target device.

[0046] The fire and smoke alarm module is configured to execute a fire and smoke alarm action in response to the fire and smoke detection result corresponding to the target image meeting the fire and smoke alarm condition.

[0047] In some embodiments, the environmental image is captured by the camera device at a first frequency.

[0048] The priority adjustment module is specifically configured to execute:

[0049] The target device is controlled to shoot at a second frequency, wherein the second frequency is greater than the first frequency.

[0050] In some embodiments, the priority adjustment module is specifically configured to perform:

[0051] An image queue for storing each of the environment images is cleared.

[0052] In some embodiments, the smoke and fire detection process is implemented by at least the following modules:

[0053] The target detection module is configured to perform target detection by inputting the environment image or target image into a pre-trained fireworks recognition model, and obtain a target detection result corresponding to the environment image or target image, wherein the target detection result includes at least one or more of the following combinations: the existence status of the target object, the detection frame of the target object, and the confidence level that the target object in the detection frame is determined to be a fireworks target.

[0054] The false alarm detection module is configured to execute, in response to the presence of a target object in the target detection result, a detection box and / or confidence corresponding to the target object in the target detection result to obtain a fireworks detection result corresponding to the environment image or the target image.

[0055] In some embodiments, the false alarm detection module is specifically configured to perform:

[0056] The size detection is performed on the detection frame in the target detection result based on a preset size detection threshold to determine a size detection result, wherein the size detection includes a combination of one or more of the following: aspect ratio detection, area detection, and intersection-over-union detection.

[0057] In response to the size detection result indicating that the corresponding detection frame meets a predetermined size condition, a confidence detection is performed on the confidence in the target detection result based on a preset confidence threshold to determine a confidence detection result.

[0058] In response to the confidence detection result indicating that the corresponding confidence meets a predetermined confidence condition, a fireworks detection result indicating that a fireworks target has been detected is generated.

[0059] In some embodiments, the confidence threshold is generated by a reference threshold and an adjustment coefficient, and the adjustment coefficient has a preset corresponding relationship with the weather type.

[0060] In some embodiments, the fireworks recognition model is trained using the following modules:

[0061] The original training set acquisition module is configured to execute acquisition of the original training set, wherein the original training set includes at least original positive samples, original negative samples and annotations corresponding to the original positive samples, the original positive samples are sample images including fireworks targets, and the original negative samples are sample images not including fireworks targets.

[0062] The initial model training module is configured to train the initial model by performing the training on the original positive sample, the original negative sample and the annotation corresponding to the original positive sample.

[0063] The target training set generation module is configured to extract a predetermined number of target negative samples from the target negative sample pool and add them to the original training set to generate a target training set, wherein the target negative sample pool includes sample images of the target scene where the camera device is installed without a pyrotechnic target.

[0064] The secondary training module is configured to perform secondary training on the initial model using the target training set to determine the fireworks recognition model.

[0065] In some embodiments, the fire alarm module is specifically configured to perform:

[0066] Performing fireworks detection processing on the target image to determine a fireworks detection result corresponding to the target image.

[0067] In response to detecting that a first number of target images with fireworks targets exist in target images captured by a single target device within a first predetermined period of time, a fireworks alarm action is executed.

[0068] In response to detecting that a second number of target images with fireworks targets exist in the target images captured by the plurality of target devices within a second predetermined period of time, a fireworks alarm action is executed.

[0069] In some embodiments, the apparatus further comprises:

[0070] The target image determination module is configured to determine a plurality of target images continuously photographed by the target device.

[0071] The first restoration module is configured to restore the target device to an initial priority level in response to the target device continuously capturing a third number of target images that pass the size detection but fail the confidence detection.

[0072] The second restoration module is configured to restore the target device to an initial priority level in response to the target device continuously capturing a fourth number of target images that fail size detection, wherein the fourth number is greater than the third number.

[0073] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer program instructions, which implement the method described in the first aspect when executed by a processor.

[0074] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0075] The embodiment of the present application can obtain an environmental image captured by at least one camera device by photographing, and perform fireworks detection processing on a single environmental image. Furthermore, when a fireworks target is detected, the embodiment of the present application can increase the priority corresponding to the target device to give priority to performing fireworks detection processing on the target image captured by the target device. Among them, since the computing power consumption of processing a single environmental image is significantly less than that of a video stream, and the embodiment of the present application can concentrate the computing power of the fireworks detection device on the target image captured by the target device at the early stage of a fireworks event (i.e., when a fireworks target is first detected), the embodiment of the present application can effectively improve the utilization rate of computing resources, reduce problems such as large amount of calculation, data processing lag, and high hardware cost, and realize fast and effective fireworks detection in any indoor or outdoor scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The above and other objects, features and advantages of the embodiments of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0077] Figure 1 A schematic diagram of a smoke and fire detection system according to an embodiment of the present application;

[0078] Figure 2 This is a flow chart of a method for detecting smoke and fire according to an embodiment of the present application;

[0079] Figure 3 Schematic diagram of each processing unit in the smoke and fire detection device according to an embodiment of the present application;

[0080] Figure 4 This is a flow chart of executing the fireworks alarm action according to an embodiment of the present application;

[0081] Figure 5 A schematic diagram of various processing units in another smoke and fire detection device according to an embodiment of the present application;

[0082] Figure 6 A flowchart of performing smoke and fire detection processing and determining the smoke and fire detection results in an embodiment of the present application;

[0083] Figure 7 Flowchart for training a fireworks recognition model according to an embodiment of the present application;

[0084] Figure 8 Flowchart for false alarm filtering in accordance with the present application;

[0085] Figure 9 A schematic diagram of various processing units in another smoke and fire detection device according to an embodiment of the present application;

[0086] Figure 10 A flowchart of restoring the priority of a target device according to an embodiment of the present application;

[0087] Figure 11 A flowchart of a method for detecting smoke and fire according to an embodiment of the present application is shown;

[0088] Figure 12 This is a schematic structural diagram of a smoke and fire detection device according to an embodiment of the present application;

[0089] Figure 13 This is a structural diagram of a smoke and fire detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0090] The present application is described below based on the following embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0091] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0092] Unless the context clearly requires otherwise, words like "include," "comprising," and the like throughout this application should be construed as including, rather than exclusive or exhaustive; that is, as meaning "including but not limited to."

[0093] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, "multiple" means two or more. In addition, the solutions described in this specification and the embodiments, if involving the processing of personal information, will be processed on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, or being necessary for the performance of the contract, etc.), and will only be processed within the scope of the regulations or agreements. The user's refusal to process personal information other than the necessary information required for basic functions will not affect the user's use of basic functions.

[0094] With the development of new energy technologies, the number of large-scale new energy devices (such as electric vehicles and motorcycles, etc.) continues to increase, and accordingly, the number of charging devices used to charge large-scale new energy devices also increases.

[0095] Due to the wide variety of charging equipment brands and varying quality, spontaneous combustion incidents caused by battery thermal runaway are common. To address this, technologies often employ smoke and fire detection based on sensors or video streams. The purpose of smoke and fire detection is to detect the presence of smoke and flames in a scene.

[0096] Sensor-based fire and smoke detection primarily relies on smoke sensors, combined with other sensors like temperature and light intensity to monitor specific environmental indicators. The sensor's detection results determine whether to trigger a fire and smoke alarm. However, triggering a smoke sensor requires a high concentration of smoke particles near the sensor. Therefore, the smoke sensor's warning sensitivity is positively correlated with the distance from the smoke sensor to the incident location. This means that sensor-based fire and smoke detection is only suitable for smaller indoor spaces and not for open outdoor areas.

[0097] Video stream-based fire and smoke detection primarily detects the presence of fire and smoke events through steps such as video signal compression and encoding, video signal transmission, video signal decoding, algorithm calculation, and output. This approach analyzes continuous video stream data, typically using methods such as optical flow, inter-frame difference, and background modeling to extract motion candidate regions within a video sequence. Features of these motion candidate regions are then extracted and classified using a classifier to determine whether the target in the video is fire and smoke. However, video stream-based fire and smoke detection relies on a continuous video stream, while intermittent video sequences or single-frame images cannot meet the algorithm's input requirements. To minimize latency, related technologies typically employ a single video acquisition unit equipped with a single computing unit, which results in high hardware costs for fire and smoke detection. Furthermore, because this approach relies on a continuous video stream (i.e., requiring a continuous video sequence as algorithm input), video stream-based fire and smoke detection incurs a high computational load and is associated with a certain degree of lag.

[0098] In other words, sensor-based fire and smoke detection methods are not suitable for open outdoor scenes, while fire and smoke detection methods based on video streams have problems such as high computational complexity, certain lags, and high hardware costs. Therefore, how to simply and effectively detect fire and smoke is an urgent problem that needs to be solved.

[0099] In order to solve the above problems, the embodiments of the present application provide a method and a system for detecting smoke and fire. Specifically, Figure 1 As shown, the smoke and fire detection system of the embodiment of the present application includes a smoke and fire detection device 11 and at least one camera device 12 ( Figure 1( 3 are shown in the figure, but the number of camera devices 12 in the embodiment of the present application can be one or more). The fire and smoke detection device 11 and each camera device 12 can transmit data via a wireless connection (such as a network connection or a near-field wireless network connection). In another case, if the distance between the fire and smoke detection device 11 and each camera device 12 is short, the fire and smoke detection device 11 and each camera device 12 can also transmit data via a wired connection.

[0100] In the embodiment of the present application, each camera device 12 can be used to capture an environmental image, and the scene corresponding to the environmental image can be any indoor scene or outdoor scene equipped with a charging device (such as a charging pile, etc.). The fire and smoke detection device 11 can be any applicable terminal device or server. The terminal device can be a smartphone, a tablet computer, or a personal computer (PC), etc. The server can be a single server, a server cluster configured in a distributed manner, or a cloud server. In the fire and smoke detection system, the fire and smoke detection device 11 can be configured to execute the above-mentioned fire and smoke detection method to achieve fast and effective fire and smoke detection.

[0101] Specifically, the fire and smoke detection device 11 of the embodiment of the present application can obtain an environmental image captured by at least one camera device, and perform fire and smoke detection processing on the environmental image to determine the fire and smoke detection result corresponding to the environmental image. Further, in response to the fire and smoke detection result indicating that a fire and smoke target has been detected, the fire and smoke detection device 11 can increase the priority corresponding to the target device (the target device is used to represent the camera device 12 that captured the fire and smoke target) (i.e., the fire and smoke detection processing is performed on the target image captured by the target device in priority). Further, the fire and smoke detection device 11 can obtain at least one target image captured by at least one target device, and in response to the fire and smoke detection result corresponding to the target image meeting the fire and smoke alarm condition, the fire and smoke detection device 11 can execute a fire and smoke alarm action.

[0102] In the embodiment of the present application, each camera device 12 captures an environmental image by taking a photo, and sends the environmental image to the fireworks detection device 11. Accordingly, after the fireworks detection device 11 obtains the environmental image, it can perform fireworks detection processing on a single environmental image, and increase the priority corresponding to the target device when a fireworks target is detected, so as to give priority to performing fireworks detection processing on the target image captured by the target device. Among them, since the computing power consumption of processing a single environmental image is significantly less than that of a video stream, and the embodiment of the present application can concentrate the computing power of the fireworks detection device 11 on the target image captured by the target device at the early stage of a fireworks event (i.e., when a fireworks target is first detected), the embodiment of the present application can effectively improve the utilization rate of computing resources, reduce problems such as large amount of calculation, data processing lag, and high hardware cost, and realize fast and effective fireworks detection in any indoor or outdoor scene.

[0103] The smoke and fire detection method of the embodiment of the present application will be described in detail below in conjunction with specific implementation methods. Figure 2 The specific steps are as follows:

[0104] In step S110 , an environment image captured by at least one camera device is acquired.

[0105] The camera device can be a device installed in any indoor scene and / or outdoor scene. In addition, since each camera device in the embodiment of the present application can remotely report the captured environmental images through a network connection or other means, the embodiment of the present application can respectively deploy one or more camera devices in multiple scenes (these scenes can include indoor scenes and / or outdoor scenes) to realize smoke and fire detection in multiple scenes.

[0106] In step S120 , smoke and fire detection processing is performed on the environment image to determine a smoke and fire detection result corresponding to the environment image.

[0107] Among them, the fireworks detection processing at least includes target detection of the content in the environmental image to determine whether the environmental image includes a fireworks target. The target detection can be achieved through a pre-trained model or other applicable methods. If the embodiment of the present application detects a fireworks target in the environmental image (that is, the fireworks detection result indicates that the fireworks target is detected), the embodiment of the present application can execute step S130 to increase the priority of the target device that photographed the fireworks target, and implement fireworks detection processing on the target image photographed by the target device in priority. Correspondingly, if the embodiment of the present application does not detect a fireworks target in the environmental image, each camera device can continue to maintain the current priority.

[0108] In step S130 , in response to the smoke and fire detection result indicating that a smoke and fire target is detected, the smoke and fire detection priority corresponding to the target device is increased.

[0109] The target device represents the camera that captured the fireworks target, and the target image is the image of the environment captured by the target device. By increasing the priority of the target device, embodiments of the present application can focus computing resources on the target image captured by the target device at the early stage of a fireworks event (i.e., when the fireworks target is first detected), effectively improving computing resource utilization.

[0110] In an optional embodiment, the environmental image of the embodiment of the present application is obtained by capturing the environmental image with a first frequency by a camera device. The first frequency can be a relatively low shooting frequency set according to actual conditions, and the frequency can be used as the initial shooting frequency of each camera device (that is, the shooting frequency corresponding to the initial priority of each camera device). For example, the first frequency can be 0.33 Hz or 0.2 Hz. By setting the first frequency, each camera device in the embodiment of the present application can capture the environmental image at a lower frequency, thereby effectively reducing the consumption of computing resources in non-fire situations.

[0111] Furthermore, the process of increasing the priority corresponding to the target device may be performed as follows: controlling the target device to shoot at the second frequency.

[0112] The second frequency in the embodiment of the present application is greater than the first frequency. For example, the second frequency may be 1 Hz, 1.5 Hz, or 2 Hz.

[0113] That is to say, the embodiment of the present application can control the priority of each camera device by setting the first frequency and the second frequency, that is, the priority corresponding to the first frequency is lower, and the priority corresponding to the second frequency is higher. When the priority of the target device is increased, the target device will shoot the target image at the higher second frequency, while other camera devices will still shoot the environmental image at the lower first frequency. At this time, the proportion of the target image in each environmental image will gradually increase, and the computing resources of the fireworks detection device will also be tilted towards the target image, thereby achieving the purpose of giving priority to the target image shot by the target device for fireworks detection processing.

[0114] In an optional implementation, the process of increasing the priority corresponding to the target device may further include: clearing an image queue for storing each environment image.

[0115] After acquiring the environmental image captured by the camera device, the embodiment of the present application can cache the environmental image into an image queue, and perform fireworks detection processing on the environmental images in the image queue in chronological order.

[0116] For example, Figure 3 As shown, Figure 3Schematic diagram of each processing unit in the smoke and fire detection device according to an embodiment of the present application. The processing unit includes at least an image acquisition unit 31 , a computing power scheduling unit 32 and an image analysis component 33 .

[0117] The image acquisition unit 31 can be used to receive environmental images captured by a camera and cache them in an image queue. The computing power scheduling unit 32 can be used to adjust the priority of each camera to implement computing power scheduling. The image analysis component 33 can be used to extract environmental images (including target images) from the image queue for smoke and fire detection, detect whether the images contain smoke and fire targets, and return the smoke and fire detection results to the computing power scheduling unit 32.

[0118] Furthermore, if the image analysis component 33 detects a fireworks target in the environmental image (i.e., the computing power scheduling unit 32 receives a fireworks detection result indicating that a fireworks target has been detected), the computing power scheduling unit 32 may increase the priority corresponding to the target device (i.e., the camera device that detects the fireworks target) so that the image analysis component 33 may prioritize fireworks detection processing on the target image captured by the target device.

[0119] Specifically, the computing power scheduling unit 32 can, on the one hand, control the target device to perform up-sampling (i.e., control the target device's shooting frequency to increase from a first frequency to a second frequency), so that the image analysis component 33 can prioritize the target image for smoke and fire detection. On the other hand, the computing power scheduling unit 32 can also clear the image queue used to store various environmental images to ensure that the target image is stored at the head of the image queue as soon as possible, so that the image analysis component 33 can prioritize the target image for smoke and fire detection.

[0120] It should be noted that for the two methods of increasing priority, namely controlling the camera device to increase the shooting frequency and clearing the image queue, the embodiment of the present application can choose to execute either one, or can choose to execute both methods at the same time to make the effect of increasing the priority more obvious.

[0121] In step S140 , at least one target image captured by at least one target device is acquired.

[0122] In step S150 , in response to the smoke and fire detection result corresponding to the target image satisfying the smoke and fire alarm condition, a smoke and fire alarm action is executed.

[0123] After acquiring a target image, embodiments of the present application can perform fire and smoke detection processing on the target image to determine a fire and smoke detection result corresponding to the target image. Fire and smoke alarm actions can include any applicable alarm actions, such as message push (e.g., SMS message push, network message push, and application message push) and phone notification.

[0124] The embodiment of the present application can obtain an environmental image captured by at least one camera device by photographing, and perform fireworks detection processing on a single environmental image. Furthermore, when a fireworks target is detected, the embodiment of the present application can increase the priority corresponding to the target device to give priority to performing fireworks detection processing on the target image captured by the target device. Among them, since the computing power consumption of processing a single environmental image is significantly less than that of a video stream, and the embodiment of the present application can concentrate the computing power of the fireworks detection device on the target image captured by the target device at the early stage of a fireworks event (i.e., when a fireworks target is first detected), the embodiment of the present application can effectively improve the utilization rate of computing resources, reduce problems such as large amount of calculation, data processing lag, and high hardware cost, and realize fast and effective fireworks detection in any indoor or outdoor scene.

[0125] In an optional embodiment, as Figure 4 As shown, the above step S140 may include the following steps:

[0126] In step S141 , smoke and fire detection processing is performed on the target image to determine a smoke and fire detection result corresponding to the target image.

[0127] In step S142 , in response to detecting that a first number of target images with fireworks targets exist in target images captured by a single target device within a first predetermined time period, a fireworks alarm action is executed.

[0128] The first predetermined duration and the first number can be reasonably set according to actual conditions. For example, in an embodiment of the present application, the first predetermined duration can be set to 60 seconds, and the first number can be set to 3. That is, if the embodiment of the present application detects that a single target device captures three target images with fireworks within 60 seconds, the embodiment of the present application can execute the above-mentioned fireworks alarm action.

[0129] In step S143 , in response to detecting that a second number of target images with fireworks targets exist in the target images captured by the plurality of target devices within the second predetermined time period, a fireworks alarm action is executed.

[0130] The multiple target devices are target devices set in the same scene, and the second predetermined duration and the second number can also be reasonably set according to actual conditions. For example, in an embodiment of the present application, the second predetermined duration can be set to 30 seconds and the second number can be set to 3. In other words, if the embodiment of the present application detects that multiple target devices capture three target images with fireworks within 30 seconds, the embodiment of the present application can execute the above-mentioned fireworks alarm action.

[0131] It should be noted that steps S142 and S143 are independent judgment steps set up for different situations. Step S142 applies to situations where only one camera device captures a firework target in a scene, while step S143 applies to situations where multiple cameras capture a firework target in a scene. In other words, embodiments of the present application can set up multiple cameras in a scene to perform cross-shooting. If multiple cameras capture a firework target, embodiments of the present application can quickly trigger a firework alarm action through step S143, achieving the purpose of a rapid response.

[0132] In an optional implementation, the embodiment of the present application may also add false alarm filtering processing during the smoke and fire detection process to reduce the number of false alarms. Specifically, the embodiment of the present application may set a corresponding processing unit in the smoke and fire detection device to implement false alarm filtering processing.

[0133] like Figure 5 As shown, in the embodiment of the present application, a target detection unit 331 , a false alarm filtering unit 332 and an event aggregation unit 333 may be set in the image analysis component 33 .

[0134] The target detection unit 331 can perform target detection on the environment image or the target image, determine the target detection result, and send the target detection result to the false alarm filtering unit 332. After receiving the target detection result, the false alarm filtering unit 332 can perform false alarm filtering on the target detection result to determine the smoke and fire detection result, and send the smoke and fire detection result to the event aggregation unit 333. The false alarm filtering process can include at least size filtering and confidence filtering.

[0135] Furthermore, the event aggregation unit 333 can make an alarm judgment based on each received fire and smoke detection result. If the fire and smoke detection result meets the fire and smoke alarm condition, the event aggregation unit 333 can execute the fire and smoke alarm action. Figure 4 Steps S142 and S143 in the process determine whether to execute the fireworks alarm action.

[0136] In the above process, if Figure 6 As shown, the process of performing smoke and fire detection processing and determining the smoke and fire detection result in the embodiment of the present application may include the following steps:

[0137] In step S210 , the environment image or the target image is input into a pre-trained fireworks recognition model to perform target detection, and a target detection result corresponding to the environment image or the target image is obtained.

[0138] Among them, the target detection result includes at least one or more of the following combinations: the existence state of the target object, the detection frame of the target object, and the confidence that the target object in the detection frame is determined to be a fireworks target. The target object can be used to characterize the object of the suspected fireworks target detected by the fireworks recognition model. The existence state of the target object can be used to characterize the specific type of fireworks target (such as smoke or fire), the detection frame is used to identify the target object in the environment image or the target image, and the confidence is used to characterize the probability that the target object in the detection frame is a fireworks target. The greater the confidence, the greater the probability that the target object in the detection frame is a fireworks target. In addition, the fireworks recognition model can be a model constructed based on a deep learning network or other applicable models. For example, the fireworks recognition model can be a yolov8 model, etc. The embodiment of the present application can train the fireworks recognition model through a training set with positive samples and negative samples, and use the trained fireworks recognition model for fireworks target detection.

[0139] In an optional implementation, the embodiment of the present application can improve the pertinence of the fireworks recognition model by means of secondary training. Specifically, Figure 7 As shown, the fireworks recognition model of the embodiment of the present application can be trained by the following steps:

[0140] In step S310, an original training set is obtained.

[0141] The original training set includes at least original positive samples, original negative samples, and annotations corresponding to the original positive samples. The original positive samples are sample images including fireworks targets, and the original negative samples are sample images not including fireworks targets.

[0142] The embodiments of the present application can collect and screen sample images that meet the requirements through legal channels. Specifically, the embodiments of the present application can collect a number (e.g., more than 10,000) of images containing fireworks targets as candidate sets of original positive samples. These images can be images or video frames from the Internet, video frames sampled from real accident videos, and video frames sampled from self-produced fireworks simulation videos.

[0143] Furthermore, in the embodiment of the present application, the images in the candidate set can be labeled or filtered according to the flame instance labeling rules and the smoke instance labeling rules to determine the original positive samples. The labeling tool can be any applicable labeling tool such as LabelMe.

[0144] Taking the flame instance annotation rule as an example, the embodiment of the present application can frame the minimum bounding rectangle of the flame instance in the image and filter out flame instance images whose minimum bounding rectangle is smaller than a first predetermined screen ratio (e.g., 0.14, 0.15, or 0.2) and exclude them from model training. Furthermore, the embodiment of the present application can annotate flame instance images that meet the requirements and use them as original positive samples.

[0145] Taking the smoke instance annotation rule as an example, the embodiment of the present application can frame the minimum bounding rectangle of the smoke instance in the image. During the screening process, the embodiment of the present application can first filter out smoke instance images whose smoke outlines are unclear, and smoke instance images whose smoke outlines are clear but are severely disturbed (for example, light smoke with a low smoke concentration, which is more easily disturbed by the background environment). Furthermore, the embodiment of the present application can filter smoke instance images whose proportion of the obscured part of the smoke is greater than or equal to a predetermined obscuration ratio (for example, 0.2, 0.25 or 0.3, etc.). At the same time, the embodiment of the present application can also filter smoke instance images whose minimum bounding rectangle is less than a second predetermined screen ratio (for example, a ratio of 0.14, 0.18 or 0.2), and not use them for model training. Furthermore, the embodiment of the present application can annotate smoke instance images that meet the requirements as original positive samples.

[0146] After screening and labeling the original positive samples, the embodiment of the present application can train the initial model using the original positive samples, the original negative samples, and the labels corresponding to the original positive samples. The original negative samples can be environmental images that do not contain fireworks targets.

[0147] In step S320, an initial model is trained using the original positive samples, the original negative samples, and the annotations corresponding to the original positive samples.

[0148] Among them, the trained initial model already has the ability to identify fireworks targets. In order to further improve the performance of the fireworks recognition model, the embodiment of the present application can perform secondary training on the initial model to increase the pertinence of the fireworks recognition model.

[0149] In step S330 , a predetermined number of target negative samples are extracted from the target negative sample pool and added to the original training set to generate a target training set.

[0150] The target negative sample pool includes sample images of the target scene where the camera is installed, without fireworks. That is, since the camera in this embodiment of the application is used to detect the presence of fireworks in indoor and / or outdoor scenes, this embodiment of the application can use environmental images of the target scene where the camera is installed as negative samples to increase the specificity of the fireworks recognition model.

[0151] Specifically, the predetermined number of target negative samples extracted in the embodiment of the present application can be reasonably set according to the number of samples in the original training set. For example, the predetermined number can be 0.1 times the number of samples in the original training set.

[0152] In step S340 , the initial model is trained again using the target training set to determine a fireworks recognition model.

[0153] The fireworks recognition model obtained after the secondary training can have a better recognition effect on the target scene (that is, the scene where the corresponding camera equipment is installed), thereby further improving the effect of fireworks detection.

[0154] In step S220 , in response to the presence of the target object in the target detection result, the detection frame and / or confidence level corresponding to the target object in the target detection result is detected to obtain a fireworks detection result corresponding to the environment image or the target image.

[0155] Among them, the fireworks detection result can be used to characterize whether the target object is a fireworks target. The embodiment of the present application can select any one of the detection box and confidence level for detection, or can choose to detect both the detection box and confidence level to improve the effect of false alarm filtering.

[0156] In an optional embodiment, as Figure 8 As shown, the above step S220 may specifically include the following steps:

[0157] In step S221 , the size of the detection frame in the target detection result is detected based on a preset size detection threshold to determine a size detection result.

[0158] Among them, in order to effectively utilize storage resources, the embodiment of the present application can only retain the target detection results within a certain period of time (for example, 60 seconds or 90 seconds, etc.) to avoid the target detection results occupying too many cache resources.

[0159] In the embodiments of the present application, the purpose of size detection is to filter out target detection results whose size characteristics do not match those of the fireworks instance, thereby improving the accuracy of fireworks detection. Size detection can include one or more combinations of the following: aspect ratio detection, area detection, and intersection over union (IoU) detection. Accordingly, the size detection threshold can include one or more combinations of the following: aspect ratio threshold, area ratio threshold, and IoU threshold.

[0160] Regarding aspect ratio filtering, embodiments of the present application can perform aspect ratio detection by setting an aspect ratio threshold, where the aspect ratio threshold can include an upper aspect ratio limit and a lower aspect ratio limit. If the aspect ratio corresponding to the detection frame in the target detection result is between the upper aspect ratio limit and the lower aspect ratio limit, the aspect ratio detection passes; otherwise, it fails. For example, the lower aspect ratio limit in embodiments of the present application can be set to 0.19, 0.2, or other reasonable values, and the upper aspect ratio limit can be set to 8.48, 8.5, or other reasonable values.

[0161] For area detection, the embodiment of the present application can perform area detection through the area ratio of the detection frame in the environment image or the target image, and the corresponding area ratio threshold. Specifically, the area ratio threshold may include an area ratio upper limit value and an area ratio lower limit value. If the area ratio of the detection frame in the target detection result in the environment image or the target image is between the area ratio upper limit value and the area ratio lower limit value, the area detection passes, otherwise it fails. For example, the area ratio lower limit value of the embodiment of the present application can be set to 0.004, 0.005 or other reasonable values, and the area ratio upper limit value can be set to 0.95, 0.96 or other reasonable values.

[0162] For IoU detection, the embodiment of the present application can perform IoU detection by setting an IoU threshold, wherein the IoU corresponding to the detection frame can be determined by the following formula:

[0163]

[0164] Among them, V iou It is used to characterize the intersection-over-union ratio corresponding to the detection frame, c is used to characterize the environment image or target image corresponding to the detection frame, and c obj is used to represent the current detection frame, and Ω(c) is used to represent all detection frames detected by the camera device corresponding to the detection frame within the above-mentioned certain time length (for example, 60 seconds or 90 seconds, etc.).

[0165] Furthermore, if the intersection-over-union ratio (V iou ) is less than or equal to the intersection-and-union ratio threshold (i.e. the overlap between the fireworks target detected this time and the fireworks target detected in the past is low), then the intersection-and-union ratio test passes. Correspondingly, if the intersection-and-union ratio (V iou ) is greater than the intersection-in-union (IoU) threshold (i.e., the overlap between the fireworks target detected this time and the fireworks target detected previously is high), the IoU test fails. The IoU threshold can be set to 0.9, 0.92, or other reasonable values.

[0166] In real fireworks events, fireworks spread quickly, while objects that are prone to false alarms (such as obstructions in front of the camera equipment) often do not have the characteristic of a fast diffusion speed. Therefore, the embodiment of the present application can filter out target objects that do not undergo morphological changes through intersection-over-union detection, thereby further improving the detection accuracy of fireworks targets.

[0167] In step S222 , in response to the size detection result indicating that the corresponding detection frame meets a predetermined size condition, a confidence detection is performed on the confidence in the target detection result based on a preset confidence threshold to determine a confidence detection result.

[0168] In step S223 , in response to the confidence detection result indicating that the corresponding confidence meets a predetermined confidence condition, a fireworks detection result indicating that a fireworks target is detected is generated.

[0169] Among them, the confidence threshold of the embodiment of the present application can be a fixed value (such as 0.7 or 0.8, etc.), or it can be a dynamically adjustable value. If the confidence in the target detection result is greater than the confidence threshold, it means that the confidence detection is passed (that is, the confidence detection result represents that the corresponding confidence meets the predetermined confidence condition). At this time, the embodiment of the present application can generate a fireworks detection result for characterizing the detected fireworks target. The fireworks detection result can include relevant information such as the corresponding image, target type, and detection frame. In addition, if the embodiment of the present application generates a fireworks detection result for characterizing the detected fireworks target for the first time, the environmental image corresponding to the fireworks detection result can be returned to the computing power scheduling unit as an alarm frame, so that the computing power scheduling unit increases the priority of the camera device corresponding to the alarm frame.

[0170] The embodiment of the present application adds false alarm filtering processing (i.e., size detection and confidence detection) during the smoke and fire detection process, which can effectively reduce the number of false alarms and improve the accuracy of smoke and fire detection.

[0171] In an optional implementation, the embodiment of the present application may also dynamically adjust the confidence threshold according to meteorological information when performing confidence filtering, so as to reduce the impact of different weather conditions on fireworks detection, thereby further improving the accuracy of fireworks detection.

[0172] Specifically, the confidence threshold of the embodiment of the present application can be generated by a base threshold and an adjustment coefficient, and the adjustment coefficient has a preset correspondence with the weather type. Among them, the base threshold can be a fixed value set according to actual conditions (for example, 0.7 or 0.8, etc.). The embodiment of the present application can adjust the base threshold by adjusting the adjustment coefficient to dynamically adjust the confidence threshold.

[0173] like Figure 9 As shown, the smoke and fire detection device of the embodiment of the present application may further include a weather collection unit 34. The weather collection unit 34 may periodically obtain weather information (e.g., once every hour) via an external interface and send the weather information to the image analysis component 33, so that the false alarm filtering unit 332 in the image analysis component 33 dynamically adjusts the confidence threshold based on the weather information. The weather information may include at least the weather type at the location of each camera device, for example, the weather type may include strong wind, snow, rain, fog, or other types.

[0174] Furthermore, after obtaining meteorological information, the embodiment of the present application can determine the adjustment coefficient according to the correspondence between the weather type and the adjustment coefficient, and adjust the baseline threshold by the adjustment coefficient to dynamically adjust the confidence threshold.

[0175] For example, the confidence threshold in the embodiment of the present application can be expressed by the following formula:

[0176]

[0177] Among them, T e Used to characterize the confidence threshold, T b Used to characterize the benchmark threshold, Used to represent the adjustment coefficient. Taking the baseline threshold of 0.7 as an example, the corresponding relationship between the adjustment coefficient and the weather type can be expressed as follows:

[0178]

[0179] Among them, weather_type is used to represent the weather type, and the value before weather_type is used to represent the corresponding adjustment coefficient. Specifically, weather type 1 (i.e. weather_type = 1) is other weather, and its corresponding adjustment coefficient is 1. Weather type 2 is windy weather, and its corresponding adjustment coefficient is 1.2. Weather type 3 is snowy weather, and its corresponding adjustment coefficient is 1.2. Weather type 4 is rainy weather, and its corresponding adjustment coefficient is 1.3. Weather type 5 is foggy weather, and its corresponding adjustment coefficient is 1.35.

[0180] By dynamically adjusting the confidence threshold using meteorological information, the embodiment of the present application can reduce the impact of different weather conditions on firework detection, thereby further improving the accuracy of firework detection. It should be noted that in the process of dynamically adjusting the confidence threshold, the adjusted confidence threshold should be less than or equal to the upper limit of the confidence threshold (e.g., 0.94 or 0.95). That is, in the above formula, In addition, the above formula and corresponding relationship are only an example of the embodiment of the present application. In actual application, the embodiment of the present application may also adopt other applicable formulas and corresponding relationships.

[0181] For example, the confidence threshold in the embodiment of the present application can also be expressed by the following formula:

[0182]

[0183] Correspondingly, taking the benchmark threshold of 0.7 as an example, the corresponding relationship can also be expressed as:

[0184]

[0185] The adjusted confidence threshold should be less than or equal to the upper limit of the confidence threshold (such as 0.94 or 0.95, etc.), that is, Should be less than or equal to the upper limit.

[0186] In an optional implementation, the embodiment of the present application can also restore the priority of the target device under certain conditions, specifically, Figure 10 As shown, the process may include the following steps:

[0187] In step S410 , a plurality of target images continuously captured by a target device are determined.

[0188] In step S420 , in response to the target device continuously capturing a third number of target images that pass the size detection but fail the confidence detection, the target device is restored to the initial priority level.

[0189] Taking the above embodiment as an example, a method of restoring the initial priority may be to restore the shooting frequency of the target device from the second frequency to the first frequency.

[0190] The third number can be reasonably set according to actual conditions. For example, the third number can be a value such as 10, 15, or 20. That is, in step S420, the target device may continuously capture multiple images that are suspected to contain fireworks targets but are not actually fireworks targets (i.e., pass the size test but fail the confidence test). In this case, the embodiment of the present application can determine that no fireworks targets appear or that the fireworks targets are extinguished. Furthermore, the embodiment of the present application can restore the target device to its initial priority to save computing resources.

[0191] In step S430 , in response to the target device continuously capturing a fourth number of target images that fail the size detection, the target device is restored to an initial priority level.

[0192] The fourth number is greater than the third number, and the fourth number can be reasonably set according to actual conditions. For example, the fourth number can be a value such as 30, 35, or 40. That is, in step S430, the target device may continuously capture multiple images in which no fireworks target or suspected fireworks target is detected (i.e., fails size detection). In this case, the embodiment of the present application can determine that no fireworks target has appeared or that the fireworks target has been extinguished. Furthermore, the embodiment of the present application can restore the target device to its initial priority to save computing resources.

[0193] In combination with the above embodiments, Figure 11 As shown, Figure 11The smoke and fire detection device of the embodiment of the present application includes three cameras (camera A, camera B, and camera C). The smoke and fire detection device can obtain environmental images captured by these three cameras and store the environmental images in image queue 111. It should be noted that in image queue 111, each filled rectangle corresponds to an image, and rectangles with the same fill are images captured by the same camera. In other words, the embodiment of the present application uses the same fill to represent each image captured by each camera.

[0194] Furthermore, after acquiring an environmental image and storing it in image queue 111, the smoke and fire detection device can extract the environmental image from image queue 111 through an image analysis component to perform smoke and fire detection processing and determine a smoke and fire detection result. The image analysis component can include a target detection unit, a false alarm filtering unit, and an event aggregation unit. When performing confidence detection through the false alarm filtering unit, the image analysis component can dynamically adjust the confidence threshold based on the weather type in the meteorological information to improve the effectiveness of false alarm filtering.

[0195] Further, such as Figure 11 As shown, if the fireworks detection device detects that the camera device A has captured a fireworks target (i.e., the fireworks detection result corresponding to the camera device A indicates that the fireworks target has been detected), the fireworks detection device can determine the camera device A as the target device, and then clear the image queue 111 and control the camera device A to perform upscaling shooting to increase the priority of the camera device A.

[0196] After performing the above operation of increasing the priority of the camera device A, as shown in FIG. Figure 11 As shown in the image queue 112 in FIG, since the embodiment of the present application immediately clears the image queue 111 after determining camera A as the target device and controls camera A to perform upscaling shooting, the images captured by camera A (i.e., the target images) occupy the vast majority of the space in the image queue 112. In other words, the fireworks detection device will give priority to performing fireworks detection processing on the images captured by camera A, thereby effectively improving the utilization of computing resources.

[0197] Furthermore, the fire and smoke detection device can determine the fire and smoke alarm and restore the priority of the target image (i.e., the image captured by camera A) based on the fire and smoke detection results. If the fire and smoke detection results of the target image meet the fire and smoke alarm conditions, the fire and smoke detection device can perform a fire and smoke alarm action. Correspondingly, if the fire and smoke detection results of the target image meet the restoration conditions, the fire and smoke detection device can restore the priority of camera A to its initial priority.

[0198] The embodiment of the present application can obtain an environmental image captured by at least one camera device by photographing, and perform fireworks detection processing on a single environmental image. Furthermore, when a fireworks target is detected, the embodiment of the present application can increase the priority corresponding to the target device to give priority to performing fireworks detection processing on the target image captured by the target device. Among them, since the computing power consumption of processing a single environmental image is significantly less than that of a video stream, and the embodiment of the present application can concentrate the computing power of the fireworks detection device on the target image captured by the target device at the early stage of a fireworks event (i.e., when a fireworks target is first detected), the embodiment of the present application can effectively improve the utilization rate of computing resources, reduce problems such as large amount of calculation, data processing lag, and high hardware cost, and realize fast and effective fireworks detection in any indoor or outdoor scene.

[0199] Based on the same technical concept, the embodiment of the present application also provides a smoke and fire detection device, such as Figure 12 As shown, the device includes: an environment image acquisition module 121, an environment image processing module 122, a priority adjustment module 123, a target image acquisition module 124 and a smoke and fire alarm module 125.

[0200] The environment image acquisition module 121 is configured to acquire an environment image captured by at least one camera device.

[0201] The environment image processing module 122 is configured to perform smoke and fire detection processing on the environment image and determine a smoke and fire detection result corresponding to the environment image.

[0202] The priority adjustment module 123 is configured to increase the smoke detection priority corresponding to a target device in response to the smoke detection result indicating that a smoke target is detected, wherein the target device is used to indicate a camera device that photographed the smoke target.

[0203] The target image acquisition module 124 is configured to acquire at least one target image captured by at least one target device.

[0204] The fire and smoke alarm module 125 is configured to execute a fire and smoke alarm action in response to the fire and smoke detection result corresponding to the target image meeting the fire and smoke alarm condition.

[0205] In some embodiments, the environmental image is captured by the camera device at a first frequency.

[0206] The priority adjustment module 123 is specifically configured to perform:

[0207] The target device is controlled to shoot at a second frequency, wherein the second frequency is greater than the first frequency.

[0208] In some embodiments, the priority adjustment module 123 is specifically configured to perform:

[0209] An image queue for storing each of the environment images is cleared.

[0210] In some embodiments, the smoke and fire detection process is implemented by at least the following modules:

[0211] The target detection module is configured to perform target detection by inputting the environment image or target image into a pre-trained fireworks recognition model, and obtain a target detection result corresponding to the environment image or target image, wherein the target detection result includes at least one or more of the following combinations: the existence status of the target object, the detection frame of the target object, and the confidence level that the target object in the detection frame is determined to be a fireworks target.

[0212] The false alarm detection module is configured to execute, in response to the presence of a target object in the target detection result, a detection box and / or confidence corresponding to the target object in the target detection result to obtain a fireworks detection result corresponding to the environment image or the target image.

[0213] In some embodiments, the false alarm detection module is specifically configured to perform:

[0214] The size detection is performed on the detection frame in the target detection result based on a preset size detection threshold to determine a size detection result, wherein the size detection includes a combination of one or more of the following: aspect ratio detection, area detection, and intersection-over-union detection.

[0215] In response to the size detection result indicating that the corresponding detection frame meets a predetermined size condition, a confidence detection is performed on the confidence in the target detection result based on a preset confidence threshold to determine a confidence detection result.

[0216] In response to the confidence detection result indicating that the corresponding confidence meets a predetermined confidence condition, a fireworks detection result indicating that a fireworks target has been detected is generated.

[0217] In some embodiments, the confidence threshold is generated by a reference threshold and an adjustment coefficient, and the adjustment coefficient has a preset corresponding relationship with the weather type.

[0218] In some embodiments, the fireworks recognition model is trained using the following modules:

[0219] The original training set acquisition module is configured to execute acquisition of the original training set, wherein the original training set includes at least original positive samples, original negative samples and annotations corresponding to the original positive samples, the original positive samples are sample images including fireworks targets, and the original negative samples are sample images not including fireworks targets.

[0220] The initial model training module is configured to train the initial model by performing the training on the original positive sample, the original negative sample and the annotation corresponding to the original positive sample.

[0221] The target training set generation module is configured to extract a predetermined number of target negative samples from the target negative sample pool and add them to the original training set to generate a target training set, wherein the target negative sample pool includes sample images of the target scene where the camera device is installed without a pyrotechnic target.

[0222] The secondary training module is configured to perform secondary training on the initial model using the target training set to determine the fireworks recognition model.

[0223] In some embodiments, the fire alarm module 125 is specifically configured to perform:

[0224] Performing fireworks detection processing on the target image to determine a fireworks detection result corresponding to the target image.

[0225] In response to detecting that a first number of target images with fireworks targets exist in target images captured by a single target device within a first predetermined period of time, a fireworks alarm action is executed.

[0226] In response to detecting that a second number of target images with fireworks targets exist in the target images captured by the plurality of target devices within a second predetermined period of time, a fireworks alarm action is executed.

[0227] In some embodiments, the apparatus further comprises:

[0228] The target image determination module is configured to determine a plurality of target images continuously photographed by the target device.

[0229] The first restoration module is configured to restore the target device to an initial priority level in response to the target device continuously capturing a third number of target images that pass the size detection but fail the confidence detection.

[0230] The second restoration module is configured to restore the target device to an initial priority level in response to the target device continuously capturing a fourth number of target images that fail size detection, wherein the fourth number is greater than the third number.

[0231] The embodiment of the present application can obtain an environmental image captured by at least one camera device by photographing, and perform fireworks detection processing on a single environmental image. Furthermore, when a fireworks target is detected, the embodiment of the present application can increase the priority corresponding to the target device to give priority to performing fireworks detection processing on the target image captured by the target device. Among them, since the computing power consumption of processing a single environmental image is significantly less than that of a video stream, and the embodiment of the present application can concentrate the computing power of the fireworks detection device on the target image captured by the target device at the early stage of a fireworks event (i.e., when a fireworks target is first detected), the embodiment of the present application can effectively improve the utilization rate of computing resources, reduce problems such as large amount of calculation, data processing lag, and high hardware cost, and realize fast and effective fireworks detection in any indoor or outdoor scene.

[0232] Figure 13 Schematic diagram of the smoke and fire detection device according to the embodiment of the present application. Figure 13 As shown, Figure 13 The illustrated smoke and fire detection device is a general-purpose address query device comprising a general-purpose computer hardware structure, including at least a processor 131 and a memory 132. The processor 131 and the memory 132 are connected via a bus 133. The memory 132 is adapted to store instructions or programs executable by the processor 131. The processor 131 may be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 131 executes the instructions stored in the memory 132, thereby executing the method flow of the embodiment of the present application as described above to process data and control other devices. The bus 133 connects the aforementioned multiple components together and also connects them to a display controller 134, a display device, and an input / output (I / O) device 135. The input / output (I / O) device 135 may be a mouse, keyboard, modem, network interface, touch input device, somatosensory input device, printer, or other devices known in the art. Typically, the input / output device 135 is connected to the system via an input / output (I / O) controller 136.

[0233] It will be understood by those skilled in the art that the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0234] The present application is described with reference to flowcharts of methods, apparatuses (devices), and computer program products according to embodiments of the present application. It should be understood that each process in the flowcharts can be implemented by computer program instructions.

[0235] These computer program instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0236] These computer program instructions can also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0237] Another embodiment of the present application relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used to enable a computer to execute part or all of the above method embodiments.

[0238] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by specifying relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0239] Another embodiment of the present application relates to a computer program product, including a computer program / instruction, which can implement some or all of the above method embodiments when executed by a processor.

[0240] That is, those skilled in the art can understand that the embodiments of the present application can specify relevant hardware (including the processor itself) by executing a computer program product (computer program / instructions) through a processor, thereby implementing all or part of the steps in the above-mentioned embodiment method.

[0241] The foregoing is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application are intended to be within the scope of protection of the present application.

Claims

1. A method for detecting fireworks, characterized in that: The method comprises: Acquire an environment image captured by at least one camera device; Performing fire and smoke detection processing on the environment image to determine a fire and smoke detection result corresponding to the environment image; In response to the fireworks detection result indicating that a fireworks target is detected, increasing the fireworks detection priority corresponding to the target device, wherein the target device is used to indicate the camera device that captured the fireworks target; Acquiring at least one target image captured by at least one of the target devices; In response to the smoke and fire detection result corresponding to the target image meeting the smoke and fire alarm condition, a smoke and fire alarm action is executed.

2. The method according to claim 1, characterized in that The environmental image is captured by the camera device at a first frequency; Increasing the priority of the target device includes: The target device is controlled to shoot at a second frequency, wherein the second frequency is greater than the first frequency.

3. The method according to claim 1 or 2, characterized in that Increasing the priority of the target device includes: An image queue for storing each of the environment images is cleared.

4. The method according to claim 1, wherein The smoke and fire detection process comprises at least the following steps: Inputting the environment image or target image into a pre-trained fireworks recognition model for target detection, and obtaining a target detection result corresponding to the environment image or target image, wherein the target detection result includes at least one or more of the following: the presence status of the target object, the detection frame of the target object, and the confidence level that the target object in the detection frame is determined to be a fireworks target; In response to the presence of the target object in the target detection result, a detection frame and / or confidence level corresponding to the target object in the target detection result is detected to obtain a fireworks detection result corresponding to the environment image or the target image.

5. The method according to claim 4, characterized in that The detecting of the detection frame and / or the confidence level corresponding to the target object in the target detection result to obtain the smoke and fire detection result corresponding to the environment image or the target image includes: Performing size detection on the detection frame in the target detection result based on a preset size detection threshold to determine a size detection result, wherein the size detection includes a combination of one or more of the following: aspect ratio detection, area detection, and intersection-over-union detection; In response to the size detection result indicating that the corresponding detection frame meets a predetermined size condition, performing a confidence test on the confidence level of the target detection result based on a preset confidence threshold to determine a confidence test result; In response to the confidence detection result indicating that the corresponding confidence meets a predetermined confidence condition, a fireworks detection result indicating that a fireworks target has been detected is generated.

6. The method according to claim 5, characterized in that The confidence threshold is generated by a reference threshold and an adjustment coefficient, and the adjustment coefficient has a preset corresponding relationship with the weather type.

7. The method according to claim 4, characterized in that The fireworks recognition model is trained by the following steps: Obtaining an original training set, wherein the original training set includes at least original positive samples, original negative samples, and annotations corresponding to the original positive samples, the original positive samples are sample images including fireworks targets, and the original negative samples are sample images not including fireworks targets; Training an initial model using the original positive samples, the original negative samples, and the annotations corresponding to the original positive samples; Extracting a predetermined number of target negative samples from a target negative sample pool and adding them to the original training set to generate a target training set, wherein the target negative sample pool includes sample images of the target scene where the camera device is installed without pyrotechnic targets; The initial model is trained twice using the target training set to determine the fireworks recognition model.

8. The method according to claim 1, characterized in that In response to the smoke and fire detection result corresponding to the target image meeting the smoke and fire alarm condition, executing the smoke and fire alarm action includes: Performing fireworks detection processing on the target image to determine a fireworks detection result corresponding to the target image; In response to detecting that a first number of target images with fireworks targets exist in target images captured by a single target device within a first predetermined period of time, executing a fireworks alarm action; In response to detecting that a second number of target images with fireworks targets exist in the target images captured by the plurality of target devices within a second predetermined period of time, a fireworks alarm action is executed.

9. The method according to claim 5, characterized in that The method further comprises: determining a plurality of target images continuously captured by the target device; In response to the target device continuously capturing a third number of target images that pass the size detection but fail the confidence detection, restoring the target device to an initial priority level; In response to the target device continuously capturing a fourth number of target images that fail size detection, the target device is restored to an initial priority level, wherein the fourth number is greater than the third number.

10. A smoke and fire detection system, characterized in that: The system includes a smoke and fire detection device and at least one camera device; Wherein, the camera device is used to capture environmental images; The smoke and fire detection device is configured to execute the smoke and fire detection method according to any one of claims 1 to 9.

11. A smoke and fire detection device, characterized in that: The device comprises: An environmental image acquisition module is configured to acquire an environmental image captured by at least one camera device; an environmental image processing module, configured to perform smoke and fire detection processing on the environmental image and determine a smoke and fire detection result corresponding to the environmental image; a priority adjustment module configured to increase the smoke and fire detection priority corresponding to a target device in response to the smoke and fire detection result indicating that a smoke and fire target is detected, wherein the target device is used to indicate a camera device that photographed the smoke and fire target; a target image acquisition module, configured to acquire at least one target image captured by at least one target device; The fire and smoke alarm module is configured to execute a fire and smoke alarm action in response to the fire and smoke detection result corresponding to the target image meeting the fire and smoke alarm condition.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images