Smoke detection method and device based on target detection algorithm, equipment and medium

Abstract

The application relates to a smoke detection method and device based on a target detection algorithm, equipment and a medium, wherein the method comprises the following steps: acquiring a video to be detected; frame-by-frame interception is performed on the video to be detected to obtain frame-divided video data; the frame-divided video data is input frame by frame into a target detection model to perform feature extraction on each frame of image in the frame-divided video data, thereby obtaining an initial feature map corresponding to each frame of image; feature classification and boundary regression processing are performed on the initial feature map, thereby obtaining a classification result and a target boundary box; a smoke recognition result of a current frame of image is generated based on the classification result and the target boundary box; and when the recognition of all frames of image in the frame-divided video data is completed, a target smoke recognition result is output. According to the application, feature extraction is performed on each frame of image by using the target detection model, thereby obtaining an initial feature map, and feature classification and boundary regression processing are performed on the initial feature map, so that the smoke target with shape change can be coped with, and the accuracy of smoke positioning can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a smoke detection method, apparatus, device and medium based on a target detection algorithm. Background Technology

[0002] Traditional smoke detection methods are mostly based on physical sensor detection technologies, including ionization smoke sensors, gas-sensitive smoke sensors, and photoelectric smoke sensors. However, smoke sensors are not suitable for environments with high sensitivity requirements or complex, diverse, and large-scale environments. Because smoke requires a certain amount of time to propagate, these traditional smoke sensors cannot detect smoke promptly and accurately. Furthermore, since smoke can propagate in multiple directions, their efficiency is low when detecting large areas.

[0003] Existing deep learning-based object detection algorithms, such as RepPoints, are designed for general targets, not smoke targets. This makes them difficult to adapt to smoke targets with changing shapes, and ordinary bounding box encoding is inaccurate for smoke localization. Therefore, there is an urgent need for a smoke detection method that can adapt to smoke targets with changing shapes and improve the accuracy of smoke localization. Summary of the Invention

[0004] The purpose of this application is to propose a smoke detection method, apparatus, device, and medium based on a target detection algorithm, so as to adapt to smoke targets with changing shapes and improve the accuracy of smoke localization.

[0005] To address the aforementioned technical problems, embodiments of this application provide a smoke detection method based on a target detection algorithm, comprising:

[0006] Obtain the video to be tested;

[0007] The video to be detected is extracted frame by frame to obtain frame-by-frame video data;

[0008] The frame-by-frame video data is input into the target detection model to extract features from each frame of the video data, thereby obtaining an initial feature map corresponding to each frame.

[0009] The initial feature map is subjected to feature classification and boundary regression to obtain the classification result and the target bounding box;

[0010] Based on the classification results and target bounding boxes, the smoke recognition result of the current frame image is generated. When the recognition of all frame images in the segmented video data is completed, the target smoke recognition result is output.

[0011] To address the aforementioned technical problems, embodiments of this application provide a smoke detection device based on a target detection algorithm, comprising:

[0012] The video acquisition unit is used to acquire the video to be detected.

[0013] The video framing unit is used to extract frame-by-frame the video to be detected to obtain frame-by-frame video data.

[0014] The feature extraction unit is used to input the frame-by-frame video data into the target detection model to extract features from each frame of the frame-by-frame video data and obtain an initial feature map corresponding to each frame.

[0015] The feature map processing unit is used to perform feature classification and boundary regression processing on the initial feature map to obtain the classification result and the target bounding box;

[0016] The recognition result output unit is used to generate the smoke recognition result of the current frame image based on the classification result and the target bounding box. When the recognition of all frame images in the segmented video data is completed, the target smoke recognition result is output.

[0017] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide a computer device, including one or more processors; and a memory for storing one or more programs, so that the one or more processors implement the smoke detection method based on the target detection algorithm described in any one of the above-mentioned methods.

[0018] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the smoke detection method based on the target detection algorithm described above.

[0019] This invention provides a smoke detection method, apparatus, device, and medium based on a target detection algorithm. The method includes: acquiring a video to be detected; extracting the video frame by frame to obtain segmented video data; inputting the segmented video data frame by frame into a target detection model to extract features from each frame, obtaining an initial feature map for each frame; performing feature classification and boundary regression processing on the initial feature map to obtain a classification result and a target bounding box; generating a smoke recognition result for the current frame based on the classification result and the target bounding box; and outputting the target smoke recognition result when all frames in the segmented video data have been recognized. This invention, by extracting features from each frame using a target detection model to obtain an initial feature map, and then performing feature classification and boundary regression processing on the initial feature map, can handle smoke targets with changing shapes, thus improving the accuracy of smoke localization. Attached Figure Description

[0020] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the implementation of the smoke detection method based on the target detection algorithm provided in this application embodiment;

[0022] Figure 2 This is a schematic diagram of the overall network structure provided in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram of the network detection header module provided in an embodiment of this application;

[0024] Figure 4 This is a flowchart illustrating the implementation of a sub-process in the smoke detection method based on a target detection algorithm provided in this application embodiment;

[0025] Figure 5 This is a flowchart illustrating the implementation of a sub-process in the smoke detection method based on a target detection algorithm provided in this application embodiment;

[0026] Figure 6 This is a flowchart illustrating the implementation of a sub-process in the smoke detection method based on a target detection algorithm provided in this application embodiment;

[0027] Figure 7 This is a comparative schematic diagram of two bounding box encoding methods provided in the embodiments of this application;

[0028] Figure 8 This is a flowchart illustrating the implementation of a sub-process in the smoke detection method based on a target detection algorithm provided in this application embodiment;

[0029] Figure 9 This is a schematic diagram of a smoke detection device based on a target detection algorithm provided in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0032] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0034] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0035] It should be noted that the smoke detection method based on the target detection algorithm provided in this application is generally executed by a computer device, and correspondingly, the smoke detection device based on the target detection algorithm is generally configured in the computer device.

[0036] Please see Figure 1 and Figure 3 , Figure 1 This paper illustrates a specific implementation of a smoke detection method based on a target detection algorithm. Figure 2 This is a schematic diagram of the overall network structure provided in the embodiments of this application; Figure 3 This is a schematic diagram of the network detection header module provided in an embodiment of this application.

[0037] It should be noted that if substantially the same result is obtained, the method of this invention is not based on... Figure 1 Limited to the order of the processes shown, this method includes the following steps:

[0038] S1: Obtain the video to be tested.

[0039] Specifically, a video input stream is acquired from a surveillance camera and used as the video to be detected. The video input stream is a continuous sequence of video frames.

[0040] S2: Extract each frame of the video to be detected to obtain frame-by-frame video data.

[0041] Specifically, the video to be detected is extracted frame by frame to obtain frame-by-frame video data, which includes video data of each frame in the video to be detected.

[0042] S3: Input the frame-by-frame video data into the target detection model to extract features from each frame of the video data and obtain the initial feature map corresponding to each frame.

[0043] Specifically, the target detection model used in this application embodiment is the RepPoints network model, specifically the RepPoints++ network model. The frame-by-frame video data is input into the RepPoints network model to extract features from each frame of the video data, thereby obtaining an initial feature map corresponding to each frame.

[0044] Furthermore, a specific implementation of step S3 is provided: the framed video data is input frame by frame into the RepPoints model, and deformable convolution processing is performed on each frame of the framed video data through the backbone network in the RepPoints model to extract features from each frame of the image, thereby obtaining the FPN feature map corresponding to each frame of the image, and the FPN feature map is used as the initial feature map.

[0045] Specifically, convolution is the core component of neural networks. It uses a fixed-size convolutional kernel to perform sliding window computation on the input tensor, producing an output tensor. For objects with relatively regular shapes, normal convolution has good modeling capabilities, but it has inherent limitations for objects with varied shapes. This limitation stems from the geometry of convolution, which samples at fixed positions in the input feature map. Thus, within the same convolutional layer, the receptive field is the same at all positions. However, since different positions may correspond to objects of different scales or shapes, adaptation to scale or receptive field is necessary for accurate localization. Deformable convolution, a variant of convolution, is a convolutional method that can increase the receptive field size of the convolutional kernel while maintaining resolution. It can learn a deformation offset at each position of the input tensor to adjust the sampling position of the convolutional kernel, allowing it to more closely approximate the shape and size of the object, capturing more detailed information in the image, thereby improving the model's accuracy. Therefore, this embodiment requires deformable convolution processing for each frame of the image to extract features.

[0046] like Figure 2As shown, the overall network structure of the RepPoints model in this embodiment includes an FPN backbone and a head. In this embodiment, the frame-by-frame video data obtained in the above steps is input into the RepPoints model frame by frame. The backbone network in the RepPoints model performs deformable convolution processing on each frame of the frame-by-frame video data to extract features from each frame, obtaining the corresponding FPN (Feature Pyramid Network) feature map. Further, each frame obtains a corresponding initial feature map. In this embodiment, smoke recognition can be performed on the current frame before performing smoke recognition on another frame, until all images have been recognized. Specifically, each frame outputs five FPN feature maps of different sizes.

[0047] Furthermore, a specific implementation of step S42 is provided: the deformable convolution process is represented by a preset formula.

[0048] The preset formula is:

[0049]

[0050] Where p0 is the sum of the values ​​of each point in the feature map and p... n It is each position in the convolution kernel, Δp n Here, w represents the offset at each position, and x represents the weights. By using deformable convolutions for feature extraction, the network automatically focuses on features that are helpful for smoke detection. Regardless of the shape of the smoke, deformable convolutions can always find a suitable offset to capture the features.

[0051] S4: Perform feature classification and boundary regression processing on the initial feature map to obtain the classification result and the target bounding box.

[0052] Specifically, existing network detection head modules use two parallel branches for classification and bounding box regression. This approach is because the classification and regression branches focus on different aspects of the object. The classification branch focuses on areas of the image with rich semantic information, i.e., distinctive features, such as a person's face and a car's wheels. The regression branch focuses more on the overall outline of the object, i.e., the object's edge features. This dual-branch detector is indeed suitable for general objects, but for smoke, which is usually a blurry, varied in shape, and has a certain degree of transparency, blending into the background. This application embodiment visualizes the sensitive regions of smoke in the classification and regression branches, finding that the areas of interest for smoke are basically the same in both branches. This application embodiment does not require two separate branches to detect smoke. These two separate branches increase the number of parameters and force the network to focus on different areas, affecting detection accuracy. This application embodiment requires a shared structure to detect smoke.

[0053] In this embodiment, only one set of shared convolutional modules is used to extract smoke features, instead of the original detector structure which uses two independent convolutional modules to extract smoke features, one for regression and the other for classification. Figure 3 As shown, the FPN module outputs five FPN feature maps of different sizes, all of which need to pass through the same head network for feature classification and boundary regression.

[0054] Please see Figure 4 , Figure 4 A specific implementation of step S4 is shown below:

[0055] S41: Extract smoke features from the initial feature map using a shared convolution module to obtain an initial smoke feature map.

[0056] like Figure 3 As shown, in this embodiment of the application, a set of three shared convolutions are used to extract smoke features from the initial feature map to obtain the initial image. The three convolutions are all 3*3 convolutions.

[0057] S42: Perform deformable convolution processing on the initial smoke feature map to obtain the first offset information.

[0058] Specifically, in this embodiment, 3*3 and 1*1 convolutional blocks are used to perform deformable convolution processing on the initial smoke feature map, and the offset with 18 output channels is obtained, which is the first offset information.

[0059] S43: Based on the first offset information and the initial smoke feature map, feature classification and boundary regression are performed using deformable convolutional blocks to obtain the classification result and the target bounding box.

[0060] Specifically, the classification results include smoke category and non-smoke category. The target bounding box is obtained by refining the regression branch, which yields the x and y coordinates of 9 points. These coordinates are then applied to the initial bounding box to obtain the final bounding box.

[0061] Please see Figure 5 , Figure 5 A specific implementation of step S43 is shown below:

[0062] S431: Encode the first offset information into a bounding box to obtain an initial bounding box.

[0063] Specifically, the first offset information is encoded into a bounding box to obtain the initial Pseudo box, which is the initial bounding box.

[0064] Please see Figure 6 and Figure 7 , Figure 6 One specific implementation of step S431 is shown. Figure 7 This is a comparative schematic diagram of two bounding box encoding methods provided in the embodiments of this application, detailed below:

[0065] S4311: Obtain the set of smoke location points in the first offset information.

[0066] S4312: Obtain the maximum and minimum values ​​in the horizontal and vertical directions of the smoke location point set respectively, and obtain the extreme points in the horizontal and vertical directions.

[0067] S4313: Calculate the average value in the horizontal direction and the average value in the vertical direction based on the extreme points in the horizontal direction and the extreme points in the vertical direction.

[0068] S4314: Based on the average value in the horizontal direction and the average value in the vertical direction, perform bounding box encoding to obtain the initial bounding box.

[0069] Specifically, there are three existing bounding box encoding methods: (1) the minmax method finds the maximum and minimum values ​​in the x and y directions of 9 points and constructs the bounding box; (2) the partial minmax method only considers the first 4 points and performs the minmax operation; (3) the moment method calculates the mean of 9 points as the center of the bounding box, and then restores the bounding box through learnable parameters. These encoding methods are suitable for objects with little shape change, but are not very suitable for smoke. minmax and partial minmax are easily affected by extreme points, thus affecting the accuracy of the bounding box.

[0070] This application embodiment considers the characteristics of smoke: its transparency increases from the center outwards, so the edges are usually blurred and often contain other objects. If points extracted using deformable convolution are directly used for encoding, interference from extreme points is unavoidable, resulting in background information in the detection results and inaccurate smoke localization. The method adopted in this application embodiment is as follows: Obtain the set of smoke location points from the first offset information; obtain the maximum and minimum values ​​in the horizontal and vertical directions of the smoke location point set, respectively, to obtain horizontal and vertical extreme points; calculate the horizontal and vertical average values ​​based on the horizontal and vertical extreme points; and encode the bounding box based on the horizontal and vertical average values ​​to obtain the initial bounding box.

[0071] In one specific embodiment, for the extracted point set, the two points with the maximum and minimum values ​​in the x and y directions are selected respectively, and the average of these two points is used as the maximum and minimum values ​​in the x and y directions. If this average value exceeds half the distance from the center point to either the x or y direction, that is, the distance between these two points exceeds the distance from the center point to that side, then an extreme point in that direction is used as the result instead of the average value. This significantly reduces the background information contained in the pseudo-boundary box, resulting in more accurate smoke localization. This embodiment does not simply take the average of the points. When there are extreme cases in the smoke outline, only one point is at the bottom, while the other points are at the top. This aligns with common practice; smoke is always thin at the bottom and spreads out at the top. Simply taking the average would truncate the smoke at the bottom, only detecting a portion of the smoke, while the lower part, which is usually the more important smoke origin, would not be detected, which is not practical. Therefore, we do not use the average of two points, but rather the extreme value of one point for encoding. The encoding method in this embodiment provides more accurate localization and better results for smoke of various shapes.

[0072] S432: The first offset information and the initial smoke feature map are refined by the deformable convolutional block to obtain the second offset information.

[0073] Specifically, 3x3 and 1x1 convolutional blocks are used to process the first offset information ( Figure 3 The Offset1 shown is subjected to deformable convolution with the initial smoke feature map to perform a refinement regression branch, that is, to further refine the bounding box, and obtain the second offset information. Figure 3 The second offset information is shown as Offset2). This second offset information consists of the refined coordinates of the nine points.

[0074] S433: Apply the second offset information to the initial bounding box to generate the target bounding box.

[0075] Specifically, the first offset information obtained in the above steps is applied to the initial bounding box to obtain the target bounding box.

[0076] S434: The deformable convolutional block performs feature classification based on the initial smoke feature map and the first offset information to obtain the classification result.

[0077] Specifically, 3x3 and 1x1 convolutions are used to perform feature classification based on the initial smoke feature map and the first offset information to obtain the classification result. This classification result is used to determine whether the current frame image contains smoke.

[0078] Furthermore, the embodiments of this application address the following technical problems: First, the smoke features are forcibly dispersed between the two branches; using a shared convolutional module can bring the features that the two branches focus on closer together. Second, there is a lack of shared features between these two branches; using shared convolution increases the interaction between the two branches. Finally, the shared convolutional structure reduces the number of parameters by nearly half compared to the original structure, requiring less time for network computation and resulting in better detection performance, which is crucial for practical applications.

[0079] S5: Generate the smoke recognition result of the current frame image based on the classification result and the target bounding box. When the recognition of all frame images in the segmented video data is completed, output the target smoke recognition result.

[0080] Please see Figure 8 , Figure 8 A specific implementation of step S5 is shown below:

[0081] S51: Determine whether there is smoke in the current frame image based on the classification result.

[0082] S52: If smoke is present, mark the smoke in the current frame image based on the target bounding box.

[0083] S53: Determine whether there is a next frame image in the segmented video data. If there is, perform smoke recognition on the next frame image until all images in all the segmented video data are recognized, and output the target smoke recognition result.

[0084] Specifically, since the classification results for each frame include smoke and non-smoke categories, the system determines whether smoke exists in the current frame based on the classification results. If not, it performs smoke recognition for the next frame. If smoke exists, it marks the smoke based on the target bounding box corresponding to the current frame. Then, it determines whether there is a next frame in the segmented video data. If there is, it performs smoke recognition for the next frame until all images in all segmented video data are recognized, and then outputs the target smoke recognition result.

[0085] In this embodiment, a video to be detected is acquired; the video to be detected is segmented frame by frame to obtain frame-by-frame video data; the frame-by-frame video data is input into a target detection model to extract features from each frame of the video data, obtaining an initial feature map corresponding to each frame; the initial feature map is subjected to feature classification and boundary regression processing to obtain classification results and target bounding boxes; based on the classification results and target bounding boxes, a smoke recognition result for the current frame image is generated; when all frame images in the video data have been recognized, the target smoke recognition result is output. This embodiment of the invention extracts features from each frame image using a target detection model to obtain an initial feature map, and performs feature classification and boundary regression processing on the initial feature map, which can handle smoke targets with changing shapes, thus improving the accuracy of smoke localization.

[0086] Please refer to Figure 9 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of a smoke detection device based on a target detection algorithm. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0087] like Figure 9 As shown, the smoke detection device based on the target detection algorithm in this embodiment includes: a video acquisition unit 61, a video framing unit 62, a feature extraction unit 63, a feature map processing unit 64, and a recognition result output unit 65, wherein:

[0088] The video acquisition unit 61 is used to acquire the video to be detected.

[0089] The video framing unit 62 is used to extract frame by frame the video to be detected to obtain frame-segmented video data.

[0090] The feature extraction unit 63 is used to input the framed video data frame by frame into the target detection model to extract features from each frame of the framed video data and obtain an initial feature map corresponding to each frame.

[0091] Feature map processing unit 64 is used to perform feature classification and boundary regression processing on the initial feature map to obtain classification results and target bounding boxes;

[0092] The recognition result output unit 65 is used to generate the smoke recognition result of the current frame image based on the classification result and the target bounding box, and outputs the target smoke recognition result when all frame images in the segmented video data have been recognized.

[0093] Furthermore, the feature map processing unit 64 includes:

[0094] An initial smoke feature map generation unit is used to extract smoke features from the initial feature map through a shared convolutional module to obtain an initial smoke feature map.

[0095] The first offset information generation unit is used to perform deformable convolution processing on the initial smoke feature map to obtain the first offset information;

[0096] A deformable convolutional unit is used to perform feature classification and boundary regression processing based on the first offset information and the initial smoke feature map through deformable convolutional blocks to obtain the classification result and the target bounding box.

[0097] Furthermore, deformable convolution includes:

[0098] A bounding box encoding unit is used to encode the first offset information into a bounding box to obtain an initial bounding box.

[0099] The second offset information generation unit is used to perform a refined regression branch process on the first offset information and the initial smoke feature map through the deformable convolution block to obtain the second offset information;

[0100] A target bounding box generation unit is used to apply the second offset information to the initial bounding box to generate the target bounding box;

[0101] The feature classification unit is used to perform feature classification based on the initial smoke feature map and the first offset information using the deformable convolutional block to obtain the classification result.

[0102] Furthermore, the bounding box encoding unit includes:

[0103] The smoke location point set extraction unit is used to obtain the smoke location point set in the first offset information;

[0104] The extreme point acquisition unit is used to acquire the maximum and minimum values ​​in the horizontal and vertical directions of the smoke location point set, respectively, to obtain the extreme points in the horizontal and vertical directions.

[0105] The average value calculation unit is used to calculate the average value in the horizontal direction and the average value in the vertical direction based on the horizontal extreme points and the vertical extreme points;

[0106] An initial bounding box generation unit is used to encode the bounding box based on the horizontal average value and the vertical average value to obtain the initial bounding box.

[0107] Furthermore, the feature extraction unit 63 includes:

[0108] The FPN feature map extraction unit is used to input the framed video data frame by frame into the RepPoints model, and to perform deformable convolution processing on each frame of the framed video data through the backbone network in the RepPoints model to extract features from each frame of the image, thereby obtaining the FPN feature map corresponding to each frame of the image, and using the FPN feature map as the initial feature map.

[0109] Furthermore, the deformable convolution process is represented by a preset formula.

[0110] The preset formula is:

[0111]

[0112] Where p0 is each point in the feature map, p n It is each position in the convolution kernel, Δp n It is the offset of each position.

[0113] Furthermore, the recognition result output unit 65 includes:

[0114] A smoke detection unit is used to determine whether smoke exists in the current frame image based on the classification result;

[0115] A smoke marking unit is used to mark the smoke in the current frame image based on the target bounding box if smoke is present.

[0116] The target smoke recognition result generation unit is used to determine whether there is a next frame image in the segmented video data. If there is, smoke recognition is performed on the next frame image until all images in all the segmented video data are recognized, and the target smoke recognition result is output.

[0117] In this embodiment, a video to be detected is acquired; the video to be detected is segmented frame by frame to obtain frame-by-frame video data; the frame-by-frame video data is input into a target detection model to extract features from each frame of the video data, obtaining an initial feature map corresponding to each frame; the initial feature map is subjected to feature classification and boundary regression processing to obtain classification results and target bounding boxes; based on the classification results and target bounding boxes, a smoke recognition result for the current frame image is generated; when all frame images in the video data have been recognized, the target smoke recognition result is output. This embodiment of the invention extracts features from each frame image using a target detection model to obtain an initial feature map, and performs feature classification and boundary regression processing on the initial feature map, which can handle smoke targets with changing shapes, thus improving the accuracy of smoke localization.

[0118] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 10 , Figure 10 This is a basic structural block diagram of the computer device in this embodiment.

[0119] Computer device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected via a system bus. It should be noted that only a computer device 7 with these three components (memory 71, processor 72, and network interface 73) is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0120] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0121] The memory 71 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the computer device 7, such as the hard disk or memory of the computer device 7. In other embodiments, the memory 71 may also be an external storage device of the computer device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 7. Of course, the memory 71 may include both internal storage units and external storage devices of the computer device 7. In this embodiment, the memory 71 is typically used to store the operating system and various application software installed on the computer device 7, such as the program code of a smoke detection method based on a target detection algorithm. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or will be output.

[0122] In some embodiments, processor 72 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 72 is typically used to control the overall operation of the computer device 7. In this embodiment, processor 72 is used to run program code stored in memory 71 or process data, for example, to run the program code of the smoke detection method based on the target detection algorithm described above, to implement various embodiments of the smoke detection method based on the target detection algorithm.

[0123] The network interface 73 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 7 and other electronic devices.

[0124] This application also provides another embodiment, namely, providing a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the smoke detection method based on the target detection algorithm described above.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0126] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A smoke detection method based on a target detection algorithm, characterized in that, include: Obtain the video to be tested; The video to be detected is extracted frame by frame to obtain frame-by-frame video data; The frame-by-frame video data is input into the target detection model to extract features from each frame of the video data, thereby obtaining an initial feature map corresponding to each frame. The initial feature map is subjected to feature classification and boundary regression to obtain the classification result and the target bounding box; Based on the classification results and target bounding boxes, generate the smoke recognition result of the current frame image. When the recognition of all frame images in the segmented video data is completed, output the target smoke recognition result. The step of performing feature classification and boundary regression on the initial feature map to obtain classification results and target bounding boxes includes: The initial smoke feature map is obtained by extracting smoke features from the initial feature map using a shared convolution module. The initial smoke feature map is subjected to deformable convolution processing to obtain the first offset information; Obtain the set of smoke location points from the first offset information; Obtain the maximum and minimum values ​​in the horizontal and vertical directions of the smoke location point set, respectively, to obtain the extreme points in the horizontal and vertical directions; Calculate the average value in the horizontal direction and the average value in the vertical direction based on the extreme points in the horizontal direction and the extreme points in the vertical direction; Boundary box encoding is performed based on the horizontal average value and the vertical average value to obtain the initial bounding box. The first offset information and the initial smoke feature map are refined by a regression branch process using deformable convolutional blocks to obtain the second offset information; The second offset information is applied to the initial bounding box to generate the target bounding box; The deformable convolutional block performs feature classification based on the initial smoke feature map and the first offset information to obtain the classification result; The target detection model is the RepPoints model. The step of inputting the frame-by-frame video data into the target detection model to extract features from each frame of the video data and obtain an initial feature map corresponding to each frame includes: The frame-segmented video data is input frame by frame into the RepPoints model. The backbone network in the RepPoints model performs deformable convolution processing on each frame of the frame-segmented video data to extract features from each frame, thereby obtaining the FPN feature map corresponding to each frame, and using the FPN feature map as the initial feature map.

2. The smoke detection method based on target detection algorithm according to claim 1, characterized in that, The deformable convolution process is represented by a preset formula; The preset formula is: ； in, It is each point in the feature map. It is each position in the convolution kernel. It is the offset of each position.

3. The smoke detection method based on a target detection algorithm according to any one of claims 1 to 2, characterized in that, The process of generating smoke recognition results for the current frame image based on the classification results and target bounding boxes, and outputting target smoke recognition results when all frame images in the segmented video data have been recognized, includes: Based on the classification results, determine whether there is smoke in the current frame image; If smoke is present, the smoke in the current frame image is marked based on the target bounding box; Determine whether there is a next frame image in the segmented video data. If there is, perform smoke recognition on the next frame image until all images in all the segmented video data have been recognized, and output the target smoke recognition result.

4. A smoke detection device based on a target detection algorithm, characterized in that, include: The video acquisition unit is used to acquire the video to be detected. The video framing unit is used to extract frame-by-frame the video to be detected to obtain frame-by-frame video data. The feature extraction unit is used to input the frame-by-frame video data into the target detection model to extract features from each frame of the frame-by-frame video data and obtain an initial feature map corresponding to each frame. The feature map processing unit is used to perform feature classification and boundary regression processing on the initial feature map to obtain the classification result and the target bounding box; The recognition result output unit is used to generate the smoke recognition result of the current frame image based on the classification result and the target bounding box, and output the target smoke recognition result when all frame images in the segmented video data have been recognized. The feature map processing unit includes: An initial smoke feature map generation unit is used to extract smoke features from the initial feature map through a shared convolutional module to obtain an initial smoke feature map. The first offset information generation unit is used to perform deformable convolution processing on the initial smoke feature map to obtain the first offset information; The smoke location point set extraction unit is used to obtain the smoke location point set in the first offset information; The extreme point acquisition unit is used to acquire the maximum and minimum values ​​in the horizontal and vertical directions of the smoke location point set, respectively, to obtain the extreme points in the horizontal and vertical directions. The average value calculation unit is used to calculate the average value in the horizontal direction and the average value in the vertical direction based on the horizontal extreme points and the vertical extreme points; The initial bounding box generation unit is used to perform bounding box encoding based on the horizontal average value and the vertical average value to obtain an initial bounding box. The second offset information generation unit is used to perform a refined regression branch process on the first offset information and the initial smoke feature map through the deformable convolution block to obtain the second offset information; A target bounding box generation unit is used to apply the second offset information to the initial bounding box to generate the target bounding box; The feature classification unit is used to perform feature classification based on the initial smoke feature map and the first offset information using the deformable convolutional block to obtain the classification result. The target detection model is the RepPoints model, and the feature extraction unit includes: The FPN feature map extraction unit is used to input the framed video data frame by frame into the RepPoints model, and to perform deformable convolution processing on each frame of the framed video data through the backbone network in the RepPoints model to extract features from each frame of the image, thereby obtaining the FPN feature map corresponding to each frame of the image, and using the FPN feature map as the initial feature map.

5. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the smoke detection method based on the target detection algorithm as described in any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the smoke detection method based on the target detection algorithm as described in any one of claims 1 to 2.

Images