Smoke and fire target detection method and device, electronic equipment and computer storage medium

Abstract

The invention relates to a smoke and fire target detection method and device, electronic equipment and a computer storage medium, and belongs to the technical field of image recognition, and the smoke and fire target detection method comprises the steps: obtaining a plurality of continuous frames of to-be-recognized images for the same scene, and processing the adjacent frames of to-be-recognized images through an inter-frame difference method, determining smoke and fire candidate pixel points in each to-be-identified image, clustering the smoke and fire candidate pixel points by adopting a clustering algorithm, and determining a smoke and fire candidate area in each to-be-identified image; and extracting bottom-layer statistical characteristics and texture characteristics of the smoke and fire candidate region, performing decision making on the bottom-layer statistical characteristics and the texture characteristics by adopting a decision-making tree, and determining a smoke and fire target region. According to the invention, aiming at the characteristic that the firework target is rapidly diffused and has more dynamic information compared with cloud and mist and light, a dynamic multi-frame dynamic tracking method is used for auxiliary identification of the firework target, and multi-dimensional features of the firework target are combined, so that identification of the firework target is more accurate.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, electronic equipment and computer storage medium for detecting fireworks targets. Background Art

[0002] In environmental monitoring and disaster warning, the recognition of fireworks is crucial, because timely and accurate detection of fireworks can effectively prevent the spread of fire and reduce potential economic and human losses. However, the recognition of fireworks targets faces many challenges in practical applications, especially when detecting in the wild and in large-scale environments.

[0003] Existing firework detection schemes often rely on a single band as a parameter for pattern recognition, usually based on the color or grayscale characteristics of the visible light band. Although this method can work effectively under ideal conditions, it is easily affected by a variety of interference factors in the actual environment. For example, in daily environments, white clouds, drifting white fog, tree shadows reflected on the water, and orange-red lights may all produce similar visual features to firework targets, resulting in false alarms in the system. Especially at night, when there is insufficient light or some areas are overexposed, white radial spots may also be mistaken for firework targets, further increasing the false alarm rate.

[0004] This shows that the accuracy of existing technologies in identifying fireworks in complex and changing environments is not high. Summary of the invention

[0005] In view of this, it is necessary to provide a fireworks target detection method, device, electronic device and computer storage medium to solve the problem that the existing technology has low accuracy in fireworks recognition in complex and changeable scenes.

[0006] In order to solve the above problems, the present invention provides a method for detecting fireworks targets, comprising: Acquire a plurality of continuous frames of to-be-recognized images for the same scene, wherein the to-be-recognized images include a to-be-recognized fireworks target; Using the inter-frame difference method to process the images to be identified in adjacent frames, determine the candidate pixels of fireworks in each image to be identified, and using a clustering algorithm to cluster the candidate pixels of fireworks, determine the candidate areas of fireworks in each image to be identified; The underlying statistical features and texture characteristics of the fireworks candidate area are extracted, and a decision tree is used to make decisions on the underlying statistical features and texture characteristics to determine the fireworks target area.

[0007] In a possible implementation manner, after acquiring a plurality of consecutive frames of images to be recognized for the same scene, the method includes: Downsample the pixels in the image to be recognized to obtain a downsampled feature map; Perform median filtering on the downsampled feature map to obtain a grayscale image of the image to be identified.

[0008] In a possible implementation manner, the process of processing the images to be identified of adjacent frames by using the inter-frame difference method to determine the candidate pixels of fireworks in each image to be identified includes: The fireworks candidate pixel points in each image to be identified are calculated using the fireworks candidate pixel point determination formula for the grayscale images corresponding to the images to be identified in each adjacent frame. The fireworks candidate pixel point determination formula is:

[0009]

[0010]

[0011] in, is a first grayscale image corresponding to the first image to be identified in the adjacent frames of images to be identified, is a second grayscale image corresponding to the second image to be identified in the adjacent frame of images to be identified, is a difference map of the pixel values ​​of the corresponding pixels in the first grayscale image and the second grayscale image, is the pixel value of the pixel with the smallest pixel value in the difference map, is the pixel value of the pixel with the largest pixel value in the difference map, is the pixel point in the difference map ( i , j ), Pixels ( i , j ) of the fireworks judgment value, Represents a pixel ( i , j ) is the pixel value after binarization, It is the fireworks determination threshold of the preset fireworks candidate pixel point.

[0012] In a possible implementation, the clustering algorithm is a DBSCAN clustering algorithm, and the clustering algorithm is used to cluster the fireworks candidate pixels to determine the fireworks candidate regions in each image to be identified, including: The DBSCAN clustering algorithm is used to cluster the connected fireworks candidate pixels in the image to be identified to obtain multiple fireworks candidate regions of preset shapes, wherein the fixed parameters in the DBSCAN clustering algorithm are calculated using a parameter determination formula, and the parameter determination formula is:

[0013]

[0014] in, It refers to m The firework candidate regions correspond to eps Parameter value, Indicates that m There are N candidate firework pixels, It refers to m Candidate fireworks areas, , It refers to the m The first k The coordinates of the candidate pixels of the fireworks, , It refers to the m The first h The coordinates of the candidate pixels of the fireworks, It refers to the m The firework candidate regions correspond to Parameter value.

[0015] In a possible implementation, extracting underlying statistical features and texture features of the fireworks candidate area includes: Acquire polarization angle images and infrared images corresponding to the plurality of continuous frames of images to be identified, wherein the polarization angle images record polarization angle information of each pixel point; Based on the mean and covariance of the polarization angles of each pixel in the fireworks candidate area; Determine a highlight pixel point in the infrared image, the pixel value of the highlight pixel point is greater than a preset highlight determination threshold, and calculate the minimum distance between each pixel point in the fireworks candidate area and the highlight pixel point; The texture features of the fireworks candidate area are calculated using a preset gray level co-occurrence matrix.

[0016] In a possible implementation manner, determining highlight pixel points in the infrared image includes: Adjacent frames of infrared images are processed using an inter-frame difference method to determine highlight pixel points in the infrared images.

[0017] In a possible implementation manner, the using a decision tree to make a decision on the underlying statistical features and texture characteristics to determine the fireworks target area includes: A preset decision tree is used to make a decision on the mean and covariance of the polarization angle, the minimum distance between each pixel in the fireworks candidate area and the highlight pixel, and the texture feature to obtain an authenticity index of the fireworks candidate area; The fireworks candidate area whose authenticity index is greater than a preset index threshold is determined as the fireworks target area.

[0018] The embodiment of the present invention further provides a smoke target detection device, comprising: An image acquisition module, used to acquire a plurality of continuous frames of to-be-recognized images for the same scene, wherein the to-be-recognized images include fireworks targets to be recognized; A candidate region determination module is used to process the images to be identified of adjacent frames using an inter-frame difference method to determine candidate pixels of fireworks in each image to be identified, and cluster the candidate pixels of fireworks using a clustering algorithm to determine candidate regions of fireworks in each image to be identified; The fireworks target recognition module is used to extract the underlying statistical features and texture characteristics of the fireworks candidate area, and use a decision tree to make decisions on the underlying statistical features and texture characteristics to determine the fireworks target area.

[0019] The present invention also provides an electronic device, comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory, and is used to execute the program stored in the memory to implement the steps in the method for detecting fireworks targets described in any one of the above embodiments.

[0020] The present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, wherein the program or instruction, when executed by a processor, can implement the steps in the method for detecting fireworks targets described in any of the above embodiments.

[0021] The beneficial effects of the present invention are as follows: the fireworks target detection method provided by the present invention collects multiple frames of continuous images to be identified for the same scene, and processes the images to be identified by using the inter-frame difference method to obtain fireworks candidate pixel points in the images to be identified, and then clusters the fireworks candidate pixel points by using a clustering algorithm to obtain multiple fireworks candidate areas, and based on the underlying statistical characteristics and texture characteristics of each fireworks candidate area, a decision tree is used to make a decision to determine the fireworks target area, and in view of the fact that fireworks targets spread rapidly and have more dynamic information than clouds and lights, a dynamic multi-frame dynamic tracking method is used to perform auxiliary identification of fireworks targets, and the multi-dimensional characteristics of fireworks targets are combined to make the identification of fireworks targets more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] FIG1 is a flow chart of a method for detecting fireworks targets according to an embodiment of the present invention; Figure 2 A schematic diagram of a flow chart of an image preprocessing method provided by an embodiment of the present invention; Figure 3 A flowchart of a method for extracting multidimensional features provided by an embodiment of the present invention; Figure 4 A flowchart of a method for determining a fireworks target area provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a fireworks target detection device provided by an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of the present invention and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0025] The descriptions of "first", "second", etc. involved in the embodiments of the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the technical features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0026] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0027] A specific embodiment of the present invention, as Figure 1 As shown, a method for detecting fireworks targets is disclosed, the method comprising: S101, acquiring a plurality of continuous frames of to-be-recognized images of the same scene, wherein the to-be-recognized images include a to-be-recognized fireworks target; S102, using an inter-frame difference method to process the images to be identified in adjacent frames, determining candidate fireworks pixels in each image to be identified, and using a clustering algorithm to cluster the candidate fireworks pixels to determine candidate fireworks regions in each image to be identified; S103, extracting underlying statistical features and texture characteristics of the fireworks candidate area, and using a decision tree to make decisions on the underlying statistical features and texture characteristics to determine the fireworks target area.

[0028] In an embodiment of the present invention, multiple continuous frames of images to be identified of the same scene refer to multiple continuous frames of images captured for the same surveillance scene, which can be understood as the position and acquisition angle of the image acquisition device remain unchanged, and continuous acquisition is performed on the same position, such as multiple continuous frames of images in a video image of a scene captured by a video surveillance device. Furthermore, the same scene may be a scene containing multiple influencing factors, such as the wild, a city at night, etc. The fireworks target to be identified refers to the fireworks target existing in the scene, which may be a single smoke target or a target with both smoke and flames.

[0029] In an embodiment of the present invention, a fireworks candidate pixel point refers to a pixel point in an image to be identified that may be a fireworks target, and the pixel value of the pixel point is generally higher than the pixel value of a non-fireworks target. A fireworks candidate region refers to a possible fireworks target region obtained after clustering the fireworks candidate pixel points. Specifically, for the multiple continuous frames of images to be identified, the inter-frame difference method is used to process the adjacent frame images, and the corresponding fireworks candidate pixel points in the adjacent frame images can be determined. The specific calculation process of the inter-frame difference method will be described in detail later in the present invention. After determining the fireworks candidate pixel points, the fireworks candidate pixel points are clustered to obtain multiple fireworks candidate regions. The specific clustering process of the fireworks candidate pixel points will be described later in the present invention.

[0030] In an embodiment of the present invention, after determining the fireworks candidate region in the image to be identified, the multi-dimensional features of the fireworks candidate region are extracted, including the underlying statistical features and texture features. The specific information of the underlying statistical features and texture features and the extraction method are described later in the present invention. The fireworks target region in the image to be identified can be obtained by making decisions on the multi-dimensional features through a preset decision tree. The fireworks target detection method provided by the present invention collects multiple frames of continuous images to be identified for the same scene, and uses the inter-frame difference method to process the images to be identified to obtain the fireworks candidate pixels in the image to be identified, and then uses a clustering algorithm to cluster the fireworks candidate pixels to obtain multiple fireworks candidate regions. Based on the underlying statistical features and texture features of each fireworks candidate region, a decision tree is used to make a decision to determine the fireworks target region. In view of the rapid diffusion of fireworks targets, which have more dynamic information than clouds and lights, a dynamic multi-frame dynamic tracking method is used to perform auxiliary recognition of fireworks targets, and the multi-dimensional features of fireworks targets are combined to more accurately recognize fireworks targets.

[0031] In a possible implementation of the present invention, Figure 2 As shown, after obtaining multiple consecutive frames of images to be recognized for the same scene, the following steps are included: S201, down-sampling the pixels in the image to be recognized to obtain a down-sampled feature map; S202, performing median filtering on the downsampled feature map to obtain a grayscale image of the image to be identified.

[0032] In an embodiment of the present invention, for the multiple continuous frames of images to be identified that are collected, in order to reduce the workload of subsequent processing, the pixels of the images to be identified can be downsampled first to obtain a downsampled feature map. The specific downsampling method can be determined according to the actual application scenario, and the present invention does not impose any restrictions on this.

[0033] Furthermore, for each pixel in the downsampled feature map, a median filter can be used to obtain a grayscale image of the downsampled image. The median filter formula is: (1) in, For the output image, is the input image, a, b represent the coordinates of a pixel, c is the window size of the median filter, It refers to the function operation of removing the median of the pixel point in the center of the window. For each pixel point (a, b) in the favorite sampled image, a window of size c is constructed with the pixel point (a, b) as the center, and the median of the pixel values ​​of all pixels in the window is calculated. The pixel value of the pixel point (a, b) is modified to the median value to obtain the grayscale image of the downsampled image.

[0034] The embodiment of the present invention can improve the speed of firework target recognition by preprocessing the image to be recognized.

[0035] As a possible implementation of the present invention, in this implementation, the images to be identified of adjacent frames are processed using an inter-frame difference method to determine candidate pixels of fireworks in each image to be identified, including: The fireworks candidate pixel point determination formula is used to calculate the fireworks candidate pixel points in each image to be identified for the grayscale image corresponding to each adjacent frame of the image to be identified. The fireworks candidate pixel point determination formula is: (2) (3) (4) in, is a first grayscale image corresponding to the first image to be identified in the adjacent frames of images to be identified, is a second grayscale image corresponding to the second image to be identified in the adjacent frame of images to be identified, is a difference map of the pixel values ​​of the corresponding pixels in the first grayscale image and the second grayscale image, is the pixel value of the pixel with the smallest pixel value in the difference map, is the pixel value of the pixel with the largest pixel value in the difference map, is the pixel point in the difference map ( i , j ), Pixels ( i , j ) of the fireworks judgment value, Represents a pixel ( i , j ) is the pixel value after binarization, is the preset fireworks decision threshold of the candidate fireworks pixel, which is usually 200.

[0036] In the embodiment of the present invention, based on the above formulas (2) to (4), the grayscale images corresponding to the images to be identified in each adjacent frame can be calculated to obtain the candidate pixels of fireworks. Specifically, the pixel values ​​of the corresponding pixels in the grayscale images of the adjacent frames are subtracted, and the absolute value of the obtained difference is removed to obtain the difference map. Then, the pixel values ​​of each pixel in the difference map are calculated using formula (3) to obtain the fireworks determination value of each pixel in the difference map. The fireworks determination value is used to determine whether the pixel is a candidate pixel of fireworks. Then, based on the fireworks determination value, all the pixels in the difference map are binarized, and the pixel values ​​of the pixels whose fireworks determination values ​​are greater than the fireworks determination threshold are set to 255, otherwise set to 0, wherein the fireworks determination threshold is set according to the actual situation, such as 200, 210, etc.

[0037] The embodiment of the present invention processes the grayscale images of the adjacent frames of the to-be-recognized images by using the inter-frame difference method, and can accurately determine the candidate pixels of fireworks.

[0038] As a possible implementation of the present invention, in this implementation, the clustering algorithm is a DBSCAN clustering algorithm, and the clustering algorithm is used to cluster the candidate pixels of fireworks to determine the candidate areas of fireworks in each image to be identified, including: The DBSCAN clustering algorithm is used to cluster the connected fireworks candidate pixels in the image to be identified, and multiple fireworks candidate regions of preset shapes are obtained. The fixed parameters in the DBSCAN clustering algorithm are calculated using the parameter determination formula, which is: (6) (7) in, It refers to m The firework candidate regions correspond to eps Parameter value, Indicates that m There are N candidate firework pixels, It refers to m Candidate fireworks areas, , It refers to the m The first k The coordinates of the candidate pixels of the fireworks, , It refers to the m The first h The coordinates of the candidate pixels of the fireworks, It refers to the m The firework candidate regions correspond to Parameter value.

[0039] In an embodiment of the present invention, the fireworks candidate pixel points determined in the aforementioned embodiment need to be clustered to obtain a fireworks candidate area. Generally, the connected fireworks candidate pixel points are clustered separately to obtain a fireworks candidate pixel point area, and then the fireworks candidate pixel point area is divided to obtain a fireworks candidate area of ​​a preset shape. Specifically, the fireworks candidate area is generally a rectangle. For a certain fireworks candidate pixel point area, the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate and maximum vertical coordinate of the pixels in the pixel point area are obtained, and the pixels corresponding to the four coordinates are used as the four vertices of the rectangle to obtain the fireworks candidate area. Among them, when clustering connected fireworks candidate pixels, the DBSCAN clustering algorithm can be used. For the fixed parameters of the DBSCAN clustering algorithm, eps and , can be calculated using formula (5) and formula (6).

[0040] The embodiment of the present invention clusters the candidate pixels of fireworks by using a clustering algorithm, and can obtain a plurality of candidate areas of fireworks with preset shapes, which is convenient for the subsequent recognition of fireworks targets.

[0041] As a possible implementation mode of the present invention, in this implementation mode, Figure 3 As shown, the underlying statistical features and texture features of the fireworks candidate area are extracted, including: S301, obtaining polarization angle images and infrared images corresponding to a plurality of consecutive frames of images to be identified, wherein the polarization angle image records polarization angle information of each pixel point; S302, based on the mean and covariance of the polarization angles of each pixel in the fireworks candidate area; S303, determining a highlight pixel in the infrared image, the pixel value of which is greater than a preset highlight determination threshold, and calculating the minimum distance between each pixel in the fireworks candidate area and the highlight pixel; S304, using a preset gray level co-occurrence matrix to calculate texture features of the fireworks candidate area.

[0042] In the embodiment of the present invention, the multi-dimensional features of the fireworks candidate region include underlying statistical features and texture features. The underlying statistical features are determined based on the polarization angle image of the fireworks candidate region. In the polarization angle image, the fireworks target usually exhibits high brightness and flat local characteristics. These features can effectively distinguish the fireworks target from the background noise and can be calculated using the following formula: (8) (9) in, M is the total number of pixels in the fireworks candidate area, is the pixel point in the fireworks candidate area m The pixel value of is the average value of the pixel values ​​of each pixel in the fireworks candidate area, is the variance of the pixel values ​​of each pixel in the fireworks candidate area.

[0043] Furthermore, the distance between each pixel and the highlighted pixel in the fireworks candidate area can be calculated through the infrared image of the fireworks candidate area. For example, the minimum distance between each pixel and the highlighted pixel in each target area can be calculated using formula (10): (10) in, Refers to the highlighted pixels in the image to be identified. There may be multiple highlighted pixels. is the coordinate of the highlighted pixel, The first i The coordinates of the pixels, The first i The minimum distance between a pixel and a highlighted pixel.

[0044] Furthermore, the texture features of the fireworks candidate area are calculated through the gray level co-occurrence matrix, which is defined as follows:

[0045] in, is the texture feature of the fireworks candidate area, is the contrast feature, is the correlation feature, For performance characteristics, Homogeneous characteristics.

[0046] The embodiment of the present invention recognizes fireworks targets by extracting multi-dimensional features of fireworks candidate areas, thereby improving the accuracy of fireworks target recognition.

[0047] As a possible implementation manner of the present invention, determining highlight pixel points in an infrared image includes: Adopt the inter-frame difference method to process the adjacent frames of infrared images and determine the highlight pixels in the infrared images.

[0048] In the embodiment of the present invention, the method for determining the highlight pixel points in the infrared image is similar to the method for determining the candidate pixel points of fireworks in the image to be identified. The highlight pixel points in each infrared image are calculated using the highlight pixel point determination formula for the grayscale images corresponding to the infrared images of each adjacent frame. The highlight pixel point determination formula is: (2) (3) (4) in, is the first grayscale image corresponding to the first infrared image in the adjacent frame infrared image, is the second grayscale image corresponding to the second infrared image in the adjacent frame infrared image, is a difference map of the pixel values ​​of the corresponding pixels in the first grayscale image and the second grayscale image, is the pixel value of the pixel with the smallest pixel value in the difference map, is the pixel value of the pixel with the largest pixel value in the difference map, is the pixel point in the difference map ( i , j ), Pixels (i , j ) is the highlight point judgment value, Represents a pixel ( i , j ) is the pixel value after binarization, The threshold for determining the highlight point of the preset highlight point pixel is usually 250.

[0049] The embodiment of the present invention processes adjacent frames of infrared images by using an inter-frame difference method, and can determine highlight pixel points, which is convenient for subsequent identification of fireworks targets.

[0050] Further, such as Figure 4 As shown in the figure, a decision tree is used to make decisions based on the underlying statistical features and texture characteristics to determine the target area of ​​fireworks, including: S401, using a preset decision tree to make a decision on the mean and covariance of the polarization angle, the minimum distance between each pixel point and the highlight pixel point in the fireworks candidate area, and the texture feature, to obtain an authenticity index of the fireworks candidate area; S402: Determine the fireworks candidate area whose authenticity index is greater than a preset index threshold as the fireworks target area.

[0051] In the embodiment of the present invention, following the above embodiment, a decision tree is used to make a decision on the above multi-dimensional features to obtain the authenticity index of the fireworks candidate area, which is defined as follows:

[0052] in, is the authenticity indicator of the fireworks candidate area as a fireworks target, For a decision tree, When it is greater than the preset index threshold, it indicates that the fireworks candidate area is a fireworks target area.

[0053] The embodiment of the present invention makes a decision on the multi-dimensional features of the fireworks candidate area through a decision tree, so that the fireworks target area can be identified more accurately.

[0054] In order to better implement the fireworks target detection method in the embodiment of the present invention, based on the fireworks target detection method, correspondingly, Figure 5 As shown, the embodiment of the present invention further provides a smoke and fire target detection device, and the smoke and fire target detection device 500 includes: An image acquisition module 501 is used to acquire a plurality of continuous frames of to-be-recognized images of the same scene, wherein the to-be-recognized images include to-be-recognized fireworks targets; The candidate region determination module 502 is used to process the images to be identified of adjacent frames by using the inter-frame difference method to determine the candidate pixels of fireworks in each image to be identified, and cluster the candidate pixels of fireworks by using the clustering algorithm to determine the candidate regions of fireworks in each image to be identified; The fireworks target recognition module 503 is used to extract the underlying statistical features and texture characteristics of the fireworks candidate area, and use a decision tree to make decisions on the underlying statistical features and texture characteristics to determine the fireworks target area.

[0055] The fireworks target detection device 500 provided in the above embodiment can implement the technical solution described in the above fireworks target detection method embodiment. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above fireworks target detection method embodiment, which will not be repeated here.

[0056] The fireworks target detection device provided by the present invention collects multiple frames of continuous images to be identified for the same scene, and processes the images to be identified by using the inter-frame difference method to obtain fireworks candidate pixel points in the images to be identified, and then clusters the fireworks candidate pixel points by using a clustering algorithm to obtain multiple fireworks candidate areas. Based on the underlying statistical characteristics and texture characteristics of each fireworks candidate area, a decision tree is used to make a decision to determine the fireworks target area. In view of the fact that fireworks targets spread rapidly and have more dynamic information than clouds and lights, a dynamic multi-frame dynamic tracking method is used to perform auxiliary identification of fireworks targets, and the multi-dimensional characteristics of fireworks targets are combined to make the identification of fireworks targets more accurate.

[0057] like Figure 6 As shown, the present invention also provides an electronic device 600. The electronic device 600 includes a processor 601, a memory 602 and a display 603. Figure 6 Only some components of the electronic device 600 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0058] In some embodiments, the processor 601 may be a central processing unit (CPU), a microprocessor or other data processing chip, and is used to run program codes or process data stored in the memory 602, such as the fireworks target detection method of the present invention.

[0059] In some embodiments, the processor 601 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 601 may be local or remote. In some embodiments, the processor 601 may be implemented in a cloud platform. In some embodiments, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-cloud, etc., or any combination thereof.

[0060] In some embodiments, the memory 602 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the electronic device 600. In other embodiments, the memory 602 may also be an external storage device of the electronic device 600, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 600.

[0061] Furthermore, the memory 602 may include both an internal storage unit of the electronic device 600 and an external storage device. The memory 602 is used to store application software installed in the electronic device 600 and various data.

[0062] In some embodiments, the display 603 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 603 is used to display information of the electronic device 600 and to display a visual user interface. The components 601-603 of the electronic device 600 communicate with each other through a system bus.

[0063] In some embodiments, when the processor 601 executes the fireworks target detection program in the memory 602, the following steps may be implemented: Acquire multiple continuous frames of to-be-recognized images of the same scene, wherein the to-be-recognized images include to-be-recognized fireworks targets; The images to be identified of adjacent frames are processed by using the inter-frame difference method to determine the candidate pixels of fireworks in each image to be identified, and the candidate pixels of fireworks are clustered by using the clustering algorithm to determine the candidate areas of fireworks in each image to be identified; The underlying statistical features and texture characteristics of the fireworks candidate area are extracted, and a decision tree is used to make decisions on the underlying statistical features and texture characteristics to determine the fireworks target area.

[0064] It should be understood that: when the processor 601 executes the fireworks target detection program in the memory 602, in addition to the above functions, other functions may also be implemented, and the details may refer to the description of the corresponding method embodiment above.

[0065] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 600 mentioned, and the electronic device 600 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, etc. Exemplary embodiments of portable electronic devices include but are not limited to portable electronic devices equipped with IOS, Android, Microsoft or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 600 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0066] Accordingly, an embodiment of the present invention further provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps or functions of the fireworks target detection method provided in the above-mentioned method embodiments can be implemented.

[0067] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0068] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting fireworks targets, characterized in that: include: Acquire multiple consecutive frames of to-be-recognized images of the same scene, wherein the to-be-recognized images include a to-be-recognized fireworks target; Using the inter-frame difference method to process the images to be identified in adjacent frames, determine the candidate pixels of fireworks in each image to be identified, and using a clustering algorithm to cluster the candidate pixels of fireworks, determine the candidate areas of fireworks in each image to be identified; The underlying statistical features and texture characteristics of the fireworks candidate area are extracted, and a decision tree is used to make decisions on the underlying statistical features and texture characteristics to determine the fireworks target area.

2. The method for detecting fireworks targets according to claim 1, characterized in that: After obtaining a plurality of consecutive frames of images to be recognized for the same scene, the method includes: Downsample the pixels in the image to be recognized to obtain a downsampled feature map; Perform median filtering on the downsampled feature map to obtain a grayscale image of the image to be identified.

3. The method for detecting fireworks targets according to claim 2, characterized in that: The method of processing the images to be identified in adjacent frames by using the inter-frame difference method to determine the candidate pixels of fireworks in each image to be identified includes: The fireworks candidate pixel points in each image to be identified are calculated using the fireworks candidate pixel point determination formula for the grayscale images corresponding to the images to be identified in each adjacent frame. The fireworks candidate pixel point determination formula is: in, is a first grayscale image corresponding to the first image to be identified in the adjacent frames of images to be identified, is a second grayscale image corresponding to the second image to be identified in the adjacent frame of images to be identified, is a difference map of the pixel values ​​of the corresponding pixels in the first grayscale image and the second grayscale image, is the pixel value of the pixel with the smallest pixel value in the difference map, is the pixel value of the pixel with the largest pixel value in the difference map, is the pixel point in the difference map ( i , j ), Pixels ( i , j ) of the fireworks judgment value, Represents a pixel ( i , j ) is the pixel value after binarization, It is the fireworks determination threshold of the preset fireworks candidate pixel point.

4. The method for detecting fireworks targets according to claim 3, characterized in that: The clustering algorithm is a DBSCAN clustering algorithm. The clustering algorithm is used to cluster the candidate fireworks pixels to determine the candidate fireworks area in each image to be identified, including: The DBSCAN clustering algorithm is used to cluster the connected fireworks candidate pixels in the image to be identified to obtain multiple fireworks candidate regions of preset shapes, wherein the fixed parameters in the DBSCAN clustering algorithm are calculated using a parameter determination formula, and the parameter determination formula is: in, It refers to m The firework candidate regions correspond to eps Parameter value, Indicates that m There are N candidate firework pixels, It refers to m Candidate fireworks areas, , It refers to the m The first k The coordinates of the candidate pixels of the fireworks, , It refers to the m The first h The coordinates of the candidate pixels of the fireworks, It refers to the m The firework candidate regions correspond to Parameter value.

5. The method for detecting fireworks targets according to claim 4, characterized in that: The step of extracting the underlying statistical features and texture features of the fireworks candidate area includes: Acquire polarization angle images and infrared images corresponding to the plurality of continuous frames of images to be identified, wherein the polarization angle images record polarization angle information of each pixel point; Based on the mean and covariance of the polarization angles of each pixel in the fireworks candidate area; Determine a highlight pixel point in the infrared image, the pixel value of the highlight pixel point is greater than a preset highlight determination threshold, and calculate the minimum distance between each pixel point in the fireworks candidate area and the highlight pixel point; The texture features of the fireworks candidate area are calculated using a preset gray level co-occurrence matrix.

6. The method for detecting fireworks targets according to claim 5, characterized in that: The step of determining highlight pixel points in the infrared image includes: Adjacent frames of infrared images are processed using an inter-frame difference method to determine highlight pixel points in the infrared images.

7. The method for detecting fireworks targets according to claim 5, characterized in that: The method of using a decision tree to make a decision on the underlying statistical features and texture characteristics to determine the fireworks target area includes: A preset decision tree is used to make a decision on the mean and covariance of the polarization angle, the minimum distance between each pixel in the fireworks candidate area and the highlight pixel, and the texture feature to obtain an authenticity index of the fireworks candidate area; The fireworks candidate area whose authenticity index is greater than a preset index threshold is determined as the fireworks target area.

8. A fireworks target detection device, characterized in that: include: An image acquisition module, used to acquire a plurality of continuous frames of to-be-recognized images for the same scene, wherein the to-be-recognized images include fireworks targets to be recognized; A candidate region determination module is used to process the images to be identified of adjacent frames using an inter-frame difference method to determine candidate pixels of fireworks in each image to be identified, and cluster the candidate pixels of fireworks using a clustering algorithm to determine candidate regions of fireworks in each image to be identified; The fireworks target recognition module is used to extract the underlying statistical features and texture characteristics of the fireworks candidate area, and use a decision tree to make decisions on the underlying statistical features and texture characteristics to determine the fireworks target area.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the fireworks target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the fireworks target detection method described in any one of claims 1 to 7.

Images