Fireworks false detection removal method, device, equipment and medium

Abstract

The present application discloses a method, apparatus, device and medium for eliminating false detection of fireworks, including: extracting a preset number of frame images from video stream data; wherein the interval number of frames between two adjacent frame images is the same; detecting the last frame image using a preset target detection algorithm to obtain a target rectangular frame, and obtaining vertex coordinates and center point coordinates based on the target rectangular frame; calculating the coordinate positions of corresponding points in the remaining number of frame images using an optical flow method based on the vertex coordinates and center point coordinates to obtain a first trajectory map corresponding to each vertex and center point; mapping the first trajectory map to a target drawing board according to a preset proportional relationship to obtain a second trajectory map, and judging whether the motion trajectory of the corresponding coordinate point is regular based on the first and second trajectory maps; determining the number of coordinate points with regular motion trajectories among the vertices and center points; and judging whether there is a false detection in the current fireworks detection using a predefined judgment rule based on the number of coordinate points. False detection caused by similar light sources can be avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and recognition, and in particular to a method, device, equipment and medium for removing false detection of fireworks. Background Art

[0002] With the development of AI (artificial intelligence) technology, traditional smoke and fire detection solutions using sensors are being gradually replaced by machine vision. However, machine vision can lead to false detections when using light sources similar to those of fireworks, such as car lights, sunlight, and streetlights. This type of false detection has always been a difficult problem to solve in the field of machine vision.

[0003] In traffic systems, accidents such as vehicle fires or smoke can have a significant impact on traffic safety. Monitoring and identifying these incidents with cameras can provide strong support for traffic management. However, during the detection process, regularly moving light sources such as headlights from moving vehicles and light sources carried by pedestrians can cause false detections of fireworks.

[0004] In summary, how to avoid false detection of fireworks caused by light sources that are similar to fireworks is a problem that needs to be solved. Summary of the Invention

[0005] In view of this, the present invention aims to provide a method, device, equipment, and medium for eliminating false detection of fireworks, which can avoid false detection caused by regularly moving light sources. The specific solution is as follows:

[0006] In a first aspect, the present application discloses a method for removing false detection of fireworks, comprising:

[0007] Extracting a preset number of frame images from the video stream data; wherein the interval number of frames between two adjacent frame images is the same;

[0008] Detect the last frame of the image using a preset target detection algorithm to obtain a target rectangular frame, and obtain vertex coordinates and center point coordinates based on the target rectangular frame;

[0009] Based on the vertex coordinates and the center point coordinates, the coordinate positions of corresponding points in the remaining number of frame images are calculated using an optical flow method to obtain first trajectory graphs corresponding to the vertices and the center point respectively;

[0010] Mapping the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph, and determining whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph;

[0011] The number of coordinate points with regular motion trajectories among the vertices and the center point is determined, and based on the number of coordinate points, a predefined determination rule is used to determine whether there is a false detection in the current fireworks detection.

[0012] Optionally, the detecting the last frame image using a preset detection algorithm to obtain a target rectangular frame, and acquiring vertex coordinates and center point coordinates based on the target rectangular frame, includes:

[0013] The last frame of the image is detected using the YOLOv4 algorithm to obtain a set of coordinate points of the target rectangle, and the coordinates of the four vertices and the center point are extracted from the set of coordinate points;

[0014] Accordingly, the coordinate positions of corresponding points in the remaining number of frame images are calculated using the optical flow method based on the vertex coordinates and the center point coordinates to obtain the first trajectory graph corresponding to the vertex and the center point respectively, including:

[0015] Based on the coordinates of the four vertices and the center point, the coordinate positions of corresponding points in the remaining number of frame images are calculated using the optical flow method to obtain first trajectory graphs corresponding to the four vertices and the center point.

[0016] Optionally, mapping the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph includes:

[0017] Counting the maximum and minimum values of the coordinate values of all the track points in the first track graph in the horizontal direction, and counting the maximum and minimum values of the coordinate values of all the track points in the first track graph in the vertical direction, so as to generate a circumscribed rectangle of the first track graph;

[0018] Obtaining the size of the target drawing board, and calculating a preset ratio between the circumscribed rectangle and the size;

[0019] Based on the coordinate values of all the track points in the first track map, the coordinate values of the corresponding track points in the target drawing board are calculated using the preset proportional relationship to obtain a second track map.

[0020] Optionally, the process of determining whether there is a false detection in the current fireworks detection based on the number of coordinate points using a predefined determination rule further includes:

[0021] Determining whether the number of the coordinate points exceeds a preset threshold;

[0022] If the number of the coordinate points exceeds the preset threshold, it is determined that the current fireworks detection may have a false detection.

[0023] Optionally, after determining that the current fireworks detection may have a false detection, the method further includes:

[0024] Re-performing the step of extracting a preset number of frame images from the video stream data, and again determining whether the number of the coordinate points exceeds the preset threshold;

[0025] If the number of the coordinate points exceeds the preset threshold, it is determined that there is a false detection in the current fireworks detection.

[0026] Optionally, judging whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph includes:

[0027] Evaluating the first trajectory graph using a covariance evaluation method to obtain a first evaluation value;

[0028] Performing classification fitting on the second trajectory graph using a classification network to obtain a second evaluation value;

[0029] Assigning corresponding weight coefficients to the first evaluation value and the second evaluation value, and calculating a total evaluation value;

[0030] The total evaluation value is used to determine whether the motion trajectory of the coordinate point is regular.

[0031] Optionally, evaluating the first trajectory graph using a covariance evaluation method to obtain a first evaluation value includes:

[0032] Obtaining the coordinate values of each coordinate point in the first frame image and the last frame image, and fitting a straight line equation;

[0033] The linear equation is used to calculate the predicted y-coordinate value corresponding to the original x-coordinate value of each coordinate point in the preset number of frame images, and the corresponding mean square error and variance are calculated based on the predicted y-coordinate value and the original y-coordinate value, and a first evaluation value is obtained based on the mean square error and variance.

[0034] In a second aspect, the present application discloses a fireworks false detection removal device, comprising:

[0035] An image extraction module is used to extract a preset number of frames of images from the video stream data; wherein the interval between two adjacent frames of images is the same;

[0036] The target detection module is used to detect the last frame of the image using a preset target detection algorithm to obtain a target rectangular frame, and obtain vertex coordinates and center point coordinates based on the target rectangular frame;

[0037] A position calculation module is used to calculate the coordinate positions of corresponding points in the remaining number of frame images based on the vertex coordinates and the center point coordinates using an optical flow method to obtain a first trajectory map corresponding to each vertex and center point;

[0038] a trajectory judgment module, configured to map the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph, and to judge whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph;

[0039] The false detection determination module is used to determine the number of coordinate points with regular motion trajectories between the vertices and the center point, and determine whether there is a false detection in the current fireworks detection based on the number of coordinate points using a predefined determination rule.

[0040] In a third aspect, the present application discloses an electronic device, comprising:

[0041] Memory, used to store computer programs;

[0042] The processor is configured to execute the computer program to implement the steps of the aforementioned method for removing false detection of fireworks.

[0043] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the aforementioned disclosed fireworks false detection removal method are implemented.

[0044] It can be seen that the present application first extracts a preset number of frame images from the video stream data; wherein the interval number of frames between two adjacent frame images is the same; the last frame image is detected using a preset target detection algorithm to obtain a target rectangular frame, and the vertex coordinates and center point coordinates are obtained based on the target rectangular frame; the coordinate positions of the corresponding points in the remaining number of frame images are calculated using the optical flow method based on the vertex coordinates and the center point coordinates to obtain the first trajectory graph corresponding to the vertex and the center point respectively; the first trajectory graph is mapped to the target canvas according to a preset proportional relationship to obtain a second trajectory graph, and whether the motion trajectory of the corresponding coordinate point is regular is judged based on the first trajectory graph and the second trajectory graph; the number of coordinate points with regular motion trajectories among the vertex and the center point is determined, and based on the number of coordinate points, a predefined judgment rule is used to determine whether there is a false detection in the current fireworks detection. It can be seen that the present application first extracts a preset number of frame images with the same frame interval from the video stream data, and uses a preset target detection algorithm to detect the last frame image to obtain the vertex coordinates and center point coordinates, and then uses the optical flow method to calculate the corresponding coordinate positions of the vertex and center point in the remaining number of frame images to obtain the first corresponding trajectory map, and then obtains the second trajectory map based on the preset proportional relationship through the first trajectory map, and finally determines whether the motion trajectory in the first trajectory map and the second trajectory map is regular to determine whether there is a false detection in the current fireworks detection. The above technical solution can eliminate the false detection caused by the regular moving light source to the fireworks detection, and by extracting a preset number of frame images for detection instead of starting the detection and judgment from the current frame image, it can improve the operating efficiency and ensure the real-time reporting of the detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0046] Figure 1 This is a flow chart of a fireworks false detection method disclosed in this application;

[0047] Figure 2 This is a flow chart of a specific fireworks false detection method disclosed in this application;

[0048] Figure 3 This is a flow chart of another specific fireworks false detection removal method disclosed in this application;

[0049] Figure 4 A motion trajectory distribution diagram disclosed in this application;

[0050] Figure 5 This is a structural diagram of a smoke and fire detection device disclosed in this application;

[0051] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] Currently, in the transportation system, accidents such as vehicle fires or smoke on the road can have a significant impact on traffic safety. Monitoring and identifying these accidents through cameras can provide strong support for traffic management. However, during the detection process, regularly moving light sources such as the headlights of moving vehicles and the light sources carried by pedestrians may cause certain false detections in fireworks detection. To this end, the embodiments of the present application disclose a method, device, equipment, and medium for false detection of fireworks, which can avoid false detections caused by regularly moving light sources in fireworks detection.

[0054] See also Figure 1 As shown, the embodiment of the present application discloses a method for removing false detection of fireworks, which includes:

[0055] Step S11: extracting a preset number of frame images from the video stream data; wherein the interval number of frames between two adjacent frame images is the same.

[0056] In this embodiment, N frames are extracted from the video stream data and cached, with the same number of frames between each adjacent frame. It should be noted that the number of cached frames needs to be set based on the current storage capacity; in this embodiment, the preset number N is set to 25. Furthermore, the interval between each adjacent frame cannot be too large and needs to be determined based on the current chip capabilities.

[0057] Step S12: Detect the last frame of image using a preset target detection algorithm to obtain a target rectangular frame, and obtain vertex coordinates and center point coordinates based on the target rectangular frame.

[0058] In this embodiment, a preset target detection algorithm is used to detect the last frame of the N extracted frames to obtain a target rectangular frame. The vertex coordinates and center point coordinates are then extracted from the target rectangular frame. It should be noted that after detection by the preset target detection algorithm, there may be a single target rectangular frame or a group of target rectangular frames. When there is a group of target rectangular frames, it is recorded as VecRect{Rect1,Rect2,…}.

[0059] Step S13: Calculate the coordinate positions of corresponding points in the remaining number of frame images using the optical flow method based on the vertex coordinates and the center point coordinates to obtain first trajectory graphs corresponding to the vertices and the center point respectively.

[0060] In this embodiment, the corresponding coordinate positions of the vertices and the center point in the remaining N-1 frames are calculated by using the optical flow method through the vertex coordinates and the center point coordinates of each target rectangular frame in the last frame image, and are recorded as U i P j , where i∈1~N-1, j represents the number of rectangles, and P represents the coordinate position of the center point of each rectangle. Then, according to the coordinate positions of the vertices and center points in each frame image, the first trajectory graph corresponding to each vertex and center point is obtained.

[0061] Step S14: mapping the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph, and determining whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph.

[0062] In this embodiment, the first trajectory map is mapped to a 128×128 target canvas according to a preset ratio to obtain the corresponding trajectory on the target canvas, i.e., the second trajectory map. The first and second trajectory maps are then used to determine whether the motion trajectory of each coordinate point is regular.

[0063] Step S15: determining the number of coordinate points with regular motion trajectories among the vertices and the center point, and determining whether there is any false detection in the current fireworks detection using a predefined determination rule based on the number of coordinate points.

[0064] In this embodiment, the number of coordinate points with regular motion trajectories among the vertices and center points is determined, and a predefined judgment rule is used to determine whether the current fireworks detection has a false detection. The above process of determining whether the current fireworks detection has a false detection based on the number of coordinate points using a predefined judgment rule may include: determining whether the number of coordinate points exceeds a preset threshold; if the number of coordinate points exceeds the preset threshold, determining that the current fireworks detection may have a false detection. It is understandable that, assuming that 4 vertices and one center point are extracted, the preset threshold is set to 3, that is, if the motion trajectories of 3 or more of the 5 points conform to regularity, then it is considered that the target in the target rectangular box is a target with regular motion, and it is determined that the current fireworks detection may have a false detection.

[0065] In this embodiment, it should be noted that after determining that the current fire and smoke detection may have a false detection, the process may further include: re-executing the step of extracting a preset number of frames of images from the video stream data, and again determining whether the number of coordinate points exceeds the preset threshold; if the number of coordinate points exceeds the preset threshold, determining that the current fire and smoke detection has a false detection. In other words, after determining that the current fire and smoke detection may have a false detection, the process may need to re-execute the above steps to re-determine whether the current fire and smoke detection has a false detection.

[0066] It can be seen that the present application first extracts a preset number of frame images from the video stream data; wherein the interval number of frames between two adjacent frame images is the same; the last frame image is detected using a preset target detection algorithm to obtain a target rectangular frame, and the vertex coordinates and center point coordinates are obtained based on the target rectangular frame; the coordinate positions of the corresponding points in the remaining number of frame images are calculated using the optical flow method based on the vertex coordinates and the center point coordinates to obtain the first trajectory graph corresponding to the vertex and the center point respectively; the first trajectory graph is mapped to the target canvas according to a preset proportional relationship to obtain a second trajectory graph, and whether the motion trajectory of the corresponding coordinate point is regular is judged based on the first trajectory graph and the second trajectory graph; the number of coordinate points with regular motion trajectories among the vertex and the center point is determined, and based on the number of coordinate points, a predefined judgment rule is used to determine whether there is a false detection in the current fireworks detection. It can be seen that the present application first extracts a preset number of frame images with the same frame interval from the video stream data, and uses a preset target detection algorithm to detect the last frame image to obtain the vertex coordinates and center point coordinates, and then uses the optical flow method to calculate the corresponding coordinate positions of the vertex and center point in the remaining number of frame images to obtain the first corresponding trajectory map, and then obtains the second trajectory map based on the preset proportional relationship through the first trajectory map, and finally determines whether the motion trajectory in the first trajectory map and the second trajectory map is regular to determine whether there is a false detection in the current fireworks detection. The above technical solution can eliminate the false detection caused by the regular moving light source to the fireworks detection, and by extracting a preset number of frame images for detection instead of starting the detection and judgment from the current frame image, it can improve the operating efficiency and ensure the real-time reporting of the detection results.

[0067] See also Figure 2 As shown, the embodiment of this application discloses a specific method for removing false detection of fireworks. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0068] Step S21: extracting a preset number of frame images from the video stream data; wherein the interval number of frames between two adjacent frame images is the same.

[0069] Step S22: Detect the last frame of image using the YOLOv4 algorithm to obtain a coordinate point set of the target rectangle, and extract the coordinates of four vertices and the center point from the coordinate point set.

[0070] In this embodiment, the target detection algorithm used is YOLOv4, and the four vertex coordinates and the center point coordinates are extracted from the coordinate point set of the target rectangle. The four vertex coordinates and the center point coordinates extracted from each rectangle are marked as VecPoints{point1,Point2,…}; i ={(x 左上 ,y左上 ),(x 右上 ,y 右上 ),(x 左下 ,y 左下 ),(x 右下 ,y 右下 ), (x 中心 ,y 中心 )},i=1,2,….

[0071] Step S23: Based on the coordinates of the four vertices and the center point, the coordinate positions of corresponding points in the remaining number of frame images are calculated using the optical flow method to obtain first trajectory graphs corresponding to the four vertices and the center point.

[0072] Step S24: Counting the maximum and minimum values of the coordinate values of all track points in the first track graph in the horizontal direction, and counting the maximum and minimum values of the coordinate values of all track points in the first track graph in the vertical direction, to generate a circumscribed rectangle of the first track graph.

[0073] In this embodiment, taking the center point as an example, the maximum and minimum values of the coordinate values of all track points in the first track graph of the center point in the horizontal and vertical directions are counted to obtain the maximum circumscribed rectangle of the first track graph.

[0074] Step S25: obtaining the size of the target drawing board, and calculating a preset ratio between the circumscribed rectangle and the size.

[0075] In this embodiment, the size of the target artboard is set to 128×128, and a preset ratio relationship between the maximum circumscribed rectangle and the target artboard is calculated.

[0076] Step S26: Based on the coordinate values of all trajectory points in the first trajectory graph, the coordinate values of the corresponding trajectory points in the target canvas are calculated using the preset proportional relationship to obtain a second trajectory graph, and based on the first trajectory graph and the second trajectory graph, it is determined whether the motion trajectory of the corresponding coordinate point is regular.

[0077] In this embodiment, the coordinate values of the corresponding track points in the target canvas of all track points in the first track map are calculated according to a preset proportional relationship to obtain a second track map after normalized mapping. In this way, a converted track distribution map on the target canvas is obtained.

[0078] Step S27: determining the number of coordinate points with regular motion trajectories among the vertices and the center point, and determining whether there is any false detection in the current fireworks detection using a predefined determination rule based on the number of coordinate points.

[0079] For more specific processing procedures of the above steps S21, S23 and S27, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be elaborated here.

[0080] It can be seen that after the embodiment of the present application obtains the coordinates of the four vertices and the center point in each target rectangular frame, it uses the optical flow method to calculate the coordinate positions of the corresponding points in the remaining number of frame images to obtain the first trajectory map of each coordinate point, and obtains the maximum circumscribed rectangle of the trajectory point according to the first trajectory map; then, through the preset proportional relationship between the maximum circumscribed rectangle and the target canvas, the second trajectory map in the target canvas is obtained, and finally, based on the first trajectory map and the second trajectory map, it is determined whether the motion trajectory of the corresponding coordinate point is regular. By normalizing and mapping the first trajectory map, the second trajectory map is drawn to avoid affecting the accuracy of subsequent processing and analysis due to the excessive or excessive range of the motion trajectory map.

[0081] See also Figure 3 As shown, the embodiment of this application discloses a specific method for removing false detection of fireworks. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, it includes:

[0082] Step S31: extracting a preset number of frame images from the video stream data; wherein the interval number of frames between two adjacent frame images is the same.

[0083] Step S32: Detect the last frame of image using a preset target detection algorithm to obtain a target rectangular frame, and obtain vertex coordinates and center point coordinates based on the target rectangular frame.

[0084] Step S33: Calculate the coordinate positions of corresponding points in the remaining number of frame images using the optical flow method based on the vertex coordinates and the center point coordinates to obtain first trajectory graphs corresponding to the vertices and the center point respectively.

[0085] Step S34: mapping the first trajectory graph to a target canvas according to a preset proportional relationship to obtain a second trajectory graph, and evaluating the first trajectory graph using a covariance evaluation method to obtain a first evaluation value; and classifying and fitting the second trajectory graph using a classification network to obtain a second evaluation value.

[0086] In this embodiment, it is necessary to evaluate the motion trajectories of the coordinate points in the first trajectory diagram and the second trajectory diagram respectively to determine whether the motion trajectories are regular. The first trajectory diagram is evaluated by a covariance evaluation method to obtain a first evaluation value; and the second trajectory diagram is classified and fitted by a classification network to obtain a second evaluation value. The above-mentioned evaluation of the first trajectory diagram using the covariance evaluation method to obtain the first evaluation value may include: obtaining the coordinate values of each coordinate point in the first frame image and the last frame image, and fitting a straight line equation; using the straight line equation to calculate the predicted y coordinate value corresponding to the original x coordinate value of each coordinate point in the preset number of frame images, and calculating the corresponding mean square error and variance based on the predicted y coordinate value and the original y coordinate value, and obtaining the first evaluation value based on the mean square error and variance. It can be understood that a straight line equation is fitted by the coordinates of the first and last points in the first trajectory diagram, which is recorded as y=ax+b; then the horizontal original x coordinate values of the remaining trajectory points in the first trajectory diagram are sequentially substituted into the straight line equation y=ax+b to calculate the predicted y coordinate value, which is recorded as Then calculate the mean square error and variance. The mean square error formula and variance calculation formula are as follows:

[0087]

[0088]

[0089] Among them, y represents the original y coordinate value; Represents the predicted y-coordinate value after linear fitting; Represents the mean of all original y-coordinate values;

[0090] Finally, the first evaluation value u is calculated, where It should be pointed out that the closer the value of u is to 0, the better the linear law is.

[0091] When calculating the second evaluation value, classification processing is used. Specifically, the second trajectory map is input into the classification network for classification fitting to obtain the second evaluation value; the final distribution map is as follows Figure 4 As shown, Figure 4 The left is the coordinate point with a more regular motion trajectory. Figure 4 On the right are the coordinate points of the irregular motion trajectory.

[0092] Step S35: assigning corresponding weight coefficients to the first evaluation value and the second evaluation value, and calculating a total evaluation value; using the total evaluation value to determine whether the motion trajectory of the coordinate point is regular.

[0093] In this embodiment, corresponding weight coefficients of 0.2 and 0.8 are assigned to the first evaluation value and the second evaluation value respectively, and then the total evaluation value is calculated. The total evaluation value is used to determine whether the motion trajectory of the coordinate point is regular. It can be understood that the better the regularity, the less likely it is fireworks.

[0094] Step S36: determining the number of coordinate points with regular motion trajectories among the vertices and the center point, and determining whether there is any false detection in the current fireworks detection using a predefined determination rule based on the number of coordinate points.

[0095] In this embodiment, since five coordinate points are obtained in a target rectangular frame, if three or more of them meet regularity, it is considered that the motion trajectory in the target rectangular frame is regular, and it is considered that there is a false detection in the current fireworks detection.

[0096] For more specific processing procedures of the above steps S31, S32 and S33, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0097] As can be seen, when calculating the first evaluation value, the embodiment of the present application designs a linear fitting method to calculate whether the moving target moves in a linear fitting manner, thereby determining whether the target moves regularly. When calculating the second evaluation value, a classification network method is used for detection, and then corresponding weight coefficients are assigned to the first and second evaluation values to obtain a total evaluation value, and finally the total evaluation value is used for final judgment. Through the above technical solution, it is possible to effectively avoid false detections in fireworks detection caused by regularly moving light sources.

[0098] See also Figure 5 As shown, the embodiment of the present application discloses a fireworks false detection device, comprising:

[0099] The image extraction module 11 is used to extract a preset number of frames of images from the video stream data; wherein the interval between two adjacent frames of images is the same;

[0100] The target detection module 12 is used to detect the last frame of the image using a preset target detection algorithm to obtain a target rectangular frame, and obtain vertex coordinates and center point coordinates based on the target rectangular frame;

[0101] A position calculation module 13 is configured to calculate the coordinate positions of corresponding points in the remaining number of frame images using an optical flow method based on the vertex coordinates and the center point coordinates, so as to obtain a first trajectory map corresponding to each vertex and the center point;

[0102] a trajectory judgment module 14 for mapping the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph, and judging whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph;

[0103] The false detection determination module 15 is configured to determine the number of coordinate points with regular motion trajectories between the vertices and the center point, and determine whether there is a false detection in the current fireworks detection based on the number of coordinate points using a predefined determination rule.

[0104] It can be seen that the present application first extracts a preset number of frame images from the video stream data; wherein the interval number of frames between two adjacent frame images is the same; the last frame image is detected using a preset target detection algorithm to obtain a target rectangular frame, and the vertex coordinates and center point coordinates are obtained based on the target rectangular frame; the coordinate positions of the corresponding points in the remaining number of frame images are calculated using the optical flow method based on the vertex coordinates and the center point coordinates to obtain the first trajectory graph corresponding to the vertex and the center point respectively; the first trajectory graph is mapped to the target canvas according to a preset proportional relationship to obtain a second trajectory graph, and whether the motion trajectory of the corresponding coordinate point is regular is judged based on the first trajectory graph and the second trajectory graph; the number of coordinate points with regular motion trajectories among the vertex and the center point is determined, and based on the number of coordinate points, a predefined judgment rule is used to determine whether there is a false detection in the current fireworks detection. It can be seen that the present application first extracts a preset number of frame images with the same frame interval from the video stream data, and uses a preset target detection algorithm to detect the last frame image to obtain the vertex coordinates and center point coordinates, and then uses the optical flow method to calculate the corresponding coordinate positions of the vertex and center point in the remaining number of frame images to obtain the first corresponding trajectory map, and then obtains the second trajectory map based on the preset proportional relationship through the first trajectory map, and finally determines whether the motion trajectory in the first trajectory map and the second trajectory map is regular to determine whether there is a false detection in the current fireworks detection. The above technical solution can eliminate the false detection caused by the regular moving light source to the fireworks detection, and by extracting a preset number of frame images for detection instead of starting the detection and judgment from the current frame image, it can improve the operating efficiency and ensure the real-time reporting of the detection results.

[0105] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Specifically, the device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps of the fireworks false detection removal method performed by a computer device as disclosed in any of the aforementioned embodiments.

[0106] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the computer device 20; the communication interface 24 can create a data transmission channel between the computer device 20 and external devices. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0107] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0108] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include an operating system 221, a computer program 222 and data 223, etc. The storage method can be temporary storage or permanent storage.

[0109] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the computer device 20, enabling the processor 21 to calculate and process the massive amount of data 223 in the memory 22. It can be run under Windows, Unix, Linux, or other operating systems. In addition to including computer programs capable of implementing the fireworks false detection method performed by the computer device 20 as disclosed in any of the aforementioned embodiments, the computer programs 222 may also include computer programs capable of performing other specific tasks. Data 223 may include data received by the computer device from external devices, as well as data collected by its own input / output interface 25.

[0110] Furthermore, an embodiment of the present application also discloses a storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method steps performed in the fireworks false detection process disclosed in any of the aforementioned embodiments are implemented.

[0111] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0112] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0114] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0115] The above is a detailed introduction to the fireworks false detection method, device, equipment and storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

Claims

1. A method for removing false detection of fireworks, characterized in that: include: Extracting a preset number of frame images from the video stream data; wherein the interval between two adjacent frame images is the same; Detect the last frame of the image using a preset target detection algorithm to obtain a target rectangular frame, and obtain vertex coordinates and center point coordinates based on the target rectangular frame; Based on the vertex coordinates and the center point coordinates, the coordinate positions of corresponding points in the remaining number of frame images are calculated using an optical flow method to obtain first trajectory graphs corresponding to the vertices and the center point respectively; Mapping the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph, and determining whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph; Determining the number of coordinate points with regular motion trajectories between the vertices and the center point, and determining whether there is a false detection in the current fireworks detection using a predefined determination rule based on the number of coordinate points; The determining whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph includes: Evaluating the first trajectory graph using a covariance evaluation method to obtain a first evaluation value; Performing classification fitting on the second trajectory graph using a classification network to obtain a second evaluation value; assigning corresponding weight coefficients to the first evaluation value and the second evaluation value, and calculating a total evaluation value; The total evaluation value is used to determine whether the motion trajectory of the coordinate point is regular.

2. The method for removing false detection of fireworks according to claim 1, characterized in that: The method of detecting the last frame of the image using a preset target detection algorithm to obtain a target rectangular frame, and obtaining vertex coordinates and center point coordinates based on the target rectangular frame, includes: The last frame of the image is detected using the YOLOv4 algorithm to obtain a set of coordinate points of the target rectangle, and the coordinates of the four vertices and the center point are extracted from the set of coordinate points; Accordingly, the coordinate positions of corresponding points in the remaining number of frame images are calculated using the optical flow method based on the vertex coordinates and the center point coordinates to obtain the first trajectory graph corresponding to the vertex and the center point respectively, including: Based on the coordinates of the four vertices and the center point, the coordinate positions of corresponding points in the remaining number of frame images are calculated using the optical flow method to obtain first trajectory graphs corresponding to the four vertices and the center point.

3. The method for removing false detection of fireworks according to claim 1, characterized in that: Mapping the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph includes: Counting the maximum and minimum values of the coordinate values of all the track points in the first track graph in the horizontal direction, and counting the maximum and minimum values of the coordinate values of all the track points in the first track graph in the vertical direction, so as to generate a circumscribed rectangle of the first track graph; Obtaining the size of the target drawing board, and calculating a preset ratio between the circumscribed rectangle and the size; Based on the coordinate values of all the track points in the first track map, the coordinate values of the corresponding track points in the target drawing board are calculated using the preset proportional relationship to obtain a second track map.

4. The method for removing false detection of fireworks according to claim 1, characterized in that: The process of determining whether there is a false detection in the current fireworks detection based on the number of the coordinate points using a predefined determination rule also includes: Determining whether the number of the coordinate points exceeds a preset threshold; If the number of the coordinate points exceeds the preset threshold, it is determined that the current fireworks detection may have a false detection.

5. The method for removing false detection of fireworks according to claim 4, characterized in that: After determining that the current fireworks detection may have a false detection, the method further includes: Re-performing the step of extracting a preset number of frame images from the video stream data, and again determining whether the number of the coordinate points exceeds the preset threshold; If the number of the coordinate points exceeds the preset threshold, it is determined that there is a false detection in the current fireworks detection.

6. The method for removing false detection of fireworks according to claim 1, characterized in that: The evaluating the first trajectory graph by using a covariance evaluation method to obtain a first evaluation value includes: Obtaining the coordinate values of each coordinate point in the first frame image and the last frame image, and fitting a straight line equation; The linear equation is used to calculate the predicted y-coordinate value corresponding to the original x-coordinate value of each coordinate point in the preset number of frame images, and the corresponding mean square error and variance are calculated based on the predicted y-coordinate value and the original y-coordinate value, and a first evaluation value is obtained based on the mean square error and variance.

7. A fireworks false detection device, characterized in that: include: An image extraction module is used to extract a preset number of frames of images from the video stream data; wherein the interval between two adjacent frames of images is the same; The target detection module is used to detect the last frame of the image using a preset target detection algorithm to obtain a target rectangular frame, and obtain vertex coordinates and center point coordinates based on the target rectangular frame; A position calculation module is used to calculate the coordinate positions of corresponding points in the remaining number of frame images based on the vertex coordinates and the center point coordinates using an optical flow method to obtain a first trajectory map corresponding to each vertex and center point; a trajectory judgment module, configured to map the first trajectory graph to a target drawing board according to a preset proportional relationship to obtain a second trajectory graph, and to judge whether the motion trajectory of the corresponding coordinate point is regular based on the first trajectory graph and the second trajectory graph; a false detection determination module, configured to determine the number of coordinate points with regular motion trajectories between the vertices and the center point, and determine whether there is a false detection in the current fireworks detection using a predefined determination rule based on the number of coordinate points; Among them, the trajectory judgment module is specifically used to evaluate the first trajectory graph using the covariance evaluation method to obtain a first evaluation value; use the classification network to classify and fit the second trajectory graph to obtain a second evaluation value; assign corresponding weight coefficients to the first evaluation value and the second evaluation value, and calculate the total evaluation value; use the total evaluation value to determine whether the motion trajectory of the coordinate point is regular.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the fireworks false detection removal method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that Used to store a computer program; wherein, when the computer program is executed by a processor, the steps of the fireworks false detection removal method according to any one of claims 1 to 6 are implemented.

Images