Firework recognition method and device, electronic equipment and storage medium

Abstract

The application provides a firework recognition method and device, electronic equipment and storage medium, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring a to-be-recognized image; performing image definition evaluation on the to-be-recognized image; determining a target confidence interval based on the comparison result of the definition of the to-be-recognized image and a preset definition condition; predicting the firework category of the to-be-recognized image, acquiring the predicted firework category of the to-be-recognized image, the target confidence corresponding to the predicted firework category and the image information of the to-be-recognized image; acquiring the position of the target confidence in the target confidence interval, and determining the firework category corresponding to the to-be-recognized image based on the position. The application improves the accuracy, robustness and adaptability of firework target recognition by analyzing the to-be-recognized image multiple times, and reduces the false positive rate.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, electronic device, and storage medium for identifying fireworks. Background Art

[0002] Forest fires, a sudden, destructive, and difficult-to-handle natural disaster, pose a significant threat to forest resources and ecological security, as well as the safety of people's lives and property. Currently, many applications rely on smoke, heat, light, and composite detectors, video surveillance, and infrared thermal imaging for fire identification. However, these technologies often fail to accurately and timely identify fire targets, resulting in low fire identification accuracy. Summary of the Invention

[0003] The embodiments of the present application provide a method, device, electronic device, and storage medium for identifying fireworks, which can solve the problem of low accuracy in fireworks identification in related technologies.

[0004] In order to solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for identifying fireworks, the method comprising:

[0006] Obtain the image to be recognized;

[0007] Performing image clarity evaluation on the image to be identified;

[0008] Determining a target confidence interval based on a comparison result of the clarity of the image to be identified with a preset clarity condition;

[0009] Predicting the fireworks category of the image to be identified, obtaining the predicted fireworks category of the image to be identified, a target confidence level corresponding to the predicted fireworks category, and information about the image to be identified;

[0010] The position of the target confidence level in the target confidence level interval is obtained, and the fireworks category corresponding to the image to be identified is determined based on the position.

[0011] In a second aspect, an embodiment of the present application further provides a fireworks recognition device, the fireworks recognition device comprising:

[0012] An acquisition module, used to acquire an image to be identified;

[0013] An evaluation module, configured to evaluate the image clarity of the image to be identified;

[0014] a determination module, configured to determine a target confidence interval based on a comparison result of the clarity of the image to be identified with a preset clarity condition;

[0015] a prediction module, configured to predict the fireworks category of the image to be identified, and obtain the predicted fireworks category of the image to be identified, a target confidence level corresponding to the predicted fireworks category, and image information of the image to be identified;

[0016] The discrimination module is configured to obtain a position of the target confidence level in the target confidence level interval, and determine a fireworks category corresponding to the image to be identified based on the position.

[0017] In a third aspect, an embodiment of the present application further provides a fireworks recognition device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the above-mentioned fireworks recognition method when executed by the processor.

[0018] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned fireworks recognition method are implemented.

[0019] In an embodiment of the present application, by setting different confidence intervals, the image category to be identified is predicted and the corresponding confidence of the predicted category is obtained. Based on this confidence, the fireworks category of the image to be identified is determined at the position in the aforementioned confidence interval, thereby improving the accuracy and recall rate of the fireworks recognition and classification algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is one of the flow charts of a method for identifying fireworks provided in an embodiment of the present application;

[0022] Figure 2 This is the second flow chart of a method for identifying fireworks provided in an embodiment of the present application;

[0023] Figure 3 This is a structural diagram of a fireworks identification device provided in an embodiment of the present application;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] Unless otherwise defined, technical or scientific terms used in this application should have the ordinary meanings understood by persons of ordinary skill in the art to which this application belongs. The terms "first," "second," and similar expressions used in this application do not denote any order, quantity, or importance, but are used only to distinguish different components. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0027] The present application embodiment provides a method for identifying fireworks. Figure 1 , Figure 1 This is one of the flow charts of the fireworks identification method provided in the embodiment of the present application. Figure 1 As shown, the following steps are included:

[0028] Step 11, obtaining the image to be identified;

[0029] Step 12: evaluating the image clarity of the image to be recognized;

[0030] Step 13: determining a target confidence interval based on a comparison result of the clarity of the image to be identified with a preset clarity condition;

[0031] Step 14: predicting the fireworks category of the image to be identified, obtaining the predicted fireworks category of the image to be identified, the target confidence corresponding to the predicted fireworks category, and image information of the image to be identified;

[0032] Step 15: Obtain the position of the target confidence level in the target confidence level interval, and determine the fireworks category corresponding to the image to be identified based on the position.

[0033] The fireworks recognition method described in the embodiments of the present application uses an image clarity detection algorithm to evaluate image clarity, and determines a target confidence interval based on a comparison result between the clarity of the image to be recognized and a preset clarity condition, which is beneficial to the recognition of fireworks targets. The method predicts the fireworks category of the image to be recognized and obtains the target confidence of the predicted category, and finally compares the target confidence at different positions in the target confidence interval. The image to be recognized is fused and analyzed using the target confidence at different positions, thereby reducing the algorithm's consumption of system resources and improving the accuracy, robustness, and adaptability of fireworks recognition.

[0034] The specific embodiments of the present application are applicable to the identification and classification of fireworks targets in the field of video surveillance. The present application is also applicable to the identification and classification of other types of targets in the field of mid- and high-level video surveillance.

[0035] The image clarity evaluation described in step 12 can use the Tenengrad function and variance function to evaluate the clarity of the detected image:

[0036] The Tenengrad function is a gradient-based function that uses the Sobel operator to extract horizontal and vertical gradient values. The average grayscale value of the image processed by the Sobel operator is the larger the value, the clearer the image. The Tenengrad function calculation formula is as follows:

[0037]

[0038] Where M×N represents the resolution of the image to be recognized;

[0039] (x,y) represents the pixel coordinates of the point in the image to be identified;

[0040] I(x,y) represents the pixel value of the point in the image to be identified;

[0041] f t (I) represents the clarity evaluation value of the Tenengrad function on the image to be recognized.

[0042] The variance function is calculated by calculating the square of the difference between the pixel values ​​of all pixels in the grayscale image and the average grayscale value. The variance is the degree of dispersion of the data (deviation from the mean). The greater the deviation from the mean, the greater the variance, indicating more information, greater energy, and clearer the image. The variance function calculation formula is as follows:

[0043]

[0044] Among them, I mean Represents the grayscale average of the image;

[0045] I(x,y) represents the pixel value of the point in the image to be identified;

[0046] f v (I) represents the clarity evaluation value of the image to be recognized by the variance function.

[0047] Optionally, in step 13, determining a target confidence interval based on a comparison result of the clarity of the image to be identified with a preset clarity condition includes:

[0048] When the clarity of the image to be recognized meets a preset clarity condition, determining the target confidence interval to be a first confidence interval;

[0049] When the clarity of the image to be recognized does not meet a preset clarity condition, the target confidence interval is determined to be a second confidence interval.

[0050] In this step, the preset clarity condition can be a preset specific numerical value or a relative numerical range. By judging whether the clarity evaluation values ​​of the image to be identified by the Tenengrad function and the variance function respectively meet the preset clarity condition, different target confidence intervals are selected and confirmed. The target confidence intervals include a first confidence interval and a second confidence interval.

[0051] In the embodiment of the present application, the first confidence interval can be (0.55, 0.83), and the second confidence interval can be (0.7, 0.9). The clearer the image, the smaller the confidence value requirement, and the blurrier the image, the larger the confidence value requirement. If the image clarity evaluation value meets the preset clarity condition, the first confidence interval is set, and the image credibility requirement is lower; if the image clarity evaluation value does not meet the preset clarity condition, the second confidence interval is set, and the image credibility requirement is higher.

[0052] Optionally, step 13 further includes:

[0053] If the clarity of the image to be identified does not meet the preset clarity condition, the image to be identified is evaluated for clarity again, and if the clarity of the image to be identified does not meet the preset clarity condition, the target confidence interval is determined to be the second confidence interval.

[0054] Specifically, the clarity of the image to be identified is evaluated again. If the image clarity evaluation value meets the preset clarity condition, the target confidence interval is determined to be the first confidence interval; if the image clarity evaluation value still does not meet the preset clarity condition, the target confidence interval is determined to be the second confidence interval from the start of this clarity evaluation to the final identification of the fireworks category.

[0055] If the image clarity evaluation value does not meet the preset clarity conditions after the first evaluation, the image clarity evaluation can be repeated multiple times to confirm the target confidence interval. This ensures accurate detection results and ensures high accuracy and robustness of fireworks classification using the deep learning algorithm.

[0056] Optionally, in step 15, obtaining a position of the target confidence level in the target confidence level interval, and determining the fireworks category corresponding to the image to be identified based on the position, includes:

[0057] determining whether the first predicted fireworks category is the first category based on the fireworks category recognition model;

[0058] When the first predicted fireworks category is the first category, obtaining a first comparison result between the first confidence level and the target confidence level interval, and obtaining first image information;

[0059] determining a fireworks category corresponding to the image to be identified based on the first comparison result;

[0060] The target confidence includes the first confidence.

[0061] In the above steps, the image to be identified that is input into the deep learning algorithm can be a picture that has been scaled to a fixed size. However, if the aspect ratio needs to be kept unchanged, the image to be identified is scaled to a fixed size. Scaling the image to be identified to a fixed size is beneficial to saving algorithm resources and is conducive to more accurate prediction and identification of fireworks types.

[0062] In the above steps, the fireworks type recognition model was generated by modifying YOLOX's image enhancement strategy, pre-generating a batch of fireworks images, and then training the model using the YOLOX algorithm. As shown in the figure, this specifically includes:

[0063] Data collection;

[0064] Create a dataset;

[0065] Annotated datasets;

[0066] Modify YOLOX's image enhancement strategy;

[0067] The model is generated by training samples of computer graphics processing units (GPUs).

[0068] In this embodiment, images of different regions, seasons, weather conditions, application scenarios, types, and targets in China are collected. This embodiment designs 10,000 positive labels and 3,000 negative labels. The positive labels are divided into five types: ordinary smoke, chimney smoke, black smoke, fire, and house smoke. The negative labels are divided into two types: light and fog. The collected positive and negative samples are labeled. For the labeled data set, the image enhancement strategy of YOLOX is modified, and 5,000 outdoor fireworks images are pre-generated for outdoor scenes. By modifying parameters such as the learning rate and the number of iterations, the samples are trained using a computer GPU, and the model is saved after approximately 300 training iterations. Finally, the model is tested. When the test set meets the high recognition rate of the image to be recognized, the training is completed. Otherwise, the sample training is continued until the recognition requirements are met.

[0069] Wherein, in the sample training stage, positive samples and negative samples with obvious distinction from the positive samples are set up, and the accuracy and recall rate of the model are improved through continuous updating of the negative samples.

[0070] It should be noted that in the embodiments of the present application, the positive class label of fireworks is provided with five categories of ordinary smoke, chimney smoke, black smoke, fire and house smoke. Since it is found in the early stage of research and development of the present application that there are more false positives caused by light and cloud, the light and cloud are separately listed as negative class labels when the label system is designed again, so as to eliminate the false positives of light and cloud in actual application.

[0071] In the present embodiment, the positive class label is set as the first category, and the negative class label is set as the second category. In some other embodiments of the present application, categories can also be set according to different needs of different scene application types.

[0072] In the embodiments of the present application, a deep learning algorithm is called for target detection, which can be a target detection YOLOX algorithm. The YOLOX algorithm is used to train various targets of outdoor fireworks, so that the deep neural network can recognize the firework targets and classify them. The target detection YOLOX algorithm is introduced into the field of outdoor firework target recognition, which can detect targets without candidate boxes, and has good detection effect on targets with irregular shape, too large or too small size. At the same time, through training and testing of the model, the image enhancement strategy of the YOLOX algorithm is modified, so that the YOLOX algorithm is more suitable for detecting firework targets.

[0073] In the above steps, the deep learning algorithm is called for category prediction based on the captured image to be identified, the target confidence T of the predicted category is determined, and the image information of the image to be identified is recorded, which can include the pixel (X, Y) of the detected target, and the width W and height H of the target. If the predicted category of the firework type in the image to be identified is the first category, the confidence is compared with the target confidence interval determined in the foregoing, and the final firework recognition category is confirmed according to the comparison result.

[0074] The target confidence interval used in this step is based on the evaluation of image clarity. The image to be identified that meets the preset clarity condition is suitable for the first confidence interval, and the image to be identified that does not meet the preset clarity condition is suitable for the second confidence interval.

[0075] Wherein, the predicted firework category obtained in this step is the first predicted firework category C1, the target confidence obtained by the first category prediction of the image to be identified is the first confidence T1, the pixel of the detected target in the identified image is (X1, Y1), and the width W1 and height H1 of the target.

[0076] Optionally, step 15 further includes:

[0077] When the first confidence level is greater than or equal to the maximum value of the target confidence level interval, determining that the fireworks category corresponding to the image to be identified is the first category;

[0078] When the first confidence level is within the target confidence level interval, the area of ​​the image to be identified is obtained and compared with a preset area, and the fireworks category corresponding to the image to be identified is determined based on the area comparison result.

[0079] In this step, if the first confidence level is greater than or equal to the maximum value of the target confidence interval, it is considered that there is a high possibility that the image to be identified contains fireworks of this category, and the fireworks category in the image to be identified is determined to be the first category, and detailed type information of the fireworks in the first category can be output; if the first confidence level is less than the maximum value of the target confidence interval and greater than the minimum value of the target confidence interval, that is, it is within the target confidence interval, then it is necessary to judge the area information of the image to be identified.

[0080] The area of ​​the image to be identified may be obtained by the width W and height H in the aforementioned image information.

[0081] Optionally, step 15 further includes:

[0082] When the first confidence level is within the target confidence level interval, if the area of ​​the image to be identified is larger than the preset area, performing the first operation twice at a preset time interval, and determining the fireworks category corresponding to the image to be identified based on parameters obtained by the first operation;

[0083] If the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified, and re-identifying the fireworks category of the magnified image to be identified to determine the fireworks category corresponding to the image to be identified;

[0084] The first operation includes predicting the fireworks category of the image to be identified, obtaining the predicted fireworks category of the image to be identified, the target confidence corresponding to the predicted fireworks category, and image information of the image to be identified.

[0085] In this step, if the area of ​​the image to be identified is less than or equal to the preset area, it indicates that the image to be identified is a small target image. When determining and predicting the fireworks type, the image to be identified can be magnified, and the fireworks type of the magnified image can be predicted again to ensure the accuracy of the algorithm's identification of the fireworks type. If the area of ​​the image to be identified is larger than the preset area, the PTZ value of the current image to be identified is obtained and compared with the PTZ value of the initial image to be identified. If they are equal, the first operation is repeated twice with a preset time interval. If not, the PTZ value of the image to be identified is converted to the initial PTZ value, and the first operation is repeated twice.

[0086] In this embodiment of the present application, the first operation includes predicting the fireworks category of the image to be identified, obtaining the fireworks category, the confidence level of the fireworks category, and image information of the image, thereby jumping to steps 14 and 15 for execution. Performing the first operation two or more times allows the deep learning algorithm to more accurately predict the fireworks category of the image to be identified, significantly reducing the misclassification rate of the fireworks category in the image to be identified.

[0087] Optionally, when the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified by a multiple, and re-identifying the magnified image to be identified to determine the fireworks category corresponding to the image to be identified, including:

[0088] When the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified to obtain a magnified image to be identified;

[0089] Predicting the magnified image to be identified, obtaining a fireworks identification category of the magnified image to be identified and a confidence level corresponding to the fireworks identification category;

[0090] The position of the confidence level corresponding to the fireworks recognition category in the target confidence level interval is obtained, and the fireworks category corresponding to the amplified image to be recognized is determined based on the position.

[0091] In this step, if the area is smaller than the preset area requirement, a local adaptive image-based algorithm can be used to magnify the area surrounding the image. Specifically, a 320×320 pixel area can be magnified by a factor of 3. Fireworks category prediction is performed again on the magnified image to obtain a second predicted fireworks category and a second confidence level.

[0092] Optionally, determining the fireworks category corresponding to the image to be identified based on the parameters obtained by the first operation includes:

[0093] Obtaining a first target result corresponding to the first operation for the first time, where the first target result includes a second confidence level, a second predicted fireworks category, and second image information;

[0094] Obtaining a second target result corresponding to the second first operation, where the second target result includes a third confidence level, a third predicted fireworks category, and third image information;

[0095] If the first predicted fireworks category, the second predicted fireworks category, and the third predicted fireworks category are all the same category, determining that the fireworks category corresponding to the image to be identified is the first category;

[0096] When the first predicted fireworks category, the second predicted fireworks category, and the third predicted fireworks category are not the same category, the fireworks category of the image to be identified is determined based on a comparison result of the first confidence level, the second confidence level, the third confidence level, and the target confidence level interval.

[0097] In this embodiment, two corresponding target results are obtained based on the two first operations. The target results include target confidence, predicted fireworks category, and image information. If the first, second, and third predicted fireworks categories are all the same, then the three fireworks predictions are relatively reliable and have a low error rate. Fireworks information of that category can be output, and the algorithm is complete. If the three predicted fireworks categories differ, a confidence assessment is required to determine the fireworks category that is most similar to the true category.

[0098] In this step, the first operation is performed twice based on a preset time interval, and the image to be identified is again called into the deep learning algorithm to predict the fireworks category. The parameter result is obtained by performing the first operation, and the parameter result includes the second predicted fireworks category C2 and the second confidence T2 of the second predicted fireworks category obtained by the first first operation, as well as the second image information. The second image information includes the pixels (X2, Y2) of the detected image, and the width W2 and height H2 of the image; the third predicted fireworks category C3 and the third confidence T3 of the third predicted fireworks category, as well as the third image information are obtained by performing the second first operation. The third image information includes the pixels (X3, Y3) of the detected image, and the width W3 and height H3 of the image.

[0099] Optionally, determining the fireworks category of the image to be identified based on a comparison result of the first confidence level, the second confidence level, the third confidence level, and the target confidence level interval includes:

[0100] When the first confidence level, the second confidence level, and the third confidence level are all smaller than the minimum value of the target confidence level interval, determining that the fireworks category of the image to be identified is the second category;

[0101] When any two of the first confidence level, the second confidence level, and the third confidence level are greater than a maximum value of the target confidence level interval, determining that the fireworks category of the image to be identified is the first category;

[0102] When any one of the first confidence level, the second confidence level, and the third confidence level is greater than the minimum value of the target confidence level interval, and at most one of the first confidence level, the second confidence level, and the third confidence level is greater than the maximum value of the target confidence level interval, the first overlap level, the second overlap level, and the third overlap level are obtained to determine the fireworks category of the image to be confirmed.

[0103] In this embodiment, when the first predicted fireworks category, the second predicted fireworks category, and the third predicted fireworks category are not the same category, the first confidence level, the second confidence level, and the third confidence level are compared with the target confidence level interval.

[0104] Among them, the first confidence level, the second confidence level, and the third confidence level are all less than the minimum value of the target confidence level interval, indicating that the possibility of the fireworks type in the image to be identified being the first category is small, and it can be considered that the feasibility of the second category is higher, and the result of the second category fireworks type is output; when any two of the first confidence level, the second confidence level, and the third confidence level are greater than the maximum value of the target confidence level interval, the feasibility of determining the fireworks type of the image to be identified as the fireworks type in the first category is higher, and it can be determined as the fireworks type of the first category; in other cases, the credibility of using confidence level to determine the fireworks type is not high, the misjudgment rate is high, and re-judgment is performed based on the overlap.

[0105] Optionally, obtaining the first overlap degree, the second overlap degree, and the third overlap degree to determine the fireworks category of the image to be confirmed includes:

[0106] acquiring a first overlap degree based on the first image information and the second image information, acquiring a second overlap degree based on the second image information and the third image information, and acquiring a third overlap degree based on the first image information and the third image information;

[0107] When the first overlap degree, the second overlap degree, and the third overlap degree are not equal, determining that the fireworks category of the image to be identified is the first category;

[0108] When the first overlap degree, the second overlap degree, and the third overlap degree are equal, the fireworks category of the image to be identified is determined to be the second category.

[0109] In the embodiment of the present application, equal overlap indicates that the image to be identified is relatively stable and changes little. Unequal overlap indicates that the image to be identified has changed significantly over the three operations. Various static objects such as night lights, reflections from accumulated water, and mist on rainy days have a significant impact on the identification and prediction of fireworks types, resulting in a low accuracy of the identified fireworks types. In this step, the overlap is obtained from the image information obtained through the three operations. For example, the first overlap is obtained based on the first image information and the second image information. Specifically, the area of ​​the image to be identified is obtained from the first image information and the second image information, and the overlap is calculated for the two obtained area values. The calculation formula is as follows:

[0110]

[0111] Where Iou represents the degree of overlap;

[0112] A∩B represents the intersection of the image areas obtained from any two image information of the image to be identified;

[0113] A∪B represents the union of the image areas obtained from any two pieces of image information of the image to be identified.

[0114] The technical solution provided in the embodiments of the present application improves the accuracy and recall rate of fireworks recognition and classification by designing different categories of fireworks to distinguish them, and uses image clarity evaluation to determine different confidence intervals according to the clarity of different images to improve the accuracy of fireworks recognition types. The YOLOX algorithm is introduced to train various types of fireworks targets, so that the deep neural network can identify and classify fireworks targets, making the improved YOLOX algorithm more suitable for detecting fireworks targets. The present application performs fusion analysis on single-frame or multi-frame images according to different confidence thresholds, which reduces the algorithm's consumption of system resources, and at the same time overcomes the influence of static targets such as night lights, reflections from accumulated water, and lens blur on recognition, thereby improving the accuracy, robustness, and adaptability of fireworks target recognition.

[0115] See also Figure 2 , Figure 2 This is a second flow chart of a method for identifying fireworks provided in an embodiment of the present application, as shown in FIG. Figure 2 As shown, the method includes the following steps:

[0116] Step 21, obtaining an image to be identified;

[0117] Step 22, evaluating the image clarity of the image to be recognized;

[0118] Step 23, determining a target confidence interval based on the image clarity evaluation result;

[0119] Specifically, step 23 further includes:

[0120] Step 231: When the image clarity meets a preset clarity condition, determine the target confidence interval as a first confidence interval;

[0121] Step 232 : When the image clarity does not meet the preset clarity condition, the image clarity evaluation is performed again. If the image clarity still does not meet the preset clarity condition, the target confidence interval is determined to be the second confidence interval.

[0122] Step 24: Call the YOLOX model file to obtain the predicted fireworks type;

[0123] Step 25, determining the predicted fireworks type;

[0124] Specifically, step 25 includes:

[0125] Step 251: when the fireworks type is predicted to be the first category, obtaining a first confidence level and first image information;

[0126] Step 252: When the predicted fireworks type is the second category, output the second category.

[0127] Step 26, confidence judgment;

[0128] Step 27 : When the first confidence level is within the target confidence level interval, the area of ​​the image to be identified is obtained and compared with a preset area, and the fireworks category corresponding to the image to be identified is determined based on the area comparison result.

[0129] Specifically, step 27 includes:

[0130] Step 271: If the first confidence level is greater than or equal to the maximum value of the target confidence level interval, then the fireworks category corresponding to the image to be identified is determined to be the first category;

[0131] Step 272 : When the area of ​​the image to be identified is less than or equal to the preset area, the image to be identified is magnified to obtain the magnified image to be identified.

[0132] Step 28 : Perform two more fireworks category predictions on the enlarged image to obtain a second predicted fireworks category, a second confidence level, and a third predicted fireworks category, and a third confidence level.

[0133] Specifically, step 28 includes:

[0134] Step 281, predicting the magnified image to be identified;

[0135] Step 282: Obtain the fireworks recognition category and the confidence level corresponding to the fireworks recognition category of the magnified image to be recognized;

[0136] In step 283, a position of the confidence degree corresponding to the firework recognition category in the target confidence degree interval is obtained, and a firework category corresponding to the magnified to-be-recognized image is determined based on the position.

[0137] In step 29, an overlap degree is determined, and a to-be-recognized firework type is determined based on a determination result of the overlap degree.

[0138] Specifically, step 29 includes:

[0139] In step 291, second image information and third image information are obtained, and first, second, and third overlap degrees are obtained based on the first, second, and third image information.

[0140] In step 292, a relationship among the first, second, and third overlap degrees is determined, and a to-be-recognized firework type is determined.

[0141] The specific implementation process of the firework recognition method provided in the embodiments of the present application can refer to the description of the method embodiments of the present application described above, and details are not described herein again. In the embodiments of the present application, through multiple analyses of the to-be-recognized image, the influence of static targets such as night light, water reflection, and rainwater mist on recognition is overcome, the accuracy, robustness, and adaptability of the firework target recognition are improved, and the false positive rate is reduced. Figure 1

[0142] Referring to Figure 3 , Figure 3 A structural diagram of a firework recognition device provided in the embodiments of the present application is shown in FIG. 3. As shown in FIG. 3, the firework recognition device 30 includes: Figure 3

[0143] The obtaining module 31 is configured to obtain a to-be-recognized image.

[0144] The evaluation module 32 is configured to perform image definition evaluation on the to-be-recognized image.

[0145] The determination module 33 is configured to determine a target confidence degree interval based on a comparison result of the definition of the to-be-recognized image and a preset definition condition.

[0146] The prediction module 34 is configured to predict a firework category of the to-be-recognized image, and obtain a predicted firework category of the to-be-recognized image, a target confidence degree corresponding to the predicted firework category, and image information of the to-be-recognized image.

[0147] The discrimination module 35 is configured to obtain a position of the target confidence degree in the target confidence degree interval, and determine a firework category corresponding to the to-be-recognized image based on the position.

[0148] Optionally, the determination module 33 includes:

[0149] ​​a first confidence interval unit, configured to determine that the target confidence interval is a first confidence interval when the clarity of the image to be recognized satisfies a preset clarity condition;

[0150] The second confidence interval unit is configured to determine that the target confidence interval is a second confidence interval when the clarity of the image to be identified does not meet a preset clarity condition.

[0151] Optionally, the second confidence interval unit is further used to:

[0152] When the clarity of the image to be identified does not meet the preset clarity condition, it is used to re-evaluate the image clarity of the image to be identified, and when the clarity of the image to be identified does not meet the preset clarity condition, it is used to determine that the target confidence interval is the second confidence interval.

[0153] Optionally, the determination module 35 is further configured to:

[0154] determining whether the first predicted fireworks category is the first category based on the fireworks category recognition model;

[0155] When the first predicted fireworks category is the first category, obtaining a first comparison result between the first confidence level and the target confidence level interval, and obtaining first image information;

[0156] determining a fireworks category corresponding to the image to be identified based on the first comparison result;

[0157] The target confidence includes the first confidence.

[0158] Optionally, the determination module 35 is further configured to:

[0159] When the first confidence level is greater than or equal to the maximum value of the target confidence level interval, determining that the fireworks category corresponding to the image to be identified is the first category;

[0160] When the first confidence level is within the target confidence level interval, the area of ​​the image to be identified is obtained and compared with a preset area, and the fireworks category corresponding to the image to be identified is determined based on the area comparison result.

[0161] Optionally, the determination module 35 is further configured to:

[0162] When the first confidence level is within the target confidence level interval, if the area of ​​the image to be identified is larger than the preset area, performing the first operation twice at a preset time interval, and determining the fireworks category corresponding to the image to be identified based on parameters obtained by the first operation;

[0163] If the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified, and re-identifying the fireworks category of the magnified image to be identified to determine the fireworks category corresponding to the image to be identified;

[0164] The first operation includes predicting the fireworks category of the image to be identified, obtaining the predicted fireworks category of the image to be identified, the target confidence corresponding to the predicted fireworks category, and image information of the image to be identified.

[0165] Optionally, the determination module 35 is further configured to:

[0166] When the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified to obtain a magnified image to be identified;

[0167] Predicting the magnified image to be identified, obtaining a fireworks identification category of the magnified image to be identified and a confidence level corresponding to the fireworks identification category;

[0168] The position of the confidence level corresponding to the fireworks recognition category in the target confidence level interval is obtained, and the fireworks category corresponding to the amplified image to be recognized is determined based on the position.

[0169] Optionally, the determination module 35 is further configured to:

[0170] Obtaining a first target result corresponding to the first operation for the first time, where the first target result includes a second confidence level, a second predicted fireworks category, and second image information;

[0171] Obtaining a second target result corresponding to the second first operation, where the second target result includes a third confidence level, a third predicted fireworks category, and third image information;

[0172] When the first predicted fireworks category, the second predicted fireworks category, and the third predicted fireworks category are all the same category, determining that the fireworks category corresponding to the image to be identified is the first category;

[0173] When the first predicted fireworks category, the second predicted fireworks category, and the third predicted fireworks category are not the same category, the fireworks category of the image to be identified is determined based on a comparison result of the first confidence level, the second confidence level, the third confidence level, and the target confidence level interval.

[0174] Optionally, the determination module 35 is further configured to:

[0175] in a case where the first confidence degree, the second confidence degree, and the third confidence degree are all less than the minimum value of the target confidence degree interval, the first class is determined as the firework class of the to-be-identified image;

[0176] in a case where any two of the first confidence degree, the second confidence degree, and the third confidence degree are greater than the maximum value of the target confidence degree interval, the first class is determined as the firework class of the to-be-identified image;

[0177] in a case where any one of the first confidence degree, the second confidence degree, and the third confidence degree is greater than the minimum value of the target confidence degree interval, and at most one of the first confidence degree, the second confidence degree, and the third confidence degree is greater than the maximum value of the target confidence degree interval, the first overlap degree, the second overlap degree, and the third overlap degree are obtained, and the firework class of the to-be-identified image is determined.

[0178] Optionally, the determining module 35 is further configured to:

[0179] obtain the first overlap degree based on the first image information and the second image information, obtain the second overlap degree based on the second image information and the third image information, and obtain the third overlap degree based on the first image information and the third image information;

[0180] in a case where the first overlap degree, the second overlap degree, and the third overlap degree are not equal, the first class is determined as the firework class of the to-be-identified image;

[0181] in a case where the first overlap degree, the second overlap degree, and the third overlap degree are equal, the second class is determined as the firework class of the to-be-identified image.

[0182] The firework identification device provided in the embodiment of the present application obtains the to-be-identified image and transmits the to-be-identified image to the evaluation module, the evaluation module informs the prediction module to capture the image, the prediction module predicts the firework type, and finally transmits the to-be-identified image to the determining module to determine the firework class according to the position of the confidence degree in the target confidence degree interval and output. The firework identification device provided in the embodiment of the present application improves the accuracy of firework identification, reduces the misjudgment rate, and can be applied to the field of forest fire prevention, straw burning, and the like to produce economic benefits.

[0183] Referring to Figure 4 The embodiment of the present application further provides an electronic device 40, which comprises a processor 41, a memory 42, a computer program stored in the memory 42 and executable on the processor, and the computer program is executed by the processor 41 to implement each process of the firework identification method embodiment and achieve the same technical effect. To avoid repetition, details are not described herein.

[0184] The present application also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the various processes of the aforementioned fireworks recognition method embodiment and achieves the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0185] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0186] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0187] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A method for identifying fireworks, characterized in that: The method comprises: Obtain the image to be recognized; Performing image clarity evaluation on the image to be identified; Determining a target confidence interval based on a comparison result of the clarity of the image to be identified with a preset clarity condition; Predicting the fireworks category of the image to be identified, obtaining the predicted fireworks category of the image to be identified, a target confidence corresponding to the predicted fireworks category, and image information of the image to be identified; Obtaining a position of the target confidence level in the target confidence level interval, and determining a fireworks category corresponding to the image to be identified based on the position; The determining of a target confidence interval based on a comparison result of the clarity of the image to be identified with a preset clarity condition includes: When the clarity of the image to be recognized meets a preset clarity condition, determining the target confidence interval to be a first confidence interval; When the clarity of the image to be recognized does not meet a preset clarity condition, the target confidence interval is determined to be a second confidence interval.

2. The method according to claim 1, characterized in that When the clarity of the image to be recognized does not meet the preset clarity condition, determining the target confidence interval as the second confidence interval includes: If the clarity of the image to be identified does not meet the preset clarity condition, the image to be identified is evaluated for clarity again, and if the clarity of the image to be identified does not meet the preset clarity condition, the target confidence interval is determined to be the second confidence interval.

3. The method according to claim 1, characterized in that The obtaining of a position of the target confidence level in the target confidence level interval, and determining the fireworks category corresponding to the image to be identified based on the position, includes: determining whether the first predicted fireworks category is the first category based on the fireworks category recognition model; When the first predicted fireworks category is the first category, obtaining a first comparison result between the first confidence level and the target confidence level interval, and obtaining first image information; determining a fireworks category corresponding to the image to be identified based on the first comparison result; The target confidence includes the first confidence.

4. The method according to claim 3, characterized in that The determining, based on the first comparison result, the fireworks category corresponding to the image to be identified includes: When the first confidence level is greater than or equal to the maximum value of the target confidence level interval, determining that the fireworks category corresponding to the image to be identified is the first category; When the first confidence level is within the target confidence level interval, the area of ​​the image to be identified is obtained and compared with a preset area, and the fireworks category corresponding to the image to be identified is determined based on the area comparison result.

5. The method according to claim 4, characterized in that When the first confidence level is within the target confidence level interval, obtaining the area of ​​the image to be identified and comparing it with a preset area, and determining the fireworks category corresponding to the image to be identified based on the area comparison result, includes: When the first confidence level is within the target confidence level interval, if the area of ​​the image to be identified is larger than the preset area, performing the first operation twice at a preset time interval, and determining the fireworks category corresponding to the image to be identified based on parameters obtained by the first operation; If the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified, and re-identifying the fireworks category of the magnified image to be identified to determine the fireworks category corresponding to the image to be identified; The first operation includes predicting the fireworks category of the image to be identified, obtaining the predicted fireworks category of the image to be identified, the target confidence corresponding to the predicted fireworks category, and image information of the image to be identified.

6. The method according to claim 5, characterized in that When the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified by a multiple, and re-identifying the magnified image to be identified to determine the fireworks category corresponding to the image to be identified, including: When the area of ​​the image to be identified is less than or equal to the preset area, magnifying the image to be identified to obtain a magnified image to be identified; Predicting the magnified image to be identified, obtaining a fireworks identification category of the magnified image to be identified and a confidence level corresponding to the fireworks identification category; The position of the confidence level corresponding to the fireworks recognition category in the target confidence level interval is obtained, and the fireworks category corresponding to the amplified image to be recognized is determined based on the position.

7. The method according to claim 5, characterized in that The determining the fireworks category corresponding to the image to be identified based on the parameters obtained by the first operation includes: Obtaining a first target result corresponding to the first operation for the first time, where the first target result includes a second confidence level, a second predicted fireworks category, and second image information; Obtaining a second target result corresponding to the second first operation, where the second target result includes a third confidence level, a third predicted fireworks category, and third image information; If the first predicted fireworks category, the second predicted fireworks category, and the third predicted fireworks category are all the same category, determining that the fireworks category corresponding to the image to be identified is the first category; When the first predicted fireworks category, the second predicted fireworks category, and the third predicted fireworks category are not the same category, the fireworks category of the image to be identified is determined based on a comparison result of the first confidence level, the second confidence level, the third confidence level, and the target confidence level interval.

8. The method according to claim 7, characterized in that The determining the fireworks category of the image to be identified based on a comparison result of the first confidence level, the second confidence level, the third confidence level, and the target confidence level interval includes: When the first confidence level, the second confidence level, and the third confidence level are all smaller than the minimum value of the target confidence level interval, determining that the fireworks category of the image to be identified is the second category; When any two of the first confidence level, the second confidence level, and the third confidence level are greater than a maximum value of the target confidence level interval, determining that the fireworks category of the image to be identified is the first category; When any one of the first confidence level, the second confidence level, and the third confidence level is greater than the minimum value of the target confidence level interval, and at most one of the first confidence level, the second confidence level, and the third confidence level is greater than the maximum value of the target confidence level interval, the first overlap level, the second overlap level, and the third overlap level are obtained to determine the fireworks category of the image to be identified.

9. The method according to claim 8, characterized in that The obtaining of the first overlap degree, the second overlap degree, and the third overlap degree to determine the fireworks category of the image to be identified includes: acquiring a first overlap degree based on the first image information and the second image information, acquiring a second overlap degree based on the second image information and the third image information, and acquiring a third overlap degree based on the first image information and the third image information; When the first overlap degree, the second overlap degree, and the third overlap degree are not equal, determining that the fireworks category of the image to be identified is the first category; When the first overlap degree, the second overlap degree, and the third overlap degree are equal, the fireworks category of the image to be identified is determined to be the second category.

10. A fireworks identification device, characterized in that: include: An acquisition module, used to acquire an image to be identified; An evaluation module, configured to evaluate the image clarity of the image to be identified; a determination module, configured to determine a target confidence interval based on a comparison result of the clarity of the image to be identified with a preset clarity condition; a prediction module, configured to predict the fireworks category of the image to be identified, and obtain the predicted fireworks category of the image to be identified, a target confidence level corresponding to the predicted fireworks category, and image information of the image to be identified; a discrimination module, configured to obtain a position of the target confidence level within the target confidence level interval, and determine a fireworks category corresponding to the image to be identified based on the position; The determination module includes: a first confidence interval unit, configured to determine that the target confidence interval is a first confidence interval when the clarity of the image to be recognized satisfies a preset clarity condition; The second confidence interval unit is configured to determine that the target confidence interval is a second confidence interval when the clarity of the image to be identified does not meet a preset clarity condition.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the fireworks recognition method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the fireworks recognition method according to any one of claims 1 to 9 are implemented.

Images