A method for pyrotechnic point recognition

Abstract

The present application relates to a kind of pyrotechnic point identification method, it includes the following steps: the scene image is input into first pyrotechnic point detection model, to output preliminary detection result;The preliminary detection result is cropped, to divide out suspected pyrotechnic point area image, and the suspected pyrotechnic point area image is input into second pyrotechnic point detection model, to output secondary detection result;Based on scene complexity, the voting weight of pyrotechnic point detection model is dynamically adjusted, and based on the voting weight of dynamic adjustment, the final pyrotechnic point detection confidence is obtained.The present application is based on the double model collaborative architecture to complete pyrotechnic point detection, simultaneously, based on the initial voting weight of two pyrotechnic point detection models under different scene complexity of reinforcement learning algorithm is dynamically adjusted, to make that weight adjustment senses scene difference, realizes the high degree of adaptation and deep connection of voting weight and scene complexity, realizes high-precision pyrotechnic point identification under different scene complexity.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for identifying smoke points. Background Technology

[0002] In existing technologies, large models are typically used to identify fireworks points, which involves fine-tuning the large models. However, the current fine-tuning of large models mostly adopts LoRA technology, which is insufficient for temporal correlation of multimodal data and cross-modal feature mining. This results in the large models having insufficient ability to identify fireworks points in complex scenarios and a high false detection rate. Summary of the Invention

[0003] The purpose of this invention is to provide a method for identifying fireworks points. This method is based on a dual-model collaborative architecture to detect fireworks points. At the same time, it uses a reinforcement learning algorithm to dynamically adjust the initial voting weights of the two fireworks point detection models under different scene complexities. This allows the weight adjustment to perceive scene differences, achieve a high degree of adaptation and deep correlation between voting weights and scene complexity, and realize high-precision fireworks point identification under different scene complexities.

[0004] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions.

[0005] A method for identifying smoke points is provided, which includes the following steps:

[0006] The scene image is input into the first fireworks detection model to perform preliminary detection of fireworks points and output the preliminary detection results.

[0007] The preliminary detection results are cropped to segment the suspected fireworks point area image, and the suspected fireworks point area image is input into the second fireworks point detection model to perform secondary detection of fireworks points, and the secondary detection results are output.

[0008] The voting weights of the fireworks detection model are dynamically adjusted based on the scene complexity, and the final confidence level of fireworks detection is obtained based on the dynamically adjusted voting weights.

[0009] Preferably, the first pyrotechnic point detection model is obtained by training a first detection model.

[0010] Preferably, the second pyrotechnic point detection model is obtained by fine-tuning a second detection model, and the fine-tuning includes the following steps:

[0011] Construct a time-series image-text dataset containing consecutive frame images and scene text;

[0012] The second detection model was fine-tuned based on QA-LoRA and cross-modal contrastive learning.

[0013] Preferably, the attention layer parameter W of the second detection model is fine-tuned based on QA-LoRA, and the attention layer parameter W = W A W B W A W B All are low-rank matrices.

[0014] Preferably, the loss function used when fine-tuning the second detection model based on cross-modal contrastive learning. as follows:

[0015]

[0016] Where I represents the image features of the image modality data in the image-text time series dataset; T represents the text features of the text modality data in the image-text time series dataset, and image feature I and text feature T are correctly paired sample pairs; T' represents the text features of the remaining text modality data in the image-text time series dataset, excluding the correctly paired text features T; T total is the set of all text modal data in the image-text time series dataset; sim(I,T) represents the similarity between image feature I and text feature T; τ is the temperature coefficient.

[0017] Preferably, the voting weights of the fireworks detection model are dynamically adjusted based on the scene complexity, and the final fireworks detection confidence is obtained based on the dynamically adjusted voting weights, including the following steps:

[0018] Get the complexity of the current scene;

[0019] The initial voting weights of the first and second fireworks detection models are determined respectively, and the initial voting weights are dynamically adjusted based on the scene perception reinforcement learning algorithm and scene complexity.

[0020] Bayesian calibration is performed on the detection confidence and judgment results of the preliminary detection results and the secondary detection results to obtain the Bayesian calibration probability;

[0021] The final fire point detection confidence level is obtained based on the adjusted voting weights of the first fire point detection model, the adjusted voting weights of the second fire point detection model, and the Bayesian calibration probability.

[0022] Preferably, the scenario complexity is obtained based on the following formula:

[0023]

[0024] Where C represents the complexity of the current scenario; ω1 and ω2 are the first and second weights, respectively, and ω1 + ω2 = 1; N int N represents the number of interference sources in the current scene. int,maxThis represents the historical maximum number of interference sources for the same scene or the total number of interference sources in a time-series image / text dataset; R ref This represents the percentage of reflective areas in the current scene image.

[0025] Preferably, the initial voting weights are dynamically adjusted based on the scene-aware reinforcement learning algorithm and the scene complexity, including the following steps:

[0026] Let w be the initial voting weight of the first firework detection model at time step t. 1,t The initial voting weight of the second firework detection model is w. 2,t And w 1,t +w 2,t =1;

[0027] Constructing the reinforcement learning state space s at time step t t =[θ t C t hist t ], where θ t The dual-model voting weighting strategy is defined at time step t, and θ is given. t =[w 1,t ,w 2,t ];C t The scene complexity at time step t; hist t Historical false positive rate;

[0028] The initial voting weights are dynamically adjusted based on the following formula:

[0029]

[0030] Where, θ t+1 The voting weight strategy for the two models at time step t+1; α is the learning rate; a t The weight adjustment action to be executed at time step t; R(s) t ,a t C t ) for reinforcement learning state space s t Scenario complexity C t Execute the weight adjustment action a t The reward received later; This is the gradient operator, used to calculate the direction of weight updates; The expected computation is used to calculate the traversal of all possible states s. t Action a t Average return after sequence;

[0031] Output the dynamically adjusted dual-model voting weight strategy θ t+1 .

[0032] Preferably, the detection confidence and judgment results of the preliminary detection results and the secondary detection results are subjected to Bayesian calibration according to the following formula to obtain the Bayesian calibration probability:

[0033]

[0034] Wherein, P(fire|d s ,d l ,c s ,c l ) represents the Bayesian calibration probability; d s c s These are the preliminary detection results for pyrotechnic point determination and the confidence level of the pyrotechnic point determination results; d l c l These are the pyrotechnic point determination results and the confidence level of the pyrotechnic point determination results from the secondary detection; P(d s ,d l |fire) represents the likelihood probability of the combination of the initial detection result and the secondary detection result when a real fire point exists in the scene; ω s ω l These are the confidence weighting coefficients for the pyrotechnic point determination results of the preliminary detection results and the secondary detection results, respectively, and ω... s =sigmoid(c s ·k))ω l =sigmoid(c l ·k), where k is the confidence scaling factor; P(fire) is the probability that a suspected fire point is actually a fire point when independent of the initial and secondary detection results; P(d) s ,d l ) represents the evidence factor, indicating that the preliminary detection result of the pyrotechnic point determination is d. s And the result of the second detection of the pyrotechnic point is d. l The probability of; Both the preliminary detection results and the secondary detection results indicate that there is no probability of any suspected pyrotechnic points. This is a binary variable used to represent the preliminary detection result of the fireworks point determination, which concludes that there are no suspected fireworks points. is a binary variable used to represent the result of the secondary detection of fireworks, indicating that there are no suspected fireworks.

[0035] Preferably, the final fire point detection confidence level is obtained according to the following formula:

[0036] C final =w1'·c s +w2'·c l ·P(fire|ds ,d l ,c s ,c l )

[0037] Among them, C final To determine the final confidence level for the detection of the pyrotechnic point, and C final ∈[0,1]; w1' and w2' are the dynamically adjusted dual-model voting weight strategies θ t+1 The adjusted voting weights of the first firework detection model and the adjusted voting weights of the second firework detection model are w1' and w2', respectively, and w1'+w2'=1.

[0038] Preferably, the smoke point identification method further includes:

[0039] If the final firework detection confidence level is less than or equal to the confidence threshold, then a false firework detection is considered to exist. If the final firework detection confidence level is greater than or equal to the confidence threshold, then no false firework detection is considered to exist. The secondary detection result is then directly used as the final firework detection result and output.

[0040] Compared with the prior art, the present invention has the following technical effects:

[0041] This invention utilizes a dual-model collaborative architecture to detect fireworks points, providing more accurate identification results. Simultaneously, it employs QA-LoRA fine-tuning technology to efficiently fine-tune the second fireworks detection model, reducing computational resource consumption while maintaining model performance. Furthermore, it constructs a time-series image-text dataset containing continuous frame images and scene text. Through cross-modal contrastive learning, the second detection model learns the differences between the dynamic features of fireworks points and the static features of interference sources, thereby improving the detection accuracy of the second detection model.

[0042] Furthermore, based on reinforcement learning algorithms, the initial voting weights of the two fireworks detection models under different scene complexities are dynamically adjusted so that the weight adjustment can perceive scene differences, achieve a high degree of adaptation and deep correlation between voting weights and scene complexity, and realize high-precision fireworks recognition under different scene complexities. Attached Figure Description

[0043] Figure 1 This is a flowchart of the steps in the smoke point identification method of the present invention;

[0044] Figure 2 This is the preliminary detection result output by the first pyrotechnic point detection model in this invention;

[0045] Figure 3 This is the secondary detection result output by the second pyrotechnic point detection model in this invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0047] Example 1:

[0048] like Figure 1 As shown, this embodiment provides a method for identifying smoke points, which includes the following steps:

[0049] S1. Use an image acquisition device (such as a camera) to acquire scene images and preprocess the images. The preprocessing includes operations such as image size adjustment, pixel value normalization, and data augmentation to improve the quality of the model's input data.

[0050] S2. The first detection model is trained based on the fireworks point dataset with completed data annotation. After the requirement of the number of iterations for training is met, the first fireworks point detection model is output. In this embodiment, the first detection model includes the YOLOv8 model.

[0051] In addition, the second detection model is fine-tuned, and after the fine-tuning is completed, the second firework detection model is output. In this embodiment, the second detection model includes the Qwen2.5_vl_7b model.

[0052] Specifically, the second detection model is fine-tuned, including the following steps:

[0053] S21. Construct a time-series image-text dataset containing continuous frame images and scene text. The continuous frame images are used to reflect the dynamic diffusion characteristics of smoke from the fireworks point, and the scene text is used to describe the relevant information of the fireworks point in the image. For example, the relevant information of the fireworks point includes one or more of the following: the time of discovery of the fireworks point, the location of the fireworks point, the type of fire source of the fireworks point, and the interference source that causes misidentification of the fireworks point (such as light spots, objects with similar textures to the fireworks point, etc.).

[0054] S22. Fine-tune the trained second detection model based on QA-LoRA (Quantized-Aware Low-Rank Adaptation) and Cross-Modal Contrast Learning (CMCL);

[0055] Specifically, in this embodiment, the attention layer parameter W of the already trained second detection model is fine-tuned based on QA-LoRA, and the attention layer parameter W = W A W B W A W B All of them are low-rank matrices, thus reducing computational resource consumption through quantization methods while maintaining model performance;

[0056] Furthermore, in this embodiment, the aforementioned image-text time-series dataset is used, and the trained second detection model is fine-tuned based on cross-modal contrastive learning. The fine-tuned second detection model is then output as the second fireworks detection model. The loss function for this fine-tuning process is... as follows:

[0057]

[0058] Where I represents the image features of the image modality data in the image-text time series dataset; T represents the text features of the text modality data in the image-text time series dataset, and image feature I and text feature T are correctly paired sample pairs; T' represents the text features of the remaining text modality data in the image-text time series dataset other than the correctly paired text features T, which can be used to compare with image feature I and construct the negative samples required for loss function calculation; T total This is the collection of all text modal data in the image-text time series dataset;

[0059] sim(I,T) represents the similarity between image feature I and text feature T, which can be measured using parameters such as cosine similarity.

[0060] τ is a temperature coefficient used to describe the distribution of similarity scores. The smaller τ is, the sharper the distribution of similarity scores, and the higher the model's requirement for the discrimination of image feature I and text feature T. The larger τ is, the smoother the distribution of similarity scores, and the lower the model's requirement for the discrimination of image feature I and text feature T. In this embodiment, the temperature coefficient τ ranges from 0.01 to 0.5.

[0061] Therefore, this embodiment can make the loss function During the minimized training process, the parameters of the second detection model are continuously adjusted based on the image and text time series dataset and cross-modal contrastive learning, so that the second detection model learns the difference between the dynamic features of the fireworks point and the static features of the interference source, thereby improving the detection accuracy of the second detection model.

[0062] Furthermore, positive and negative sample pairs are constructed during the cross-modal contrastive learning process to enhance the second detection model's ability to distinguish between dynamic and static features.

[0063] S3. Input the scene image into the first smoke and fire point detection model to perform preliminary detection of smoke and fire points, and output the preliminary detection results, as follows: Figure 2 As shown, the preliminary detection results include the location of the suspected pyrotechnic spot area and / or pyrotechnic spot category information;

[0064] S4. The preliminary detection results are cropped to segment the suspected fireworks point area image, and the suspected fireworks point area image is input into the second fireworks point detection model for secondary detection of fireworks points, and the secondary detection results are output (e.g., Figure 3 (as shown);

[0065] S5. Dynamically adjust the voting weights of the fireworks detection model based on scene complexity, and obtain the final fireworks detection confidence based on the dynamically adjusted voting weights. This includes the following steps:

[0066] S51. Based on the number of interference sources and the proportion of reflective areas in the scene image, obtain the complexity of the current scene. The method for obtaining the number of interference sources includes the following steps:

[0067] Define a feature template for interference sources, which can be determined based on prior knowledge or typical visual features of interference sources obtained through clustering.

[0068] The algorithm detects suspected interference sources in the scene image, matches the interference source feature template with the suspected interference source feature, and calculates the feature similarity. For example, in this embodiment, the feature similarity can be cosine similarity. Where F1 represents the target features of the suspected interference source, and F2 represents the template features of the interference source;

[0069] The suspected interference source target is determined to be a real interference source based on the feature similarity. For example, if the feature similarity is greater than or equal to the feature similarity, the suspected interference source target is considered to be a real interference source.

[0070] Repeat the above process of identifying suspected interference sources, feature matching, and feature similarity to count the number of real interference sources in the scene image, denoted as the number of interference sources N. int ;

[0071] Obtaining the percentage of reflective area involves the following steps:

[0072] The reflective areas in a scene image are determined by setting a color space threshold. For example, in the HSV color space, highlight areas typically have high brightness and a specific tonal range, so the brightness V ≥ V can be set. th The area was defined as the reflective area, V th This is the brightness threshold, and its value range can be set to 200-220;

[0073] Image segmentation algorithms (such as U-Net) are used to segment the scene image of the current frame in order to extract the reflective areas;

[0074] Calculate the number of pixels N in the reflective area ref Compared with the total number of pixels N in the current scene image totalThe ratio of the two values ​​is used as the proportion of the reflective area R. ref ,Right now

[0075] Furthermore, this embodiment obtains the scenario complexity based on the following weighted summation formula:

[0076]

[0077] Where C represents the complexity of the current scene, and C∈[0,1]. The larger the value of C, the more complex the scene. ω1 and ω2 are the first weight and the second weight, respectively, and ω1+ω2=1. In this embodiment, ω1 and ω2 can be determined through experimental tuning. For example, the initial values ​​of ω1 and ω2 can be 0.4 and 0.6, respectively. Then, reverse optimization is performed based on the model detection error under different complexity scenarios. N int,max This refers to the historical maximum number of interference sources for the same scene or the total number of interference sources in a time-series dataset of images and text.

[0078] S52. Determine the initial voting weights of the first fireworks detection model and the second fireworks detection model respectively, and dynamically adjust the initial voting weights (including the initial voting weights of the first fireworks detection model or the second fireworks detection model) based on the scene perception reinforcement learning algorithm and the scene complexity C.

[0079] Specifically, the initial voting weights are dynamically adjusted based on the scene-aware reinforcement learning algorithm and the scene complexity C, including the following steps:

[0080] Let w be the initial voting weight of the first firework detection model at time step t. 1,t (e.g., 0.2), the initial voting weight of the second firework detection model is w. 2,t (e.g., 0.8), and w 1,t +w 2,t =1;

[0081] Constructing the reinforcement learning state space s at time step t t =[θ t C t hist t ], where θ t The dual-model voting weighting strategy is defined at time step t, and θ is given. t =[w 1,t ,w 2,t ];C t The scene complexity at time step t; hist t Historical false positive rate;

[0082] The initial voting weights are dynamically adjusted based on the following formula:

[0083]

[0084] Where, θ t+1 The dual-model voting weight strategy at time step t+1 is the updated set of dual-model voting weight parameters adapted to the scenario complexity C; α is the learning rate, used to control the step size of weight updates. If α is too large, the weight adjustment range is too large, which may skip the optimal solution; if α is too small, the convergence speed is slow. Therefore, in this embodiment, 0.01≤α≤0.1 is taken; t Choose the weight adjustment action to execute at time step t (e.g., adjust w). 1,t Or adjust w 2,t ); R(s) t ,a t C t ) for reinforcement learning state space s t Scenario complexity C t Execute the weight adjustment action a t The reward received later; The gradient operator is used to compute the direction of weight updates, adjusting the dual-model voting weight strategy θ at time step t in a direction that maximizes the objective function (i.e., expected reward). t This allows the model to learn a better allocation of voting weights; The expected computation is used to calculate the traversal of all possible states s. t Action a t Average return after sequence;

[0085] Output the dynamically adjusted dual-model voting weight strategy θ t+1 This is used for subsequent fusion voting, and the dual-model voting weight strategy θ t+1 The adjusted voting weights w1' and w2' of the first fireworks detection model are included, and w1' + w2' = 1.

[0086] Therefore, this embodiment can dynamically adjust the initial voting weights of two fireworks detection models under different scene complexities based on reinforcement learning algorithms, so that the weight adjustment can perceive scene differences and achieve a high degree of adaptation and deep correlation between voting weights and scene complexity. For example, in scenes with high scene complexity, the voting weight of the second fireworks detection model can be increased, and in scenes with low scene complexity, the voting weights of the two models can be balanced, such as the voting weights of the two fireworks detection models being 0.5, 0.5, etc., to achieve high-precision fireworks recognition under different scene complexities.

[0087] S53. Perform Bayesian calibration on the detection confidence and judgment results of the preliminary detection results and the secondary detection results according to the following formula to obtain the Bayesian calibration probability:

[0088]

[0089] Wherein, P(fire|d s ,d l ,c s ,c l ) represents the Bayesian calibration probability; d s c s These are the preliminary detection results for determining the pyrotechnic points (d is the case when there are suspected pyrotechnic points). s =1, when there are no suspected ignition points, d s =0), confidence level of the pyrotechnic point determination result, and 0≤c s ≤1; d l c l These are the results of the second detection of pyrotechnic points (d is the result of the second detection of pyrotechnic points). l =1, when there are no suspected ignition points, d l =0), confidence level of the pyrotechnic point determination result, and 0≤c l ≤1; P(d) s ,d l |fire) represents the likelihood probability of the combination of the initial detection result and the secondary detection result when real fire points exist in the scene. It can be obtained based on historical real fire point samples. If P(d s ,d l The higher the value of |fire), the better the current fire point determination result d. s d l The combination of these values ​​has a higher matching degree with the actual fire points, and the auxiliary posterior probability (i.e., P(fire|d)) is higher. s ,d l ,c s ,c l The calculation allows the model to focus more on combinations of detection results with high matching scores; ω s ω l These are the confidence weighting coefficients for the pyrotechnic point determination results of the preliminary detection results and the secondary detection results, respectively, and ω... s =sigmoid(c s ·k))ω l =sigmoid(c l ·k), where k is the confidence scaling factor (k ranges from 0.2 to 5), thus the influence weight of the confidence of the fireworks point determination result on the likelihood probability can be enhanced through two confidence weighting coefficients; P(fire) is the probability that the suspected fireworks point is actually a real fireworks point when it does not depend on the initial detection result and the secondary detection result, reflecting the prior tendency of fireworks points to appear in the scene, which can be obtained based on historical data statistics; P(d s ,d l) represents the evidence factor, indicating that the preliminary detection result of the pyrotechnic point determination is d. s And the result of the second detection of the pyrotechnic point is d. l The probability is independent of whether there are real fireworks points in the actual scene; Both the preliminary detection results and the secondary detection results indicate that there is no probability of any suspected pyrotechnic points. It is a binary variable, i.e., d s The equivalent expression when =0 is used to indicate that the preliminary detection result of the pyrotechnic point determination is that there are no suspected pyrotechnic points. It is a binary variable, i.e., d l The equivalent expression when = 0 is used to indicate that the fireworks point determination result of the secondary detection result is that there are no suspected fireworks points. Therefore, this embodiment can use the confidence inverse weight (1-ω) s (1-ω) l The evidence factors are balanced to normalize the posterior probability, ensuring that P(fire|d) s ,d l ,c s ,c l )∈[0,1];

[0090] Therefore, in this embodiment, by fusing prior knowledge and Bayesian calibration of the two detection results, the posterior probability (i.e., P(fire|d)) of "suspected fire point is a real fire point" can be calculated more accurately. s ,d l ,c s ,c l And by combining the prior probability (P(fire)) from historical statistics and the likelihood probability (P(d)) of the detection result. s ,d l |fire)) Eliminates single-shot detection bias and improves the reliability of the final fire point judgment result, especially in complex scenarios (with many interferences and high risk of false detection). The Bayesian calibration probability obtained after calibration is closer to the real situation.

[0091] S54. Based on the adjusted voting weights w1' of the first firework detection model, w2' of the second firework detection model, and the Bayesian calibration probability P(fire|d s ,d l ,c s ,c l Obtain the final confidence level for the detection of the fireworks point;

[0092] Specifically, the final fire point detection confidence level is obtained according to the following formula:

[0093] C final =w1'·cs +w2'·c l ·P(fire|d s ,d l ,c s ,c l )

[0094] Among them, C final To determine the final confidence level for the detection of the pyrotechnic point, and C final ∈[0,1];

[0095] Therefore, in this step, the confidence level c of the two detections can be determined based on the dynamically adjusted voting weights w1' and w2'. s c l Weighting is applied, and a Bayesian calibration probability P(fire|d) is used, which integrates prior knowledge and the results of the two tests. s ,d l ,c s ,c l The weighted confidence level is adjusted to fully reflect the differences in model reliability under scene complexity (e.g., w2' is higher for higher scene complexity to emphasize the confidence of secondary detection results), ultimately improving the accuracy of fire point detection.

[0096] S6. Detect confidence level c based on the final pyrotechnic point. final To determine whether there are false detections of pyrotechnic points, specifically, when the final pyrotechnic point detection confidence level C... final If the confidence level is ≤ 0.7, then false fire detections are considered to exist, requiring further analysis of the global semantics of the image to confirm the fire detection results. When the final fire detection confidence level C... final If the confidence threshold is reached, it is considered that there is no firework detection error, and the secondary detection result is directly used as the final firework detection result and output.

[0097] For example, for Figure 2-3 The two detection results shown are used to calculate the final fire point detection confidence level C in step S5. final It is 0.5, from Figure 2-3 As can be seen, the suspected fire point area within the box does not have obvious orange-red or yellow flame colors or dynamic flashing effects, nor is there any drifting or diffuse smoke. Therefore, it can be concluded that there is no real fire point in this area, and there is a high possibility of false fire point detection. The detection results need to be reconfirmed.

[0098] In summary, this application completes the detection of smoke points based on a dual-model (i.e., a first smoke detection model and a second smoke detection model) collaborative architecture. Specifically, the first smoke detection model can quickly and initially detect suspected smoke and fire point areas in the image, and the second smoke detection model can further perform secondary discrimination to provide more accurate recognition results.

[0099] Meanwhile, the second fireworks detection model is efficiently fine-tuned using QA-LoRA fine-tuning technology, reducing computational resource consumption while maintaining model performance. Additionally, a time-series dataset containing continuous frame images and scene text is constructed. Through cross-modal contrastive learning, the second detection model learns the differences between the dynamic features of fireworks points and the static features of interference sources, thereby improving the detection accuracy of the second detection model.

[0100] Furthermore, based on reinforcement learning algorithms, the initial voting weights of the two fireworks detection models under different scene complexities are dynamically adjusted so that the weight adjustment can perceive scene differences and achieve a high degree of adaptation and deep correlation between voting weights and scene complexity. For example, in scenes with high scene complexity, the voting weight of the second fireworks detection model is increased, and in scenes with low scene complexity, the voting weights of the two models are balanced, such as the voting weights of the two fireworks detection models being 0.5 and 0.5, etc., to achieve high-precision fireworks recognition under different scene complexities.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying fireworks points, characterized in that, Includes the following steps: The scene image is input into the first fireworks detection model to perform preliminary detection of fireworks points and output the preliminary detection results. The preliminary detection results are cropped to segment the suspected fireworks point area image, and the suspected fireworks point area image is input into the second fireworks point detection model to perform secondary detection of fireworks points, and the secondary detection results are output. Get the complexity of the current scene; The initial voting weights of the first and second fireworks detection models are determined respectively, and the initial voting weights are dynamically adjusted based on the scene perception reinforcement learning algorithm and scene complexity. The Bayesian calibration probability is obtained by performing Bayesian calibration on the detection confidence and judgment results of the preliminary and secondary detection results according to the following formula: ； in, Calibrate the probabilities for Bayesian analysis; , These are the preliminary detection results for determining the pyrotechnic point and the confidence level of the pyrotechnic point determination results. , These are the results of the pyrotechnic point determination in the secondary detection and the confidence level of the pyrotechnic point determination results, respectively. ω represents the likelihood probability of the combination of the initial detection result and the secondary detection result when real pyrotechnic points exist in the scene; s ω l These are the confidence weighting coefficients for the pyrotechnic point determination results of the preliminary detection results and the confidence weighting coefficients for the pyrotechnic point determination results of the secondary detection results, respectively. k is the confidence scaling factor; To determine the probability that a suspected fireworks point is actually a fireworks point when not relying on preliminary or secondary test results; The evidence factor represents the preliminary detection result of the pyrotechnic point determination. And the result of the second detection was the determination of the pyrotechnic point. The probability of; Both the preliminary detection results and the secondary detection results indicate that there is no probability of any suspected pyrotechnic points. The variable is a binary variable used to represent the preliminary detection result of the pyrotechnic spot determination, which concludes that there are no suspected pyrotechnic spots. is a binary variable used to represent the result of the secondary detection of fireworks, indicating that there are no suspected fireworks. The final fire point detection confidence level is obtained based on the adjusted voting weights of the first fire point detection model, the adjusted voting weights of the second fire point detection model, and the Bayesian calibration probability.

2. The method for identifying fireworks points as described in claim 1, characterized in that, The first pyrotechnic point detection model is obtained by training the first detection model.

3. The method for identifying fireworks points as described in claim 1, characterized in that, The second pyrotechnic point detection model is obtained by fine-tuning the second detection model, and the fine-tuning includes the following steps: Construct a time-series image-text dataset containing consecutive frame images and scene text; The second detection model was fine-tuned based on QA-LoRA and cross-modal contrastive learning.

4. The method for identifying fireworks points as described in claim 3, characterized in that, The attention layer parameter W of the second detection model is fine-tuned based on QA-LoRA, and the attention layer parameter W = W A W B W A W B All are low-rank matrices.

5. The method for identifying smoke points as described in claim 3, characterized in that, Loss function when fine-tuning the second detection model based on cross-modal contrastive learning as follows: ； Where I represents the image features of the image modality data in the image-text time series dataset; T represents the text features of the text modality data in the image-text time series dataset, and image feature I and text feature T are correctly paired sample pairs; T' represents the text features of the remaining text modality data in the image-text time series dataset, excluding the correctly paired text features T; T total is the set of all text modal data in the image-text time series dataset; sim(I,T) represents the similarity between image feature I and text feature T; τ is the temperature coefficient.

6. The method for identifying fireworks points as described in claim 1, characterized in that, The scenario complexity is obtained using the following formula: ； Where C represents the complexity of the current scenario; They are the first weight and the second weight, respectively. ; This represents the number of interference sources in the current scene. This refers to the historical maximum number of interference sources for the same scene or the total number of interference sources in a time-series dataset of images and text. This represents the percentage of reflective areas in the current scene image.

7. The method for identifying fireworks points as described in claim 1, characterized in that, The initial voting weights are dynamically adjusted based on scene-aware reinforcement learning algorithms and scene complexity, including the following steps: Let the initial voting weights of the first firework detection model at time step t be... The initial voting weights for the second firework detection model are: ,and ; Constructing the reinforcement learning state space at time step t ,in, The dual-model voting weight strategy at time step t, and has ; Let be the scene complexity at time step t; Historical false positive rate; The initial voting weights are dynamically adjusted based on the following formula: ； in, Dual-model voting weighting strategy at time step t+1; The learning rate; Select the weight adjustment action to execute at time step t; To reinforce learning state space Scene complexity Execute weight adjustment action The reward received later; This is the gradient operator, used to calculate the direction of weight updates; The expected computation is used to calculate the traversal of all possible states. ,action Average return after sequence; Output the dynamically adjusted dual-model voting weight strategy .

8. The method for identifying fireworks points as described in claim 1, characterized in that, The final confidence level of the fire point detection is obtained according to the following formula: ； in, To determine the confidence level for the final pyrotechnic point detection, and w1' and w2' represent the dynamically adjusted dual-model voting weight strategy, respectively. The adjusted voting weights of the first firework detection model and the adjusted voting weights of the second firework detection model are w1' and w2', respectively, and w1'+w2'=1.

9. The method for identifying fireworks points as described in claim 1, characterized in that, The method for identifying ignition points also includes: If the final firework detection confidence level is less than or equal to the confidence threshold, then a false firework detection is considered to exist. If the final firework detection confidence level is greater than or equal to the confidence threshold, then no false firework detection is considered to exist. The secondary detection result is then directly used as the final firework detection result and output.

Images