A flame detection method and a feature extraction model training method and device

Abstract

This invention provides a flame detection method, a feature extraction model training method, and an apparatus, relating to the field of image processing technology. The flame detection method includes: acquiring a video to be processed; for each video frame in the video to be processed, if a potential flame region is detected in the video frame, obtaining a detection image based on the potential flame region in the video frame; inputting the detection image into a pre-trained feature extraction model to obtain a first feature vector representing the image features of the detection image; obtaining an associated image of the detection image based on the video to be processed; inputting the associated image into the feature extraction model to obtain a second feature vector representing the image features of the associated image; and determining that the detection image contains a target flame region when the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold, thereby improving the accuracy of flame detection.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a flame detection method, a feature extraction model training method, and an apparatus. Background Technology

[0002] Video flame detection algorithms offer advantages over smoke detectors (such as a wider detection radius) and over thermal detectors (such as lower cost), making them highly valuable for practical applications. In related technologies, when detecting fires in a target scene, a target video of that scene is acquired. Since flames are typically bright during a fire, each frame of the target video is analyzed to detect the presence of a bright image region. If a bright image region is detected in the frame, it is determined that a flame region exists within that frame, thus confirming that a fire has occurred in the target scene.

[0003] However, video analysis-based flame detection algorithms are susceptible to interference from luminous objects and other bright objects, leading to false alarms. For example, the brightness of light areas in a video frame is also high; flame detection based on the brightness of image regions in a video frame may mistake light areas for flame areas. Therefore, the accuracy of flame detection in related technologies is relatively low. Summary of the Invention

[0004] The purpose of this invention is to provide a flame detection method, a feature extraction model training method, and an apparatus to improve the accuracy of flame detection. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, a flame detection method is provided, the method comprising:

[0006] The process involves: acquiring a video to be processed; for each video frame in the video to be processed, if a potential flame region is detected in that video frame, obtaining a detection image based on the potential flame region in that video frame; inputting the detection image into a pre-trained feature extraction model to obtain a feature vector representing the image features of the detection image, which is used as a first feature vector; obtaining an associated image of the detection image based on an image region in a video frame located after the detection image in the video to be processed, which is at the same position as the detection image; inputting the associated image into the feature extraction model to obtain a feature vector representing the image features of the associated image, which is used as a second feature vector; and determining that the detection image contains a target flame region when the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold.

[0007] In a second aspect of the present invention, a feature extraction model training method is provided for generating the feature extraction model described in the first aspect above, the method comprising:

[0008] Multiple sample videos are acquired. For each target video frame in the sample videos, multiple sample images corresponding to the sample video are obtained based on the image regions in the target video frame whose brightness is greater than a preset brightness threshold. The multiple sample images corresponding to each sample video are divided into multiple sample image groups. For each sample image group, if the sample image group belongs to the same sample video, the sample similarity of the sample image group is determined as a first value; if the sample image group belongs to different sample videos, the sample similarity of the sample image group is determined as a second value. Training samples containing the sample image groups are generated. The training samples include positive samples and negative samples. The positive samples include sample image groups belonging to the same sample video and the sample similarity of the sample image group. The negative samples include sample image groups belonging to different sample videos and the sample similarity of the sample image group. The feature extraction model is trained based on the training samples to obtain a trained feature extraction model.

[0009] Optionally, obtaining multiple sample videos includes: dividing the original video into multiple sub-videos according to a first sampling interval, which are used as sample videos;

[0010] Before obtaining multiple sample images corresponding to each target video frame in the sample video based on the image region in the target video frame whose brightness is greater than a preset brightness threshold, the method further includes: sampling the sample video according to a second sampling interval to obtain the target video frame.

[0011] Optionally, obtaining multiple sample images corresponding to the sample video based on the image region in the target video frame whose brightness is greater than a preset brightness threshold includes: obtaining a first image to be processed based on the connected image region composed of pixels in the target video frame whose brightness is greater than the preset brightness threshold; if the boundaries of the first image to be processed are not equal, expanding the shortest boundary among the boundaries of the first image to be processed to obtain a second image to be processed; wherein the boundaries of the second image to be processed are equal; expanding the boundary of the second image to be processed to obtain a sample image; wherein the boundary of the sample image is a preset multiple of the boundary of the second image to be processed.

[0012] Optionally, obtaining the first image to be processed based on the connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame includes: determining the connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame as the third image to be processed; for each third image to be processed, if the area of ​​the third image to be processed is greater than a preset area threshold, determining the third image to be processed as the first image to be processed.

[0013] Optionally, determining the connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame as the third image to be processed includes: converting the target video frame into a grayscale image; smoothing the grayscale image based on a Gaussian blur function to obtain a fourth image to be processed; and determining the connected image region composed of pixels with brightness greater than a preset brightness threshold in the fourth image to be processed as the third image to be processed.

[0014] Optionally, training the feature extraction model based on the training samples to obtain a trained feature extraction model includes: for each sample image in the sample image group, inputting the sample image into the feature extraction model of the initial structure to obtain a feature vector representing the image features of the sample image, as a sample feature vector; calculating the similarity of the sample feature vectors of the sample images in the sample image group as a predicted similarity; calculating the function value of a preset loss function based on the predicted similarity and the sample similarity of the sample image group; and training the feature extraction model of the initial structure based on the calculated function value to obtain a trained feature extraction model.

[0015] In a third aspect of the present invention, a flame detection device is provided, the device comprising:

[0016] The video acquisition module is used to acquire videos to be processed.

[0017] The image acquisition module is used to obtain the image to be detected based on the potential flame region in each video frame of the video to be processed.

[0018] The first feature vector acquisition module is used to input the image to be detected into a pre-trained feature extraction model to obtain a feature vector representing the image features of the image to be detected, which is used as the first feature vector.

[0019] The associated image acquisition module is used to obtain the associated image of the image to be detected based on the image region in the video frame located after the image to be detected in the video to be processed, which is at the same position as the image to be detected;

[0020] The second feature vector acquisition module is used to input the associated image into the feature extraction model to obtain a feature vector representing the image features of the associated image, which is used as the second feature vector.

[0021] The determination module is used to determine that the image to be detected contains a target flame region when the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold.

[0022] In a fourth aspect of the present invention, a feature extraction model training apparatus is provided for generating the feature extraction model described in the first aspect above, the apparatus comprising:

[0023] The sample video acquisition module is used to acquire multiple sample videos;

[0024] The sample image acquisition module is used to obtain multiple sample images corresponding to each target video frame in the sample video based on the image regions in the target video frame whose brightness is greater than a preset brightness threshold.

[0025] The segmentation module is used to divide the multiple sample images corresponding to each sample video into multiple sample image groups;

[0026] The sample similarity determination module is used to determine the sample similarity of each sample image group as a first value if the sample image group belongs to the same sample video, and as a second value if the sample image group belongs to different sample videos.

[0027] A training sample generation module is used to generate training samples containing the sample image group; wherein, the training samples include positive samples and negative samples; the positive samples include sample image groups belonging to the same sample video, and the sample similarity of the sample image group; the negative samples include sample image groups belonging to different sample videos, and the sample similarity of the sample image group.

[0028] The training module is used to train the feature extraction model based on the training samples to obtain a trained feature extraction model.

[0029] Optionally, the sample video acquisition module is specifically used to: divide the original video into multiple sub-videos according to the first sampling interval, and use them as sample videos;

[0030] The device further includes:

[0031] The target video frame determination module is used to sample the sample video according to a second sampling interval to obtain the target video frame before the sample image acquisition module performs sampling for each target video frame in the sample video, based on the image region in the target video frame whose brightness is greater than a preset brightness threshold.

[0032] Optionally, the sample image acquisition module is specifically used to: obtain a first image to be processed based on a connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame; if the boundaries of the first image to be processed are not equal, expand the shortest boundary among the boundaries of the first image to be processed to obtain a second image to be processed; wherein the boundaries of the second image to be processed are equal; expand the boundary of the second image to be processed to obtain a sample image; wherein the boundary of the sample image is a preset multiple of the boundary of the second image to be processed.

[0033] Optionally, the sample image acquisition module is specifically used to: determine a connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame, as a third image to be processed; for each third image to be processed, if the area of ​​the third image to be processed is greater than a preset area threshold, determine the third image to be processed as a first image to be processed.

[0034] Optionally, the sample image acquisition module is specifically used to: convert the target video frame into a grayscale image; smooth the grayscale image based on a Gaussian blur function to obtain a fourth image to be processed; and determine the connected image region composed of pixels with brightness greater than a preset brightness threshold in the fourth image to be processed as a third image to be processed.

[0035] Optionally, the training module is specifically configured to: for each sample image in the sample image group, input the sample image into the feature extraction model of the initial structure to obtain a feature vector representing the image features of the sample image, as a sample feature vector; calculate the similarity of the sample feature vectors of the sample images in the sample image group as a predicted similarity; calculate the function value of a preset loss function based on the predicted similarity and the sample similarity of the sample image group; and train the feature extraction model of the initial structure based on the calculated function value to obtain a trained feature extraction model.

[0036] In a fifth aspect of the present invention, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor is used to execute the program stored in the memory to implement the flame detection method steps described in any of the first aspects above, or to implement the feature extraction model training method steps described in any of the second aspects above.

[0037] In a sixth aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the flame detection method steps described in any of the first aspects above, or implements the feature extraction model training method steps described in any of the second aspects above.

[0038] This invention also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the flame detection methods described above, or cause the computer to execute any of the feature extraction model training methods described above.

[0039] This invention provides a flame detection method, a feature extraction model training method, and an apparatus. The flame detection method includes: acquiring a video to be processed; for each video frame in the video to be processed, if a potential flame region is detected in the video frame, obtaining a detection image based on the potential flame region in the video frame; inputting the detection image into a pre-trained feature extraction model to obtain a feature vector representing the image features of the detection image, as a first feature vector; obtaining a related image of the detection image based on an image region in a video frame located after the detection image in the video to be processed at the same position as the detection image; inputting the related image into the feature extraction model to obtain a feature vector representing the image features of the related image, as a second feature vector; and determining that the detection image contains a target flame region when the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold.

[0040] Based on the above processing, the similarity between the first and second feature vectors can represent the similarity between the image to be detected and its associated images. Since the shape of a flame changes during combustion, when there is a significant difference between the shape features of a potential flame region in the image to be detected and the shape features of a potential flame region in the associated images, the similarity between the first and second feature vectors is low, indicating that the image to be detected contains a flame region. Conversely, the shape of image regions containing other bright objects (e.g., light areas) does not change. Therefore, when the shape features of a potential flame region in the image to be detected and the potential flame region in the associated images are not significantly different, the similarity between the first and second feature vectors is high, indicating that the image to be detected does not contain a flame region. Thus, if the similarity between the first and second feature vectors is less than a preset similarity threshold, it means that the video to be processed contains the target flame region. This reduces interference from other bright objects, avoids misidentifying light areas as flame regions, reduces the false alarm rate of flame detection, and improves the accuracy of flame detection.

[0041] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0043] Figure 1 A flowchart of a flame detection method provided in an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of a flame detection method provided in an embodiment of the present invention;

[0045] Figure 3 This is a first flowchart of a feature extraction model training method provided in an embodiment of the present invention;

[0046] Figure 4 This is a second flowchart of the feature extraction model training method provided in the embodiments of the present invention;

[0047] Figure 5 This is a third flowchart of the feature extraction model training method provided in the embodiments of the present invention;

[0048] Figure 6 A schematic diagram of a connected image region provided in an embodiment of the present invention;

[0049] Figure 7 This is a fourth flowchart of the feature extraction model training method provided in the embodiments of the present invention;

[0050] Figure 8 A target video frame provided in an embodiment of the present invention;

[0051] Figure 9a A schematic diagram of a sample image provided in an embodiment of the present invention;

[0052] Figure 9b A schematic diagram of a positive sample pair provided in an embodiment of the present invention;

[0053] Figure 9c A schematic diagram of training pairing provided in an embodiment of the present invention;

[0054] Figure 10 A flowchart of a training sample acquisition method provided in an embodiment of the present invention;

[0055] Figure 11 This is a fifth flowchart of the feature extraction model training method provided in the embodiments of the present invention;

[0056] Figure 12 A schematic diagram illustrating a training method for a feature extraction model provided in an embodiment of the present invention;

[0057] Figure 13 A structural diagram of a flame detection device provided in an embodiment of the present invention;

[0058] Figure 14 A structural diagram of a feature extraction model training device provided in an embodiment of the present invention;

[0059] Figure 15 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0061] Video analytics-based flame detection algorithms are susceptible to interference from luminous objects and other bright objects, leading to false alarms. For example, the brightness of light areas in a video frame is also high; flame detection based on the brightness of image regions in a video frame may mistake light areas for flame areas. Therefore, the accuracy of flame detection in related technologies is relatively low.

[0062] To address the aforementioned problems, this invention provides a flame detection method applied to an electronic device, which can be a terminal or a server. The electronic device acquires a video to be processed. For each video frame in the video to be processed, if a potential flame region is detected in that video frame, a detection image is obtained based on the potential flame region. The detection image is input into a pre-trained feature extraction model to obtain a feature vector representing the image features of the detection image, which serves as a first feature vector. Based on an image region in a video frame following the detection image that is at the same position as the detection image, a related image of the detection image is obtained. The related image is input into the feature extraction model to obtain a feature vector representing the image features of the related image, which serves as a second feature vector. When the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold, it is determined that the detection image contains a target flame region. This reduces interference from other bright objects, lowers the false alarm probability, and improves the accuracy of flame detection. Subsequently, when it is determined that the detection image contains a target flame region, an alarm message can be output.

[0063] See Figure 1 , Figure 1 A flowchart of a flame detection method provided in an embodiment of the present invention, the method comprising the following steps:

[0064] S101: Obtain the video to be processed.

[0065] S102: For each video frame in the video to be processed, if a potential flame region is detected in the video frame, the image to be detected is obtained based on the potential flame region in the video frame.

[0066] S103: Input the image to be detected into the pre-trained feature extraction model to obtain the feature vector representing the image features of the image to be detected, which is used as the first feature vector.

[0067] S104: Based on the image region in the video frame following the image to be detected that is at the same position as the image to be detected, obtain the associated image of the image to be detected.

[0068] S105: Input the associated image into the feature extraction model to obtain the feature vector representing the image features of the associated image, which is used as the second feature vector.

[0069] S106: When the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold, it is determined that the image to be detected contains a target flame region.

[0070] Based on the flame detection method provided in this invention, the similarity between the first feature vector and the second feature vector can represent the similarity between the image to be detected and its associated image. Since the shape of a flame changes during combustion, when there is a significant difference between the shape features of a potential flame region in the image to be detected and the shape features of a potential flame region in the associated image, the similarity between the first feature vector and the second feature vector is low, indicating that the image to be detected contains a flame region. Conversely, the shape of image regions containing other bright objects (e.g., light areas) does not change. Therefore, when the shape features of a potential flame region in the image to be detected and the potential flame region in the associated image are not significantly different, the similarity between the first feature vector and the second feature vector is high, indicating that the image to be detected does not contain a flame region. Thus, if the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold, it indicates that the video to be processed contains a target flame region. This reduces interference from other bright objects, avoids misidentifying light areas as flame regions, reduces the false alarm rate of flame detection, and improves the accuracy of flame detection.

[0071] For steps S101 and S102, the video to be processed is the video of the target scene for which fire detection is required. For each video frame in the video to be processed, the electronic device can first use a flame detection method in the prior art to detect whether the video frame contains a flame region. The existing flame detection method can be a flame detection method based on an object detection model, such as Mask-RCNN (Mask Region-based Convolutional Neural Network) or RCNN (Region-based Convolutional Neural Networks). Alternatively, the existing flame detection method can be a shape similarity-based method, such as a method based on SSIM (Structural Similarity Index).

[0072] In some embodiments, the target detection model can be trained based on a sample image containing a flame region and the target location of the flame region in the sample image. The flame region in the sample image can be a square region, and the target location can include the vertex coordinates of the four vertices of the flame region. The electronic device acquires the sample image containing the flame region and the target location of the flame region in the sample image. The electronic device inputs the sample image into the target detection model with an initial structure to obtain the predicted location of the flame region in the sample image output by the target detection model with the initial structure. Based on the target location and the predicted location, the model parameters of the target detection model with the initial structure are adjusted to obtain a trained target detection model.

[0073] If an electronic device detects a flame region in a video frame using existing flame detection methods, it can extract the flame region from the video frame based on its detected location, thus obtaining a potential flame region. However, existing flame detection techniques are not highly accurate; the extracted potential flame region might not actually be a flame region, for example, misdetecting a light area as a flame. Therefore, to improve the accuracy of flame detection, the electronic device can obtain a detection image based on the potential flame region in the video frame. Subsequently, the electronic device can perform flame detection again on this detection image.

[0074] In one implementation, for each video frame in the video to be processed, if a potential flame area is detected in the video frame according to the target detection model, the electronic device can directly identify the video frame as the image to be detected.

[0075] In another implementation, for each video frame in the video to be processed, the electronic device detects the potential flame regions in the video frame based on the object detection model. Then, the electronic device can directly extract the potential flame regions from the video frame according to the positions of the potential flame regions output by the object detection model, thus obtaining the image to be detected. Alternatively, the electronic device can expand the boundaries of the potential flame regions in the video frame according to preset width and height adjustment parameters to obtain adjusted potential flame regions, which can then be used as the image to be detected.

[0076] The preset width and height adjustment parameters are the same as those used by the feature extraction model in subsequent embodiments to adjust the brightness region (i.e., the image region with brightness greater than a preset brightness threshold) in the target video frame of the sample video. The preset width and height adjustment parameters include: a preset multiple corresponding to the width of the brightness region and a preset multiple corresponding to the height of the brightness region. The preset multiple corresponding to the width of the brightness region and the preset multiple corresponding to the height of the brightness region can be the same. For example, the preset multiple can range from (2, 2.5). The preset multiple corresponding to the width of the brightness region and the preset multiple corresponding to the height of the brightness region can also be different.

[0077] For each video frame in the video to be processed, after identifying the potential flame region in that frame, the electronic device can move the boundary along the width direction of the potential flame region by a certain number of pixels, so that the width of the adjusted potential flame region is a preset multiple of the width of the original potential flame region. It can also move the boundary along the height direction of the potential flame region by a certain number of pixels, so that the height of the adjusted potential flame region is a preset multiple of the height of the original potential flame region, thus obtaining the adjusted potential flame region. Then, the adjusted potential flame region is extracted from the video frame to obtain the image to be detected.

[0078] Alternatively, after identifying the potential flame region in the video frame, if the boundaries of the potential flame region are not equal, the electronic device can shift the other boundaries by a certain number of pixels according to the longest boundary of the potential flame region, making the boundaries of the potential flame region equal, thus obtaining a square potential flame region. Then, the electronic device can shift the boundary along the width direction of the square potential flame region by a certain number of pixels, so that the width of the adjusted potential flame region is a preset multiple of the width of the original potential flame region. And it can shift the boundary along the height direction of the square potential flame region by a certain number of pixels, so that the height of the adjusted potential flame region is a preset multiple of the height of the original potential flame region, thus obtaining the adjusted potential flame region. Finally, the adjusted potential flame region is extracted from the video frame to obtain the image to be detected.

[0079] In step S103, the electronic device can input the image to be detected into a pre-trained feature extraction model. The pre-trained feature extraction model processes the image to obtain a first feature vector representing the image features of the image to be detected. The first feature vector can represent the shape features of the potential flame region in the image to be detected. For example, the pre-trained feature extraction model can be a MobileNetv2 (Lightweight Mobile Network 2) model. The method by which the electronic device obtains the pre-trained feature extraction model can be referred to the detailed description in the following embodiments.

[0080] In step S104, the shape of the flame changes as it burns, while the shape of the image area containing other highlighted objects (e.g., the light area) remains unchanged. For example, see... Figure 2 , Figure 2 Images 1 and 2 are flame regions extracted from two adjacent video frames of the same video to be processed. Images 3 and 4 are light regions extracted from two adjacent video frames of the same video to be processed. Figure 2As can be seen, the shape of the flame area in Image 1 is different from that in Image 2. The shape of the light area in Image 3 is the same as that in Image 4.

[0081] If the image to be detected is a video frame in the video to be processed, the electronic device can identify the video frame in the video to be processed that follows the image to be detected as the associated image of the image to be detected. For example, the electronic device can identify the video frame in the video to be processed that follows the image to be detected and is separated from the image by 3 video frames as the associated image of the image to be detected.

[0082] If the image to be detected is extracted from a video frame in the video to be processed, the electronic device determines the same position as the image to be detected in the video frame following the image to be detected in the video frame, extracts an image region of the same size as the image to be detected, and obtains the associated image of the image to be detected.

[0083] For steps S105 and S106, the electronic device can input the associated image into a pre-trained feature extraction model. The pre-trained feature extraction model processes the associated image to obtain a second feature vector representing the image features of the associated image. The second feature vector can represent the shape features of the potential flame region in the associated image. The potential flame region in the associated image is, the image region of the same size extracted from the associated image at the same location as the potential flame region in the image to be detected. The method by which the electronic device obtains the pre-trained feature extraction model can be referred to the detailed description in the following embodiments.

[0084] When the image to be detected contains a flame region, the shape features of the potential flame region in the image to be detected differ from the shape features of the potential flame region in the associated image. Therefore, the first feature vector representing the image features of the image to be detected and the second feature vector representing the image features of the associated image have low similarity. Conversely, when the image to be detected does not contain a flame region but contains image regions containing other highlighted objects, the shape features of the potential flame region in the image to be detected are similar to the shape features of the potential flame region in the associated image. In this case, the first feature vector and the second feature vector have high similarity.

[0085] Therefore, the electronic device can calculate the similarity between the first feature vector and the second feature vector according to a preset similarity calculation method. When the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold, it indicates that the potential flame region in the image to be detected is the target flame region; when the similarity between the first feature vector and the second feature vector is not less than the preset similarity threshold, it indicates that the potential flame region in the image to be detected is a falsely detected flame region, such as a light area instead of a potential flame region. The preset similarity calculation method can be: a similarity calculation method based on Euclidean distance, a similarity calculation method based on Manhattan distance, a similarity calculation method based on Cosine distance, etc.

[0086] In some embodiments, see Figure 2 , Figure 2 This is a schematic diagram of a flame detection method provided in an embodiment of the present invention. Figure 2 Image 1 is the image to be detected, and image 2 is the associated image of image 1; image 3 is the image to be detected, and image 4 is the associated image of image 3. ShapeNet is the pre-trained feature extraction model in the aforementioned embodiments.

[0087] The electronic device inputs image 1 into ShapeNet to obtain the shape features of the potential flame region in image 1 output by ShapeNet. The shape features of the potential flame region in image 1 are also the first feature vector representing the image features of the image to be detected in the aforementioned embodiment. The electronic device also inputs image 2 into ShapeNet to obtain the shape features of the potential flame region in image 2 output by ShapeNet. The shape features of the potential flame region in image 2 are also the second feature vector representing the image features of the associated image in the aforementioned embodiment.

[0088] Furthermore, the electronic device can calculate the feature cosine similarity between the shape features of the potential flame region in image 1 and the shape features of the potential flame region in image 2. The feature cosine similarity is the cosine distance between the first feature vector representing the shape features of the potential flame region in image 1 and the second feature vector representing the shape features of the potential flame region in image 2. If the feature cosine similarity value between the shape features of the potential flame region in image 1 and the potential flame region in image 2 is low, meaning the similarity between the first and second feature vectors is less than a preset similarity threshold, the electronic device can determine that image 1 contains the target flame region, and that the flame shape in image 1 is dissimilar to the flame shape in image 2.

[0089] The electronic device inputs image 3 into ShapeNet to obtain the shape features of the potential flame region in image 3 output by ShapeNet. The shape features of the potential flame region in image 3 are also the first feature vector representing the image features of the image to be detected in the aforementioned embodiment. Furthermore, the electronic device also inputs image 4 into ShapeNet to obtain the shape features of the potential flame region in image 4 output by ShapeNet. The shape features of the potential flame region in image 4 are also the second feature vector representing the image features of the associated image in the aforementioned embodiment.

[0090] Furthermore, the electronic device can calculate the feature cosine similarity between the shape features of the potential flame region in image 3 and the shape features of the potential flame region in image 4. The feature cosine similarity is the cosine distance between the first feature vector representing the shape features of the potential flame region in image 3 and the second feature vector representing the shape features of the potential flame region in image 4. If the feature cosine similarity value between the shape features of the potential flame region in image 3 and the potential flame region in image 4 is high, meaning the similarity between the first and second feature vectors is not less than a preset similarity threshold, then the electronic device can determine that image 3 does not contain the target flame region, and that the shape of the light in image 3 is similar to the shape of the light in image 4.

[0091] Based on the above processing, since the shape of a flame changes during combustion, the shape features of the potential flame region in the image to be detected differ significantly from those in the associated image. However, the shape of image regions containing other bright objects (e.g., light areas) does not change, resulting in smaller differences in shape features between these regions and the potential flame regions in the associated image. Performing flame detection based on a first feature vector representing the shape features of the potential flame region in the image to be detected and a second feature vector representing the shape features of the potential flame region in the associated image can reduce interference from other bright objects, avoid misidentifying light areas as flame regions, reduce the false alarm rate of flame detection, and improve the accuracy of flame detection.

[0092] See Figure 3 , Figure 3 This is a first flowchart of a feature extraction model training method provided in an embodiment of the present invention. The method is used to generate any of the feature extraction models described in the foregoing embodiments, and may include the following steps:

[0093] S301: Acquire multiple sample videos.

[0094] S302: For each target video frame in the sample video, based on the image region in the target video frame whose brightness is greater than a preset brightness threshold, obtain multiple sample images corresponding to the sample video.

[0095] S303: Divide the multiple sample images corresponding to each sample video into multiple sample image groups.

[0096] S304: For each sample image group, if the sample image group belongs to the same sample video, the sample similarity of the sample image group is determined to be a first value; if the sample image group belongs to different sample videos, the sample similarity of the sample image group is determined to be a second value.

[0097] S305: Generate training samples containing a set of sample images.

[0098] The training samples include positive samples and negative samples; positive samples include sample image groups belonging to the same sample video, and the sample similarity of the sample image group; negative samples include sample image groups belonging to different sample videos, and the sample similarity of the sample image group.

[0099] S306: Train the extraction model based on the features of the training samples to obtain a well-trained feature extraction model.

[0100] Based on the feature extraction model training method provided in this embodiment of the invention, the feature extraction model is trained using training samples containing sample image groups. Subsequently, flame detection is performed on the video to be processed based on the trained feature extraction model, which can reduce interference from other bright objects, avoid identifying light areas as flame areas, reduce the false alarm rate of flame detection, and improve the accuracy of flame detection.

[0101] For steps S301 and S302, the sample videos include: sample videos containing flame areas, sample videos containing light areas, and sample videos containing other target areas that are easily misidentified as flame areas.

[0102] In one implementation, a technician can handheld shooting equipment to capture multiple videos in different scenarios, obtaining multiple sample videos. The duration of a single sample video can be 5 minutes, or the total duration of multiple sample videos can be 5 hours, but is not limited to these.

[0103] Furthermore, for each sample video, the electronic device can determine the target video frame within that sample video. For example, the electronic device can determine all video frames in the sample video as the target video frame, or it can determine a specific video frame in the sample video as the target video frame, such as the first video frame, the last video frame, etc., but is not limited to these.

[0104] In another implementation, technicians can handhold the camera and continuously move while filming a street scene with lights on, obtaining a video (i.e., the original video). Electronic devices then extract multiple sample videos from the original video. Figure 3 Based on this, see Figure 4 Step S301 may include the following steps:

[0105] S3011: Divide the original video into multiple sub-videos according to the first sampling interval, and use them as sample videos.

[0106] Accordingly, prior to step S302, the method may further include the following steps:

[0107] S307: Sample the sample video according to the second sampling interval to obtain the target video frame.

[0108] After acquiring the original video captured by the technician, the electronic device can divide the original video into sub-videos, starting from the first video frame, according to a first sampling interval. Each sub-video is divided into segments of one first sampling interval, serving as a sample video. Accordingly, multiple sample videos are obtained based on the original video. The first sampling interval can be preset by the technician, such as 5 seconds or 100 frames.

[0109] For example, if the first sampling interval is 100 frames, the electronic device can divide the first to the 100th video frames in the original video into a sub-video; divide the 101st to the 200th video frames in the original video into a sub-video, and so on, dividing the original video into multiple sub-videos, thus obtaining multiple sample videos.

[0110] Since multiple sample videos are derived from the same original video, they are essentially videos taken by the same device from the same angle within a similar time period. This avoids discrepancies between sample videos caused by differences in device performance and lighting conditions. Consequently, the accuracy of the feature extraction model trained using multiple sample videos is higher.

[0111] Furthermore, for each sample video, the electronic device can sample the sample video according to the second sampling interval to obtain the target video frame in the sample video. The second sampling interval can be preset by the technician, such as 3 frames or 5 frames.

[0112] For example, for each video frame in the sample video, the electronic device determines which video frame it is in the sample video and obtains its frame number. Then, it calculates the ratio of the frame number to the second sampling interval. If the ratio of the frame number to the second sampling interval is an integer, then the video frame is the target video frame.

[0113] Based on the above processing, dividing the original video into multiple sample videos can reduce the differences between the sample videos caused by factors such as device performance and lighting conditions, thereby improving the accuracy of the obtained sample videos. Subsequently, the feature extraction model trained based on the more accurate sample videos will also have higher accuracy. Sampling the target video frame from each sample video can fully consider the differences between the sample videos themselves and reduce the number of target video frames, thus improving training efficiency.

[0114] Furthermore, for each target video frame in the sample video, the electronic device can obtain multiple sample images corresponding to the sample video based on the image regions in the target video frame whose brightness is greater than a preset brightness threshold.

[0115] In one implementation, the electronic device can first divide the target video frame into multiple image regions according to a preset division method. Then, for each image region in the target video frame, the electronic device can calculate the sum of the brightness of each pixel in the image region to obtain the brightness of the image region. Subsequently, image regions with brightness greater than a preset brightness threshold are extracted from the target video frame to obtain a sample image. The preset division method can be to divide the target video frame into a preset number of image regions, such as 20, 30, etc.

[0116] In another implementation, Figure 3 Based on this, see Figure 5 Step S302 may include the following steps:

[0117] S3021: Based on the connected image region composed of pixels in the target video frame whose brightness is greater than a preset brightness threshold, a first image to be processed is obtained.

[0118] S3022: If the boundaries of the first image to be processed are not equal, the shortest boundary among the boundaries of the first image to be processed is expanded to obtain the second image to be processed.

[0119] In this case, all boundaries of the second image to be processed are equal.

[0120] S3023: Expand the boundary of the second image to be processed to obtain a sample image.

[0121] The boundary of the sample image is a preset multiple of the boundary of the second image to be processed.

[0122] For each pixel in the target video frame, the electronic device can determine whether the brightness of that pixel is greater than a preset brightness threshold. Furthermore, the electronic device can identify all pixels in the target video frame whose brightness is greater than the preset brightness threshold (which can be called bright pixels), and correspondingly, all pixels in the target video frame whose brightness is not greater than the preset brightness threshold can be called non-bright pixels. Thus, the electronic device can also identify multiple connected image regions composed of bright pixels.

[0123] For example, see Figure 6 , Figure 6 This is a schematic diagram of a connected image region provided in an embodiment of the present invention. Figure 6 In this image, a small square represents a pixel. A square displaying the number "1" indicates a highlighted pixel, and a square displaying the number "0" indicates a non-highlighted pixel. The target video frame contains multiple highlighted pixels, which form three connected image regions. Figure 6 The gray-filled connected image regions 1, 2, and 3 are shown.

[0124] In one implementation, for each connected image region, the electronic device can use that connected image region as the first image to be processed.

[0125] In another implementation, Figure 5 Based on this, see Figure 7 Step S3021 may include the following steps:

[0126] S30211: Determine the connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame, and use it as the third image to be processed.

[0127] S30212: For each third image to be processed, if the area of ​​the third image to be processed is greater than a preset area threshold, the third image to be processed is determined to be the first image to be processed.

[0128] When the connected image region is small, it may be a region with high brightness due to noise interference, meaning it contains less information. Therefore, training a feature extraction model based on this region will be ineffective, resulting in low accuracy. Conversely, when the connected image region is large, it is no longer a region with high brightness due to noise interference; it contains more information, such as background and foreground information. In this case, training a feature extraction model based on this region will be effective, leading to higher accuracy. Therefore, to improve the accuracy of the trained feature extraction model, the electronic device can first designate the identified connected image region as the third image to be processed.

[0129] In some embodiments, Figure 7 Based on this, step S30211 includes the following steps: converting the target video frame into a grayscale image; smoothing the grayscale image using a Gaussian blur function to obtain a fourth image to be processed; and determining the connected image region composed of pixels with brightness greater than a preset brightness threshold in the fourth image to be processed as the third image to be processed.

[0130] The electronic device can first convert the target video frame into a grayscale image, and then use a Gaussian blur function to smooth the grayscale image, thus performing noise reduction preprocessing to obtain a fourth image to be processed after noise reduction preprocessing. Furthermore, the electronic device can determine the connected image region composed of pixels with brightness greater than a preset brightness threshold in the fourth image to be processed, thus obtaining a third image to be processed. The third image to be processed is the connected image region after noise reduction preprocessing.

[0131] Furthermore, the electronic device can calculate the area of ​​each third image to be processed. For each third image to be processed, if the area of ​​the third image to be processed is greater than a preset area threshold, then the third image to be processed contains more information. Therefore, the feature extraction model is trained based on the third image to be processed, and the accuracy of the trained feature extraction model is high. Thus, the electronic device can identify the third image to be processed as the first image to be processed. If the area of ​​the third image to be processed is not greater than the preset area threshold, then the third image to be processed contains less information. Therefore, the accuracy of the trained feature extraction model is not high. Thus, the electronic device can choose not to identify the third image to be processed as the first image to be processed, that is, filter out the third images to be processed that contain less information.

[0132] Based on the above processing, the electronic device can filter out the third image to be processed, which contains less information. As a result, the first image to be processed contains more information. Subsequently, the feature extraction model trained based on the first image to be processed, which contains more information, will be more accurate, thus improving the accuracy of the trained feature extraction model.

[0133] Furthermore, to improve the accuracy of the trained feature extraction model, the size of the first image to be processed can be expanded to identify image regions containing more information. To reduce interference from other information during training, the size of the image regions containing more information can be limited to obtain image regions with more accurate information, i.e., to determine the image region corresponding to the sample image in the target video frame. Subsequently, sample images can be extracted from the target video frame according to the determined image regions corresponding to the sample images in the target video frame.

[0134] The electronic device can expand the boundaries of the first image to be processed in the target video frame according to preset width and height adjustment parameters to obtain an adjusted first image to be processed, which serves as a sample image. The preset width and height adjustment parameters include: a preset multiple corresponding to the width of the first image to be processed, and a preset multiple corresponding to the height of the first image to be processed. The preset multiple corresponding to the width of the first image to be processed can be the same as the preset multiple corresponding to the height of the first image to be processed. For example, the value range of the preset multiple is (2, 2.5). The preset multiple corresponding to the width of the first image to be processed can also be different from the preset multiple corresponding to the height of the first image to be processed.

[0135] For each target video frame, after determining the first image to be processed in that target video frame, the electronic device can move a certain number of pixels along the boundary of the width direction of the first image to be processed, so that the width of the adjusted first image to be processed is a preset multiple of the width of the original first image to be processed. It can also move a certain number of pixels along the boundary of the height direction of the first image to be processed, so that the height of the adjusted first image to be processed is a preset multiple of the height of the original first image to be processed, thus obtaining the adjusted first image to be processed. Then, the adjusted first image to be processed is extracted from the video frame to obtain a sample image.

[0136] Alternatively, after determining the first image to be processed in the target video frame, if the boundaries of the first image to be processed are not equal, the electronic device can move the other boundaries by a certain number of pixels according to the longest boundary of the first image to be processed, so that the boundaries of the first image to be processed are equal, that is, obtain a square first image to be processed. Then, the electronic device can move the boundary along the width direction of the square first image to be processed by a certain number of pixels, so that the width of the adjusted first image to be processed is a preset multiple of the width of the original first image to be processed. And it can move the boundary along the height direction of the square first image to be processed by a certain number of pixels, so that the height of the adjusted first image to be processed is a preset multiple of the height of the original first image to be processed, thus obtaining the adjusted first image to be processed. Then, the adjusted first image to be processed is extracted from the video frame to obtain a sample image.

[0137] For example, see Figure 8 , Figure 8 This is a target video frame provided in an embodiment of the present invention. Figure 8 The image region defined by the box is the highlight region in the target video frame. The highlight region is the image region in the target video frame that corresponds to the sample image, which is determined by the connected image region composed of pixels whose brightness is greater than a preset brightness threshold.

[0138] Regarding steps S303 and S304, for each sample video, the electronic device can determine multiple sample images corresponding to that sample video, meaning the electronic device can acquire multiple sample images. Furthermore, the electronic device can select any two sample images from the multiple sample images as a sample image group, thus dividing the multiple sample images into multiple sample image groups.

[0139] For each set of sample images, the electronic device can determine whether two sample images in the set belong to the same sample video. If the two sample images belong to the same sample video, the electronic device can determine the sample similarity of the set as a first value; if the two sample images belong to different sample videos, the electronic device can determine the sample similarity of the set as a second value. For example, the first value can be 1, and the second value can be 0.

[0140] Regarding step S305, further, for each sample image group, if the sample image group belongs to the same sample video, that is, the two sample images in the sample image group belong to the same sample video, then the sample image group and the sample similarity of the sample image group constitute a positive sample; if the sample image group belongs to different sample videos, that is, the two sample images in the sample image group belong to different sample videos, then the sample image group and the sample similarity of the sample image group constitute a negative sample.

[0141] For example, see Figure 9a , Figure 9a This is a schematic diagram of a sample image provided in an embodiment of the present invention. Figure 9a The dataset contains three rows of images, each row consisting of three sample images identified from a single sample video. In other words, each sample video corresponds to three sample images. The electronic device acquired a total of nine sample images.

[0142] Figure 9b This is a schematic diagram of a positive sample pair provided in an embodiment of the present invention. The positive sample pair is the positive sample in the aforementioned embodiment. Figure 9b In the middle, positive sample pairs 1 include from Figure 9a The two sample images identified in the first row of images, positive sample pair 2 includes those from... Figure 9a The two sample images identified in the second row of images, positive sample pair 3 includes those from... Figure 9a The two sample images identified in the third row of the image.

[0143] Figure 9c This is a schematic diagram of a training pairing provided in an embodiment of the present invention. Figure 9c In the sample image, sample image A and sample image 1 are... Figure 9b The positive sample pair 1 includes two sample images. Sample image B and sample image 2 are... Figure 9b The positive sample pair 2 includes two sample images. Sample image C and sample image 3 are... Figure 9b The positive sample pair 3 includes two sample images. These six sample images can be combined into a nine-sample-image group, which can be used... Figure 9cThe image is represented by nine square regions. Each square region records the similarity between the sample images in its row and column. When the sample images in the row and column of the square region form a positive sample, the similarity recorded by the square region is 1; when the sample images in the row and column of the square region form a negative sample, the similarity recorded by the square region is 0. The square region also records the predicted similarity between the sample images in its row and column, obtained based on a feature extraction model. When the sample images of the row containing the square region and the sample images of the column containing the square region form a positive sample, the predicted similarity between the sample images of the row containing the square region and the sample images of the column containing the square region obtained based on the feature extraction model is relatively high; when the sample images of the row containing the square region and the sample images of the column containing the square region form a negative sample, the predicted similarity between the sample images of the row containing the square region and the sample images of the column containing the square region obtained based on the feature extraction model is relatively low.

[0144] For example, if sample image A and sample image 1 form a positive sample, then the sample similarity between sample image A and sample image 1 is 1; the predicted similarity between sample image A and sample image 1 obtained based on the feature extraction model is 0.83, which is relatively high. If sample image C and sample image 1 form a negative sample, then the sample similarity between sample image C and sample image 1 is 0; the predicted similarity between sample image C and sample image 1 obtained based on the feature extraction model is 0.31, which is relatively low.

[0145] Consequently, the electronic device obtains training samples including both positive and negative samples.

[0146] See Figure 10 , Figure 10 This is a flowchart of a training sample acquisition method provided in an embodiment of the present invention.

[0147] S1001: Acquire video footage.

[0148] In this step, the video footage is the original video from the aforementioned embodiments. A technician can handhold the camera and continuously move while filming a street scene with lights on, obtaining video footage, which is the original video from the aforementioned embodiments. Alternatively, the technician can handhold the camera and film multiple videos in different scenes, obtaining multiple video clips, which are the sample videos from the aforementioned embodiments. For example, if the technician can film 10 videos, the total duration of these 10 videos can exceed 5 hours.

[0149] S1002: Video settings include sampling interval Q, sampling frame interval I, brightness threshold T, brightness area threshold S, sampling queue B, and queue length limit L.

[0150] In this step, the electronic device can acquire the relevant parameters pre-set by the technician when processing the original video to obtain training samples. The sampling interval Q is the first sampling interval in the aforementioned embodiment; the sampling frame interval I is the second sampling interval in the aforementioned embodiment; the brightness threshold T is the preset brightness threshold in the aforementioned embodiment; the brightness region area threshold S is the preset area threshold in the aforementioned embodiment; the sampling queue B includes the target video frames in the aforementioned embodiment, and the queue length L is the number of target video frames to be acquired.

[0151] S1003: Traverse video frames f from the beginning to the end, with frame number idx. If frame number idx is not divisible by the sampling frame interval I, skip this frame; otherwise, video frame f enters queue B and proceeds to S1004.

[0152] In this step, for each video frame f in the video, the frame number idx indicates which video frame it is in the video. If the frame number idx of video frame f is not divisible by the sampling frame interval I, it means that video frame f has not yet reached the sampling interval, that is, video frame f is not the target video frame that can be sampled according to the second sampling interval. In this case, video frame f can be skipped, and the (idx+1)th video frame in the video can be selected as the current video frame f, and the determination of whether the frame number idx+1 is divisible by the sampling frame interval I can be continued.

[0153] If the frame number idx of video frame f is divisible by the sampling frame interval I, it means that the video frame f has reached the sampling interval, that is, the video frame f is the target video frame that can be sampled according to the second sampling interval. The electronic device can determine that the video frame f is the sampled target video frame, add the video frame f to the queue B, and proceed to step S1004.

[0154] S1004: If the queue length of queue B exceeds the queue length limit L, delete the first frame in queue B and proceed to S1005.

[0155] In this step, after adding video frame f to queue B, the electronic device can determine whether the number of currently determined target video frames is greater than a preset number. If the number of currently determined target video frames is greater than the preset number, the electronic device can delete the first target video frame in queue B so that the number of target video frames in queue B is not greater than the preset number, and continue to execute step S1005.

[0156] S1005: If the frame number idx is not divisible by the sampling interval Q, return to S1003 and continue execution.

[0157] In this step, the electronic device determines whether the frame number idx of video frame f is divisible by the sampling interval Q, that is, whether video frame f and the previous video frame of video frame f in queue B belong to the same sub-video. When the frame number idx is not divisible by the sampling interval Q, it means that video frame f and the previous video frame of video frame f in queue B belong to the same sub-video. Then, when the electronic device extracts sample images based on video frame f, it will group the sample images extracted from video frame f and the sample images extracted from the previous video frame of video frame f in queue B into the same similar highlight region image group. The device then selects the (idx+1)th video frame in the video as the current video frame f and continues to determine whether idx+1 is divisible by the sampling frame interval I. A similar highlight region image group is an image set including multiple sample images corresponding to a sample video.

[0158] When the frame number idx is divisible by the sampling interval Q, it means that the video frame f belongs to a different sub-video from the previous video frame of the same video frame f in queue B. Then, when the subsequent electronic device extracts sample images based on the video frame f, it will divide the sample images extracted from the video frame f and the sample images extracted from the previous video frame of the same video frame f in queue B into different similar highlight area image groups.

[0159] S1006: Detect the highlighted areas in video frame f, perform cropping and sampling of the same area from each frame in queue B, obtain similar highlighted area image groups, save them, and obtain the training dataset.

[0160] In this step, the highlighted areas in video frame f are the image areas in the target video frame of the aforementioned embodiment whose brightness exceeds a preset brightness threshold. For each video frame f in queue B, which is also the target video frame in the aforementioned embodiment, the electronic device can detect the highlighted areas of video frame f.

[0161] Specifically, the electronic device can convert the video frame f into a grayscale image g; then, it performs Gaussian blur smoothing on the grayscale image g, and uses a brightness threshold T to binarize the smoothed grayscale image g to obtain a bright foreground region image h. The bright foreground region image h includes pixels whose brightness is greater than the preset brightness threshold in the aforementioned embodiment. Then, it detects connected components in the bright foreground region image h and filters out connected components whose area is smaller than the brightness region area threshold S to obtain a set of sampling regions C. The set of sampling regions C includes multiple sampling regions c, which are the first image to be processed in the aforementioned embodiment. Furthermore, the electronic device can traverse each sampling region c in the set of sampling regions C, and when the boundaries of each sampling region c are not equal, it expands the shortest boundary among the boundaries of each sampling region c to obtain a second image to be processed; and expands the boundary of the second image to be processed by a preset multiple to obtain a sample image.

[0162] Then, image regions at the locations of sample images are extracted sequentially from each target video frame in queue B, resulting in multiple sample images. Sample images belonging to the same sub-video within these multiple sample images form a group of similar highlighted region images. This group of similar highlighted region images corresponding to the sample video is the sample image corresponding to that sample video. Furthermore, based on the sample images corresponding to each sample video, training samples, including positive and negative samples, are obtained.

[0163] The final training dataset includes: TrainDate (training data); video1 (video 1); snap-data1 (snapshot data 1), including 1.jpg, 2.jpg, and 3.jpg; snap-data2 (snapshot data 2), including 1.jpg, 2.jpg, and 3.jpg; video2 (video 2), etc. TrainDate is the training sample in the aforementioned embodiment, and video1, video2, etc. are the sample videos in the aforementioned embodiment. video1 includes 1.jpg, 2.jpg, 3.jpg, etc., which are the sample images corresponding to the first sample video, and 1.jpg, 2.jpg, and 3.jpg belong to queue B in the aforementioned embodiment; video2 includes 1.jpg, 2.jpg, 3.jpg, etc., which are the sample images corresponding to the second sample video, and 1.jpg, 2.jpg, and 3.jpg belong to queue B in the aforementioned embodiment.

[0164] Based on the above processing, the electronic device can extract training samples, including positive and negative samples, from the acquired video. Subsequently, the feature extraction model of the initial structure is trained based on the training samples to obtain a trained feature extraction model. Then, flame detection is performed on the video to be processed based on the trained feature extraction model, which can reduce the interference of other bright objects, avoid identifying light areas as flame areas, reduce the false alarm rate of flame detection, and improve the accuracy of flame detection.

[0165] In some embodiments, when training the feature extraction model of the initial structure, the electronic device randomly collects a specified batch-size number of directories from the sample dataset collected from the video. For example, if the batch-size is set to 128, the electronic device will collect 128 subdirectories, which contain image datasets with specific similarity.

[0166] The video refers to the original video in the aforementioned embodiment. Specifying the batch-size (number of directories) determines how many sub-videos the original video will be divided into, which is the number of sampling queues B. When the batch-size is set to 128, the original video is divided into 128 sub-videos, resulting in 128 sample videos. Each subdirectory stores a dataset of images with specific similarities, which determines the sample images corresponding to each sample video. One sampling queue B corresponds to one subdirectory.

[0167] Then, the electronic device randomly selects two images from the directory. If the two images come from the same subdirectory, they are considered a positive sample pair; if they come from different subdirectories, they are considered a negative sample pair. A positive sample pair is the group of sample images that constitutes a positive sample in the aforementioned embodiment; a negative sample pair is the group of sample images that constitutes a negative sample in the aforementioned embodiment. In other words, the electronic device obtains multiple groups of sample images based on the multiple sample images corresponding to each sample video. For each group of sample images, if the group belongs to the same sample video, the group and its sample similarity constitute a positive sample; if the group belongs to different sample videos, its sample similarity constitutes a negative sample. Subsequently, the electronic device can use training samples, including both positive and negative samples, to train the feature extraction model of the initial structure.

[0168] Regarding step S306, in some embodiments, in Figure 3 Based on this, see Figure 11 Step S306 may include the following steps:

[0169] S3061: For each sample image in the sample image group, input the sample image into the feature extraction model of the initial structure to obtain the feature vector representing the image features of the sample image, which is used as the sample feature vector.

[0170] S3062: Calculate the similarity of the sample feature vectors of the sample images in the sample image group as the predicted similarity.

[0171] S3063: Calculate the function value of the preset loss function based on the predicted similarity and the sample similarity of the sample image group.

[0172] S3064: Train the feature extraction model of the initial structure based on the calculated function values ​​to obtain a trained feature extraction model.

[0173] The training samples consist of multiple groups of sample images, along with the sample similarity scores for each group. Each group of sample images contains two sample images. For each sample image in a group, the electronic device inputs it into the feature extraction model of the initial structure to obtain a sample feature vector representing the image features of that sample image. This sample feature vector can represent the shape features of the sample image. In other words, the electronic device can obtain the sample feature vectors of each of the two sample images in the group. The feature extraction model of the initial structure can be a MobileNetv2 (Lightweight Mobile Network 2) model.

[0174] Then, the electronic device calculates the similarity between the two sample feature vectors, thus obtaining the predicted similarity of the shape features of the two sample images. The method by which the electronic device calculates the similarity between the two sample feature vectors is similar to the method used to calculate the similarity between the first feature vector and the second feature vector in the aforementioned embodiments, and can be referred to the relevant descriptions in the aforementioned embodiments.

[0175] Furthermore, the electronic device can calculate the difference between the predicted similarity and the sample similarity of the sample image group, thus obtaining the difference between the predicted similarity of the shape features of the two sample images and the actual similarity of the shape features of the two sample images, and obtaining the function value of the preset loss function.

[0176] For example, the preset loss function can be the info-ice loss function. The electronic device can calculate the loss function value based on the following formula (1):

[0177]

[0178] in, Let q represent the loss function value, q be the feature vector representing the image features of the first sample image, and k be the value of the loss function. +Let k be the feature vector representing the image features of the second sample image that forms a positive sample with the first sample image. - Let τ be the feature vector representing the image features of the third sample image that forms a negative sample with the first sample image, and let τ represent the temperature hyperparameter.

[0179] Subsequently, the electronic device can adjust the model parameters of the initial structure's feature extraction model based on the calculated loss function value, thereby training the initial structure's feature extraction model to obtain a trained feature extraction model. Then, when it is necessary to deploy the trained feature extraction model, the electronic device can remove the network layers used to calculate the loss and adjust the model parameters of the feature extraction model.

[0180] Based on the above processing, the electronic device can train the feature extraction model of the initial structure to obtain a trained feature extraction model. Subsequently, based on the trained feature extraction model, flame detection can be performed on the video to be processed, which can reduce the interference of other bright objects, avoid identifying light areas as flame areas, reduce the false alarm rate of flame detection, and improve the accuracy of flame detection.

[0181] See Figure 12 , Figure 12 This is a schematic diagram of a training method for a feature extraction model provided in an embodiment of the present invention.

[0182] Figure 12 In the image, the leftmost image represents the input data for the feature extraction model of the initial structure. This model comprises multiple convolutional layers (conv). The first convolutional layer extracts features from the input data, resulting in a first output data with a dimension of 128*128. The second convolutional layer then extracts features from the first output data, resulting in a second output data with a dimension of 126*126. The third convolutional layer further extracts features from the second output data, resulting in a third output data with a dimension of 96 (60*60), and so on, until the final convolutional layer outputs 32 output data with a dimension of 1*1.

[0183] Then, the 32 1*1 output data are flattened to obtain shape features. Then, the shape features are processed by the info-ice loss function through fc+relu (Full Connection+rectified linear unit) to obtain the loss function value output by the traininglogits layer.

[0184] Then, based on the calculated loss function value, the model parameters of the initial structure's feature extraction model are adjusted to train the initial structure's feature extraction model, resulting in a trained feature extraction model. Then, when the trained feature extraction model needs to be deployed, the electronic device can remove the training logits layer used for calculating the loss and adjusting the model parameters of the feature extraction model.

[0185] By training a feature extraction model on an initial structure using training samples, the feature extraction model can learn the shape features of image regions. A well-trained feature extraction model can then extract the shape features of the input image. Therefore, the feature extraction model can also be called a shape feature model, and the method of training the feature extraction model on the initial structure can be called a contrastive learning method for shape feature models.

[0186] Based on the above processing, the electronic device can train the feature extraction model of the initial structure to obtain a trained feature extraction model. Subsequently, based on the trained feature extraction model, flame detection can be performed on the video to be processed, which can reduce the interference of other bright objects, avoid identifying light areas as flame areas, reduce the false alarm rate of flame detection, and improve the accuracy of flame detection.

[0187] Based on the same inventive concept as the flame detection method described above, this invention also provides a flame detection device, see [link to relevant documentation]. Figure 13 , Figure 13 This is a structural diagram of a flame detection device provided in an embodiment of the present invention. The device includes:

[0188] The video acquisition module 1301 is used to acquire the video to be processed.

[0189] The image acquisition module 1302 is used to obtain an image to be detected based on the potential flame region in each video frame of the video to be processed.

[0190] The first feature vector acquisition module 1303 is used to input the image to be detected into a pre-trained feature extraction model to obtain a feature vector representing the image features of the image to be detected, which is used as the first feature vector.

[0191] The associated image acquisition module 1304 is used to obtain the associated image of the image to be detected based on the image region in the video frame located after the image to be detected in the video to be processed, which is at the same position as the image to be detected;

[0192] The second feature vector acquisition module 1305 is used to input the associated image into the feature extraction model to obtain a feature vector representing the image features of the associated image, which is used as the second feature vector.

[0193] The determining module 1306 is used to determine that the image to be detected contains a target flame region when the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold.

[0194] Based on the flame detection device provided in this embodiment of the invention, the similarity between the first feature vector and the second feature vector can represent the similarity between the image to be detected and its associated image. Since the shape of a flame changes during combustion, when there is a significant difference between the shape features of a potential flame region in the image to be detected and the shape features of a potential flame region in the associated image, the similarity between the first feature vector and the second feature vector is low, indicating that the image to be detected contains a flame region. Conversely, the shape of image regions containing other bright objects (e.g., light areas) does not change. Therefore, when the shape features of a potential flame region in the image to be detected and the potential flame region in the associated image are not significantly different, the similarity between the first feature vector and the second feature vector is high, indicating that the image to be detected does not contain a flame region. Thus, if the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold, it indicates that the video to be processed contains a target flame region. This reduces interference from other bright objects, avoids misidentifying light areas as flame regions, reduces the false alarm rate of flame detection, and improves the accuracy of flame detection.

[0195] Based on the same inventive concept as the aforementioned feature extraction model training method, this embodiment of the invention also provides a feature extraction model training device, see [link to relevant documentation]. Figure 14 , Figure 14 This is a structural diagram of a feature extraction model training device provided in an embodiment of the present invention. The device includes:

[0196] The sample video acquisition module 1401 is used to acquire multiple sample videos;

[0197] The sample image acquisition module 1402 is used to obtain multiple sample images corresponding to each target video frame in the sample video based on the image area in the target video frame whose brightness is greater than a preset brightness threshold.

[0198] The segmentation module 1403 is used to divide the multiple sample images corresponding to each sample video into multiple sample image groups;

[0199] The sample similarity determination module 1404 is used to determine the sample similarity of each sample image group as a first value if the sample image group belongs to the same sample video, and determine the sample similarity of the sample image group as a second value if the sample image group belongs to different sample videos.

[0200] The training sample generation module 1405 is used to generate training samples containing the sample image group; wherein, the training samples include positive samples and negative samples; the positive samples include sample image groups belonging to the same sample video, and the sample similarity of the sample image group; the negative samples include sample image groups belonging to different sample videos, and the sample similarity of the sample image group.

[0201] The training module 1406 is used to train the feature extraction model based on the training samples to obtain a trained feature extraction model.

[0202] Optionally, the sample video acquisition module 1401 is specifically used to: divide the original video into multiple sub-videos according to the first sampling interval, and use them as sample videos;

[0203] The device further includes:

[0204] The target video frame determination module is used to sample the sample video according to a second sampling interval to obtain the target video frame before the sample image acquisition module 1402 performs sampling for each target video frame in the sample video, based on the image area in the target video frame whose brightness is greater than a preset brightness threshold.

[0205] Optionally, the sample image acquisition module 1402 is specifically used for: obtaining a first image to be processed based on a connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame; if the boundaries of the first image to be processed are not equal, expanding the shortest boundary among the boundaries of the first image to be processed to obtain a second image to be processed; wherein the boundaries of the second image to be processed are equal; expanding the boundary of the second image to be processed to obtain a sample image; wherein the boundary of the sample image is a preset multiple of the boundary of the second image to be processed.

[0206] Optionally, the sample image acquisition module 1402 is specifically used to: determine the connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame as the third image to be processed; for each third image to be processed, if the area of ​​the third image to be processed is greater than a preset area threshold, determine the third image to be processed as the first image to be processed.

[0207] Optionally, the sample image acquisition module 1402 is specifically used to: convert the target video frame into a grayscale image; smooth the grayscale image based on a Gaussian blur function to obtain a fourth image to be processed; and determine the connected image region composed of pixels with brightness greater than a preset brightness threshold in the fourth image to be processed as a third image to be processed.

[0208] Optionally, the training module 1406 is specifically configured to: for each sample image in the sample image group, input the sample image into the feature extraction model of the initial structure to obtain a feature vector representing the image features of the sample image, as a sample feature vector; calculate the similarity of the sample feature vectors of the sample images in the sample image group as a predicted similarity; calculate the function value of a preset loss function based on the predicted similarity and the sample similarity of the sample image group; and train the feature extraction model of the initial structure based on the calculated function value to obtain a trained feature extraction model.

[0209] Based on the feature extraction model training device provided in this embodiment of the invention, the feature extraction model is trained using training samples containing sample image groups. Subsequently, flame detection is performed on the video to be processed based on the trained feature extraction model, which can reduce interference from other bright objects, avoid identifying light areas as flame areas, reduce the false alarm rate of flame detection, and improve the accuracy of flame detection.

[0210] This invention also provides an electronic device, such as... Figure 15 As shown, the system includes a processor 1501, a communication interface 1502, a memory 1503, and a communication bus 1504. The processor 1501, the communication interface 1502, and the memory 1503 communicate with each other through the communication bus 1504. The memory 1503 is used to store computer programs. When the processor 1501 executes the program stored in the memory 1503, it implements any of the flame detection method steps described above, or implements any of the feature extraction model training method steps described above.

[0211] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above electronic devices and other devices.

[0212] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0213] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0214] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described flame detection methods, or implements the steps of any of the above-described feature extraction model training methods.

[0215] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the flame detection methods in the above embodiments, or to implement any of the feature extraction model training methods described above.

[0216] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0217] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0218] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0219] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A flame detection method, characterized in that, The method includes: Get the video to be processed; For each video frame in the video to be processed, if a potential flame region is detected in the video frame, the image to be detected is obtained based on the potential flame region in the video frame; The image to be detected is input into a pre-trained feature extraction model to obtain a feature vector representing the image features of the image to be detected, which is used as the first feature vector. Based on the image region in the video frame following the image to be detected that is at the same position as the image to be detected, the associated image of the image to be detected is obtained; The associated image is input into the feature extraction model to obtain a feature vector representing the image features of the associated image, which is used as the second feature vector. When the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold, it is determined that the image to be detected contains a target flame region. For each video frame in the video to be processed, if a potential flame region is detected in that video frame, a detection image is obtained based on the potential flame region in that video frame, including: For each video frame in the video to be processed, the video frame is detected based on the target detection model to obtain the potential flame region in the video frame; the boundary along the width direction of the potential flame region is moved by a certain number of pixels so that the width of the adjusted potential flame region is a preset multiple of the width of the original potential flame region; and the boundary along the height direction of the potential flame region is moved by a certain number of pixels so that the height of the adjusted potential flame region is a preset multiple of the height of the original potential flame region, to obtain the adjusted potential flame region; the adjusted potential flame region is extracted from the video frame to obtain the image to be detected; Alternatively, for each video frame in the video to be processed, the potential flame region in the video frame is detected based on the target detection model; if the boundaries of the potential flame region are not equal, the other boundaries are moved by a certain number of pixels according to the longest boundary of the potential flame region, so that the boundaries of the potential flame region are equal, resulting in a square potential flame region; the boundary along the width direction of the square potential flame region is moved by a certain number of pixels, so that the width of the adjusted potential flame region is a preset multiple of the width of the original potential flame region; and the boundary along the height direction of the square potential flame region is moved by a certain number of pixels, so that the height of the adjusted potential flame region is a preset multiple of the height of the original potential flame region, resulting in an adjusted potential flame region; the adjusted potential flame region is extracted from the video frame to obtain the image to be detected.

2. A method for training a feature extraction model, characterized in that, The method for generating the feature extraction model of claim 1 includes: Acquire multiple sample videos; For each target video frame in the sample video, based on the image region in the target video frame whose brightness is greater than a preset brightness threshold, multiple sample images corresponding to the sample video are obtained; The multiple sample images corresponding to each sample video are divided into multiple sample image groups; For each sample image group, if the sample image group belongs to the same sample video, the sample similarity of the sample image group is determined as a first value; if the sample image group belongs to different sample videos, the sample similarity of the sample image group is determined as a second value. Generate training samples containing the sample image groups; wherein the training samples include positive samples and negative samples; the positive samples include sample image groups belonging to the same sample video, and the sample similarity of the sample image groups; the negative samples include sample image groups belonging to different sample videos, and the sample similarity of the sample image groups; The feature extraction model is trained based on the training samples to obtain a well-trained feature extraction model.

3. The method according to claim 2, characterized in that, The acquisition of multiple sample videos includes: The original video is divided into multiple sub-videos according to the first sampling interval, which are used as sample videos; Before obtaining multiple sample images corresponding to each target video frame in the sample video based on the image regions in the target video frame whose brightness is greater than a preset brightness threshold, the method further includes: The sample video is sampled according to the second sampling interval to obtain the target video frame.

4. The method according to claim 2, characterized in that, The method involves obtaining multiple sample images corresponding to the sample video based on image regions in the target video frame whose brightness exceeds a preset brightness threshold, including: The first image to be processed is obtained based on the connected image region composed of pixels in the target video frame whose brightness is greater than a preset brightness threshold. If the boundaries of the first image to be processed are not equal, the shortest boundary among the boundaries of the first image to be processed is expanded outward to obtain the second image to be processed; wherein, the boundaries of the second image to be processed are equal. The boundary of the second image to be processed is expanded to obtain a sample image; wherein the boundary of the sample image is a preset multiple of the boundary of the second image to be processed.

5. The method according to claim 4, characterized in that, The first image to be processed is obtained by forming a connected image region based on pixels in the target video frame whose brightness is greater than a preset brightness threshold, including: The connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame is identified as the third image to be processed. For each third image to be processed, if the area of ​​the third image to be processed is greater than a preset area threshold, the third image to be processed is determined to be the first image to be processed.

6. The method according to claim 5, characterized in that, The step of determining the connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame as the third image to be processed includes: Convert the target video frame into a grayscale image; The grayscale image is smoothed using a Gaussian blur function to obtain a fourth image to be processed. The connected image region composed of pixels whose brightness is greater than a preset brightness threshold in the fourth image to be processed is determined as the third image to be processed.

7. The method according to claim 2, characterized in that, The step of training the feature extraction model based on the training samples to obtain the trained feature extraction model includes: For each sample image in the sample image group, the sample image is input into the feature extraction model of the initial structure to obtain a feature vector representing the image features of the sample image, which is used as the sample feature vector; Calculate the similarity of the sample feature vectors of the sample images in the sample image group, and use it as the predicted similarity; Based on the predicted similarity and the sample similarity of the sample image group, the function value of the preset loss function is calculated; The feature extraction model of the initial structure is trained based on the calculated function values ​​to obtain a trained feature extraction model.

8. A flame detection device, characterized in that, The device includes: The video acquisition module is used to acquire videos to be processed. The image acquisition module is used to obtain the image to be detected based on the potential flame region in each video frame of the video to be processed. The first feature vector acquisition module is used to input the image to be detected into a pre-trained feature extraction model to obtain a feature vector representing the image features of the image to be detected, which is used as the first feature vector. The associated image acquisition module is used to obtain the associated image of the image to be detected based on the image region in the video frame located after the image to be detected in the video to be processed, which is at the same position as the image to be detected; The second feature vector acquisition module is used to input the associated image into the feature extraction model to obtain a feature vector representing the image features of the associated image, which is used as the second feature vector. The determination module is used to determine that the image to be detected contains a target flame region when the similarity between the first feature vector and the second feature vector is less than a preset similarity threshold. The image acquisition module is specifically used for: For each video frame in the video to be processed, the video frame is detected based on the target detection model to obtain the potential flame region in the video frame; the boundary along the width direction of the potential flame region is moved by a certain number of pixels so that the width of the adjusted potential flame region is a preset multiple of the width of the original potential flame region; and the boundary along the height direction of the potential flame region is moved by a certain number of pixels so that the height of the adjusted potential flame region is a preset multiple of the height of the original potential flame region, to obtain the adjusted potential flame region; the adjusted potential flame region is extracted from the video frame to obtain the image to be detected; Alternatively, for each video frame in the video to be processed, the potential flame region in the video frame is detected based on the target detection model; if the boundaries of the potential flame region are not equal, the other boundaries are moved by a certain number of pixels according to the longest boundary of the potential flame region, so that the boundaries of the potential flame region are equal, resulting in a square potential flame region; the boundary along the width direction of the square potential flame region is moved by a certain number of pixels, so that the width of the adjusted potential flame region is a preset multiple of the width of the original potential flame region; and the boundary along the height direction of the square potential flame region is moved by a certain number of pixels, so that the height of the adjusted potential flame region is a preset multiple of the height of the original potential flame region, resulting in an adjusted potential flame region; the adjusted potential flame region is extracted from the video frame to obtain the image to be detected.

9. A feature extraction model training device, characterized in that, The apparatus for generating the feature extraction model of claim 1, comprising: The sample video acquisition module is used to acquire multiple sample videos; The sample image acquisition module is used to obtain multiple sample images corresponding to each target video frame in the sample video based on the image regions in the target video frame whose brightness is greater than a preset brightness threshold. The segmentation module is used to divide the multiple sample images corresponding to each sample video into multiple sample image groups; The sample similarity determination module is used to determine the sample similarity of each sample image group as a first value if the sample image group belongs to the same sample video, and as a second value if the sample image group belongs to different sample videos. A training sample generation module is used to generate training samples containing the sample image group; wherein, the training samples include positive samples and negative samples; the positive samples include sample image groups belonging to the same sample video, and the sample similarity of the sample image group; the negative samples include sample image groups belonging to different sample videos, and the sample similarity of the sample image group. The training module is used to train the feature extraction model based on the training samples to obtain a trained feature extraction model.

10. The apparatus according to claim 9, characterized in that, The sample video acquisition module is specifically used for: The original video is divided into multiple sub-videos according to the first sampling interval, which are used as sample videos; The device further includes: The target video frame determination module is used to sample the sample video according to a second sampling interval to obtain the target video frame before the sample image acquisition module performs sampling for each target video frame in the sample video, based on the image region in the target video frame whose brightness is greater than a preset brightness threshold.

11. The apparatus according to claim 9, characterized in that, The sample image acquisition module is specifically used for: The first image to be processed is obtained based on the connected image region composed of pixels in the target video frame whose brightness is greater than a preset brightness threshold. If the boundaries of the first image to be processed are not equal, the shortest boundary among the boundaries of the first image to be processed is expanded outward to obtain the second image to be processed; wherein, the boundaries of the second image to be processed are equal. The boundary of the second image to be processed is expanded to obtain a sample image; wherein the boundary of the sample image is a preset multiple of the boundary of the second image to be processed.

12. The apparatus according to claim 11, characterized in that, The sample image acquisition module is specifically used for: The connected image region composed of pixels with brightness greater than a preset brightness threshold in the target video frame is identified as the third image to be processed. For each third image to be processed, if the area of ​​the third image to be processed is greater than a preset area threshold, the third image to be processed is determined to be the first image to be processed.

13. The apparatus according to claim 12, characterized in that, The sample image acquisition module is specifically used for: Convert the target video frame into a grayscale image; The grayscale image is smoothed using a Gaussian blur function to obtain a fourth image to be processed. The connected image region composed of pixels whose brightness is greater than a preset brightness threshold in the fourth image to be processed is determined as the third image to be processed.

14. The apparatus according to claim 9, characterized in that, The training module is specifically used for: For each sample image in the sample image group, the sample image is input into the feature extraction model of the initial structure to obtain a feature vector representing the image features of the sample image, which is used as the sample feature vector; Calculate the similarity of the sample feature vectors of the sample images in the sample image group, and use it as the predicted similarity; Based on the predicted similarity and the sample similarity of the sample image group, the function value of the preset loss function is calculated; The feature extraction model of the initial structure is trained based on the calculated function values ​​to obtain a trained feature extraction model.

15. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of claim 1, or any one of claims 2-7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of claim 1, or any one of claims 2-7.

Images