Method and system for improving generalization of computer vision small model based on text graph model

Abstract

The invention relates to the technical field of artificial intelligence, and provides a method and system for improving generalization of a small computer vision model based on a text graph model, and the method comprises the steps: S1, constructing and optimizing a large text graph model; s2, constructing positive and negative sample text descriptions according to a text graph large model on the basis of features and scene requirements of a target detection object; s3, inputting the positive and negative sample text descriptions into the optimized text graph large model to generate corresponding positive and negative sample images, labeling the positive and negative sample images, and carrying out preprocessing and data enhancement on the labeled positive and negative sample images; s4, loading a CV small model, and performing model training according to the preprocessed and data enhanced positive and negative sample images; s5, evaluating the trained CV small model, and if an evaluation result does not meet a preset condition, adjusting hyper-parameters and returning to the step S4 for re-training; if a preset condition is met, the trained CV small model is stored, and the problems that training data is insufficient and samples lack real scene diversity are solved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method and system for improving the generalization of small computer vision models based on text-based graph models. Background Technology

[0002] In the field of computer vision, flame detection technology is widely used in industrial safety monitoring, forest fire prevention, and home fire protection, playing a vital role in preventing fire accidents and protecting life and property. Currently, deep learning-driven flame detection models have become the mainstream technology solution due to their powerful feature extraction and pattern recognition capabilities. However, the performance of such models is highly dependent on high-quality, large-scale training data.

[0003] Existing flame detection datasets generally suffer from significant limitations. First, data acquisition largely relies on surveillance cameras or specific imaging equipment, resulting in limited scene coverage. They primarily focus on typical scenarios such as laboratory environments, traffic accidents, forest fires, and industrial furnaces, failing to encompass real-world, complex environments like home kitchens, complex factory offices, and old buildings. Second, the data samples are highly homogenized, mostly consisting of images of large-scale, high-intensity flames, with insufficient coverage of early-stage fires such as small flames and weak fires. However, small flame detection is crucial for early fire warning, a requirement that traditional datasets cannot meet. Summary of the Invention

[0004] This disclosure provides a method and system for improving the generalization of small computer vision models based on text-based graph models. It aims to solve the problems of insufficient training data and lack of real-world scene diversity in traditional techniques due to difficulties in data collection and limited enhancement effects.

[0005] According to a first aspect of this disclosure, a method for improving the generalization of small computer vision models based on text-based graph models is provided, comprising:

[0006] S1: Construct and optimize the large model of text-based graphs;

[0007] S2: Based on the features of the target detection object and the scene requirements, construct positive sample text descriptions and negative sample text descriptions according to the text-generated image big model. The positive sample text descriptions contain target detection object and scene information, while the negative sample text descriptions have the same scene information as the positive sample text descriptions but do not contain target detection objects.

[0008] S3: Input the positive sample text and negative sample text into the optimized text-to-image large model to generate corresponding positive sample images and negative sample images, label the generated positive sample images and negative sample images, and preprocess and augment the labeled positive sample images and negative sample images.

[0009] S4: Load the small CV model and train the model based on the preprocessed and data-augmented positive and negative sample images;

[0010] S5: Evaluate the trained CV mini-model. If the evaluation result does not meet the preset conditions, adjust the hyperparameters and return to step S4 to retrain. If the evaluation result meets the preset conditions, save the trained CV mini-model.

[0011] In a preferred implementation, step S1, constructing and optimizing the text-based image large model, includes:

[0012] The Stable Diffusion model was chosen as the basic text-to-image model.

[0013] The selected text-to-text model is pre-trained on a large-scale image-text dataset to obtain the pre-trained basic text-to-text model.

[0014] Collect real flame images containing the target detection object, and perform secondary fine-tuning based on the real flame images and the pre-trained basic text image model to obtain the optimized text image model.

[0015] In a preferred embodiment, the pre-trained basic textural image model is fine-tuned a second time based on the real flame image to obtain an optimized textural image model, including:

[0016] The real flame images are filtered based on the characteristics of the target object under different environmental conditions, different degrees of occlusion, and various morphological variations;

[0017] A refined text description is constructed for the filtered real flame images, wherein the refined text description includes the color, shape, size of the target object, the relative positional relationship between the target object and surrounding objects, and interfering elements in the environment;

[0018] A fine-tuning sample set is formed by combining real flame images with corresponding refined text descriptions, and the fine-tuning sample set is divided into a training subset and a validation subset according to a preset ratio.

[0019] Set fine-tuning training parameters and use a loss function to quantify the differences between the generated image and the real flame image in terms of color, shape, size, and scene integration.

[0020] The pre-trained basic text-to-image model is iteratively trained based on the training subset, and the basic text-to-image model after secondary fine-tuning is verified through the verification subset.

[0021] Training stops when the generated image and the real flame image reach a preset threshold in terms of color, shape, size, and scene fusion, resulting in an optimized text image model.

[0022] In a preferred embodiment, the loss function is a weighted combination of the cross-entropy loss function and the perceptual loss function.

[0023] In a preferred embodiment, step S3 involves labeling the generated positive and negative sample images, and then preprocessing and data augmenting the labeled positive and negative sample images, including:

[0024] The target detection objects in the positive sample images are labeled with bounding boxes to indicate their location information, and the negative sample images are labeled as having no targets.

[0025] The labeled positive and negative sample images are normalized to obtain preprocessed positive and negative sample images.

[0026] Basic enhancement and physical simulation enhancement are performed on the preprocessed positive and negative sample images to obtain enhanced positive and negative sample images.

[0027] In a preferred embodiment, the step of performing basic enhancement and physical simulation enhancement on the preprocessed positive and negative sample images to obtain enhanced positive and negative sample images includes:

[0028] Basic enhancement is performed on the preprocessed positive and negative sample images by random rotation, scaling, and brightness adjustment.

[0029] Physical simulation enhancement is performed on the base enhanced positive and negative sample images to add interference terms, resulting in enhanced positive and negative sample images. This includes: generating a realistic smoke layer based on a fluid dynamics model, simulating multi-source interference using a ray tracing algorithm, and adding dynamic occlusions through a generative adversarial network.

[0030] In a preferred embodiment, the CV mini-model uses the YOLOv5 model, and before model training in step S4, the anchor point parameters of the YOLOv5 model are specifically optimized, including:

[0031] The bounding box width and height data of the target detection objects in the generated positive sample images are statistically analyzed, the aspect ratio distribution range is calculated, and valid samples are selected.

[0032] The K-means clustering algorithm is used to perform cluster analysis on the bounding boxes of the effective samples to generate optimized microscale anchor boxes;

[0033] The optimized microscale anchor boxes are bound to the deepest feature map of the YOLOv5 model.

[0034] In a preferred embodiment, step S4, which involves training the model based on the preprocessed and data-enhanced positive and negative sample images, further includes:

[0035] After each training cycle, the detection loss value of all training samples is calculated, and the detection loss values ​​are sorted to generate a detection loss value sequence.

[0036] The detection loss value sequence is truncated according to a preset ratio, and the sample corresponding to the truncated detection loss value is determined as the original difficult sample;

[0037] Based on the visual feature defects of the original difficult samples, a quantitative description of the defect features is added to the original text description to generate an enhanced text description set.

[0038] The enhanced text description set is input into the optimized text-generated graph model to generate a set of difficult enhancement samples.

[0039] The original hard samples and the enhanced hard sample set are mixed and then incorporated into the total sample set for the next round of training according to a preset mixing ratio.

[0040] In a preferred embodiment, step S4, evaluating the trained CV mini-model includes:

[0041] A tiered test sample dataset is constructed for the target detection object and its surrounding scene;

[0042] Calculate the recall rate for each level of sample data and the corresponding detection decay rate, wherein the recall rate includes low-level recall rate or high-level recall rate;

[0043] If the evaluation results do not meet the preset conditions, the specific level that did not meet the standard is located, and supplementary samples of the corresponding level are generated through the optimized Wenshengtu large model before returning to step S4 for retraining.

[0044] If the evaluation results meet the preset conditions, the trained CV mini-model is saved.

[0045] The preset conditions are that the lowest level recall rate is greater than the first preset threshold, the highest level recall rate is greater than the second preset threshold, and the detection decay rate is not higher than the third threshold.

[0046] According to a second aspect of this disclosure, a system for improving the generalization of small computer vision models based on text-based graph models is provided, comprising:

[0047] The Wensheng Image Large Model Construction and Optimization Module is used to construct and optimize the Wensheng Image Large Model.

[0048] The text description generation module is used to construct positive sample text descriptions and negative sample text descriptions based on the features of the target detection object and the scene requirements, according to the text-generated image big model. The positive sample text descriptions contain target detection object and scene information, while the negative sample text descriptions have the same scene information as the positive sample text descriptions but do not contain the target detection object.

[0049] The sample image generation and enhancement module is used to input the positive sample text description and the negative sample text description into the optimized text-to-image large model to generate corresponding positive sample images and negative sample images, to annotate the generated positive sample images and negative sample images, and to preprocess and enhance the annotated positive sample images and negative sample images.

[0050] The CV mini-model training module is used to load CV mini-models and train them based on preprocessed and data-augmented positive and negative sample images.

[0051] The CV small model evaluation module is used to evaluate the trained CV small model. If the evaluation result does not meet the preset conditions, the hyperparameters are adjusted and the process returns to step S4 for retraining. If the evaluation result meets the preset conditions, the trained CV small model is saved.

[0052] According to a third aspect of this disclosure, an electronic device is provided, comprising at least one processor and a memory communicatively connected to said at least one processor; wherein the memory stores a computer program executable by said at least one processor, said computer program being executed by said at least one processor to enable said at least one processor to perform the method as described in any of the preceding claims.

[0053] Compared with the prior art, this disclosure achieves the following beneficial effects:

[0054] (1) This disclosure applies the text-based image model to the generation of training data for the flame detection model. Through carefully designed text descriptions, it accurately generates diverse image samples containing small flames, improves data quality, and enhances the generalization of the model to small flames, so as to detect fires in the early stages of a fire and issue an alarm.

[0055] (2) This disclosure directly generates positive samples that conform to the characteristics of the initial stage of a fire by using refined text description (including quantitative features such as the color, shape, and size of small flames), which solves the problem of difficult data collection of small flames in real scenes. In addition, it combines physical simulation enhancement (smoke, multiple light sources, dynamic occlusion, etc.) to generate complex scene samples that are highly matched with the real environment, far exceeding the limitations of traditional data augmentation.

[0056] (3) Based on the bounding box data of the generated samples, this disclosure uses K-means clustering to customize the anchor box adapted to small targets and binds it to the deep feature map of the model to enhance the ability to extract subtle features and solve the problem of missing small targets. In addition, difficult examples (such as occlusion or small flames in low light) are screened by loss value, and enhanced samples are generated and trained in a targeted manner to improve the model's adaptability to high-difficulty scenes.

[0057] (4) This disclosure significantly improves the generalization of CV small models in real-world scenarios through the full-process design of “generating textual samples - optimizing small models - closed-loop evaluation and iteration”. Compared with traditional data augmentation methods, the model’s detection accuracy for small flames has been significantly improved, and it can accurately identify fires in the early stages of a fire. In addition, the solution disclosed in this disclosure is applicable to diverse scenarios such as kitchens, forests, and industrial plants, avoiding “overfitting” caused by training in a single scenario.

[0058] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0059] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0060] Figure 1 A flowchart illustrating a method for improving the generalization of small computer vision models based on a text-based graph model, according to an embodiment of this disclosure, is shown.

[0061] Figure 2 A flowchart illustrating sample collection and training according to an embodiment of this disclosure is shown;

[0062] Figure 3 A block diagram of a system for improving the generalization of small computer vision models based on a text-based graph model, according to an embodiment of this disclosure, is shown.

[0063] Figure 4 A schematic diagram of an exemplary electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0065] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0066] like Figure 1 The diagram shown is a flowchart of a method for improving the generalization of small computer vision models based on text-based graph models, according to an embodiment of this disclosure. The method 100 includes:

[0067] S1: Construct and optimize the large model of text-based graphs;

[0068] In some embodiments, the construction and optimization process of the Wensheng graph large model is as follows:

[0069] The StableDiffusion model was chosen as the basic text-to-text model. It was first pre-trained on a large-scale general image-text dataset to enable it to generate basic images and understand the visual mapping relationship of basic concepts such as "flame" and "kitchen".

[0070] Based on this, and considering the specific characteristics of the flame detection task, the model is further optimized: real images containing the target detection object and corresponding text descriptions are collected, and then the pre-trained basic text-based image model is fine-tuned again using the collected real images of the target detection object and corresponding text descriptions to enhance its accuracy in generating target features and complex scenes, ultimately resulting in an optimized large text-based image model.

[0071] For example, 1000-2000 real flame images containing small flames are collected, including initial fire photos in various scenes such as home kitchens, industrial plants, and forests. A precise text description is constructed for each image, such as: "A 2 cm high orange-yellow flickering flame is located next to a greasy paper towel in the kitchen. The background light is dim, and there is a shadow of a metal water pipe to the left of the flame." The pre-trained model is then fine-tuned by collecting real images of the target detection object and corresponding text descriptions. This enhances the model's ability to capture the core features of small flames (color gradient: from orange-yellow to faint dark red; dynamic shape: irregular flickering 2-3 times per second; size boundary: pixel-level definition below 3 cm) and complex scene elements (such as oil smoke obstruction and multi-light source reflection). Ultimately, this improves the visual matching degree between the small flame images generated by the model and real samples.

[0072] It should be noted that, in addition to the Stable Diffusion model, other mainstream text-to-image models can also be used in this embodiment. The specific choice can be made based on factors such as scene requirements, generation accuracy, and computing power costs. For example, the ControlNet model (which uses edge detection, depth maps, and other control conditions to accurately constrain the position and shape of the small flame in the image, making it suitable for space-constrained scenarios such as gaps in industrial equipment) and the LLaVA-1.5 model (which combines the logical understanding capabilities of a large language model to parse multi-condition descriptions such as "the flame is partially obscured by a semi-transparent plastic bag in a light breeze," improving the accuracy of complex text generation) can be used. If lightweight deployment is required, open-source fine-tuning models such as DreamShaper can also be selected (which has a small number of parameters and can be fine-tuned on consumer-grade GPUs, making it suitable for edge sample generation).

[0073] In some embodiments, the pre-trained basic textural image model is fine-tuned based on real flame images to obtain an optimized textural image model, including:

[0074] (a) Real image screening: From the collected real flame images, samples covering different environmental conditions were screened to ensure the diversity of fine-tuning data. The different environmental conditions include different light intensities and ambient humidity (light intensity: 0.1-0.8 lux; humidity: 30%-80%), different degrees of occlusion (no occlusion, 10%-50% partial occlusion, more than 50% severe occlusion) and various morphological variations (point-shaped, strip-shaped, and diffuse flames).

[0075] (b) Refined text construction: Refined text description is constructed for the filtered real flame images, wherein the refined text description includes the color, shape, size of the target object, the relative positional relationship between the target object and surrounding objects, and interference elements in the environment.

[0076] For example: Color: "Orange-yellow gradient with faint red edges"; Shape: "Irregularly swaying with a forked top"; Size: "Height 2.5 cm, width 1.8 cm"; Relative position: "The flame is located in the upper left corner of the wooden tabletop, 3 cm away from the metal container on the right"; Interfering elements: "The background shows the silhouette of a worker in blue overalls, and there is a flashing incandescent light spot in the upper right corner".

[0077] (c) Dataset partitioning: The real flame images and corresponding refined text descriptions are combined to form a fine-tuning sample set, and the fine-tuning sample set is divided into a training subset and a validation subset according to a preset ratio of 7:3;

[0078] (d) Fine-tuning parameter settings: The AdamW optimizer is used to quantify the differences between the generated image and the real flame image in terms of color, shape, size and scene blending by using a loss function;

[0079] (e) Iterative training and verification: The pre-trained basic text image model is iteratively trained according to the training subset, and the basic text image after secondary fine-tuning is verified through the verification subset; when the generated image and the real flame image reach the preset threshold (90%) in terms of color, shape, size and scene fusion, training is stopped and the optimized text image model is obtained.

[0080] It is worth noting that in step (d) above, a weighted combination of the cross-entropy loss function and the perceptual loss function is used as the fine-tuning loss function, and the specific formula is as follows:

[0081] L=α*L CE +βL Perceptual ;

[0082] Among them, L CE This is the cross-entropy loss function, used to quantify the pixel-level differences between the generated image and the real image;

[0083] L Perceptual To achieve the perceptual loss function, features are extracted through a pre-trained VGG network to quantify the differences in high-level visual semantics (such as flame texture and scene atmosphere).

[0084] α and β are weighting coefficients, and α + β = 1.

[0085] By combining the cross-entropy loss function with the perceptual loss function, we can balance pixel-level accuracy with high-level semantic consistency, thereby more accurately optimizing the model's ability to generate target features such as small flames and complex scenes.

[0086] S2: Based on the features of the target detection object and the scene requirements, construct positive sample text descriptions and negative sample text descriptions according to the text-generated image model. The positive sample text descriptions contain target detection object and scene information, while the negative sample text descriptions have the same scene information as the positive sample text descriptions but do not contain target detection objects.

[0087] In some embodiments, the positive sample text description adopts the structure of "target features + scene details + interference factors".

[0088] Positive Sample Text Description Generation: Write a text description that includes the key features of a small flame against a complex background. For example: "In the gap under the kitchen cabinet, there is a small, flickering orange-yellow flame, about 2 cm high, surrounded by greasy paper towels. The background is the dimly lit interior of the cabinet." Input this type of text into the Wensheng Image Model to generate an image containing the small flame.

[0089] The negative sample text description follows the principle of "full scene reuse + target feature replacement" to ensure that the scene interference conditions are completely consistent with those of the positive sample, and only the target is replaced with a non-flame object that is easily confused.

[0090] Negative sample text description generation: Write text for the same scene but without flames, for example: "Greasy paper towels are scattered in the gap under the kitchen cabinet, the background is the dimly lit interior of the cabinet, workers in red or gray-white work clothes, red or yellow lights, etc." to generate negative sample images.

[0091] Through the above-mentioned structured text construction method, positive samples can accurately guide the text-generated image large model to generate diverse samples that conform to the characteristics of small flames, while negative samples, through highly similar scenes and easily confused targets, enhance the model's ability to distinguish between "flames" and "non-flames", providing high-quality positive and negative sample comparison data for subsequent CV small model training.

[0092] S3: Input the positive sample text and negative sample text into the optimized text-based image model to generate corresponding positive sample images and negative sample images. Label the generated positive sample images and negative sample images, and preprocess and augment the labeled positive sample images and negative sample images.

[0093] In some embodiments, the generated positive and negative sample images are first labeled. Specifically, the generated positive sample images need to be labeled with the position of the small flames (e.g., coordinates (x1, y1, x2, y2)) using bounding boxes, and the negative sample images are labeled as "no target".

[0094] Secondly, pixel value normalization processing is performed on the labeled positive and negative sample images, including: resizing the images to 640×640 pixels and normalizing the pixel values ​​to the [0, 1] interval;

[0095] Next, basic enhancement and physical simulation enhancement are performed on the preprocessed positive and negative sample images. Basic enhancement includes random rotation of 15°, scaling by 0.8-1.2 times, and brightness adjustment by 0.7-1.3 times. Physical simulation enhancement includes adding smoke layers of different concentrations based on a fluid dynamics model, simulating multi-source light reflections through ray tracing, and generating dynamic occlusions such as floating plastic bags using GANs.

[0096] It is worth mentioning that, in order to ensure sample quality, a quality assessment mechanism for data augmentation samples was established. For example, the proportion of flame pixels within the labeled box is ≥60%, otherwise the sample is discarded. The color histogram, target location distribution, and other features of the augmented samples are statistically analyzed to ensure data diversity.

[0097] Through the above data augmentation process, the core features of the small flame can be preserved while simulating complex interference factors in real-world scenarios, providing high-quality and diverse training samples for CV small models and improving the model's ability to detect small flames in different scenarios.

[0098] S4: Load the CV mini-model and train the model based on the preprocessed and data-augmented positive and negative sample images.

[0099] In some embodiments, the above-mentioned CV small model adopts the YOLOv5 model, and the adaptability for small object detection is improved by anchor point parameter optimization before model training. During training, the model performance is dynamically optimized by combining a hard sample reinforcement strategy. The specific process is as follows:

[0100] (a) Valid sample screening: Extract the bounding box data of all small flames from the generated positive sample images, calculate the aspect ratio distribution (screening range 0.5-2.0, remove abnormal samples with aspect ratio <0.5 or >2.0), avoid interference with clustering results by extreme aspect ratio samples (such as thin strips or super large targets), and make the anchor points more in line with the real shape distribution of micro-scale targets such as small flames.

[0101] (b) K-means++ clustering analysis: The K-means clustering algorithm is used to perform clustering analysis on the bounding boxes of the effective samples (the number of clusters is set to 9, matching the 3 detection scales of YOLOv5) to generate anchor boxes that are more in line with the target detection characteristics. For targets of 10-50 pixels such as small flames, optimized micro-scale anchor boxes (such as [5×5, 8×8, 12×12]) are generated.

[0102] (c) Detection branch structure adjustment: Adjust the detection branch structure of the YOLOv5 model, bind the optimized micro-scale anchor boxes to the deepest feature map of the model, and enhance the ability to extract subtle features of small flames by increasing the number of convolutional layer channels in this branch structure.

[0103] In some embodiments, model training is performed based on preprocessed and data-augmented positive and negative sample images, employing a two-stage strategy of "basic training + hard sample augmentation," specifically including the following steps:

[0104] After each training cycle, the detection loss value of all training samples is calculated and sorted to generate a detection loss value sequence. The detection loss value sequence is truncated according to a preset ratio, and the samples corresponding to the truncated detection loss values ​​are determined as the original hard samples. Based on the visual feature defects of the original hard samples, quantitative descriptions of the defect features are added to the original text descriptions to generate an enhanced text description set. The enhanced text description set is input into the optimized text-to-image large model to generate an enhanced hard sample set. The original hard samples and the enhanced hard sample set are mixed and integrated into the total sample set of the next training cycle according to a preset mixing ratio.

[0105] For example, after each round of training, the detection loss value of all samples is calculated. Positive samples focus on samples with "missed flames" (classification loss > 0.8) and "location bias" (CIoU < 0.3), while negative samples focus on samples "misclassified as flames" (classification loss > 0.7). The loss values ​​are sorted from largest to smallest to generate a detection loss value sequence. The top 30% of the loss value sequence are selected as the original hard samples, including typical hard examples such as "flames 70% obscured by thick smoke", "flames confused with the color of red fabric", and "flames with only the edges visible in low light".

[0106] Next, to address the visual feature deficiencies of the original difficult samples, quantitative parameters are added to the original text description, for example:

[0107] Original text: "A small flame in the kitchen crack";

[0108] Enhanced text: "A small flame, 1.5 cm high, obscured by 70% dense smoke in a kitchen crevice, with only 30% of its edge visible, under ambient light of 0.21ux (extremely weak light)."

[0109] Original text: "A flame on a dead branch in the forest";

[0110] Enhanced text: "A flame, 2 centimeters high, on a dead branch in the forest, its color blends with the orange-red fallen leaves, its shape overlapping with the fallen leaves by 60%, with fallen leaves drifting in the background."

[0111] Building upon this foundation, the enhanced text input and optimized text-to-image model are further used to generate enhanced hard samples in a 1:1 ratio with the original hard samples (e.g., 500 enhanced samples are generated from 500 original hard samples). The original hard samples and enhanced samples are then mixed (mixing ratio 1:1) and incorporated into the next round of training at a ratio of 30% of the total sample size (e.g., 3,000 hard samples in a mixed set of 10,000 total samples). The learning rate of this mixed set is temporarily increased by 50% (e.g., from 0.0005 to 0.00075) to accelerate the model's learning of hard examples.

[0112] This embodiment improves the adaptability of small target detection by optimizing anchor point parameters and addresses the "missed detection and misjudgment" problems of the model by combining dynamic reinforcement strategy for difficult samples. Ultimately, the YOLOv5 model improves the detection accuracy of small flames to over 92%, and its generalization in complex scenes is significantly better than traditional training methods.

[0113] S5: Evaluate the trained CV mini-model. If the evaluation result does not meet the preset conditions, adjust the hyperparameters and return to step S4 to retrain. If the evaluation result meets the preset conditions, save the trained CV mini-model.

[0114] In some embodiments, the evaluation of the trained CV mini-model specifically includes:

[0115] A tiered test set is constructed for the target detection object (such as a small flame) and its surrounding scene. The target is divided into "ultra-small target" (<10×10 pixels) and "small target" (10×10-30×30 pixels) according to its size, and into "no interference", "weak interference" and "strong interference" according to the interference intensity.

[0116] Calculate the low-level recall, high-level recall, and adjacent-level decay rate for each level of target detection object. The low-level recall rate refers to the recall rate of the level with larger size (such as small to medium-sized targets) or weaker interference (such as no interference), while the high-level recall rate refers to the recall rate of the level with smaller size (such as ultra-small targets) or stronger interference (such as strong interference). The detection decay rate is calculated using the formula: (low-level recall rate - high-level recall rate) / low-level recall rate, and is only calculated for adjacent levels (such as small targets and ultra-small targets, weak interference and strong interference).

[0117] If the evaluation does not meet the preset conditions, such as the recall rate being too low for strong interference levels, then the enhanced samples for the scene (such as "small flames obscured by multiple light sources and thick smoke") are generated by supplementing the text-based large model, and the learning rate is adjusted before retraining.

[0118] If the preset conditions are met, the model that meets the criteria will be saved.

[0119] According to the above embodiments of this disclosure, the following technical effects are achieved:

[0120] (1) This disclosure applies a text-based image model to the generation of training data for a flame detection model. Through carefully designed text descriptions, it accurately generates diverse image samples containing small flames, improving data quality and enhancing the model's generalization ability to detect small flames, thus enabling early detection of fires and issuing alarms. Compared to traditional solutions, the model trained in this disclosure improves the accuracy of small flame detection by 23% and the recall rate by 31%, significantly enhancing early fire warning capabilities and reducing fire risk.

[0121] (2) This disclosure directly generates positive samples that conform to the characteristics of the initial stage of a fire by using refined text description (including quantitative features such as the color, shape, and size of small flames), which solves the problem of difficult data collection of small flames in real scenes. In addition, it combines physical simulation enhancement (smoke, multiple light sources, dynamic occlusion, etc.) to generate complex scene samples that are highly matched with the real environment, far exceeding the limitations of traditional data augmentation.

[0122] (3) Based on the bounding box data of the generated samples, this disclosure uses K-means clustering to customize the anchor box adapted to small targets and binds it to the deep feature map of the model to enhance the ability to extract subtle features and solve the problem of missing small targets. In addition, difficult examples (such as occlusion or small flames in low light) are screened by loss value, and enhanced samples are generated and trained in a targeted manner to improve the model's adaptability to high-difficulty scenes.

[0123] (4) This disclosure significantly improves the generalization of CV small models in real-world scenarios through the full-process design of “generating textual samples - optimizing small models - closed-loop evaluation and iteration”. Compared with traditional data augmentation methods, the model’s detection accuracy for small flames has been significantly improved, and it can accurately identify fires in the early stages of a fire. In addition, the solution disclosed in this disclosure is applicable to diverse scenarios such as kitchens, forests, and industrial plants, avoiding “overfitting” caused by training in a single scenario.

[0124] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0125] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0126] Figure 3A block diagram of a system for improving the generalization of small computer vision models based on text-based graph models, according to embodiments of the present disclosure, is shown. Figure 3 As shown, system 300 includes:

[0127] The Wenshengtu large model construction and optimization module 310 is used to construct and optimize the Wenshengtu large model;

[0128] The text description generation module 320 is used to construct positive sample text descriptions and negative sample text descriptions based on the features of the target detection object and the scene requirements, according to the text-generated image big model. The positive sample text descriptions contain target detection object and scene information, while the negative sample text descriptions have the same scene information as the positive sample text descriptions but do not contain the target detection object.

[0129] The sample image generation and enhancement module 330 is used to input the positive sample text description and the negative sample text description into the optimized text-to-image large model to generate corresponding positive sample images and negative sample images, to annotate the generated positive sample images and negative sample images, and to preprocess and enhance the annotated positive sample images and negative sample images.

[0130] The CV small model training module 340 is used to load CV small models and train the models based on preprocessed and data-augmented positive and negative sample images.

[0131] The CV small model evaluation module 350 is used to evaluate the trained CV small model. If the evaluation result does not meet the preset conditions, the hyperparameters are adjusted and the process returns to step S4 for retraining. If the evaluation result meets the preset conditions, the trained CV small model is saved.

[0132] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0133] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0134] Figure 4 A schematic block diagram of an electronic device 400 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0135] Electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in ROM 402 or a computer program loaded into RAM 403 from storage unit 408. RAM 403 can also store various programs and data required for the operation of electronic device 400. The computing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. I / O interface 405 is also connected to bus 404.

[0136] Multiple components in electronic device 400 are connected to I / O interface 405, including: input unit 406, such as keyboard, mouse, etc.; output unit 407, such as various types of displays, speakers, etc.; storage unit 408, such as disk, optical disk, etc.; and communication unit 409, such as network card, modem, wireless transceiver, etc. Communication unit 409 allows electronic device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0137] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by the computing unit 401, one or more steps of method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform method 100 by any other suitable means (e.g., by means of firmware).

[0138] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0140] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0142] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0143] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0144] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0145] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for improving the generalization of small computer vision models based on text-based graph models, characterized in that, include: S1: Construct and optimize the large model of text-based graphs; S2: Based on the features of the target detection object and the scene requirements, construct positive sample text descriptions and negative sample text descriptions according to the text-generated image big model. The positive sample text descriptions contain target detection object and scene information, while the negative sample text descriptions have the same scene information as the positive sample text descriptions but do not contain target detection objects. S3: Input the positive sample text description and negative sample text description into the optimized text-generated image large model to generate corresponding positive sample images and negative sample images, label the generated positive sample images and negative sample images, and preprocess and data augment the labeled positive sample images and negative sample images; S4: Load the small CV model and train the model based on the preprocessed and data-augmented positive and negative sample images; S5: Evaluate the trained CV mini-model. If the evaluation result does not meet the preset conditions, adjust the hyperparameters and return to step S4 to retrain. If the evaluation result meets the preset conditions, save the trained CV mini-model.

2. The method according to claim 1, characterized in that, In step S1, constructing and optimizing the large-scale text-to-image model includes: The Stable Diffusion model was chosen as the basic text-to-image model. The selected text-to-text model is pre-trained on a large-scale image-text dataset to obtain the pre-trained basic text-to-text model. Collect real flame images containing the target detection object, and perform secondary fine-tuning based on the real flame images and the pre-trained basic text image model to obtain the optimized text image model.

3. The method according to claim 2, characterized in that, The pre-trained basic texturing image model is fine-tuned based on the real flame image to obtain an optimized texturing image model, including: The real flame images are filtered based on the characteristics of the target object under different environmental conditions, different degrees of occlusion, and various morphological variations; A refined text description is constructed for the filtered real flame images, wherein the refined text description includes the color, shape, size of the target object, the relative positional relationship between the target object and surrounding objects, and interfering elements in the environment; A fine-tuning sample set is formed by combining real flame images with corresponding refined text descriptions, and the fine-tuning sample set is divided into a training subset and a validation subset according to a preset ratio. Set fine-tuning training parameters and use a loss function to quantify the differences between the generated image and the real flame image in terms of color, shape, size, and scene integration. The pre-trained basic text-to-image model is iteratively trained based on the training subset, and the basic text-to-image model after secondary fine-tuning is verified through the verification subset. Training stops when the generated image and the real flame image reach a preset threshold in terms of color, shape, size, and scene fusion, resulting in an optimized text image model.

4. The method according to claim 3, characterized in that, The loss function is a weighted combination of the cross-entropy loss function and the perceptual loss function, specifically: L=α*L CE +βL Perceptual ; Among them, L CE Let L be the cross-entropy loss function. Perceptual Let α be the perceptual loss function, and let β be the weight coefficients, and let α + β = 1.

5. The method according to claim 1, characterized in that, In step S3, the generated positive and negative sample images are labeled, and the labeled positive and negative sample images are preprocessed and data augmented, including: The target detection objects in the positive sample images are labeled with bounding boxes to indicate their location information, and the negative sample images are labeled as having no targets. The labeled positive and negative sample images are normalized to obtain preprocessed positive and negative sample images. Basic enhancement and physical simulation enhancement are performed on the preprocessed positive and negative sample images to obtain enhanced positive and negative sample images.

6. The method according to claim 5, characterized in that, The process of performing basic enhancement and physical simulation enhancement on the preprocessed positive and negative sample images to obtain enhanced positive and negative sample images includes: Basic enhancement is performed on the preprocessed positive and negative sample images by random rotation, scaling, and brightness adjustment. The enhanced positive and negative sample images are subjected to physical simulation enhancement to add interference terms, resulting in enhanced positive and negative sample images. The physical simulation enhancement to add interference terms includes: generating a realistic smoke layer based on a fluid dynamics model, simulating multi-source interference using a ray tracing algorithm, and adding dynamic occlusions through a generative adversarial network.

7. The method according to claim 1, characterized in that, The CV mini-model uses the YOLOv5 model, and before model training in step S4, the anchor point parameters of the YOLOv5 model are specifically optimized, including: The bounding box width and height data of the target detection objects in the generated positive sample images are statistically analyzed, the aspect ratio distribution range is calculated, and valid samples are selected. The K-means clustering algorithm is used to perform cluster analysis on the bounding boxes of the effective samples to generate optimized microscale anchor boxes; The optimized microscale anchor boxes are bound to the deepest feature map of the YOLOv5 model.

8. The method according to claim 1, characterized in that, In step S4, model training is performed based on the preprocessed and data-augmented positive and negative sample images, which also includes: After each training cycle, the detection loss value of all training samples is calculated, and the detection loss values ​​are sorted to generate a detection loss value sequence. The detection loss value sequence is truncated according to a preset ratio, and the sample corresponding to the truncated detection loss value is determined as the original difficult sample; Based on the visual feature defects of the original difficult samples, a quantitative description of the defect features is added to the original text description to generate an enhanced text description set. The enhanced text description set is input into the optimized text-generated graph model to generate a set of difficult enhancement samples. The original hard samples and the enhanced hard sample set are mixed and then incorporated into the total sample set for the next round of training according to a preset mixing ratio.

9. The method according to claim 1, characterized in that, In step S5, the evaluation of the trained CV small model includes: A tiered test sample dataset is constructed for the target detection object and its surrounding scene; Calculate the recall rate for each level of sample data and the corresponding detection decay rate, wherein the recall rate includes low-level recall rate or high-level recall rate; If the evaluation results do not meet the preset conditions, the specific level that did not meet the standard is located, and supplementary samples of the corresponding level are generated through the optimized Wenshengtu large model before returning to step S4 for retraining. If the evaluation results meet the preset conditions, the trained CV mini-model is saved. The preset conditions are that the lowest level recall rate is greater than the first preset threshold, the highest level recall rate is greater than the second preset threshold, and the detection decay rate is not higher than the third threshold.

10. A system for improving the generalization ability of small computer vision models based on text-based graph models, characterized in that, include: The Wensheng Image Large Model Construction and Optimization Module is used to construct and optimize the Wensheng Image Large Model. The text description generation module is used to construct positive sample text descriptions and negative sample text descriptions based on the features of the target detection object and the scene requirements, according to the text-generated image big model. The positive sample text descriptions contain target detection object and scene information, while the negative sample text descriptions have the same scene information as the positive sample text descriptions but do not contain the target detection object. The sample image generation and enhancement module is used to input the positive sample text description and the negative sample text description into the optimized text-to-image large model to generate corresponding positive sample images and negative sample images, to annotate the generated positive sample images and negative sample images, and to preprocess and enhance the annotated positive sample images and negative sample images. The CV mini-model training module is used to load CV mini-models and train them based on preprocessed and data-augmented positive and negative sample images. The CV small model evaluation module is used to evaluate the trained CV small model. If the evaluation result does not meet the preset conditions, the hyperparameters are adjusted and the process returns to step S4 for retraining. If the evaluation result meets the preset conditions, the trained CV small model is saved.

Images