Smoke image analysis method and system, storage medium, equipment and product
By combining a fine-grained smoke detection model with a multimodal large model and utilizing text spatial feature similarity comparison, the smoke hazard level and its handling plan can be determined quickly and accurately. This solves the problems of cumbersome detection steps and poor text readability in existing technologies, and improves the accuracy and efficiency of smoke detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2026-03-10
AI Technical Summary
In existing technologies, the output of smoke detection models needs to be compared with a database to determine the hazard level. This process is cumbersome and cannot meet the timeliness requirements of fire source control. Furthermore, the text information is poorly readable, which delays fire rescue opportunities.
By combining a fine-grained smoke detection model with a multimodal large model, and processing smoke detection box images and text information, the smoke hazard level and its response plan can be quickly and accurately determined by comparing text spatial feature similarity.
It improves the accuracy and efficiency of smoke detection, provides highly readable auxiliary information, and meets the requirements for timely fire source control.
Smart Images
Figure CN121640357A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of image processing, and particularly relates to a smoke image analysis method, system, storage medium, device and product. BACKGROUND
[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.
[0003] With the wide application of deep learning algorithms in various industries, a smoke target detection model based on deep learning can process the set scene image, so that a smoke detection frame image or concentration type information is generally obtained. If it is desired to understand the current smoke level and the corresponding solution measures, the traditional method is to compare the output result of the deep learning model with the corresponding dangerous level image, and then determine the dangerous level, and then according to the dangerous level, the corresponding solution measures are matched and searched in the set database, so that the steps are relatively complicated, time-consuming, and cannot meet the requirements of timeliness and rationality of fire source control.
[0004] In addition, the corresponding solution measures of smoke of different threat levels currently rely on the database constructed based on historical data, and lack analysis of specific smoke in specific scenes. Moreover, the result after processing the set scene image contains text information, but the readability of the text information is poor, so that the on-site disposal personnel cannot quickly obtain valuable auxiliary disposal information from the text information, thereby delaying the opportunity of fire rescue. SUMMARY
[0005] In order to solve the above technical problems, the present application provides a smoke image analysis method, system, storage medium, device and product, which can quickly and accurately detect the threat level of smoke to achieve the purpose of timely controlling the fire source.
[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:
[0007] The first aspect of the present application provides a smoke image analysis method.
[0008] In one or more embodiments, a smoke image analysis method is provided, comprising:
[0009] The smoke image to be processed is acquired, the smoke fine-grained detection model is used to process the smoke image to be processed, a smoke detection frame image and smoke concentration text information are generated, and then the smoke concentration text information is filled into a preset condition disposal problem template to form a text problem instruction.
[0010] The smoke detection frame image and the text question command are processed using a multimodal large model to obtain the hazard level status and handling plan corresponding to the smoke image to be processed;
[0011] The aforementioned hazard levels and their corresponding handling solutions are mapped to the text space to form text space features;
[0012] Several preset standard spatial features are compared with the text spatial features for similarity. The preset structured text corresponding to the standard spatial feature with the highest similarity to the text spatial feature is selected as the final analysis result of the smoke image to be processed.
[0013] In one implementation, the fine-grained smoke detection model includes a smoke detection model and a smoke concentration classification model; the smoke detection model is used to mark smoke detection boxes in the smoke image to be processed; the smoke concentration classification model is used to classify the image within the smoke detection box into different concentrations of smoke.
[0014] The advantage of the above technical solution is that by using a fine-grained smoke detection model, not only can smoke be detected, but also smoke of different concentrations can be distinguished, thus improving the accuracy of smoke detection.
[0015] As one implementation method, a large text model encoder is used to map the hazard level and its handling plan into the text space.
[0016] The advantage of the above technical solution is that it solves the problem of the difficulty in processing and analyzing unstructured text content output by multimodal large models, laying the foundation for subsequent structured text processing.
[0017] As one implementation method, a text similarity measurement model is used to measure the similarity between the standard spatial features and the text spatial features.
[0018] The advantage of the above technical solution is that it can quickly and accurately filter out the corresponding structured text, improve the efficiency of smoke disposal, and provide valuable auxiliary disposal information for on-site personnel.
[0019] As one implementation method, the text similarity calculation method in the text similarity measurement model is as follows:
[0020] The vector corresponding to the text space feature is multiplied on the left by the transpose of the vector corresponding to the standard space feature; wherein the vector corresponding to the text space feature and the vector corresponding to the standard space feature are both one-dimensional row vectors or one-dimensional column vectors with the same structure.
[0021] The advantage of the above technical solution is that it uses the product of one-dimensional vectors corresponding to two spatial features to represent text similarity, which simplifies the calculation process and ensures the accuracy of the screening results.
[0022] As one implementation, the situation handling problem template includes questions about smoke hazard levels and handling solutions.
[0023] The advantage of the above technical solution is that the problems in the situation handling problem template can be preset by those skilled in the art according to the actual situation to meet personalized needs.
[0024] A second aspect of the present invention provides a smoke image analysis system.
[0025] In one or more embodiments, a smoke image analysis system includes:
[0026] The fine-grained detection module is used to acquire the smoke image to be processed, process the smoke image to be processed using the fine-grained smoke detection model, generate a smoke detection box image and smoke concentration text information, and then fill the smoke concentration text information into a preset situation handling problem template to form a text problem instruction.
[0027] A multimodal processing module is used to process the smoke detection box image and the text question command using a multimodal large model to obtain the hazard level status and disposal plan corresponding to the smoke image to be processed.
[0028] The text mapping module is used to map the hazard level and its handling plan to a text space to form text space features;
[0029] The similarity comparison module is used to compare the similarity of several preset standard spatial features with the text spatial features, and select the preset structured text corresponding to the standard spatial feature with the highest similarity to the text spatial features as the final analysis result of the smoke image to be processed.
[0030] In one implementation, the fine-grained detection module includes a smoke detection model and a smoke concentration classification model; the smoke detection model is used to mark smoke detection boxes in the smoke image to be processed; the smoke concentration classification model is used to classify the image within the smoke detection box into different concentrations of smoke.
[0031] The advantage of the above technical solution is that by using a fine-grained smoke detection model, not only can smoke be detected, but also smoke of different concentrations can be distinguished, thus improving the accuracy of smoke detection.
[0032] In other embodiments, a smoke image analysis system includes:
[0033] Surveillance devices, image processing devices, and monitoring devices;
[0034] The monitoring device is used to capture smoke images of a set scene to be processed and transmit them to the image processing device;
[0035] The image processing apparatus is configured as follows:
[0036] The smoke image to be processed is processed using a fine-grained smoke detection model to generate a smoke detection box image and smoke concentration text information. The smoke concentration text information is then filled into a preset situation handling question template to form a text question instruction.
[0037] The smoke detection frame image and the text question command are processed using a multimodal large model to obtain the hazard level status and handling plan corresponding to the smoke image to be processed;
[0038] The aforementioned hazard levels and their corresponding handling solutions are mapped to the text space to form text space features;
[0039] Several preset standard spatial features are compared with the text spatial features for similarity. The preset structured text corresponding to the standard spatial feature with the highest similarity to the text spatial feature is selected as the final analysis result of the smoke image to be processed and transmitted to the monitoring device.
[0040] The monitoring device is used to receive and display the final analysis results of the smoke image to be processed.
[0041] A third aspect of the present invention provides an electronic device.
[0042] In one or more embodiments, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the smoke image analysis method described above.
[0043] A fourth aspect of the present invention provides a computer-readable storage medium.
[0044] In one or more embodiments, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the smoke image analysis method as described above.
[0045] A fifth aspect of the present invention provides a computer program product.
[0046] In one or more embodiments, a computer program product includes a computer program that, when executed by a processor, implements the steps of the smoke image analysis method as described above.
[0047] Compared with the prior art, the beneficial effects of the present invention are:
[0048] This invention utilizes a combination of a fine-grained smoke detection model and a multimodal large model to analyze smoke concentration, smoke hazard levels, and corresponding response plans. This approach fully leverages the rational allocation and optimization of resources from both large and small models to achieve fine-grained smoke detection. The combination of fine-grained smoke detection and the multimodal large model enables hazard level classification. Furthermore, by employing text space and feature similarity comparison, the multimodal large model achieves structured text output, which not only improves the readability of information on smoke hazard situations and corresponding response measures but also meets the requirement for timely fire source control. Attached Figure Description
[0049] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0050] Figure 1 This is a schematic flowchart of a smoke image analysis method according to an embodiment of the present invention;
[0051] Figure 2 This is a flowchart of the smoke fine-grained detection model processing smoke images according to an embodiment of the present invention;
[0052] Figure 3 This is a flowchart of the multimodal large model output structured text according to an embodiment of the present invention;
[0053] Figure 4 This is a schematic diagram of the structure of a smoke image analysis system according to an embodiment of the present invention;
[0054] Figure 5 This is a schematic diagram of another smoke image analysis system structure according to an embodiment of the present invention;
[0055] Figure 6 This is a schematic diagram of the electronic device structure according to an embodiment of the present invention. Detailed Implementation
[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0057] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0058] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0059] Example 1
[0060] Figure 1 This is a flowchart illustrating a smoke image analysis method according to an embodiment of the present invention, as shown below. Figure 1 The smoke image analysis method shown in this embodiment may include:
[0061] S101, Obtain the smoke image to be processed, process the smoke image to be processed using the fine-grained smoke detection model, generate a smoke detection box image and smoke concentration text information, and then fill the smoke concentration text information into a preset situation handling problem template to form a text problem instruction.
[0062] S102, using a multimodal large model to process the smoke detection box image and the text question command to obtain the hazard level status and handling plan corresponding to the smoke image to be processed;
[0063] S103, map the aforementioned danger level and its handling plan to the text space to form text space features;
[0064] S104, compare the similarity of several preset standard spatial features with the text spatial features, and select the preset structured text corresponding to the standard spatial feature with the highest similarity to the text spatial features as the final analysis result of the smoke image to be processed.
[0065] This embodiment combines a fine-grained smoke detection model with a multimodal large model to analyze smoke concentration, smoke hazard levels, and corresponding response plans. This fully utilizes the reasonable allocation and optimization of resources from both large and small models to achieve fine-grained smoke detection. The combination of fine-grained smoke detection and the multimodal large model enables hazard level classification. Furthermore, by using text space and feature similarity comparison, the multimodal large model achieves structured text output, which not only improves the readability of information on smoke hazard situations and corresponding response measures but also meets the requirement of timely fire source control.
[0066] It should be noted that the smoke image can be a smoke image of a set scene, such as a scene image of a power transmission line, a scene image inside a substation, or a scene image of other areas to be monitored. Those skilled in the art can set it according to the actual situation, which will not be described in detail here.
[0067] The following section uses smoke images in a power transmission line scenario as an example to illustrate the specific implementation process of the smoke image analysis method in this embodiment.
[0068] Figure 2 A flowchart of the smoke fine-grained detection model for processing smoke images in this embodiment is provided.
[0069] In step S101, according to Figure 2 The fine-grained smoke detection model in this embodiment includes a smoke detection model and a smoke concentration classification model. The smoke detection model is used to mark smoke detection boxes in the smoke image to be processed. The smoke concentration classification model is used to classify the smoke within the smoke detection boxes into different concentrations of smoke. Thus, using the fine-grained smoke detection model not only detects smoke but also distinguishes between smoke of different concentrations, improving the accuracy of smoke detection.
[0070] The smoke detection model can be trained using a YoloV8 model, or other target detection network models can be selected according to the actual situation.
[0071] Smoke concentration classification models can be constructed using lightweight classification models such as PPHGNet or MobileNet. Furthermore, the smoke concentration classification model outputs a smoke concentration category using color; for example, when the smoke concentration is greater than or equal to 80%, it is represented as black smoke.
[0072] This approach, combining lightweight target detection with cascaded classification, enables the rational allocation and optimized use of resources, as well as the detection of fine-grained smoke.
[0073] It should be noted that both the smoke detection model and the smoke concentration classification model can be trained based on a set of samples.
[0074] In step S101, the situation handling problem template includes questions about smoke hazard levels and handling solutions. The questions in the situation handling problem template can be pre-set by those skilled in the art to meet individual needs based on actual circumstances.
[0075] For example, a situation handling question template is: Does the "***" in the picture pose a threat to the power transmission line? How should this situation be handled?
[0076] The "***" in the situation handling template is filled with the text information of smoke concentration output by the fine-grained smoke detection model. For example, the text information of smoke concentration output by the fine-grained smoke detection model is the black smoke within the red box.
[0077] In step S102, the multimodal large model can be constructed using multimodal large models such as LLava or QwenVL, and those skilled in the art can set it according to the actual situation.
[0078] It should be noted here that multimodal large models can be pre-trained based on a preset sample set.
[0079] Specifically, the hazard level of the smoke can be predetermined based on the actual situation. For example, it can be set to level 0, level 1, level 2, level 3, etc., according to the hazard level increasing step by step.
[0080] Each hazard level has a corresponding corresponding response plan. For example, when the hazard level is 0, no professional personnel are required to handle the situation on-site.
[0081] In step S103, the text content C output by the multimodal large model is unstructured data, such as... Figure 3 As shown, unstructured data exists in flexible and diverse forms, making it difficult to process and analyze using traditional database management systems. To reduce the complexity of subsequent text processing and analysis, it is necessary to structure the text content output by a large multimodal model.
[0082] In some alternative embodiments, a large text model encoder is used to map the hazard level and its handling scheme to a text space. For example, the large text model encoder is constructed using the CLIP text encoder model.
[0083] Specifically, the text large model encoder is represented by f; the text content output by the multimodal large model, i.e., the danger level and its handling plan, is represented by C; and the text space features are represented by the vector y.
[0084] y=f(C)(1)
[0085] In step S104, a text similarity measurement model is used to measure the similarity between the standard spatial features and the text spatial features. This enables the rapid and accurate filtering of corresponding structured text, improving the efficiency of smoke control and providing valuable auxiliary information for on-site personnel.
[0086] The text similarity measurement model is represented in the following way:
[0087] Text similarity is calculated as the left product of the vector corresponding to the text spatial feature and the transpose of the vector corresponding to the standard spatial feature; wherein the vectors corresponding to the text spatial features and the vectors corresponding to the standard spatial features are both one-dimensional row / column vectors with the same structure. This method of representing text similarity through the product of the one-dimensional vectors corresponding to two spatial features simplifies the calculation process and ensures the accuracy of the selection results.
[0088] Specifically, text similarity is calculated using S(y,y) i To represent:
[0089]
[0090] Among them, y i This represents the vector corresponding to the i-th preset standard spatial feature; This represents the transpose of the vector corresponding to the i-th preset standard spatial feature.
[0091] Each preset standard spatial feature corresponds to a structured text, such as Figure 3 The structured text in the text includes scenarios such as: Scenario 1: The current accident poses a high threat to the conductor; Scenario 2: The current accident poses a low threat to the conductor.
[0092] It is understood here that in other embodiments, text similarity may also be measured in other ways, such as cosine distance or Euclidean distance. Those skilled in the art can choose the specific form of text similarity measurement according to the actual situation, which will not be described in detail here.
[0093] This embodiment utilizes a text similarity calculation model to calculate the similarity score between the text generated by the multimodal large model and other preset structured texts. The structured text with the highest similarity score to the preset structured text is selected as the structured text generated by the multimodal large model, thereby completing the structuring of the multimodal large model-generated text.
[0094] Taking power transmission scenarios as an example, among 5,000 data images of smoke alarms collected in power transmission scenarios, the COCOAPI testing tool showed that the mAP (mean average precision) of fine-grained smoke detection was 0.61, and the accuracy rate of hazard level classification was 94%.
[0095] This embodiment utilizes a fine-grained smoke detection model and a multimodal large model to process smoke images, obtain the hazard level and its handling plan, and then combines the multimodal large model with a text large model encoder and a text similarity measurement model to improve the efficiency of smoke handling and provide valuable auxiliary handling information for on-site personnel.
[0096] Example 2
[0097] Figure 4 This is a schematic diagram of a smoke image analysis system according to an embodiment of the present invention. This embodiment is similar to... Figure 1 Corresponding to the smoke image analysis methods, such as Figure 4 As shown, the smoke image analysis system in this embodiment may include:
[0098] The fine-grained detection module 401 is used to acquire a smoke image to be processed, process the smoke image to be processed using a fine-grained smoke detection model, generate a smoke detection box image and smoke concentration text information, and then fill the smoke concentration text information into a preset situation handling problem template to form a text problem instruction.
[0099] The multimodal processing module 402 is used to process the smoke detection box image and the text question command using a multimodal large model to obtain the hazard level status and disposal plan corresponding to the smoke image to be processed.
[0100] The text mapping module 403 is used to map the hazard level and its handling plan to a text space to form text space features;
[0101] The similarity comparison module 404 is used to compare the similarity of several preset standard spatial features with the text spatial features, and select the preset structured text corresponding to the standard spatial feature with the highest similarity to the text spatial features as the final analysis result of the smoke image to be processed.
[0102] Specifically, in the fine-grained detection module 401, the smoke fine-grained detection model includes a smoke detection model and a smoke concentration classification model; the smoke detection model is used to mark smoke detection boxes in the smoke image to be processed; the smoke concentration classification model is used to classify the image within the smoke detection box into different concentrations of smoke. In this way, the smoke fine-grained detection model can not only detect smoke but also distinguish smoke of different concentrations, improving the accuracy of smoke detection.
[0103] Specifically, in the text mapping module 403, the hazard level and its handling plan are mapped to the text space using a large text model encoder. This solves the problem of the difficulty in processing and analyzing unstructured text content output by multimodal large models, laying the foundation for subsequent structured text.
[0104] In the text mapping module 403, a text similarity measurement model is used to measure the similarity between the standard spatial features and the text spatial features. This enables the rapid and accurate filtering of corresponding structured text, improving the efficiency of smoke control and providing valuable auxiliary information for on-site personnel.
[0105] The text similarity calculation method in the text similarity measurement model is as follows:
[0106] The vector corresponding to the text space feature is left-multiplied by the transpose of the vector corresponding to the standard space feature;
[0107] Wherein, the vectors corresponding to the text space features and the vectors corresponding to the standard space features are both one-dimensional row vectors or one-dimensional column vectors with the same structure.
[0108] It should be noted that the four modules in this embodiment—fine-grained detection module 401, multimodal processing module 402, text mapping module 403, and similarity comparison module 404—correspond one-to-one with steps S101 to S104 in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.
[0109] Example 3
[0110] like Figure 5 As shown, this embodiment provides a smoke image analysis system, including:
[0111] The monitoring device 501, the image processing device 502, and the surveillance device 503;
[0112] The monitoring device 501 is used to capture smoke images of a set scene to be processed and transmit them to the image processing device;
[0113] The image processing device 502 is configured as follows:
[0114] The smoke image to be processed is processed using a fine-grained smoke detection model to generate a smoke detection box image and smoke concentration text information. The smoke concentration text information is then filled into a preset situation handling question template to form a text question instruction.
[0115] The smoke detection frame image and the text question command are processed using a multimodal large model to obtain the hazard level status and handling plan corresponding to the smoke image to be processed;
[0116] The aforementioned hazard levels and their corresponding handling solutions are mapped to the text space to form text space features;
[0117] Several preset standard spatial features are compared with the text spatial features for similarity. The preset structured text corresponding to the standard spatial feature with the highest similarity to the text spatial feature is selected as the final analysis result of the smoke image to be processed and transmitted to the monitoring device.
[0118] The monitoring device 503 is used to receive and display the final analysis results of the smoke image to be processed.
[0119] In this embodiment, the monitoring device 501 can be implemented using a camera.
[0120] The image processing device 502 can be implemented using a chip with image processing capabilities;
[0121] The monitoring device 503 can be implemented using electronic devices with display or alarm functions.
[0122] The image processing device 502 can be connected to the monitoring device 503 via a network device 504 (such as a wireless communication device or a wired communication extension device).
[0123] It is understood here that the image processing device 502 may also be integrated with the monitoring device 501, and those skilled in the art can make specific settings according to the actual situation.
[0124] When the image processing device 502 and the monitoring device 501 are integrated into one unit, the monitoring device 501 can be directly connected to the monitoring device 503 through the network device 504.
[0125] In this embodiment, each step in the image processing device 502 corresponds one-to-one with each step in Embodiment 1, and their specific implementation process is the same, so it will not be described in detail here.
[0126] Example 4
[0127] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements... Figure 1 The steps of the smoke image analysis method shown are as follows.
[0128] Reference Figure 6 A schematic diagram of an electronic device is provided. It should be noted that... Figure 6 The electronic device 600 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
[0129] like Figure 6 As shown, the electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for system operation. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0130] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a local area network (LAN) card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.
[0131] When the central processing unit 601 in the electronic device of this embodiment executes the program, it achieves the following: Figure 1 The steps in the smoke image analysis method shown.
[0132] Example 5
[0133] This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following... Figure 1 The steps of the smoke image analysis method shown are as follows.
[0134] in, Figure 1 The computer program instructions corresponding to the method shown may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0135] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0136] Example 6
[0137] Specifically, according to embodiments of this application, the processes described in the above-referenced flowcharts can be implemented as computer software programs. This embodiment provides a computer program product including a computer program carried on a computer-readable medium, the computer program comprising methods for executing... Figure 1 The program code for the method shown. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit 601, it performs the various functions defined in the apparatus of this application.
[0138] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method of smoke image analysis, characterized by, The method comprises the following steps: acquiring a smoke image to be processed, processing the smoke image to be processed by using a smoke fine-grained detection model, generating a smoke detection frame image and smoke concentration text information, filling the smoke concentration text information into a preset condition handling problem template to form a text problem instruction; processing the smoke detection frame image and the text problem instruction by using a multi-modal large model to obtain a dangerous level condition and a handling scheme corresponding to the smoke image to be processed; mapping the dangerous level condition and the handling scheme to a text space to form a text space feature; comparing a plurality of preset standard space features with the text space feature in terms of similarity, and screening a preset structured text corresponding to the standard space feature with the highest similarity to the text space feature as a final analysis result of the smoke image to be processed.
2. The smoke image analysis method of claim 1, wherein, The smoke fine-grained detection model comprises a smoke detection model and a smoke concentration classification model; the smoke detection model is used for labeling a smoke detection frame in the smoke image to be processed; and the smoke concentration classification model is used for classifying different smoke concentrations in the smoke detection frame.
3. The smoke image analysis method of claim 1, wherein, The dangerous level condition and the handling scheme are mapped to the text space by using a text large model encoder.
4. The smoke image analysis method of claim 1, wherein, A text similarity measurement model is used to measure the similarity between the standard space feature and the text space feature.
5. The smoke image analysis method of claim 4, wherein, The text similarity calculation manner in the text similarity measurement model is that: a vector corresponding to the text space feature is left multiplied by a transpose of a vector corresponding to the standard space feature; wherein the vector corresponding to the text space feature and the vector corresponding to the standard space feature are one-dimensional row vectors or one-dimensional column vectors with the same structure.
6. The smoke image analysis method of claim 1, wherein, The condition handling problem template comprises a smoke dangerous level problem and a handling scheme problem.
7. A smoke image analysis system characterized by, The method comprises the following steps: a fine-grained detection module is used for acquiring a smoke image to be processed, processing the smoke image to be processed by using a smoke fine-grained detection model, generating a smoke detection frame image and smoke concentration text information, and filling the smoke concentration text information into a preset condition handling problem template to form a text problem instruction; a multi-modal processing module is used for processing the smoke detection frame image and the text problem instruction by using a multi-modal large model to obtain a dangerous level condition and a handling scheme corresponding to the smoke image to be processed; a text mapping module is used for mapping the dangerous level condition and the handling scheme to a text space to form a text space feature; a similarity comparison module is used for comparing a plurality of preset standard space features with the text space feature in terms of similarity, and screening a preset structured text corresponding to the standard space feature with the highest similarity to the text space feature as a final analysis result of the smoke image to be processed.
8. The smoke image analysis system of claim 7, wherein, In the fine-grained detection module, the smoke fine-grained detection model comprises a smoke detection model and a smoke concentration classification model; the smoke detection model is used for labeling a smoke detection frame in the to-be-processed smoke image; and the smoke concentration classification model is used for classifying different smoke concentrations of the image in the smoke detection frame.
9. A smoke image analysis system characterized by, Comprise: A monitoring device, an image processing device, and a monitoring device; The monitoring device is used for shooting a to-be-processed smoke image of a set scene and transmitting the image to the image processing device; The image processing device is configured to: Process the to-be-processed smoke image by using a smoke fine-grained detection model, generate a smoke detection frame image and smoke concentration text information, fill the smoke concentration text information into a preset condition handling problem template to form a text problem instruction; Process the smoke detection frame image and the text problem instruction by using a multi-modal large model to obtain a dangerous level condition and a handling scheme corresponding to the to-be-processed smoke image; Map the dangerous level condition and the handling scheme to a text space to form a text space feature; Compare a plurality of preset standard space features with the text space feature in similarity, filter out a preset structured text corresponding to the standard space feature with the highest similarity to the text space feature as a final analysis result of the to-be-processed smoke image and transmit the result to the monitoring device; The monitoring device is used for receiving and displaying the final analysis result of the to-be-processed smoke image.
10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the smoke image analysis method in any one of claims 1-6.
11. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the steps of the smoke image analysis method in any one of claims 1-6.
12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the smoke image analysis method in any one of claims 1-6.