Building anomaly detection data set generation method based on multi-modal model

Through the data set generation method based on multimodal model, the problem of difficulty in obtaining abnormal detection data sets and insufficient coverage in smart buildings is solved, and the automatic generation of high-quality and diversified data sets is realized, which improves detection accuracy and system intelligence. change.

CN119068285AInactive Publication Date: 2024-12-03DEQING ALPHA INNOVATION RES INST
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410982439.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing smart building anomaly detection technology is difficult to obtain high-quality anomaly event image data sets, and existing data cannot fully cover all possible anomaly event types, resulting in insufficient detection accuracy and robustness, and privacy and data protection become challenges.

Method used

Using a data set generation method based on multimodal model, the data library of exceptional scenes is collected by establishing a multimodal model, and the image data set is automatically generated by using the model to verify and apply it to existing algorithms, enhancing the diversity of data and Coverage.

Benefits of technology

It improves the accuracy and robustness of the anomaly detection algorithm, generates a rich and diverse data set, reduces the difficulty and cost of data acquisition, enhances the intelligence and automation of the system, and improves the response speed and processing capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068285A_ABST
    Figure CN119068285A_ABST
Patent Text Reader

Abstract

The invention relates to a building anomaly detection data set generation method based on a multi-modal model. The method comprises the following steps: 1) establishing a multi-modal large model; 2) collecting an abnormal scene text library, and adding the abnormal scene text library into the large model to serve as a basis for generating image data; according to the method, the accuracy and the robustness of an anomaly detection algorithm can be enhanced, various abnormal conditions can be identified and responded more comprehensively, and a large amount of high-quality and diversified abnormal image data can be automatically generated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of smart buildings and artificial intelligence, and in particular relates to a method for generating a building anomaly detection data set based on a multimodal model. Background Art

[0002] At present, anomaly detection in smart buildings mainly relies on sensor monitoring or monitoring systems based on image recognition. The monitoring system collects real-time image data through cameras installed inside the building and processes these image data using computer vision technology to detect and identify abnormal events such as smoke and fire. These processing technologies usually include steps such as mobile target detection, image segmentation, feature extraction, and pattern recognition. Some systems also use deep learning-based algorithms, such as convolutional neural networks (CNNs), to further improve detection accuracy.

[0003] However, there are several problems with these existing technologies. First, it is difficult to obtain high-quality abnormal event image datasets because abnormal events are not common in reality. Second, in actual applications, abnormal events are of various types, and existing data cannot fully cover all possible abnormal event types, resulting in the system being unable to effectively identify unseen abnormal types. Finally, privacy and data protection are also challenges faced by existing technologies, especially when it comes to the storage and processing of surveillance image data.

[0004] In terms of data collection, traditional data enhancement methods such as image rotation, flipping, cropping, and color adjustment can enrich the data set to a certain extent, but may not be enough to simulate complex scenes in the real world. Therefore, there is still a lot of room for improvement in the accuracy, robustness, and data coverage of anomaly detection in existing technologies.

[0005] To this end, a building anomaly detection dataset generation method based on a multimodal model is designed to overcome the above problems. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for generating a building anomaly detection data set based on a multimodal model. The present invention can enhance the accuracy and robustness of the anomaly detection algorithm, achieve more comprehensive recognition and response to various abnormal situations, and automatically generate a large amount of high-quality and diverse abnormal image data.

[0007] The present invention is implemented by the following technical solution: a method for generating a building anomaly detection data set based on a multimodal model, the method comprising the following steps: 1) Establish a large multimodal model; 2) Collect abnormal scene text libraries and add them to the large model as the basis for generating image data; 3) Automatically generate image datasets; 4) Verification and application of generated data sets.

[0008] Preferably, the method for establishing the multimodal large model in step 1) is to use an open source multimodal large model, and fine-tune or train it in combination with image and text data of various scenes to ensure that the model can adapt to specific abnormal scenes of smart buildings.

[0009] Preferably, the step 2) collects an abnormal scene text library and adds it to the large model as the basis for generating image data. The specific method is as follows: by collecting detailed description texts of various building abnormalities, a comprehensive abnormal scene text library is constructed, which covers a variety of abnormal types, including smoke, fire, flooding, and illegal intrusion, as the basis for generating image data; the description text is a picture downloaded from the Internet, existing documents, events that have occurred or news reports, text extracted from voice, and text converted from video extracted voice.

[0010] Preferably, the method for automatically generating an image data set in step 3) is as follows: inputting scene text prompt words for abnormality detection into the multimodal large model, and using the model to generate corresponding image data.

[0011] As a preferred embodiment, the specific method for verifying and applying the generated data set in step 4) is: 1) Multiple image data will be generated through step 3); 2) Perform preliminary screening on the generated multiple graphic data, and filter out unqualified images through filtering algorithms, such as uncleaned, erroneous images, and images with abnormal pixels; 3) Apply it to the anomaly detection algorithm corresponding to existing smart buildings for training and testing.

[0012] Compared with the existing building anomaly detection dataset acquisition technology, the method and system for generating a building anomaly detection dataset based on a multimodal model provided by the present invention have the following beneficial effects: 1. Improved data diversity: The system is able to generate a large number of diverse anomaly images, providing a richer training dataset than what can be collected in reality.

[0013] 2. Enhanced anomaly detection performance: Due to the richness and diversity of training data, the accuracy and robustness of anomaly detection algorithms in detecting abnormal events in smart buildings are improved.

[0014] 3. Reduce the difficulty and cost of data collection: Automatically generating data sets can reduce the difficulty of collecting data in the field, especially collecting data sets in dangerous scenarios, reducing costs and improving the efficiency and safety of data acquisition.

[0015] 4. Enhance the intelligence and automation of the system: The data generated by this system can assist in building a more intelligent and automated building management system to improve response speed and processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the process structure of the present invention. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to more clearly understand the objectives, technical solutions and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0018] In the description of the present invention, it is necessary to understand that the orientation or position relationship indicated by the terms such as "up", "down", "left", "right", "inside", "outside", "horizontal" and "vertical" is based on the orientation or position relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention, rather than indicating or implying that the device or original referred to must have a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0019] The present invention will be described in detail below with reference to the accompanying drawings: Figure 1 As shown, a method for generating a building anomaly detection dataset based on a multimodal model comprises the following steps: 1) Establish a large multimodal model; 2) Collect abnormal scene text libraries and add them to the large model as the basis for generating image data; 3) Automatically generate image datasets; 4) Verification and application of generated data sets.

[0020] The method for establishing the multimodal large model in step 1) is to use an open source multimodal large model, and fine-tune or train it in combination with image and text data of various scenes to ensure that the model can adapt to specific abnormal scenes of smart buildings.

[0021] The specific method of step 2) collecting abnormal scene text library and adding it to the large model as the basis for generating image data is as follows: by collecting detailed description texts of various building abnormal conditions, a comprehensive abnormal scene text library is constructed, which covers a variety of abnormal types, including smoke, fire, flooding, and illegal intrusion, as the basis for generating image data; the description text is a picture downloaded from the Internet, existing documents, events that have occurred or news reports, text extracted from voice, and text converted from voice extracted from video.

[0022] The method of automatically generating the image data set in step 3) is: inputting scene text prompt words for abnormal detection into the multimodal large model, and using the model to generate corresponding image data.

[0023] The specific method for verifying and applying the generated data set in step 4) is as follows: 1) Multiple image data will be generated through step 3); 2) Perform preliminary screening on the generated multiple graphic data, and filter out unqualified images through filtering algorithms, such as uncleaned, erroneous images, and images with abnormal pixels; 3) Apply it to the anomaly detection algorithm corresponding to existing smart buildings for training and testing.

[0024] The method for generating a building anomaly detection dataset based on a multimodal model designed by the present invention can achieve improved data diversity: the system can generate a large number of diverse abnormal situation images, thereby providing a richer training dataset than the actual collection. The present invention can achieve enhanced anomaly detection performance: due to the richness and diversity of the training data, the accuracy and robustness of the anomaly detection algorithm in detecting abnormal events in smart buildings are improved. The present invention can reduce the difficulty and cost of data collection: the automatic generation of datasets can reduce the difficulty of field data collection, especially the collection of datasets in dangerous scenarios, reduce costs and improve the efficiency and safety of data acquisition. The present invention can enhance the intelligence and automation of the system: the data generated by this system can assist in building a more intelligent and automated building management system to improve response speed and processing capabilities.

[0025] The specific embodiments described herein are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for generating a building anomaly detection dataset based on a multimodal model, characterized in that: The method comprises the following steps: 1) Establish a large multimodal model; 2) Collect abnormal scene text libraries and add them to the large model as the basis for generating image data; 3) Automatically generate image datasets; 4) Verification and application of generated data sets.

2. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The method for establishing the multimodal large model in step 1) is to use an open source multimodal large model, and fine-tune or train it in combination with image and text data of various scenes to ensure that the model can adapt to specific abnormal scenes of smart buildings.

3. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The specific method of step 2) collecting abnormal scene text library and adding it to the large model as the basis for generating image data is as follows: by collecting detailed description texts of various building abnormal conditions, a comprehensive abnormal scene text library is constructed, which covers a variety of abnormal types, including smoke, fire, flooding, and illegal intrusion, as the basis for generating image data; the description text is a picture downloaded from the Internet, existing documents, events that have occurred or news reports, text extracted from voice, and text converted from voice extracted from video.

4. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The method of automatically generating the image data set in step 3) is: inputting scene text prompt words for abnormal detection into the multimodal large model, and using the model to generate corresponding image data.

5. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The specific method for verifying and applying the generated data set in step 4) is as follows: 1) Multiple image data will be generated through step 3); 2) Preliminary screening of the generated multiple graphic data, filtering out unqualified images, such as those that are not cleaned or have errors, through filtering algorithms; Images with abnormal pixels; 3) Apply it to the anomaly detection algorithm corresponding to existing smart buildings for training and testing.

Citation Information

Cited By

  • Abnormity early warning method and system based on multi-modal large model, medium and equipment

    CN121349717A