Building anomaly detection data set generation method based on multi-modal model
Through the data set generation method based on multimodal model, the problem of difficulty in obtaining abnormal detection data sets and insufficient coverage in smart buildings is solved, and the automatic generation of high-quality and diversified data sets is realized, which improves detection accuracy and system intelligence. change.
Patent Information
- Application Number
- CN202410982439.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing smart building anomaly detection technology is difficult to obtain high-quality anomaly event image data sets, and existing data cannot fully cover all possible anomaly event types, resulting in insufficient detection accuracy and robustness, and privacy and data protection become challenges.
Using a data set generation method based on multimodal model, the data library of exceptional scenes is collected by establishing a multimodal model, and the image data set is automatically generated by using the model to verify and apply it to existing algorithms, enhancing the diversity of data and Coverage.
It improves the accuracy and robustness of the anomaly detection algorithm, generates a rich and diverse data set, reduces the difficulty and cost of data acquisition, enhances the intelligence and automation of the system, and improves the response speed and processing capabilities.
Smart Images

Figure CN119068285A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of smart buildings and artificial intelligence, and in particular relates to a method for generating a building anomaly detection data set based on a multimodal model. Background Art
[0002] At present, anomaly detection in smart buildings mainly relies on sensor monitoring or monitoring systems based on image recognition. The monitoring system collects real-time image data through cameras installed inside the building and processes these image data using computer vision technology to detect and identify abnormal events such as smoke and fire. These processing technologies usually include steps such as mobile target detection, image segmentation, feature extraction, and pattern recognition. Some systems also use deep learning-based algorithms, such as convolutional neural networks (CNNs), to further improve detection accuracy.
[0003] However, there are several problems with these existing technologies. First, it is difficult to obtain high-quality abnormal event image datasets because abnormal events are not common in reality. Second, in actual applications, abnormal events are of various types, and existing data cannot fully cover all possible abnormal event types, resulting in the system being unable to effectively identify unseen abnormal types. Finally, privacy and data protection are also challenges faced by existing technologies, especially when it comes to the storage and processing of surveillance image data.
[0004] In terms of data collection, traditional data enhancement methods such as image rotation, flipping, cropping, and color adjustment can enrich the data set to a certain extent, but may not be enough to simulate complex scenes in the real world. Therefore, there is still a lot of room for improvement in the accuracy, robustness, and data coverage of anomaly detection in existing technologies.
[0005] To this end, a building anomaly detection dataset generation method based on a multimodal model is designed to overcome the above problems. Summary of the invention
[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for generating a building anomaly detection data set based on a multimodal model. The present invention can enhance the accuracy and robustness of the anomaly detection algorithm, achieve more comprehensive recognition and response to various abnormal situations, and automatically generate a large amount of high-quality and diverse abnormal image data.
[0007] The present invention is implemented by the following technical solution: a method for generating a building anomaly detection data set based on a multimodal model, the method comprising the following steps: 1) Establish a large multimodal model; 2) Collect abnormal scene text libraries and add them to the large model as the basis for generating image data; 3) Automatically generate image datasets; 4) Verification and application of generated data sets.
[0008] Preferably, the method for establishing the multimodal large model in step 1) is to use an open source multimodal large model, and fine-tune or train it in combination with image and text data of various scenes to ensure that the model can adapt to specific abnormal scenes of smart buildings.
[0009] Preferably, the step 2) collects an abnormal scene text library and adds it to the large model as the basis for generating image data. The specific method is as follows: by collecting detailed description texts of various building abnormalities, a comprehensive abnormal scene text library is constructed, which covers a variety of abnormal types, including smoke, fire, flooding, and illegal intrusion, as the basis for generating image data; the description text is a picture downloaded from the Internet, existing documents, events that have occurred or news reports, text extracted from voice, and text converted from video extracted voice.
[0010] Preferably, the method for automatically generating an image data set in step 3) is as follows: inputting scene text prompt words for abnormality detection into the multimodal large model, and using the model to generate corresponding image data.
[0011] As a preferred embodiment, the specific method for verifying and applying the generated data set in step 4) is: 1) Multiple image data will be generated through step 3); 2) Perform preliminary screening on the generated multiple graphic data, and filter out unqualified images through filtering algorithms, such as uncleaned, erroneous images, and images with abnormal pixels; 3) Apply it to the anomaly detection algorithm corresponding to existing smart buildings for training and testing.
[0012] Compared with the existing building anomaly detection dataset acquisition technology, the method and system for generating a building anomaly detection dataset based on a multimodal model provided by the present invention have the following beneficial effects: 1. Improved data diversity: The system is able to generate a large number of diverse anomaly images, providing a richer training dataset than what can be collected in reality.
[0013] 2. Enhanced anomaly detection performance: Due to the richness and diversity of training data, the accuracy and robustness of anomaly detection algorithms in detecting abnormal events in smart buildings are improved.
[0014] 3. Reduce the difficulty and cost of data collection: Automatically generating data sets can reduce the difficulty of collecting data in the field, especially collecting data sets in dangerous scenarios, reducing costs and improving the efficiency and safety of data acquisition.
[0015] 4. Enhance the intelligence and automation of the system: The data generated by this system can assist in building a more intelligent and automated building management system to improve response speed and processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a schematic diagram of the process structure of the present invention. DETAILED DESCRIPTION
[0017] In order to enable those skilled in the art to more clearly understand the objectives, technical solutions and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0018] In the description of the present invention, it is necessary to understand that the orientation or position relationship indicated by the terms such as "up", "down", "left", "right", "inside", "outside", "horizontal" and "vertical" is based on the orientation or position relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention, rather than indicating or implying that the device or original referred to must have a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0019] The present invention will be described in detail below with reference to the accompanying drawings: Figure 1 As shown, a method for generating a building anomaly detection dataset based on a multimodal model comprises the following steps: 1) Establish a large multimodal model; 2) Collect abnormal scene text libraries and add them to the large model as the basis for generating image data; 3) Automatically generate image datasets; 4) Verification and application of generated data sets.
[0020] The method for establishing the multimodal large model in step 1) is to use an open source multimodal large model, and fine-tune or train it in combination with image and text data of various scenes to ensure that the model can adapt to specific abnormal scenes of smart buildings.
[0021] The specific method of step 2) collecting abnormal scene text library and adding it to the large model as the basis for generating image data is as follows: by collecting detailed description texts of various building abnormal conditions, a comprehensive abnormal scene text library is constructed, which covers a variety of abnormal types, including smoke, fire, flooding, and illegal intrusion, as the basis for generating image data; the description text is a picture downloaded from the Internet, existing documents, events that have occurred or news reports, text extracted from voice, and text converted from voice extracted from video.
[0022] The method of automatically generating the image data set in step 3) is: inputting scene text prompt words for abnormal detection into the multimodal large model, and using the model to generate corresponding image data.
[0023] The specific method for verifying and applying the generated data set in step 4) is as follows: 1) Multiple image data will be generated through step 3); 2) Perform preliminary screening on the generated multiple graphic data, and filter out unqualified images through filtering algorithms, such as uncleaned, erroneous images, and images with abnormal pixels; 3) Apply it to the anomaly detection algorithm corresponding to existing smart buildings for training and testing.
[0024] The method for generating a building anomaly detection dataset based on a multimodal model designed by the present invention can achieve improved data diversity: the system can generate a large number of diverse abnormal situation images, thereby providing a richer training dataset than the actual collection. The present invention can achieve enhanced anomaly detection performance: due to the richness and diversity of the training data, the accuracy and robustness of the anomaly detection algorithm in detecting abnormal events in smart buildings are improved. The present invention can reduce the difficulty and cost of data collection: the automatic generation of datasets can reduce the difficulty of field data collection, especially the collection of datasets in dangerous scenarios, reduce costs and improve the efficiency and safety of data acquisition. The present invention can enhance the intelligence and automation of the system: the data generated by this system can assist in building a more intelligent and automated building management system to improve response speed and processing capabilities.
[0025] The specific embodiments described herein are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.
Claims
1. A method for generating a building anomaly detection dataset based on a multimodal model, characterized in that: The method comprises the following steps: 1) Establish a large multimodal model; 2) Collect abnormal scene text libraries and add them to the large model as the basis for generating image data; 3) Automatically generate image datasets; 4) Verification and application of generated data sets.
2. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The method for establishing the multimodal large model in step 1) is to use an open source multimodal large model, and fine-tune or train it in combination with image and text data of various scenes to ensure that the model can adapt to specific abnormal scenes of smart buildings.
3. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The specific method of step 2) collecting abnormal scene text library and adding it to the large model as the basis for generating image data is as follows: by collecting detailed description texts of various building abnormal conditions, a comprehensive abnormal scene text library is constructed, which covers a variety of abnormal types, including smoke, fire, flooding, and illegal intrusion, as the basis for generating image data; the description text is a picture downloaded from the Internet, existing documents, events that have occurred or news reports, text extracted from voice, and text converted from voice extracted from video.
4. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The method of automatically generating the image data set in step 3) is: inputting scene text prompt words for abnormal detection into the multimodal large model, and using the model to generate corresponding image data.
5. The method for generating a building anomaly detection dataset based on a multimodal model according to claim 1, characterized in that: The specific method for verifying and applying the generated data set in step 4) is as follows: 1) Multiple image data will be generated through step 3); 2) Preliminary screening of the generated multiple graphic data, filtering out unqualified images, such as those that are not cleaned or have errors, through filtering algorithms; Images with abnormal pixels; 3) Apply it to the anomaly detection algorithm corresponding to existing smart buildings for training and testing.
Citation Information
Cited By
Abnormity early warning method and system based on multi-modal large model, medium and equipment
CN121349717A
