Infrared image data classification and filing method and platform

Abstract

The invention relates to the technical field of image processing and data management, and discloses an infrared image data classification filing method and platform, and the method comprises the following steps: 1, collecting infrared image data from a plurality of different types of infrared sensors, the sensors including an infrared thermal imaging camera, a visible light camera and a radar; 2, the collected infrared image data are preprocessed, noise removal, temperature correction and image enhancement are included, and wavelet transform and a deep convolutional neural network are adopted for noise removal; through intelligent classification and management, the infrared image data processing efficiency is improved, manual intervention is reduced, the error rate is reduced, the data storage and retrieval speed is optimized by adopting multi-level storage and distributed management, the storage cost is reduced, the data safety and reliability are guaranteed through redundant backup and data de-duplication, and the data storage and retrieval efficiency is improved. And long-time reliable storage and efficient recovery of the data are ensured.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing and data management, and particularly to an infrared image data classification and archiving method and platform. Background Art

[0002] At present, infrared imaging technology has been widely applied in various fields, such as security monitoring, medical detection, industrial inspection, etc. With the improvement of the performance of infrared imaging devices, the amount of infrared image data generated has increased rapidly. To cope with the growing amount of image data, many problems have gradually emerged in traditional image data management methods, especially in aspects such as data classification, storage, retrieval, and security management.

[0003] Existing image data management systems generally rely on traditional file systems or relational databases and use manual classification or simple image indexing mechanisms for storage. When dealing with a large number of infrared images, these systems face problems such as wasted storage space, data redundancy, low classification efficiency, and slow retrieval speed. For example, the manual classification method is not only time-consuming but also error-prone and difficult to meet the processing requirements of large-scale data; while the systems based on traditional databases have bottlenecks in storage structure and retrieval efficiency and are difficult to support complex image data queries and fast access. In addition, existing systems lack efficient protection for data security, and the risk of data loss or damage is relatively high. Therefore, there are obvious deficiencies in the classification, management, and retrieval of infrared image data in the prior art, and it cannot meet the requirements of high efficiency, intelligence, and security in modern infrared image data management. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides an infrared image data classification and archiving method and platform, which solves the problems of inefficient manual classification, wasted storage space, difficult data retrieval, and insufficient security existing in the management process of infrared image data in the prior art.

[0005] To achieve the above objectives, the present invention is realized through the following technical solutions: An infrared image data classification and archiving method and platform, including the following steps: Step 1: Collect infrared image data from multiple different types of infrared sensors, where the sensors include infrared thermal imaging cameras, visible light cameras, and radars; Step 2: Preprocess the collected infrared image data, including noise removal, temperature correction, and image enhancement. Among them, wavelet transform and deep convolutional neural network are used for noise removal, and temperature correction is based on the non-linear mapping of the difference between the ambient temperature and the image temperature; Step 3: Standardize the image data from multiple sensors to unify the pixel value range of the image data; Step 4: Extract features from the standardized image. The feature extraction includes manual feature extraction and deep learning feature extraction. Among them, manual feature extraction uses Histogram of Oriented Gradients (HOG) and Scale-Invariant Feature Transform (SIFT), and deep learning feature extraction is performed through a deep convolutional neural network; Step 5: Perform multimodal data fusion on the extracted image features, and use a weighted fusion method and a generative adversarial network to optimize the feature alignment between modalities; Step 6: Optimize the fused features, and update the network weights through the backpropagation algorithm to improve the accuracy of image classification; Step 7: Classify and label the optimized image features, generate labels and automatically update the label library; Step 8: Store the classified images and corresponding labels in a distributed storage system, and store the metadata of the images in a relational database to establish an index for the image data; Step 9: Through an intelligent retrieval system, retrieve the stored images based on the image content or labels. The retrieval methods include similarity retrieval based on image features and retrieval based on labels.

[0006] Preferably, the preprocessing of the image data includes image temperature correction through a deep neural network (DNN). Specifically: where, is the corrected image, is the ambient temperature, is the temperature of the image scene, is the correction coefficient.

[0007] Preferably, the feature extraction extracts high-level semantic features of the image through a deep convolutional neural network. The CNN network structure includes a self-attention mechanism to improve the feature representation ability.

[0008] Preferably, the multimodal data fusion is performed through a weighted fusion method. The weighted fusion method is calculated using the following formula: where, is the image feature of the th modality, is the weighted coefficient of modality , is the fused feature.

[0009] Preferably, the image classification uses a deep convolutional neural network and adopts a multi-task learning framework to simultaneously perform image classification, object detection, and segmentation tasks.

[0010] Preferably, the total loss function of the multi-task learning is: Among them, is the classification loss, is the detection loss, is the segmentation loss, and are the loss weights.

[0011] Preferably, the intelligent retrieval system is based on a content-based image retrieval method, and retrieves by calculating the cosine similarity between image features. The calculation formula is: An infrared image data classification and archiving platform, comprising: A data acquisition module, configured to acquire infrared image data from multiple different types of infrared sensors; A data preprocessing module, configured to denoise, temperature correct, image enhance, and standardize the acquired image data; A feature extraction module, configured to extract manual features and high-level features based on a deep convolutional neural network from the image; A multimodal data fusion module, configured to perform weighted fusion on data from different sensors to generate fused features; An image classification and annotation module, configured to classify the fused image features and generate corresponding image labels; A data storage and retrieval module, configured to store the classified images and labels in a distributed storage system and provide an intelligent retrieval function based on content and labels.

[0012] Preferably, the data storage and retrieval module includes a distributed storage system and a relational database. The relational database is used to store the metadata of the images and provide a fast query function.

[0013] Preferably, the image classification and annotation module is trained through a multi-task learning framework, supports simultaneously performing image classification, object detection, and image segmentation tasks, and automatically updates the label library according to the detection results.

[0014] The present invention provides an infrared image data classification and archiving method and platform. It has the following beneficial effects: 1. By combining the metadata of the image and the image content features, the present invention realizes the intelligent classification and precise management of infrared image data. Through machine learning algorithms and image processing technologies, the platform can automatically identify and classify and store different types of infrared image data, avoiding the inefficiency and error rate in the traditional manual classification and storage process. This method can effectively reduce manual intervention, improve data management efficiency, and ensure the standardization and automation of the archiving process. Especially in a large-scale data environment, it can greatly improve the processing ability of the system.

[0015] 2. The present invention adopts a multi-level storage strategy and a distributed database management mechanism. The system intelligently classifies and stores data according to the data access frequency and importance, ensuring that high-frequency data can be quickly responded to, while low-frequency data is archived through low-cost storage devices. This storage mode not only ensures the high efficiency of data storage but also reduces the storage cost. In addition, through the combination of keyword indexing and image content indexing, the system provides a powerful data retrieval function, enabling users to quickly locate and access the required image data according to multiple conditions, greatly improving the convenience and accuracy of data retrieval.

[0016] 3. By combining distributed redundant backup, snapshot backup, and cloud storage technologies, the present invention ensures the high security and reliability of infrared image data. The system designs a multiple backup mechanism that can cope with the risks of storage device failures or data corruption, ensuring that data can be completely restored under any circumstances. In addition, through a deduplication algorithm, redundant data storage is avoided, which not only saves storage resources but also improves the efficiency of data management. The overall security guarantee measures enable long-term reliable preservation during the data storage and management process, providing users with stable and secure data protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the specification of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0019] Embodiment: Please refer to the attached Figure 1 , the embodiment of the present invention provides an infrared image data classification and archiving method and platform, including the following steps: Step 1: Collect infrared image data from multiple different types of infrared sensors, where the sensors include infrared thermal imaging cameras, visible light cameras, and radars; In this embodiment, data collection is the primary step of the infrared image data classification and archiving method, aiming to obtain infrared image data from multiple different types of sensors. This step is of great significance for subsequent image processing, feature extraction, and classification. Specifically, the data collection module is mainly responsible for collecting image data from different types of infrared sensor devices, and the sensors include but are not limited to infrared thermal imaging cameras, visible light cameras, lidar (LiDAR), etc.

[0020] As an option, an infrared thermal imaging camera can generate infrared images by capturing the temperature radiation information on the surface of an object. It has good capabilities for low-light and long-range detection, and is particularly suitable for applications such as fault detection and night monitoring in different industries. Specifically, an infrared thermal imaging camera can generate thermal images by converting the thermal energy radiated by an object into an electrical signal, and these images can be used for precise analysis of temperature changes.

[0021] In another possible implementation, in addition to the infrared thermal imaging camera, other sensors such as visible light cameras can also be used. This device can provide information complementary to the infrared images. In some application scenarios, such as environmental monitoring and traffic management, the detailed information provided by the visible light images can work together with the infrared images to further improve the accuracy of image recognition and analysis.

[0022] It should be noted that the data of different sensors have different forms of representation. For example, each pixel point in the image generated by an infrared thermal imaging camera represents the temperature value of an object, while a visible light camera captures images within the visible light spectrum range. There are significant differences in their resolution, data format, color model, etc. Therefore, when processing this data, it is necessary to ensure that the image data of different sensors can be effectively combined and processed within the same framework.

[0023] During the implementation of the present invention, the data acquisition modules of all sensors include a time synchronization function to ensure the consistency of the data of different sensors in the time dimension. For example, a method based on precise clock synchronization is adopted to calibrate each captured image with a timestamp, so that the image data collected from different sensors at the same moment can be accurately aligned, avoiding data mismatch caused by time errors.

[0024] Specifically, the data acquisition module adopts a unified timestamp mechanism in different sensors. Each image data will be attached with its corresponding timestamp when it is generated, which is convenient for pairing the data of different sensors in the subsequent processing stage. In some embodiments, the acquisition accuracy of the data can be ensured by means of hardware clock synchronization or software synchronization in a high-dynamic environment.

[0025] During the data acquisition process, the quality of the infrared image data may also be affected by external environmental factors. For example, the image may be interfered by factors such as external temperature changes, sensor noise, and object movement. To address this issue, the data acquisition module can perform selective acquisition and obtain redundant data in combination with the characteristics of different sensors, thereby improving the reliability and stability of the image data.

[0026] As an option, when collecting infrared image data, the environmental monitoring system can also record parameters such as environmental temperature and humidity, and store this data as auxiliary information in the metadata. This can provide effective reference for subsequent processing steps such as temperature correction, image enhancement, and noise removal, thereby improving the accuracy and efficiency of the entire data processing process.

[0027] Exemplarily, by summarizing and storing the collected data from different sensors, the data acquisition module can complete the aggregation of multi-modal data in a unified data integration platform, facilitating subsequent feature extraction, classification, and archiving processes. This platform supports the management of multi-sensor data and provides data access through a unified interface, providing a convenient data source for data analysis and subsequent processing.

[0028] It should be noted that although infrared thermal imaging cameras and visible light cameras are mainly mentioned as examples of sensors in the present invention, in practical applications, different types of infrared sensors and other auxiliary sensors can also be used according to specific requirements, even including lidar, ultrasonic sensors, millimeter wave radars, etc. These different types of sensors can not only enrich the content of data collection, but also provide redundant information under different environmental conditions, thereby enhancing the robustness of data analysis.

[0029] Therefore, the data acquisition scheme provided by the present invention has high flexibility and scalability, and can adapt to the needs of different application scenarios. Through the data fusion and synchronous acquisition of multiple sensors, it can provide a solid data foundation for subsequent infrared image data processing, classification, and archiving, ensuring the efficiency and accuracy of the entire system in large-scale data processing.

[0030] Step 2: Preprocess the collected infrared image data, including noise removal, temperature correction, and image enhancement. Among them, wavelet transform and deep convolutional neural network are used for noise removal, and temperature correction is based on the non-linear mapping of the difference between environmental temperature and image temperature; In this embodiment, feature extraction and multi-modal data fusion are key steps in the infrared image data classification and archiving method, aiming to extract effective feature information from the collected original images and effectively fuse the data from different sensors, thereby improving the accuracy and robustness of subsequent classification and analysis. The feature extraction and fusion process combines traditional handcrafted features and deep learning methods, and uses the complementarity of different sensors to enhance the feature representation ability.

[0031] In the feature extraction stage, the present invention combines traditional image processing methods with deep learning techniques to obtain valuable low-level and high-level features from images. Exemplarily, in terms of traditional handcrafted feature extraction, classic methods such as Histogram of Oriented Gradients (HOG) and Scale-Invariant Feature Transform (SIFT) are adopted. These methods have been verified in many computer vision tasks and can effectively extract local features such as edges, corners, and textures from infrared images.

[0032] The Histogram of Oriented Gradients (HOG) method can extract the shape features of an image by calculating the gradient direction and magnitude of local regions of the image. HOG features perform well in object detection and image classification tasks. The calculation formula of HOG is as follows: Where, is the feature description at the position of the image, is the gradient value at the direction and is the number of direction bins. By converting the statistical information of the local gradient direction distribution into a feature vector, HOG can effectively represent the shape features of an object.

[0033] The Scale-Invariant Feature Transform (SIFT) can extract key point features in an image and has scale and rotation invariance. The SIFT method has strong robustness by finding local extreme points and performing feature description based on these extreme points. The calculation formula for the key points of SIFT is: Where, is the calculated feature descriptor, is the weight, and are the pixel values of two points in the image respectively, is the total number of points. In this way, SIFT can extract stable local features from an image to help improve the accuracy of classification tasks.

[0034] In terms of deep learning feature extraction, the present invention adopts a deep convolutional neural network (CNN), especially the Residual Network (ResNet). CNN automatically learns high-level semantic features from the original image through multiple convolutional and pooling operations. Compared with traditional handcrafted features, CNN can automatically recognize patterns in images when processing complex image data and can effectively improve performance on large-scale datasets.

[0035] The core idea of the Residual Network (ResNet) is to solve the problem of vanishing gradients in the training of deep networks by introducing residual connections. Specifically, ResNet forms "residual" learning by directly adding the input of a certain layer to the output of the next layer through skip connections. The formula is described as follows: Among them, is the output, is the input, is the residual function learned through the network, is the weight parameter of the network. Through this structure, ResNet can effectively alleviate the problem of vanishing gradients in the training of deep networks and enhance the learning ability of the network.

[0036] As an option, when extracting high-level features, a model based on the Self-Attention mechanism, such as Transformer, can also be used to further improve the efficiency and accuracy of feature learning. The Self-Attention mechanism can optimize the feature extraction process by calculating the global feature dependencies and assigning different weights to the features at each position.

[0037] Specifically, the goal of the multi-modal data fusion stage is to effectively combine the data from different sensors to construct a more comprehensive and accurate feature representation. In this embodiment, the image data comes from multiple sensors, including infrared thermal imaging cameras, visible light cameras, etc. The data feature expression forms provided by each sensor may vary greatly.

[0038] To achieve multi-modal data fusion, the present invention adopts a weighted fusion method to perform weighted summation on the image features of different sensors. Specifically, assuming there are features of different modalities , its weighted fusion formula is: Among them, is the feature of the -th modality, is the weighting coefficient of the -th modality, is the fused feature. The weighting coefficient is dynamically adjusted according to the quality and importance of each modality data. In practical applications, can be set based on the accuracy of the sensor, environmental conditions, and task requirements.

[0039] It should be noted that this weighted fusion method can integrate the advantages of various modal data, utilize the complementary information provided by different sensors, and enhance the expression ability of the final fused features. For certain specific scenarios, such as object detection and anomaly detection, the fused features can significantly improve the accuracy of classification and detection.

[0040] In a possible implementation, to further optimize the multi-modal data fusion process, the present invention also introduces a Generative Adversarial Network (GAN) ** for feature alignment. The GAN optimizes the similarity between features of different modalities by training a generator and a discriminator, making the fused features more stable and consistent. In this way, the system can effectively avoid the problem of unstable fusion caused by feature differences between different modalities.

[0041] For example, in the fusion process of infrared images and visible light images, infrared images may contain more temperature information, while visible light images contain more spatial structure information. Through the GAN model, the system can automatically adjust the fusion method of the two, so that the final feature representation contains both the temperature information of the infrared image and can utilize the detailed information of the visible light image, thereby enhancing the classification and analysis capabilities.

[0042] It can be understood that the advantage of multi-modal data fusion is that it can maximize the integration of feature information from different sensors, so as to better adapt to complex and changing environments and task requirements. Especially in the processing of large-scale image data, fusing data from multiple modalities can significantly improve the expression ability of features, providing a more accurate and reliable basis for subsequent classification and archiving.

[0043] Through the above steps of feature extraction and multi-modal data fusion, the present invention can effectively extract useful information from multi-source data and further enhance the expression ability of features by using fusion technology, laying a solid foundation for subsequent classification and archiving tasks.

[0044] Step 3: Standardize the image data from multiple sensors to unify the pixel value range of the image data; In this embodiment, data classification and archiving are key steps in the infrared image data management system. The aim is to efficiently classify the image data based on the aforementioned feature extraction and multi-modal data fusion results, and structurally store and archive the classified data for subsequent query, retrieval, and use. This step uses a variety of technical means, combining machine learning and data storage strategies, to ensure that large-scale infrared image data can be efficiently managed while ensuring accuracy.

[0045] During the data classification process, in this embodiment, a classification algorithm based on deep learning is adopted, especially convolutional neural networks (CNNs) and their variant models such as ResNet, VGGNet, etc., to classify the fused image features. Through these deep learning models, different categories of target or scene information can be identified from complex multi-modal image data.

[0046] Specifically, during the image classification process, the feature vector of the input image is fed into the trained classification network for processing. Assuming the output of the classification network is , the goal of the classification task is to optimize the model parameters so that can accurately predict the category to which the image belongs. The output of the classification model can use the Softmax function to calculate the probability of each category: where, is the predicted probability that the image belongs to category , is the input feature vector, and are the weights and biases corresponding to category , is the total number of categories.

[0047] In some embodiments, transfer learning is adopted during the network training process. By using a pre-trained model (such as a model trained on large-scale datasets like ImageNet), and then fine-tuning the model according to the target task. This approach can greatly reduce the training time and improve the performance of the model on the target dataset.

[0048] It should be noted that data classification is not limited to the recognition of target categories, but can also be classified more finely according to the requirements of the task. For example, for different temperature ranges in infrared images, they can be classified into different temperature segments to focus on abnormal high or low temperature areas. At this time, the classification model not only needs to consider the type of the target, but also needs to consider the attribute information of the target, such as temperature, position, etc.

[0049] As an option, the classification module of the present invention can also be combined with traditional machine learning methods such as support vector machines (SVMs) or random forests (Random Forests) ** for auxiliary classification. These methods can combine the output of the deep learning model to further optimize the classification results and improve the robustness and accuracy of the classification.

[0050] Data archiving refers to the reasonable storage and management of classification results and original data after data classification. The goal of archiving is to ensure the efficiency of data retrieval and access in the future, while ensuring the integrity and scalability of the storage process.

[0051] In this embodiment, structured storage is adopted for data archiving, that is, the classified image data and related metadata are stored in a database through a database system for fast indexing and querying. Specifically, for each infrared image data, the system stores the following information: Image data: Store the binary data of the image or the URL of the image for subsequent viewing or further processing.

[0052] Classification label: The category label to which the image belongs, such as "normal temperature", "abnormal high temperature", "target detection", etc.

[0053] Metadata: Include sensor information, acquisition time, environmental parameters (such as temperature, humidity, etc.) and other auxiliary information. These metadata provide support for subsequent data retrieval and image analysis.

[0054] Feature vector of the image: The high-dimensional feature vector obtained through feature extraction and multi-modal data fusion, which can also be used for similar image retrieval.

[0055] All these data are stored in the database and accelerated retrieval is performed through hash indexing or inverted indexing. For example, for the feature vector of the image, methods such as k-d tree (k-dimensional tree) or LSH (Locality-Sensitive Hashing) can be used for similarity retrieval. For classification labels, fast queries based on categories can be supported by establishing a classification label index.

[0056] Specifically, to ensure the efficiency and scalability of data archiving, the present invention can also adopt a distributed storage system, such as a distributed file system (such as HDFS) or a database system based on a cloud computing platform, to achieve the efficient storage of a large amount of infrared image data. For different scale application scenarios, the storage strategy can be dynamically adjusted according to requirements, and horizontal expansion or distributed computing can be adopted to process a large amount of data.

[0057] Exemplarily, in a distributed storage system, image data can be split into multiple data blocks, each data block is stored on different nodes, and data exchange is performed between nodes through network communication. In this way, the system can support the efficient storage and management of a large amount of infrared image data.

[0058] It should be noted that the image data in the archiving system is not limited to storing the original images, but also includes the results of data analysis. For example, by detecting temperature anomaly regions, these regions can be marked and stored in the archiving system for subsequent analysts or systems to view and further analyze. The system can also customize data archiving strategies according to different application requirements. For example, for image data of abnormally high temperature regions, dedicated storage and annotation are performed for subsequent fault detection and alarm.

[0059] As an option, the archiving process can also incorporate a version control mechanism to ensure that the historical records of image data and its analysis results can be completely archived, supporting version backtracking and traceability analysis. Especially in industrial application scenarios, the historical versions and evolution processes of image data are of great significance and can help analyze and solve potential fault problems.

[0060] It can be understood that data classification and archiving are not just about simply storing images. It is a core part of the entire image data management system. Through reasonable classification and efficient storage and archiving strategies, the present invention can ensure the management and retrieval efficiency of large-scale image data throughout its life cycle. Whether it is for classification requirements in a specific scenario or for efficient archiving requirements for a specific task, the system can provide stable and efficient solutions.

[0061] Through the data classification and archiving steps in this embodiment, infrared image data can be accurately classified and stored and managed in an efficient manner, providing strong support for subsequent retrieval, analysis, and decision-making.

[0062] Step 4: Extract features from the standardized images. The feature extraction includes manual feature extraction and deep learning feature extraction. Among them, manual feature extraction uses histogram of oriented gradients and scale-invariant feature transform, and deep learning feature extraction is performed through a deep convolutional neural network; In this embodiment, the automatic annotation and update of infrared image data based on intelligent algorithms are important steps in the infrared image data management system, aiming to update the labels and automatically annotate the image data stored in the system through automated means, thereby improving the quality and accuracy of the data. This step combines advanced algorithms such as deep learning and reinforcement learning to achieve intelligent image annotation and can continuously update the label information of existing data based on newly acquired data.

[0063] During the process of automatic data annotation, this embodiment adopts a deep learning model, especially based on convolutional neural network (CNN) and object detection algorithms, to perform automatic label prediction on infrared image data. Specifically, the image feature vectors extracted based on the foregoing steps are input into a pre-trained model for classification or annotation. The goal of this annotation process is to automatically assign appropriate labels to each image through the model for subsequent classification, archiving, and analysis work.

[0064] Exemplarily, during the image annotation process, object detection algorithms used include YOLO (You Only Look Once), Faster R-CNN, and RetinaNet, etc. These algorithms can identify and locate the positions of objects in the image and assign labels to them simultaneously. The YOLO algorithm performs object detection by dividing the image into multiple grids and making predictions for each grid, which can be specifically represented by the following formula: where, represents the probability that the object belongs to class y given the input x, W is the weight, and b is the bias term. Through this method, YOLO can quickly and accurately identify multiple objects in the infrared image and generate prediction results with bounding boxes.

[0065] In addition, Faster R-CNN adopts a Region Proposal Network (RPN) to extract the target regions, and then uses a convolutional network for object classification and bounding box regression. The output of this algorithm includes the class of the object and precise localization information, which is of great significance for object analysis in infrared images.

[0066] As an option, in addition to object detection, image annotation can also be based on the context information of the image. Natural Language Processing (NLP) technology can be combined with image recognition to utilize the text information contained in the image (such as annotations or marks in the image) to further supplement or update the labels. For example, for the device status in an infrared image, the label information can be updated by parsing the corresponding annotation text information in the image (such as device fault prompts), thereby enhancing the accuracy and richness of the annotation.

[0067] Training and Optimization of the Annotation Model During the training process of the annotation model, this embodiment adopts a transfer learning strategy. Transfer learning refers to migrating the model parameters that have been trained on a large-scale dataset (such as ImageNet) to the target task and fine-tuning them with a small amount of task-specific data. This method can effectively improve the training efficiency of the model and still obtain good results even when the amount of data is limited.

[0068] Specifically, the transfer learning method used in this embodiment includes the following steps: Select a network (such as ResNet, InceptionV3, etc.) trained on ImageNet or a similar dataset as the pre-trained model.

[0069] According to the requirements of the target task, fine-tune the last few layers of the pre-trained model. These last fully connected layers are usually responsible for task-specific feature learning, so they need to be trained according to the label information of the new task.

[0070] Train the dataset of the new task through supervised learning to optimize the model parameters.

[0071] The training objective in this process is to minimize the difference between the model's predicted labels and the actual labels. Usually, the cross-entropy loss function is used for optimization, and the formula is as follows: Among them, represents the loss between the label y and the predicted result The number of categories is is the number of categories, is the actual label, is the probability value predicted by the model. By minimizing this loss function, the model can learn the best parameters on the given training data, thereby improving the accuracy of image annotation.

[0072] It should be noted that in the management system of infrared image data, as new image data is continuously collected and stored, the annotation of existing data may need to be continuously updated to adapt to new changes and situations. For this reason, this embodiment combines online learning and reinforcement learning methods to achieve continuous update of data annotation.

[0073] In the online learning process, the system will continuously feed the newly collected image data back into the annotation model for incremental training. Specifically, the system will regularly use the new annotation data to adjust the model parameters, so that the model can adapt to the new data distribution and feature changes. This method can effectively avoid the performance degradation of the model in long-term use by continuously introducing new training data.

[0074] As an option, in the reinforcement learning framework, the system can give rewards and punishments according to the quality and accuracy of image annotation. The goal of reinforcement learning is to continuously optimize the annotation strategy of the model through interaction with the environment. Assuming that each image annotation process can be regarded as a state-action pair, then the goal of the system is to maximize the cumulative reward. Its mathematical representation can be as follows: Among them, is the cumulative reward starting from the current time t, is the discount factor, is the immediate reward at each moment. By maximizing the cumulative reward, the system can continuously optimize the performance of the annotation model according to the actual annotation effect of the image.

[0075] Specifically, this step can achieve automated and intelligent infrared image data annotation and continuously optimize the annotation accuracy. Through the combination of deep learning and reinforcement learning, this embodiment can achieve efficient and accurate automatic annotation in different application scenarios and can continuously update the existing annotations according to new data, improving the adaptability and robustness of the system.

[0076] It can be understood that the data automatic annotation and update step in this embodiment can not only improve the efficiency of image data management, but also provide high-quality annotation information for subsequent image analysis and processing. In the fields of industrial inspection, security monitoring, etc., with the continuous increase of infrared image data, the automatic annotation and update technology can effectively reduce the burden of manual annotation and improve the automation degree of data processing.

[0077] Through the intelligent algorithm annotation and update step in this embodiment, the system can annotate image data in real time and accurately, and continuously improve the annotation accuracy through continuous learning, providing strong support for subsequent steps such as classification, archiving, and analysis of infrared image data.

[0078] Step 5: Perform multi-modal data fusion on the extracted image features, and use the weighted fusion method and the generative adversarial network to optimize the feature alignment between modalities; In this embodiment, the intelligent retrieval and analysis of image data is one of the key steps in the infrared image data management system. By adopting an efficient image retrieval algorithm and analysis model, the system can quickly and accurately retrieve the data that meets the requirements in the large-scale infrared image data storage and conduct in-depth analysis on the data to support subsequent applications such as decision-making, monitoring, and alarm. This step combines content-based image retrieval (CBIR) and deep learning analysis models to achieve fast positioning, analysis, and efficient utilization of image data.

[0079] During the image data retrieval process, this embodiment uses the content-based image retrieval (CBIR) method to combine the visual features of the image, such as texture, color, shape, etc., to retrieve similar images. The core of the CBIR method is to use the feature vectors of the image for similarity calculation to find the image data similar to the query image.

[0080] Specifically, after the feature extraction and multi-modal data fusion of the image in the foregoing steps, it will be converted into a high-dimensional feature vector , which represents the visual features of the image. For the input query image, its feature vector will be compared with the feature vectors of all images in the repository and common similarity measurement methods include Euclidean distance, cosine similarity, etc. Exemplarily, the cosine similarity is used to calculate the similarity between two feature vectors, and its formula is as follows: where, and are the feature vectors of the query image and the repository image respectively, represents the norm of the vector, and · represents the dot product of vectors. By calculating the cosine similarity, the similarity between the query image and other images in the database can be obtained, and the image with the highest similarity is returned.

[0081] It should be noted that during the image retrieval process, in order to improve the retrieval efficiency, the system can adopt the inverted index technology or the locality-sensitive hashing (LSH) ** technology to accelerate the retrieval of feature vectors. These technologies can compress the high-dimensional feature space into a low-dimensional index structure, thereby achieving fast retrieval on large-scale datasets.

[0082] As an option, in the retrieval for specific targets or scenarios, the system can further utilize deep learning models for the retrieval of specific categories or targets. For example, deep learning technologies such as object detection and semantic segmentation are used to label specific regions or targets in the image, thereby performing precise target-based retrieval.

[0083] After completing the image retrieval, the image data analysis process in this embodiment will further deeply mine the information in the image and perform tasks such as detailed scene analysis, anomaly detection, and change analysis. Specifically, the analysis process may include the following aspects: Anomaly detection: For infrared images, anomaly detection is particularly important and is usually used to detect regions with too high or too low temperatures, or in industrial inspections, to detect abnormal behaviors of equipment. In this embodiment, an algorithm combining self-supervised learning and the "convolutional neural network (CNN)" method is adopted, and by analyzing the thermal distribution characteristics of different regions in the image, it automatically determines whether there are anomalies. Assume that the temperature distribution of the infrared image is a two-dimensional matrix , where represents the coordinate points in the image, represents the temperature value at that point. The goal of anomaly detection is to determine whether there are obvious temperature changes or phenomena deviating from the normal value in a certain region. This task can be carried out by calculating the temperature mean and variance of local regions in the image: where, is the average temperature of region , is the temperature variance of the region, and N is the number of pixels in the region. By setting a threshold, if the temperature of a certain region exceeds the set range, it can be determined as an abnormal region.

[0084] Target tracking: In a dynamic monitoring scenario, it may be necessary to track and analyze the target. In this embodiment, a target tracking algorithm based on a convolutional neural network (CNN) is adopted. This algorithm determines the movement trajectory of the target through feature matching between multiple frames of images. By extracting spatio-temporal features from consecutive frames of images, accurate tracking of dynamic targets can be achieved.

[0085] Change detection: For scenarios that require long-term monitoring (such as the health monitoring of facilities and equipment), the system can perform change detection on infrared images of multiple time periods to analyze the state changes of the equipment. Specifically, the system can compare the images taken of the same target at different time points, calculate the difference between the two images, and determine whether there are structural changes. Common change detection methods include the image difference method, principal component analysis (PCA), etc.

[0086] Exemplarily, the calculation formula for change detection is as follows: where, are two images and at position The difference value. By calculating the difference value of each pixel, the degree of change of the entire image can be obtained, thereby determining whether a significant change has occurred.

[0087] Visualization of image analysis results and decision support After the analysis is completed, the analysis results of the image will be presented to the user through a visualization interface. For example, the system can overlay annotation boxes on the image to highlight abnormal regions or targets, or overlay the temperature data with the image through color mapping to form an intuitive heat map.

[0088] It should be noted that in terms of decision support, the system can combine the image analysis results to provide an automated alarm mechanism. For example, when a temperature anomaly or a target deviates from the normal trajectory is detected, the system can automatically trigger an alarm notification to remind the user to take further intervention.

[0089] As an option, in specific application scenarios (such as equipment maintenance), the system can also combine external data (such as the working status of the equipment, historical fault records, etc.) for comprehensive analysis to provide optimized maintenance suggestions or predict the risk of faults.

[0090] Specifically, to improve the performance and scalability of the system, this embodiment adopts a distributed computing architecture, divides the data analysis tasks into multiple subtasks for parallel execution, reduces the computing burden on a single node, and improves the processing efficiency. The system also supports cloud computing platforms and can dynamically expand computing resources as needed to meet the processing requirements of large-scale image data.

[0091] It can be understood that the image retrieval and analysis module in this embodiment not only supports standard image queries but also can perform high-level semantic analysis to help users extract valuable information from a large amount of infrared image data, thus supporting intelligent applications such as decision-making, alarm, and monitoring.

[0092] Step 6: Optimize the fused features, and update the network weights through the backpropagation algorithm to improve the accuracy of image classification; In this embodiment, step 6 involves the data dynamic update and feedback mechanism in the infrared image data management system. With the continuous accumulation of infrared image data and the growing demand for real-time monitoring, the system must be able to update data in real time or periodically and process data changes to ensure the timeliness, accuracy, and integrity of the data. In addition, the system also needs to be able to optimize the data storage, processing strategies, and the accuracy of result analysis according to the actual operation feedback.

[0093] During the process of image data management, as the infrared imaging device keeps running, new data is continuously generated, and the original data repository needs to accept new image data in real time. To improve the efficiency of data update, this embodiment adopts an incremental data update mechanism, which can process only the newly added infrared image data instead of reprocessing all historical data.

[0094] Specifically, the system periodically checks for new images in the image data source, identifies these new data, and performs processing tasks such as feature extraction, annotation, and classification on the new images through an image feature update algorithm. For example, for a newly input infrared image , the system first extracts the heat map features by calculating the temperature distribution of each region in the image : Then, based on the heat map and other features of the image (such as texture, color, shape, etc.), the system uses a trained machine learning model (such as a convolutional neural network, CNN) for image classification and feature vector generation. The calculation method of the feature vector is similar to that mentioned in step 3, and uses inference after training with a deep learning network to obtain a vector representing the most important features of the image.

[0095] It should be noted that, compared with the traditional full - scale update, this incremental update mechanism not only improves the efficiency of data processing, but also ensures that the system can respond to new monitoring requirements in real - time when processing large - scale image data. Especially in scenarios such as dynamic monitoring and real - time alarm, it can effectively reduce the system load.

[0096] To improve the quality of data during the update process, this embodiment also introduces an intelligent data optimization algorithm. This algorithm combines factors such as the quality detection of images, the timeliness requirements of data, and the usage frequency of historical data to intelligently determine which data needs to be processed or updated preferentially. For example, some images with large differences in thermal distribution (such as images of overheated or malfunctioning equipment) may be marked as high - priority data and archived and analyzed subsequently.

[0097] Specifically, in this embodiment, by analyzing the historical usage of image data, a weight value is assigned to each image . This represents the usage frequency or importance of the image. The calculation of this weight value can be based on the following factors: Among them, is the usage frequency of the image in the past period of time, is the priority score of the image (based on image content or abnormal conditions), while and are parameters that control the importance of these two factors respectively. Through this algorithm, the system can identify which data is the most valuable and preferentially update or process it.

[0098] After completing the dynamic update of the data, the system will provide feedback to the user or system administrator according to the changes in the data. The feedback mechanism can not only help users understand the monitoring status in a timely manner, but also guide users to take necessary actions.

[0099] In this embodiment, the system feeds back the status of the updated data through an integrated alarm system and a monitoring interface. For example, when a newly added image shows that a certain device is abnormal (such as too high temperature), the system will trigger the alarm mechanism and send warning messages to relevant personnel in real - time via email, text message, or system push. To improve the accuracy and timeliness of the feedback, the system will also conduct intelligent analysis based on historical data and the current status, provide optimization suggestions or alarm levels to help users make quick decisions.

[0100] As an option, the system is also capable of integrating external data (such as environmental changes, device status, etc.), performing intelligent analysis through a rules engine, automatically determining whether there are potential faults or risks in the device, and predicting the possible types of faults that the device may encounter in the future based on historical data. These prediction results can not only help users understand the health status of the device in a timely manner, but also guide maintenance personnel to carry out preventive measures and regular inspections in advance.

[0101] This embodiment also designs a continuous optimization mechanism, which gradually optimizes the image data management and processing algorithms by collecting feedback information of the system. For example, the system can learn from the monitoring results of user feedback, thereby adjusting the algorithm parameters of modules such as image classification, object detection, and anomaly detection, and then improving the overall performance and accuracy of the system.

[0102] Specifically, the system continuously adjusts its model parameters or weight values through learning from each feedback, implementing dynamic optimization. The data of user feedback will automatically adjust the hyperparameters of model training through subsequent data processing modules to further improve the accuracy of image analysis and retrieval.

[0103] For example, assume that the system has a high false alarm rate (such as false alarms) for a certain type of image during a certain period. Then, the system analyzes these false alarm data and automatically adjusts the weight parameters in the classifier or detector to reduce the occurrence probability of this type of false alarm, thereby improving the accuracy of the system.

[0104] It can be understood that with the long-term operation of the system, the continuous optimization of the feedback mechanism can help the system gradually adapt to the characteristics of new infrared image data, making the system more and more intelligent and capable of efficiently coping with changing application scenarios.

[0105] To enhance the adaptability of the system in different application scenarios, the dynamic update and feedback mechanism of this embodiment supports flexible expansion. For example, the system can support the access of different image data sources according to actual needs, meeting different requirements from single-device monitoring to large-scale monitoring systems. In this case, the system can flexibly process image data from different devices and perform corresponding priority assignments according to the working status of different devices.

[0106] Exemplarily, the system can be docked with different types of infrared imaging devices by supporting multiple data interfaces (such as APIs, standard data transfer protocols, etc.). For example, in some industrial applications, it may be necessary to access data provided by multiple devices. The system will automatically perform data preprocessing according to the data characteristics of each device and fuse the data of each device for unified management and analysis.

[0107] It should be noted that the system can also achieve cross - regional and cross - device data update and feedback through the cloud computing platform, further improving the timeliness of data processing and the scalability of the system.

[0108] Step 7: Classify and label the optimized image features, generate labels and automatically update the label library; In this embodiment, Step 7 involves the storage and archiving management of infrared image data. Data storage and archiving are the core components of the infrared image data management system, ensuring the efficient access, retrieval and secure storage of image data during long - term operation. This step provides an automated and scalable management method through the classified storage and efficient retrieval of image data.

[0109] In this embodiment, distributed storage systems are used for data storage to meet the storage requirements of large - scale infrared image data. The system classifies and stores image data in multiple dimensions such as type, time, event, etc., and sets different storage strategies according to different types of image data. For example, for data with different priorities (such as abnormal alarm images and regular monitoring images), different storage media and storage paths can be adopted to improve data access efficiency.

[0110] As an option, image data can be distributedly stored and managed through a cloud storage platform. This storage method can be dynamically adjusted according to the access frequency and importance of the data. In the cloud storage platform, the system can store hot data on high - speed storage media (such as solid - state drives) through an automated mechanism, while storing cold data (such as historical monitoring images) on low - cost large - capacity storage media (such as mechanical hard drives or object storage).

[0111] Specifically, the system stores the relevant metadata of each image (such as timestamp, device information, image classification label, etc.) in a dedicated database. The image files themselves are stored in a distributed file During the process of image data accumulation, the system needs to formulate corresponding archiving strategies according to the value and importance of the images. For image data that has not been accessed for a long time or no longer has real - time monitoring value, it can be transferred to a low - cost storage area through archiving to free up high - efficiency storage space.

[0112] Lifecycle label can be calculated based on the access frequency of the image and importance score , for example: Among them, and are the weighted coefficients of access frequency and priority respectively. The system dynamically adjusts the lifecycle of images through such formulas to ensure the optimal utilization of storage resources.

[0113] It should be noted that in some scenarios, the system can also automatically adjust the archiving time and archiving strategy of image data according to external environmental changes (such as long-term idling of devices, end of monitoring cycle, etc.). For example, when a device is no longer used for a long time, the system will automatically archive the historical image data related to the device and move it to the cold data storage area to reduce the real-time storage pressure.

[0114] To ensure the security and high availability of image data, a redundant backup strategy is adopted for image data in this embodiment. Redundant backup ensures that in the event of a storage medium failure, the image data can still be restored through other backup copies.

[0115] As an option, the system realizes data redundancy through RAID (Redundant Array of Independent Disks) technology. Specifically, for the hard disk array storing important image data, the system ensures the storage of at least two copies through solutions such as RAID 1 or RAID 5, so that the data can be restored from other copies when a single hard disk is damaged. For cloud storage, the system adopts a multi-copy storage strategy, and further improves the reliability of data by distributing each image data on multiple cloud nodes.

[0116] In addition, the system also ensures the integrity and consistency of data through data check codes (such as CRC check, hash value, etc.). Each time image data is written, the corresponding check code will be calculated and stored in the database. Each time image data is read, the system will compare the check code to ensure that the read data has not been tampered with or damaged.

[0117] For the large amount of infrared image data stored in the system, the system must have efficient retrieval and access functions. To achieve this goal, a data retrieval system based on an indexing mechanism is designed in this embodiment. By creating multi-dimensional indexes for image data (such as establishing indexes according to time, device, event type, etc.), users can quickly find relevant images.

[0118] Specifically, each piece of image data will generate corresponding multi-dimensional index information, and these index information will be stored in the database and managed through efficient retrieval structures such as hash tables and B+ trees. The retrieval of images can be based on the following query conditions: Time range query: Users can retrieve images according to a specific time range.

[0119] Device query: Users can retrieve all images of a device based on the device identifier.

[0120] Event type query: Users can query images related to specific events (such as device failure, overheating, etc.).

[0121] Exemplarily, the system uses a time index and a device index to quickly find the image requested by the user within a short time. For example, given a certain device ID and a time range , the system can quickly extract all the image data completed by the device within that time period from the index, and sort the results through search algorithms (such as binary search, B+ tree search, etc.).

[0122] It can be understood that through this efficient indexing mechanism, the system can support the rapid query of large-scale image data, avoid the inefficient method of full-scale scanning, and improve the overall performance of the system.

[0123] During the data storage and archiving process, the security and privacy protection of data are also important considerations in this embodiment. To ensure that the infrared image data is not illegally accessed or tampered with, the system protects the data through a multi-layer encryption mechanism. When the image data is stored, the AES (Advanced Encryption Standard) encryption algorithm is used to encrypt and store the image file; while during the data transmission process, TLS (Transport Layer Security Protocol) is used for encrypted transmission to prevent the data from being stolen or tampered with during transmission.

[0124] It should be noted that the system also supports role-based access control, and only authorized users can access specific data. The permissions of each user will be dynamically allocated and adjusted through the system management module to ensure that the data access permissions meet the actual needs and are effectively audited.

[0125] Through the above storage and archiving management strategies, this embodiment effectively solves problems such as insufficient storage space, inefficient data access, and security risks in the management of infrared image data, and improves the stability, reliability, and efficiency of the system.

[0126] Step 8: Store the classified images and their corresponding tags in a distributed storage system, and store the metadata of the images in a relational database to establish an index for the image data; In this embodiment, Step 8 involves the analysis and intelligent decision-making functions of infrared image data. This step aims to automatically identify and push important information related to infrared images through efficient data processing and intelligent analysis techniques, thereby assisting in decision-making and optimizing the working efficiency of the system.

[0127] In this embodiment, data analysis mainly relies on deep learning and machine learning technologies. By analyzing a large amount of stored infrared image data, the system can automatically extract effective information from the images, identify potential problems, and generate corresponding analysis reports. The analysis framework can be divided into two main parts: image preprocessing and feature extraction, and intelligent analysis and decision support.

[0128] First, the system preprocesses the infrared image data, removes noise, enhances the image quality, and performs normalization processing for subsequent feature extraction and analysis. During the preprocessing, methods such as median filtering and Gaussian filtering can be used to eliminate the influence caused by environmental factors or device noise and improve the clarity of the image.

[0129] Specifically, after denoising, the system extracts the key information regions in the image through image segmentation techniques (such as threshold segmentation, region growing method, etc.), such as target objects, hot spots, or temperature anomaly points. For infrared images, the processing of temperature distribution images is often involved. Therefore, the system also calibrates the temperature of the image and converts the pixel values in the infrared image into actual temperature values.

[0130] As an option, there may be multiple heat source regions in the image. The system classifies and labels these heat source regions, and identifies each independent heat source region through region extraction algorithms (such as connected component analysis), thus providing more accurate input data for subsequent target analysis.

[0131] It should be noted that the feature data generated at this stage is not only a simple analysis of the image content, but also includes some statistical features of the image (such as the mean and standard deviation of temperature distribution, etc.) and spatial features of the image (such as the shape, size, and position of the object, etc.), providing rich information for subsequent analysis.

[0132] After image feature extraction, the system uses deep learning models (such as convolutional neural network CNN, generative adversarial network GAN, etc.) to deeply analyze the image data. These models can learn complex patterns based on a large amount of training data and make intelligent judgments based on the extracted features. Potential problems that may exist in the image (such as equipment failure, overheating, fire hazard, etc.) will be identified and classified through the trained models.

[0133] In a possible implementation, the system analyzes the infrared image data, identifies possible abnormal hot spots in the image, and uses image classification algorithms to classify these abnormal situations into different types of fault modes, such as overheating, equipment abnormality, fire, etc. Based on these analysis results, the system can actively send warning messages to remind users to handle them in a timely manner.

[0134] For example, when a certain device has a temperature anomaly, the system can calculate the degree of temperature anomaly through the following formula : where is the measured temperature value in the current image, is the predetermined temperature threshold. If When it exceeds a certain threshold, the system will trigger a fault alarm.

[0135] Exemplarily, if the operating temperature of a device exceeds the set safety threshold, the system will send an alarm to the maintenance personnel after image analysis and recommend equipment inspection or cooling measures. In addition, the system can also perform trend analysis based on the historical data of the device to predict future fault risks. For example, if the system finds that the temperature of a certain device is gradually rising and shows periodic fluctuations, it may mean that the device is aging and preventive maintenance is required.

[0136] Based on the analysis of image data, the system can provide intelligent decision-making support for operators. In addition to simple fault detection, the system can also automatically generate some optimization suggestions according to the analysis results. For example, if a certain device has been overheated for a long time, the system not only prompts the abnormality, but also can suggest specific operation suggestions such as increasing the heat dissipation of the device, reducing the load or replacing the device.

[0137] Specifically, when the system identifies temperature anomalies in the image, it will combine the operating status, maintenance history of the device and relevant environmental data to generate a comprehensive risk assessment report. The report will list the root causes that may lead to problems and recommend the most appropriate solutions. For example, if the device overheats, the system may suggest reducing the device load or regularly checking the device's heat dissipation system to reduce the risk of serious failures.

[0138] It can be understood that the intelligent decision-making module does not only make analysis through single image data. The system can also integrate data from other sensors (such as humidity, air pressure, device operating status, etc.) to perform multi-dimensional analysis of possible fault causes, thereby improving the accuracy and scientific nature of decision-making.

[0139] To improve users' understanding and application of data analysis results, this embodiment also introduces data visualization technology. By presenting the analysis results in a graphical and visual way, users can more intuitively understand the analysis process and results. For example, the system can display the temperature distribution in the infrared image in the form of a heat map, or display the fault analysis results of the device as a trend chart, pie chart, etc., to facilitate users to comprehensively grasp the operating status of the device.

[0140] As an option, the system can also automatically generate a report based on the image analysis results, including image data, analysis conclusions and recommended measures, etc. The report will be exported in formats such as PDF and Excel for easy viewing and archiving by users. When generating the report, the system can automatically fill in the corresponding analysis results according to the preset template, so that the content and format of the report are always consistent.

[0141] As the system continues to run, the data analysis module will continuously accumulate more infrared image data and optimize its analysis model through continuous learning. The system can perform self-training and optimization of the model based on historical data to improve its adaptability to new situations.

[0142] It should be noted that the deep learning model of the system may rely on expert-annotated data for training in the initial stage. However, as the usage process progresses, the system can label and train newly emerging fault types based on automated learning, enhancing the overall analysis accuracy and adaptability.

[0143] Step 9: Retrieve the stored images based on the image content or tags through an intelligent retrieval system. The retrieval methods include similarity retrieval based on image features and retrieval based on tags.

[0144] In this embodiment, Step 9 involves the archiving and storage of infrared image data. The archiving and storage of data are key components in the entire system. It ensures the efficient management and long-term preservation of large-scale infrared image data, while facilitating ready access and processing at any time. The implementation of Step 9 not only needs to ensure the integrity, reliability, and security of the data but also ensure the efficiency and scalability of data storage.

[0145] According to the present invention, the archiving of data is carried out by combining hierarchical storage and metadata management. First, the archiving system stores the data hierarchically according to different characteristics of the infrared image data (such as file size, timestamp, importance, access frequency, etc.) so as to adopt different storage strategies for different categories of data.

[0146] Specifically, for frequently used infrared image data with a high access frequency, the system will store it on high-performance storage devices such as solid-state drives (SSDs) to ensure efficient reading and processing. For image data that needs to be stored for a long time and has a low access frequency, the system will migrate it to low-cost storage devices such as cloud storage or tape storage.

[0147] As an option, the classification criteria for archived data can be defined according to factors such as device type, task importance, image content, etc. For example, infrared image data related to security monitoring may be preferentially stored as high-priority data, while other images for daily device monitoring are classified as low-priority data.

[0148] In this way, the archiving system can effectively manage data storage, reduce storage costs, and at the same time ensure that high-priority data that is needed can be quickly retrieved and accessed.

[0149] In this embodiment, in addition to storing the image data itself, the system also stores metadata related to the image data. The metadata includes, but is not limited to, information such as the generation time of the image, the device number, the physical location of the image, the image analysis result, and the abnormal event marker. These metadata can not only help users quickly retrieve the required infrared image data, but also provide context information for the data, providing strong support for subsequent analysis and decision-making.

[0150] It should be noted that in order to ensure efficient data retrieval and management, the system uses distributed database technology and stores the metadata using a NoSQL database (such as MongoDB, Cassandra, etc.). Such databases can support high-concurrency read and write operations and have good scalability in a large-scale data environment. By using a distributed architecture, the system can disperse the storage of metadata among different nodes, improving data reliability and access efficiency.

[0151] In practical applications, the metadata can also be bidirectionally index-associated with the image data, facilitating user searches based on multiple conditions. For example, users can quickly retrieve all the infrared image data of a certain device on a certain day according to the device number and date, or filter out all the image data with equipment overheating based on the abnormal event label.

[0152] Specifically, the storage format of the metadata adopts the JSON format, which has a simple structure and is easy to expand. The metadata not only contains basic information, but can also embed image processing results (such as temperature deviation, hot spot location, etc.) and information such as links to analysis reports.

[0153] To ensure data security and reliability, the system designs a multi-level backup mechanism. The storage of archived data not only has local backups, but also has remote off-site backup functions. For each piece of archived data, the system will regularly execute backup tasks and store data copies on physically isolated storage media or in remote data centers to prevent risks brought by data loss or hardware failures.

[0154] In a possible implementation, the system uses RAID (Redundant Array of Independent Disks) technology for redundant backup of local storage to ensure that data is not lost when storage devices fail. In addition, the system also uses snapshot backup technology to generate snapshot copies of the data after each data update to ensure that historical versions of the data can be restored at any time.

[0155] As an option, the system also supports storing some important data (such as infrared images of key devices) in the cloud. Cloud storage has higher reliability and flexibility and can dynamically expand the storage space according to storage requirements. In cloud storage, the access to data is also protected by encryption to ensure data confidentiality and security.

[0156] It is understandable that by using distributed storage and cloud storage technologies, the system can achieve flexible storage expansion, remote disaster recovery, and efficient data recovery, meeting the storage requirements of long-term large-scale infrared image data.

[0157] In the actual storage process, the amount of infrared image data is huge. To reduce the occupation of storage space, the system also performs compression and deduplication processing on the image data. Data compression technology can reduce the use of storage space while maintaining the integrity and recoverability of the data.

[0158] Specifically, before archiving the image data, the system will adopt common image compression algorithms such as JPEG2000 or PNG compression. These algorithms can effectively reduce the occupation of storage space while maintaining the image quality. For similar data uploaded multiple times, the system will apply a deduplication algorithm to automatically identify duplicate images and remove duplicate storage, further saving storage resources.

[0159] Exemplarily, when a user uploads a batch of infrared images, the system will calculate the hash value (such as MD5 or SHA-256) of each image and then compare it with the already stored data. If duplicate data is found, only one copy of the data will be stored, and indexes will be established for other duplicate data, pointing to the stored original image. This deduplication method not only saves storage space but also speeds up the data access speed.

[0160] After the data storage and archiving are completed, how to efficiently retrieve and access the infrared image data stored in the system is another key issue in system design. To improve the data retrieval efficiency, this embodiment adopts a hybrid retrieval method based on keyword indexing and image content indexing.

[0161] Specifically, the system will generate multiple retrieval indexes based on the metadata of the image (such as device number, timestamp, fault label, etc.) and the content features of the image (such as hot spots in the image, temperature value distribution, etc.). When performing data retrieval, the user can quickly filter out the target data based on keywords such as time range, device number, or anomaly type. At the same time, the system also supports similarity search based on image content. By calculating the similarity of image content features, other images similar to the target image can be found.

[0162] As an option, the system also supports a more intelligent retrieval method through natural language processing (NLP) technology. The user can input a natural language query instruction, and the system will automatically parse and match the relevant image data.

[0163] In this embodiment, the system also provides a visual display function for data archiving and storage. Users can view the data storage status through a graphical interface to understand the data distribution, storage capacity usage, backup status, etc. In addition, the system will generate regular storage reports to help users understand information such as the health status of the storage system, data access frequency, and consumption of storage resources.

[0164] It should be noted that the report will include important information such as the space occupancy of image data storage, data redundancy, and backup strategy execution, which is convenient for users to manage and optimize storage resources.

[0165] In this embodiment, the data archiving and storage solution in step 9 realizes the efficient storage and secure management of infrared image data through the combination of hierarchical storage, metadata management, redundant backup, compression, and deduplication technologies. The system not only supports efficient storage management and long-term archiving but also provides flexible retrieval, access, and visualization functions, ensuring data reliability, scalability, and efficiency.

[0166] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for classifying and archiving infrared image data, characterized in that: The following steps are involved: Step 1: Collect infrared image data from multiple different types of infrared sensors, including infrared thermal imaging cameras, visible light cameras, and radars; Step 2: Preprocess the collected infrared image data, including noise removal, temperature correction and image enhancement. The noise removal is performed by wavelet transform and deep convolutional neural network, and the temperature correction is performed based on the nonlinear mapping of the difference between ambient temperature and image temperature. Step 3: Standardize the image data from multiple sensors to unify the pixel value range of the image data; Step 4: extracting features from the standardized image, wherein the feature extraction includes manual feature extraction and deep learning feature extraction, wherein the manual feature extraction uses oriented gradient histogram and scale-invariant feature transformation, and the deep learning feature extraction is performed through a deep convolutional neural network; Step 5: Perform multimodal data fusion on the extracted image features, and use weighted fusion method and adversarial generative network to optimize feature alignment between modalities; Step 6: Optimize the fused features and update the network weights through the back propagation algorithm to improve the accuracy of image classification; Step 7: Classify and annotate the optimized image features, generate labels and automatically update the label library; Step 8: Store the classified images and corresponding labels in a distributed storage system, store the image metadata in a relational database, and create an index for the image data; Step 9: Using an intelligent retrieval system, the stored images are retrieved based on image content or tags. The retrieval method includes similarity retrieval based on image features and retrieval based on tags.

2. The infrared image data classification and archiving method according to claim 1, characterized in that: The preprocessing of the image data includes performing image temperature correction by a deep neural network (DNN), specifically: in, is the corrected image, is the ambient temperature, is the temperature of the image scene, is the correction factor.

3. The infrared image data classification and archiving method according to claim 1, characterized in that: The feature extraction extracts high-level semantic features of the image through a deep convolutional neural network, and the CNN network structure includes a self-attention mechanism to improve the representation ability of the features.

4. The infrared image data classification and archiving method according to claim 1, characterized in that: The multimodal data fusion is performed by a weighted fusion method, and the weighted fusion method is calculated using the following formula: in, For the The image features of the modality, For modal The weighting coefficient of The fused features.

5. The infrared image data classification and archiving method according to claim 1, characterized in that: The image classification uses a deep convolutional neural network and adopts a multi-task learning framework to simultaneously perform image classification, object detection and segmentation tasks.

6. The infrared image data classification and archiving method according to claim 5, characterized in that: The total loss function of the multi-task learning is: in, is the classification loss, To detect the loss, is the segmentation loss, and is the loss weight.

7. The infrared image data classification and archiving method according to claim 1, characterized in that: The intelligent retrieval system uses a content-based image retrieval method to perform retrieval by calculating the cosine similarity between image features. The calculation formula is: 。 8. An infrared image data classification and archiving platform, according to the infrared image data classification and archiving method according to any one of claims 1 to 7, characterized in that: include: A data acquisition module, used for acquiring infrared image data from a plurality of different types of infrared sensors; A data preprocessing module is used to perform denoising, temperature correction, image enhancement and standardization on the collected image data; Feature extraction module, used to extract manual features and advanced features based on deep convolutional neural networks from images; Multimodal data fusion module, used to perform weighted fusion on data from different sensors to generate fused features; Image classification and labeling module, used to classify the fused image features and generate corresponding image labels; The data storage and retrieval module is used to store the classified images and labels in the distributed storage system and provide intelligent retrieval functions based on content and labels.

9. The infrared image data classification and archiving platform according to claim 8, characterized in that: The data storage and retrieval module includes a distributed storage system and a relational database, wherein the relational database is used to store metadata of the image and provide a fast query function.

10. The infrared image data classification and archiving platform according to claim 8, characterized in that: The image classification and labeling module is trained through a multi-task learning framework, supports simultaneous image classification, target detection and image segmentation tasks, and automatically updates the label library according to the detection results.

Images