Thermal imaging acquisition system, method, equipment and medium based on Linux operating system
By using a thermal imaging acquisition system based on the Linux operating system, combined with network video surveillance and convolutional neural network algorithms, low-cost and high-efficiency fire monitoring has been achieved, solving the problems of high cost and small monitoring range of existing equipment, and improving the accuracy and coverage of fire monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-14
- Publication Date
- 2026-04-07
AI Technical Summary
Existing thermal imaging monitoring equipment is expensive and has a small monitoring range, making it difficult to achieve large-scale application. In addition, traditional fire monitoring has blind spots and identification errors.
Design a thermal imaging acquisition system based on the Linux operating system. Combined with network video surveillance, it uses convolutional neural network algorithms to analyze images, takes real-time photos of and identifies high-temperature objects through cameras, and compares images using neural visual networks and databases to reduce the proportion of stored data. It is suitable for large-scale forest fire monitoring.
It achieves low-cost and efficient fire monitoring, can identify high-temperature objects in real time, reduces the need for data storage, and improves the accuracy and coverage of fire monitoring.
Smart Images
Figure CN121815047A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of thermal imaging technology, and specifically to a thermal imaging acquisition system, method, device, and medium based on the Linux operating system. Background Technology
[0002] Fires pose a significant threat to people's lives and property. Therefore, early detection and rapid response to fires are crucial for fire safety management. Traditional fire monitoring relies primarily on manual inspections and the deployment of fire-fighting equipment; however, this method suffers from blind spots and identification errors. Fire monitoring and identification based on infrared thermal imaging technology offers advantages such as non-contact operation, multi-angle monitoring, real-time performance, and high resolution, improving the efficiency and accuracy of fire monitoring and identification. However, current thermal imaging monitoring equipment generally suffers from high costs and limited monitoring range. Therefore, reducing equipment costs and improving monitoring accuracy are key challenges currently facing thermal imaging monitoring equipment. Summary of the Invention
[0003] To address the aforementioned problems, this invention combines a thermal imaging acquisition system with a network video surveillance system to design a Linux-based thermal imaging acquisition system. It uses photography to collect real-time data from the site and determines whether a fire has occurred based on the acquired photos. This reduces the proportion of stored data, thereby lowering costs. It is suitable for large-scale deployment in some large forests, ultimately achieving the goal of real-time temperature monitoring and fire prevention.
[0004] The technical solution adopted in this invention is as follows:
[0005] A thermal imaging acquisition system based on the Linux operating system, developed on the IMX6ullMINI development board, is characterized by including a video acquisition module, an image analysis module, a network communication module, and a client, with the network communication module connected to the video acquisition module, the image analysis module, and the client respectively.
[0006] The video capture module is used to capture video streams of the current environment and take pictures of the current scene through the camera, push the video stream to the client module, and send the pictures to the image analysis module at the same time.
[0007] The image analysis module is used to analyze and compare the received photos, identify high-temperature objects in the photos, and send the identification results to the client module.
[0008] The network communication module is used to enable network communication between the video acquisition module, the client module, and the image analysis module.
[0009] The client is used to display the obtained video stream and recognition results, and to provide users with operation interfaces for taking photos and making analysis requests.
[0010] Furthermore, the image analysis module includes a neural visual network analysis unit and a database, wherein,
[0011] The neural network analysis unit is used to analyze and compare the received data using convolutional neural network algorithms, identify high-temperature objects in the image, and return the results to the client.
[0012] A database used to store photos taken by the camera.
[0013] Furthermore, the convolutional neural network algorithm includes the following steps:
[0014] Step 1: Retrieve image data from the MySQL database;
[0015] Step 2: Label the different images with labelimg to indicate their state under different temperature ranges;
[0016] Step 3: Train the YOLOv5 model using temperature ranges;
[0017] Step 4: Input the collected on-site photos into the trained YOLOv5 model to identify outliers;
[0018] Step 5: Perform pseudo-color processing on the image with outliers, displaying different colors for different temperatures. Label the temperature using specific colors to obtain an infrared grayscale image.
[0019] Step 6: Input the infrared grayscale image into the improved CNN model for classification to identify high-temperature objects, i.e., abnormal high bright spots in the image.
[0020] Furthermore, the improved CNN model includes a first flat layer, a convolutional layer, a max pooling layer, another convolutional layer, a max pooling layer, a second flat layer, a hidden dense layer, and a softmax dense layer connected in sequence.
[0021] The first flat layer is used to convert the labels of the test image into a binary matrix;
[0022] Convolutional layers are used to extract features from the input;
[0023] The max pooling layer is used to perform subsampling on the feature maps learned by the convolutional layers. It uses max pooling to take the maximum value within the window as the output.
[0024] The second flat layer is used to stretch the trained data;
[0025] Hidden dense layers are used to define activation functions;
[0026] The Softmax dense layer is used to perform the Softmax operation and output a classification.
[0027] Furthermore, the network communication module communicates with the video acquisition module via the I2C and CSI protocols; the network communication module communicates with the client module and the image analysis module via the TCP / IP protocol.
[0028] A thermal imaging acquisition method based on the Linux operating system, implemented based on the thermal imaging acquisition system based on the Linux operating system as described in claim 1, characterized in that...
[0029] Step 1: Capture real-time images and photos of the current indoor environment using the video capture module;
[0030] Step 2: Send the collected photos to the image analysis module. The image analysis module will analyze and compare the photos with the images and send the comparison results to the client module through the network communication module.
[0031] Step 3: The client module displays the results to the user, who then determines whether a fire has occurred based on the results. If so, an alarm is triggered.
[0032] An electronic device, characterized in that it comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the thermal imaging acquisition method as described in claim 6.
[0033] A computer-readable storage medium storing a computer program, characterized in that, when executed by a processor, the computer program implements the thermal imaging acquisition method as described in claim 6. The beneficial effects of this invention are:
[0034] First, this invention collects real-time on-site data by taking photos at regular intervals, and uses a corresponding comparison algorithm to detect the collected photos to determine whether a fire has occurred. Compared with video data collected by monitoring equipment on the market, this invention can greatly reduce the proportion of stored data, thereby saving costs and making it suitable for widespread application.
[0035] Secondly, this invention trains the model on interval temperatures using a convolutional neural network algorithm. For example, 20-50 degrees Celsius is interval a, 50-100 degrees Celsius is interval b, and above 100 degrees Celsius is interval c. After extensive training, the model can be continuously adjusted and optimized to more accurately identify whether a high-temperature object is a human body or a fire source through interval recognition, thereby enabling more precise prediction of forest fires. Attached Figure Description
[0036] Figure 1 This is the architecture diagram of this system.
[0037] Figure 2 This is a structural diagram of the hardware component of the present invention.
[0038] Figure 3 This is an architecture diagram of the client side of the present invention.
[0039] Figure 4 This is the login interface for this invention.
[0040] Figure 5 This invention provides a user interface for selecting and analyzing images.
[0041] Figure 6 This is the result return interface of the present invention; the left image is the original detection image, and the right image is the analysis result return image. Detailed Implementation
[0042] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0043] This invention discloses a thermal imaging acquisition system based on the Linux operating system, comprising software and hardware components. The hardware component uses the Zhengdian Atom IMX6ullMINI EMMC board as the basis for kernel and u-boot flashing and compilation. Figure 2 As shown.
[0044] 1. System Design
[0045] (1) Hardware section
[0046] The u-boot uses NXP's official uboot-imx-rel_imx_4.1.15_2.1.0_ga version, and the kernel uses NXP's official uboot-imx-rel_imx_4.1.15_2.1.0_ga version, with modifications made to the LCD, camera, and QT driver based on this.
[0047] The LCD uses a 7-inch RGB screen from Zhengdian Atom, with a resolution of 1024*600. After driver modification, the LCD can be powered and used normally.
[0048] The camera uses the OV5640 model. The communication between the camera and Linux adopts the I2C and CSI protocols and the V42L video architecture. The camera driver was modified accordingly, and relevant camera information parameters were added to the I2C node. The kernel and device tree were modified, and the camera's photo-taking function was added.
[0049] The QT driver uses version QT6 and leverages QT to preprocess the acquired images.
[0050] The above modifications to the driver are all conventional driver modification methods in this field and do not constitute improvements of this invention.
[0051] (2) Software Part
[0052] like Figure 1 As shown, this system is divided into a video acquisition module, an image analysis module, a network communication module, and a client module.
[0053] ① Video capture module
[0054] The video acquisition module is used to capture video streams of the current environment through the OV5640 camera. Specifically, it uses the Linux kernel's V4L2 video driver to drive the OV5640 camera to acquire image data, and uses the camera's photo-taking function to take pictures of the scene. At the same time, it pushes the data acquired by the camera to the client module through the network communication module, and displays the acquired data on the client interface. In addition, when it receives a photo-taking operation request from the client, it takes a picture and uploads the captured image data to the database in the image analysis module.
[0055] ② Image Analysis Module
[0056] The image analysis module includes a neural visual network analysis unit and a MySQL database. The neural visual network analysis unit uses a convolutional neural network algorithm to analyze and compare the received data, identify high-temperature objects in the image, and return the comparison results to the client module. The MySQL database is used to store the photos taken by the camera.
[0057] The convolutional neural network algorithm includes the following steps:
[0058] Step 1: Retrieve image data from the MySQL database;
[0059] Step 2: Label the different images with labelimg to indicate their state under different temperature ranges;
[0060] Step 3: Train the YOLOv5 model using temperature ranges;
[0061] Step 4: Input the collected on-site photos into the trained YOLOv5 model to identify outliers;
[0062] Step 5: Perform pseudo-color processing on the image with outlier points, displaying different colors for different temperatures. Label the temperature using specific colors to obtain an infrared grayscale image.
[0063] Step 6: Input the infrared grayscale image into the improved CNN model for classification to identify high-temperature objects, i.e., abnormal high bright spots in the image.
[0064] The specific steps of step 3 are as follows: Since objects at different temperatures emit different amounts of radiation, this invention trains the YOLOv5 model through different temperature ranges. For example, 20-50 degrees Celsius is range a, 50-100 degrees Celsius is range b, and greater than 100 degrees Celsius is range c. After a large amount of training, the parameters of the model are continuously adjusted and optimized.
[0065] The CNN model described in step 6 includes a first flat layer, a convolutional layer, a max pooling layer, another convolutional layer, another max pooling layer, a second flat layer, a hidden dense layer, and a softmax dense layer connected in sequence.
[0066] The first flat layer is used to convert the labels of the test image into a binary matrix;
[0067] Convolutional layers are used to extract features from the input;
[0068] The max pooling layer is used to perform subsampling on the feature maps learned by the convolutional layers. It uses max pooling to take the maximum value within the window as the output.
[0069] The second flat layer is used to stretch the trained data;
[0070] Hidden dense layers are used to define activation functions;
[0071] The Softmax dense layer is used to perform the Softmax operation and output a classification.
[0072] Compared to other deep learning architectures, convolutional neural networks (CNNs) deliver better results for images. Traditional CNN models consist of one or more convolutional layers, dense hidden layers, and pooling layers. This invention adds a flattening layer at the initial position to flatten the image data. Image features are extracted through two convolutional layers and a max-pooling layer, enabling it to handle complex scenes. A second flattening layer further stretches the data, culminating in a softmax dense layer as the final output. The learned features are then applied to classification, and the final softmax dense layer ensures optimal prediction results.
[0073] The training steps of the CNN neural network include:
[0074] S1: Import the relevant libraries or datasets;
[0075] S2: Preprocess the label data (convert them into one-hot encodings) so that optimization can be performed using the label data; normalize the data, converting the test labels into a binary class matrix;
[0076] S3: Create training and testing datasets;
[0077] S4: Initialize the model;
[0078] S5: Define the model structure
[0079] S51: Specifies the number of hidden layers;
[0080] S52: Specifies the number of nodes in the hidden layer.
[0081] S53: Specifies the activation function to be executed in the hidden layer.
[0082] S54: Specify the loss function to be minimized
[0083] S55: Specifies the optimizer that minimizes the loss function.
[0084] S6: Fitting model;
[0085] S7: Test model;
[0086] S71: Validate the model using a test set;
[0087] S72: Calculate the accuracy and loss value of the test dataset;
[0088] S8: Examine the changes in loss and accuracy within each epoch, which is helpful for understanding the model's performance during training.
[0089] ③ Client
[0090] This invention uses an LNMP (Linux-Nginx-MySQL-PHP) architecture to build a web client. The LNMP architecture is as follows: Figure 2 As shown. The client receives the data streamed by the video acquisition module and the results analyzed by the image analysis module, and displays the results on the client's page. Users can perform both photo taking and analysis operations, and these requests are returned to the video acquisition module and the image analysis module via the network communication module.
[0091] like Figure 3 As shown, when the user clicks button 1, a photo capture request is sent to the video capture module, which takes a picture of the current camera image, saves it, and uploads it to the MySQL database. When the user operates button 2, an analysis request is sent to the image analysis module. The image analysis module retrieves the captured image data from the MySQL database to perform neural visual network analysis, identifies abnormal bright spots (high-temperature objects) in the image, and returns the results to the client's result return interface. At the same time, the real-time video stream captured by the camera is pushed to the client's push display interface.
[0092] ④ Network communication module
[0093] The network communication module includes I2C protocol, CSI protocol, and TCP / IP protocol. The video acquisition module and the network communication module communicate via I2C protocol and CSI protocol. The network communication module communicates with the client module and the image analysis module via TCP / IP protocol.
[0094] Based on the aforementioned thermal imaging system, this invention implements a thermal imaging acquisition method based on the Linux operating system, which includes the following steps:
[0095] Step 1: Capture real-time images and photos of the current indoor environment using the video capture module;
[0096] Step 2: Send the collected photos to the image analysis module. The image analysis module will analyze and compare the photos with the images and send the comparison results to the client module through the network communication module.
[0097] Step 3: The client module presents the comparison results to the user. If a fire occurs, the alarm mode is triggered immediately.
[0098] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.
Claims
1. A thermal imaging acquisition system based on the Linux operating system, developed using the IMX6ullMINI development board, characterized in that... It includes a video acquisition module, an image analysis module, a network communication module, and a client. The network communication module is connected to the video acquisition module, the image analysis module, and the client, respectively. The video capture module is used to capture video streams of the current environment and take pictures of the current scene through the camera, push the video stream to the client module, and send the pictures to the image analysis module at the same time. The image analysis module is used to analyze and compare the received photos, identify high-temperature objects in the photos, and send the identification results to the client module. The network communication module is used to enable network communication between the video acquisition module, the client module, and the image analysis module. The client is used to display the obtained video stream and recognition results, and to provide users with operation interfaces for taking photos and making analysis requests.
2. The thermal imaging acquisition system based on the Linux operating system as described in claim 1, characterized in that, The image analysis module includes a neural visual network analysis unit and a database, wherein... The neural network analysis unit is used to analyze and compare the received data using convolutional neural network algorithms, identify high-temperature objects in the image, and return the results to the client. A database used to store photos taken by the camera.
3. The thermal imaging acquisition system based on the Linux operating system as described in claim 2, characterized in that, The convolutional neural network algorithm includes the following steps: Step 1: Retrieve image data from the MySQL database; Step 2: Label the different images with labelimg to indicate their state under different temperature ranges; Step 3: Train the YOLOv5 model using temperature ranges; Step 4: Input the collected on-site photos into the trained YOLOv5 model to identify outliers; Step 5: Perform pseudo-color processing on the image with outliers, displaying different colors for different temperatures. Label the temperature using specific colors to obtain an infrared grayscale image. Step 6: Input the infrared grayscale image into the improved CNN model for classification to identify high-temperature objects, i.e., abnormal high bright spots in the image.
4. The thermal imaging acquisition system based on the Linux operating system as described in claim 3, characterized in that, The improved CNN model includes a first flat layer, a convolutional layer, a max pooling layer, another convolutional layer, another max pooling layer, a second flat layer, a hidden dense layer, and a softmax dense layer connected in sequence. The first flat layer is used to convert the labels of the test image into a binary matrix; Convolutional layers are used to extract features from the input; The max pooling layer is used to perform subsampling on the feature maps learned by the convolutional layers. It uses max pooling to take the maximum value within the window as the output. The second flat layer is used to stretch the trained data; Hidden dense layers are used to define activation functions; The Softmax dense layer is used to perform the Softmax operation and output a classification.
5. The thermal imaging acquisition system based on the Linux operating system as described in claim 2, characterized in that, The network communication module communicates with the video acquisition module via I2C and CSI protocols; the network communication module communicates with the client module and the image analysis module via TCP / IP protocol.
6. A thermal imaging acquisition method based on the Linux operating system, implemented based on the thermal imaging acquisition system based on the Linux operating system as described in claim 1, characterized in that, Step 1: Capture real-time images and photos of the current indoor environment using the video capture module; Step 2: Send the collected photos to the image analysis module. The image analysis module will analyze and compare the photos with the images and send the comparison results to the client module through the network communication module. Step 3: The client module displays the results to the user, who then determines whether a fire has occurred based on the results. If so, an alarm is triggered.
7. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the thermal imaging acquisition method as described in claim 6.
8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the thermal imaging acquisition method as described in claim 6.