A low-cost high-precision heat screening method based on infrared camera

Abstract

The application discloses a low-cost high-precision fever screening method based on an infrared camera. In view of the problems of difficult face detection, low fever judgment precision and high cost in the fever screening system based on the infrared camera in the market, the application provides a low-cost high-precision fever screening method based on the infrared camera. Specifically, a visible light and infrared binocular camera system is built to assist infrared face detection; a historical difference framework is constructed to overcome the influence of environmental changes; a position enhancement module is proposed to overcome the influence of distance changes; and a self-supervised template generation network is proposed to extract effective face temperature information in the historical crowd. The application can realize high fever judgment precision only by relying on the temperature measurement sensitivity of the infrared camera, greatly reduces the cost of fever screening, and improves the adaptability to complex environments.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically a low-cost, high-precision fever screening method based on an infrared camera. Background Technology

[0002] The COVID-19 pandemic has ravaged the globe, severely impacting human life worldwide. The virus can spread through the air, making prevention and control extremely difficult, especially in densely populated areas. Without effective prevention and control measures, the consequences of a virus outbreak would be unimaginable. Fever is the most common symptom of various infectious diseases. Determining whether someone has a fever by measuring body temperature is an effective way to initially screen for potentially infected individuals. Therefore, temperature measurement plays a crucial role in epidemic prevention and control, and is typically required when entering places such as train stations, shopping malls, and universities.

[0003] Existing temperature measurement devices include mercury thermometers, temperature guns, and non-contact thermometers based on infrared cameras. Thermometers are commonly used for accurate temperature measurement of patients in hospitals, while temperature guns are cumbersome and slow. Non-contact thermometers based on infrared cameras offer fast temperature measurement and are currently widely used in public places with high traffic.

[0004] However, non-contact thermometers based on infrared cameras still face several challenges. One major issue is the difficulty of infrared face detection. Most face detection algorithms perform well in visible light images but poorly in infrared images, making face detection in infrared images a significant factor hindering the further development of thermometers. Secondly, the temperature is affected by changes in the measurement environment and distance, making it difficult to determine whether a person's body temperature is abnormal based solely on infrared facial information. The facial temperature measured by infrared cameras varies considerably under different environments and distances, making it difficult to determine whether a person is overheating using only threshold methods, resulting in extremely low accuracy in fever detection by screening devices. Some devices have attempted to improve accuracy by adding ambient temperature sensors, distance sensors, or even real-time blackbody calibration, but this undoubtedly increases the cost, failure rate, and deployment difficulty of the temperature measurement equipment. Summary of the Invention

[0005] The purpose of this invention is to address the problems existing in the prior art by providing a low-cost, high-precision heat screening method based on an infrared camera. This method constructs a visible light and infrared binocular camera system to assist in infrared face detection; builds a historical difference framework to overcome the influence of environmental changes; proposes a position enhancement module to overcome the influence of distance changes; and proposes a self-supervised template generation network to extract effective facial temperature information from historical populations.

[0006] The technical solution to achieve the objective of this invention is: a low-cost, high-precision heat screening method based on an infrared camera, the method comprising the following steps:

[0007] Step 1: Set up a visible light and infrared binocular camera system and control the two cameras to synchronously acquire images of the person being tested;

[0008] Step 2: Use a face detection algorithm to detect faces in the visible light image;

[0009] Step 3: Map the results of Step 2 onto the infrared image to detect faces in the infrared image;

[0010] Step 4: Based on the face detection results in Step 3, determine whether the body temperature of the person being tested is abnormal.

[0011] Furthermore, in step 1, when acquiring images, the infrared camera's acquisition cycle is used as a pulse signal to trigger the visible light camera to acquire data, and the time difference between the two cameras is controlled by the trigger delay time, so that the two cameras can acquire images at the same time.

[0012] Furthermore, step 3, which involves detecting faces in the infrared image based on the results of step 2, specifically includes the following process:

[0013] Step 3-1: Map the detected face bounding boxes in the visible light image to the infrared image;

[0014] Step 3-2, correct the mapping deviation. Specifically, take the face bounding box mapped to the infrared image as the reference and set several candidate boxes of the same size on the left and right of it on the same axis.

[0015] Step 3-3: Score the face completeness of each candidate box, and take the candidate box with the highest score as the final infrared face detection result.

[0016] Furthermore, in step 3-3, the face completeness of each candidate box is scored based on a scoring network model. The construction process of the scoring network model is as follows:

[0017] Construct the initial convolutional neural network;

[0018] Acquire several complete infrared face images and images containing partial infrared faces;

[0019] Using complete infrared face images as positive samples and images containing partial infrared faces as negative samples, the initial convolutional neural network is trained using the cross-entropy loss function to obtain a scoring network model. This model can output the confidence score of the infrared face in the image as a score value; the more complete the infrared face in the image, the higher the confidence score of the network output, i.e., the higher the score.

[0020] Furthermore, step 4, which involves determining whether the body temperature of the person being tested is abnormal based on the face detection results from step 3, specifically includes the following process:

[0021] Step 4-1: Arrange the pixel values ​​in the infrared face image to be tested in descending order to form a facial infrared vector;

[0022] Step 4-2: Collect facial infrared vectors of people with normal body temperature from a previous period of time and generate a template;

[0023] Step 4-3: Calculate the L1 distance (Manhattan distance) between the infrared vector of the subject's face and the template. If the value exceeds the preset threshold, it indicates that the subject is generating heat; otherwise, it indicates that the subject is not generating heat.

[0024] Furthermore, the specific process of generating the template in step 4-2 includes:

[0025] Step 4-2-1: Construct a template generation network, which consists of a fully connected neural network. The input dimension of the fully connected layer in the network is N, which decreases layer by layer until a one-dimensional value is finally output. N is the total number of people collected.

[0026] Step 4-2-2: Merge the facial infrared vectors of the historically normal body temperature population into a matrix. Each column represents the facial infrared vector of each person. N people will have N columns. Each vector has a length of M, so the matrix will have M rows and N columns.

[0027] Step 4-2-3: Assign different weights to each vector in the matrix. The weighting function is an exponential function. The input to the function is the reciprocal of the height difference between the test face and the face corresponding to the vector.

[0028] Step 4-2-4: Train the template generation network using infrared facial images of people with normal body temperature;

[0029] Step 4-2-5: Input the matrix from step 4-2-2 into the trained template generation network to generate templates.

[0030] Furthermore, step 4-2-4, which involves training the template generation network using infrared facial images of people with normal body temperature, specifically includes the following process:

[0031] The matrix formed in step 4-2-2 is used as training data, and the facial infrared vector of the person being tested is used as supervision information. The template generation network is trained by constraining the training process with L1 norm.

[0032] Furthermore, the method also includes the following steps:

[0033] Step 5: When the detection result in Step 4 indicates an abnormal body temperature, the audio-visual output module issues a warning through display and audio broadcast.

[0034] Compared with the prior art, the present invention has the following significant advantages:

[0035] 1) The method of the present invention relies solely on the sensitivity of the infrared camera to distinguish the differences in infrared faces between different people, without the need to accurately measure the temperature of each face. Therefore, a low-cost infrared camera can be used instead of a high-precision infrared camera to reduce equipment costs.

[0036] 2) This invention achieves accurate alignment of the acquisition times of infrared and visible light images by controlling the acquisition cycle of the infrared camera to trigger the acquisition of the visible light camera; and by evaluating the completeness of the infrared candidate bounding box for faces, it achieves accurate alignment of the spatial positions of infrared and visible light faces. Therefore, this invention can accurately detect the position of faces in infrared images using a binocular camera system with both visible light and infrared sensors. Compared with existing binocular camera systems, this invention has higher face detection accuracy.

[0037] 3) This invention generates templates from historical populations. Leveraging the fact that the environments of short-term historical populations and the individuals being tested are largely consistent, it overcomes the impact of environmental changes on fever screening. Furthermore, by increasing the weight of the vectors corresponding to faces at the same distance from the individual being tested within the historical population, it overcomes the impact of distance changes on infrared temperature measurement. Compared to existing methods that add distance sensors, temperature sensors, and blackbody, this invention only requires a small amount of computation at the algorithm level to solve the problem of reduced temperature measurement accuracy caused by environmental and distance changes, significantly reducing the cost and complexity of the equipment.

[0038] 4) This invention fully extracts effective infrared facial information from historical populations and generates templates by designing a specific neural network. Compared with the existing method of calculating the average temperature of historical populations, it can extract richer facial temperature information and achieve higher accuracy in fever judgment.

[0039] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0040] Figure 1 This is a diagram of the device hardware connections.

[0041] Figure 2 This is a schematic diagram of a face projection deviation correction strategy.

[0042] Figure 3 This is a schematic diagram of the historical differences framework.

[0043] Figure 4 A schematic diagram of the template generation network and its training and testing methods.

[0044] Figure 5 This is a diagram illustrating the generation correspondence between historical populations and templates.

[0045] Figure 6 This is a flowchart of the system software. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] In one embodiment, combined Figure 6 This paper provides a low-cost, high-precision heat screening method based on an infrared camera, the method comprising the following steps:

[0048] Step 1: Set up a visible light and infrared binocular camera system and control the two cameras to synchronously acquire images of the person being tested;

[0049] Here, for example, a visible light and infrared binocular camera system includes a visible light camera, an infrared camera, a computer, a monitor, speakers, and other accessories. The visible light camera and the infrared camera, as data acquisition devices, are both connected to the computer via network cables. The two cameras are fixed on the same rigid plane with parallel optical axes. Algorithms such as face detection and heat assessment are implemented on the computer. System output is achieved through the monitor and speakers. Specific hardware connection methods are as follows: Figure 1 As shown.

[0050] Here, to maintain high performance, infrared cameras need to acquire data at fixed intervals. Even when controlled by software triggering, the returned image is still the most recent image acquired within that fixed acquisition cycle, making it impossible to accurately control the infrared camera to operate in software-triggered mode. Visible light cameras have more mature imaging technology, and when operating via software triggering, they can acquire an image within a fixed time after triggering. However, due to the influence of light, visible light cameras need to automatically adjust their exposure time. Changes in exposure time lead to changes in image acquisition time, especially in low-light environments where longer exposure times are often required to ensure image brightness, resulting in longer capture times for a single image. Because of the inconsistency in the working principles of visible light and infrared cameras, it is difficult to control simultaneous image acquisition by both cameras. When capturing dynamic targets in a scene, the inconsistency in the imaging time of the two cameras may cause significant deviations in the two images of the same target, ultimately leading to target matching errors. To maximize control over synchronous data acquisition by the dual cameras, this invention treats the fixed operating cycle of the infrared camera as a clock signal. At the beginning of each cycle, the visible light camera is triggered after a delay, the delay time of which depends on the exposure time of the visible light camera. Within each cycle, one infrared image and one visible light image can be acquired. This relatively fixed acquisition cycle ensures that the two cameras acquire images at roughly the same time. Finally, the acquired images are stored in a buffer queue for later use.

[0051] Preferably, the visible light camera is a Medvision GigE color industrial camera; the infrared camera is a FOTRIC680 ​​series thermal infrared camera; the computer configuration is: Intel(R) Core(TM) i5-8400 CPU, 16GB RAM; the monitor is a regular computer monitor, and the speakers are regular speakers.

[0052] Step 2: Use a face detection algorithm to detect faces in the visible light image;

[0053] Here, the visible light face detection model is constructed by a deep neural network, mainly consisting of convolutional layers and fully connected layers. The model is trained on the WIDER-Face large face dataset, which contains faces in various scenes. The face detection model trained on this dataset can basically meet the needs of practical use.

[0054] Step 3: Map the result of Step 2 onto the infrared image to detect faces in the infrared image; the mapping process can be simplified to obtaining the face position in the infrared image by translating and scaling the face bounding box in the visible light image, and the rotation and translation parameters are calculated by the camera's intrinsic and extrinsic parameters.

[0055] Step 4: Based on the face detection results in Step 3, determine whether the body temperature of the person being tested is abnormal;

[0056] Step 5: When the detection result in Step 4 indicates an abnormal body temperature, the audio-visual output module issues a warning through a display and an audio broadcast. The fever screening device is equipped with a display and audio equipment. When the device is working, the display shows the images captured by the visible light camera and infrared camera in real time, and marks the detected faces in the image with bounding boxes. When the body temperature is normal, a green bounding box is used; when the body temperature is abnormal, a red bounding box is used, and a warning sound signal is emitted through the audio system.

[0057] Furthermore, in one embodiment, step 3, which involves detecting a face in the infrared image based on the result of step 2, specifically includes:

[0058] Step 3-1: Map the detected face bounding boxes in the visible light image to the infrared image;

[0059] Step 3-2: Correct the mapping deviation using the candidate box offset method, such as... Figure 2 As shown, specifically: taking the face bounding box mapped onto the infrared image as a reference, several candidate boxes of the same size are set on the left and right of it on the same axis, and the candidate boxes cover the face position more accurately;

[0060] Preferably, four candidate boxes are arranged on each side of the projected bounding box with a step size of 1 / 8 of the bounding box width.

[0061] Step 3-3: Score the face completeness of each candidate box, and take the candidate box with the highest score as the final infrared face detection result.

[0062] Furthermore, in one embodiment, step 3-3 scores the face completeness of each candidate box based on a scoring network model. The construction process of the scoring network model is as follows:

[0063] Construct the initial convolutional neural network;

[0064] Acquire several complete infrared face images and images containing partial infrared faces;

[0065] Using complete infrared face images as positive samples and images containing partial infrared faces as negative samples, the initial convolutional neural network is trained using the cross-entropy loss function to obtain a scoring network model. This model can output the confidence score of the infrared face in the image as a score value; the more complete the infrared face in the image, the higher the confidence score of the network output, i.e., the higher the score.

[0066] Furthermore, step 4, which determines whether the body temperature of the person being tested is abnormal based on the face detection results from step 3, specifically involves constructing a historical difference framework to overcome the influence of environmental changes, combined with... Figure 3 The process includes:

[0067] Step 4-1: Sort the pixel values ​​in the infrared face image to be tested in descending order to form a facial infrared vector. The vector contains more information than the maximum or average facial temperature.

[0068] Step 4-2: Collect facial infrared vectors of people with normal body temperature from a previous period of time and generate a template;

[0069] Step 4-3: Calculate the L1 distance (Manhattan distance) between the infrared vector of the subject's face and the template. If the value exceeds the preset threshold, it indicates that the subject is generating heat; otherwise, it indicates that the subject is not generating heat.

[0070] Here, due to the influence of face coverings such as masks and glasses, coupled with the uncertainty of facial posture, it is difficult to accurately obtain the temperature of the effective skin area of ​​the face. To solve this problem, this invention proposes a facial infrared vector. The heat from the covering usually comes from the facial skin, so its temperature is often lower than the skin temperature. In the facial infrared vector, the pixel values ​​of the skin area are usually ranked first, and the pixel values ​​of the non-skin area are usually ranked last. This processing method can effectively obtain the effective temperature information of the facial skin area.

[0071] Traditional screening devices only extract the average or maximum temperature of a certain area of ​​the face and use it as a feature for judging fever. If the temperature is above a threshold, it is considered a fever; if it is below the threshold, it is considered normal. This method of judging fever is easily affected by factors such as the temperature measurement environment and measurement distance. The historical difference framework proposed in this invention effectively solves the above problems.

[0072] Furthermore, the specific process of generating the template in step 4-2 includes:

[0073] Step 4-2-1: Construct a template generation network, which consists of a fully connected neural network. The input dimension of the fully connected layer in the network is N, which decreases layer by layer until a one-dimensional value is finally output. N is the total number of people collected.

[0074] Step 4-2-2: Merge the facial infrared vectors of the historically normal body temperature population into a matrix. Each column represents the facial infrared vector of each person. N people will have N columns. Each vector has a length of M, so the matrix will have M rows and N columns.

[0075] Step 4-2-3: Assign different weights to each vector in the matrix. The weighting function is an exponential function. The input to the function is the reciprocal of the height difference between the test face and the face corresponding to the vector.

[0076] Here, the face height difference is calculated as follows: the distance from the face to the infrared camera is estimated based on the size of the face bounding box, the distance difference between the person being tested and each person in the historical group is calculated, and this distance difference is used as the height difference;

[0077] Here, the infrared camera calculates temperature based on the intensity of infrared radiation emitted by a face. Since infrared radiation attenuates as it travels through the air, and the degree of attenuation is affected by factors such as ambient temperature, humidity, and particulate matter, it exhibits varying degrees of positive correlation with the propagation path length. Therefore, the infrared camera will have varying degrees of deviation when measuring faces at different distances in different environments. To overcome the influence of the face-to-lens distance on temperature measurement, further improvements were made based on the historical difference representation. By assigning weights, the contribution of infrared face vectors with smaller distance differences from the person being measured within the historical crowd to the template is highlighted, while vectors with larger distance differences are weakened. This emphasizes the role of infrared face vectors at the same distance as the person being measured within the historical crowd, ultimately limiting the historical crowd to the same distance and environment, thus simultaneously overcoming the influence of environment and distance on inaccurate temperature measurement. Furthermore, the method of this invention relies solely on the sensitivity of the infrared camera to distinguish infrared face differences between different individuals, without needing to accurately measure the temperature of each face. Therefore, a low-cost infrared camera can be used instead of a high-precision infrared camera to reduce equipment costs.

[0078] Step 4-2-4: Train the template generation network using infrared facial images of people with normal body temperature;

[0079] Step 4-2-5: Input the matrix from step 4-2-2 into the trained template generation network to generate templates.

[0080] Furthermore, in one embodiment, a self-supervised template generation network is proposed to extract effective facial temperature information from historical populations. The self-supervised template generation network can generate templates from infrared face vectors of historical populations and consists of fully connected layers, with the specific structure as follows: Figure 4 As shown, the network output is consistent with the size of the infrared face vector of the person being tested. The fully connected layers are applied sequentially to all vectors of the historical population from high temperature to low temperature, i.e., high temperature regions generate high temperature regions of the template, and low temperature regions generate low temperature regions of the template. In step 4-2-4, when training the template generation network using infrared face images of people with normal body temperature, infrared face images of several people with normal body temperature were collected as training data, and the infrared face vector of the person being tested was used as the supervision information for generating the template, with the L1 norm constraining the training process.

[0081] Here, a self-supervised template generation network is used to encode historical population data, generating normal infrared face vectors of individuals at the same distance from the test subject in the current environment. Fully connected layers are used to encode the high-temperature to low-temperature regions of the population vectors sequentially, such as... Figure 5 As shown, different positions of the template vector are generated by a fully connected network from the positions corresponding to historical crowd vectors. The data collected for training the network includes infrared face vectors from different environments to improve the network's generalization ability.

[0082] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A low-cost high-precision heat-emitting screening method based on an infrared camera, characterized in that, The method comprises the following steps: Step 1, build a visible light and infrared binocular camera system, control the synchronous acquisition of the double cameras to obtain the image of the person to be measured; Step 2, detect the face in the visible light image using a face detection algorithm; Step 3, map the result of step 2 to the infrared image to detect the face in the infrared image; Step 4, judge whether the body temperature of the person to be measured is abnormal based on the face detection result of step 3; the specific process comprises: Step 4-1, arrange the pixel values in the infrared face image of the person to be measured in descending order to form a facial infrared vector; Step 4-2, collect the facial infrared vectors of the historical normal temperature crowd measured in a period of time before the person to be measured and generate a template; Step 4-3, calculate the L1 distance, i.e. Manhattan distance, between the facial infrared vector of the person to be measured and the template; if the distance exceeds a preset threshold, it indicates fever, otherwise it indicates no fever; The specific process of generating the template in step 4-2 comprises: Step 4-2-1, construct a template generation network, which is composed of a fully connected neural network, the input dimension of the fully connected layer in the network is N, then the dimension is reduced layer by layer, and finally a one-dimensional value is output; the N is the total number of people collected; Step 4-2-2, combine the collected facial infrared vectors of the historical normal temperature crowd measured into a matrix, each column represents the facial infrared vector of each person, N people are N columns, and each vector length is M, so the matrix is M rows and N columns; Step 4-2-3, assign different weights to each vector in the matrix, the weight function is an exponential function, and the input of the function is the inverse of the height difference between the face to be measured and the face corresponding to each vector; Step 4-2-4, train the template generation network using the infrared face image of normal temperature; Step 4-2-5, input the matrix of step 4-2-2 into the trained template generation network to generate a template.

2. The low cost high accuracy heat screening method based on infrared camera according to claim 1, characterized in that, In step 1, when collecting images, the acquisition cycle of the infrared camera is used as a pulse signal to trigger the visible light camera to collect data, and the time difference between the two cameras is controlled through the delay time of the trigger to realize the acquisition of images at the same time by the two cameras.

3. The low cost high accuracy heat emitting screening method based on infrared camera as claimed in claim 1, wherein, The specific process of detecting the face in the infrared image based on the result of step 2 in step 3 comprises: Step 3-1, map the face bounding box detected in the visible light image to the infrared image; Step 3-2, correct the mapping deviation, specifically: set several candidate boxes of the same specification on the left and right of the face bounding box mapped to the infrared image as the reference; Step 3-3, score the completeness of the face in each candidate box, and take the candidate box with the highest score as the final infrared face detection result.

4. The low cost high accuracy heat emitting screening method based on infrared camera according to claim 3, characterized in that, In step 3-3, the completeness of the face in each candidate box is scored based on a scoring network model, and the construction process of the scoring network model is as follows: Construct an initial convolutional neural network; Collect several complete infrared face images and images containing partial infrared faces; The initial convolutional neural network is trained by taking a complete infrared face image as a positive sample, taking an image containing a partial infrared face as a negative sample, and using a cross-entropy loss function, to obtain a scoring network model. The model can output the confidence of the infrared face in the image as a score. The more complete the infrared face contained in the image, the higher the confidence output by the network, that is, the higher the score.

5. The low cost high accuracy heat emitting screening method based on infrared camera as claimed in claim 1, wherein, The template generation network is trained using infrared face images with normal body temperature according to step 4-2-4, and the specific process includes: The matrix formed in step 4-2-2 is used as training data, and the facial infrared vector of the person to be tested is used as supervision information to train the template generation network with L1 norm constraint.

6. The low cost high accuracy heat emitting screening method based on infrared camera according to claim 1, wherein, The method further includes the following steps: Step 5: When the detection result of step 4 is abnormal body temperature, the sound-light output module issues a warning through the display and sound broadcast.

Images