A non-contact body temperature measurement method and system

Abstract

The present invention discloses a non-contact body temperature measurement method and system, belonging to the field of temperature measurement technology. In view of the problems that infrared temperature measurement in the prior art has large restrictions on the measurement position and low measurement accuracy, the present invention provides a non-contact body temperature measurement method and system, which uses a visible light camera and a thermal camera for data collection at the same time, detects the image data collected by the camera through a face detection algorithm, and then performs face matching on the face data collected by the visible light camera and the face data collected by the thermal camera, reads the forehead temperature set in combination with the range of the forehead, processes the forehead temperature data, and obtains the final measured temperature. The measurement result can also be displayed or alarmed through a human-computer interaction device. The present invention can realize long-distance non-contact body temperature measurement, and has high measurement accuracy and strong reliability, and is particularly suitable for security inspection in public places such as stations and airports.

Description

Technical Field

[0001] The present invention relates to the technical field of temperature measurement, and more specifically, to a non-contact body temperature measurement method and system. Background Art

[0002] In the prior art, infrared cameras are mainly used for temperature measurement in crowded public places. The measurement principle is to use an infrared camera to capture the shape of the human body, then detect the position of the human body, and use infrared technology to measure the temperature. During the temperature measurement process, the person being measured needs to stand in a designated position, and the distance between the person being measured and the infrared camera should generally not exceed 1 meter, so the temperature measurement efficiency is relatively low. At the same time, due to the influence of environmental factors, the infrared measurement of the prior art has low measurement accuracy, insufficient reliability, and poor practicality. Summary of the invention

[0003] 1. Technical issues to be solved

[0004] In view of the problems existing in the prior art such as large restrictions on the measurement position and low measurement accuracy of infrared temperature measurement, the present invention provides a non-contact body temperature measurement method and system, which can realize long-distance non-contact body temperature measurement with high measurement accuracy, and is particularly suitable for security inspection in public places such as stations and airports.

[0005] 2. Technical solution

[0006] The purpose of the present invention is achieved through the following technical solutions.

[0007] A non-contact body temperature measurement method, using a visible light camera to collect visible light images, using a thermal camera to collect thermal images and temperature, performing face detection on the collected visible light images and thermal images respectively, performing face pairing on the visible light face image and the thermal face image obtained after face detection, matching the temperature collected by the thermal camera to the successfully matched face image, obtaining the forehead temperature of the face image and processing it, and obtaining body temperature data. The present invention matches the image collected by the visible light camera with the image collected by the thermal camera, reads the temperature collected by the thermal camera based on the image collected by the visible light camera, and realizes non-contact measurement. The present invention has no requirements for the measurement environment or measurement distance, can realize mobile non-contact measurement, and has high measurement accuracy and strong reliability.

[0008] Furthermore, the face detection model used in face detection is built using a deep convolutional neural network detection algorithm.

[0009] Furthermore, the collected visible light face image is input to the visible light face detection model for detecting the position of the face in the visible light image and determining whether the face is wearing a mask; the collected thermal face image is input to the thermal face detection model for detecting the position of the face in the thermal image. The face detection model of the present invention includes a visible light face detection model and a thermal face detection model. During face detection, the face detection model will be retrained according to the collected data to obtain a more accurate face detection result.

[0010] Furthermore, the epipolar geometry algorithm is used for face matching. The thermal face image is matched with the visible light face image based on the face center of the visible light face image and the epipolar line of the corresponding thermal face image. When matching faces, the epipolar line in the thermal camera is calculated from the face center detected in the visible light image. If there is a face detected near the epipolar line in the thermal image, then the face closest to the epipolar line is the corresponding face, thus completing the matching of the faces detected in the visible light image and the thermal image. Generally speaking, in an image with a resolution of 320*240, the epipolar line refers to a distance of 0-40 pixels from the epipolar line.

[0011] Furthermore, when matching faces, the spatial correspondence between the visible light camera and the thermal camera reconstructed by the stereo vision algorithm is used to calculate the epipolar lines. The correspondence is represented by a basic matrix, which is calculated using a checkerboard calibration method. During the checkerboard calibration, the thermal camera is irradiated with sunlight to form a temperature difference. Due to the high absorption temperature of black and high reflection temperature of white, when sunlight is used to illuminate the checkerboard for a long time, a significant temperature difference will appear. The obvious temperature difference causes the thermal camera to capture different display images. The basic matrix is ​​calculated using the checkerboard calibration algorithm.

[0012] Furthermore, according to the ratio of the upper and lower border lines of the forehead in the visible light face image in the rectangular frame of the face, the temperature in the corresponding ratio area of ​​the thermal face image is read, that is, the forehead temperature of the face image. First, a large amount of data is collected through the visible light camera, and the ratio of the upper and lower border lines of the forehead in the detected face rectangular frame is calculated. Combined with the thermal image to detect the face image, the forehead position is estimated according to the statistical ratio of the upper and lower borders of the forehead in the rectangular frame, and the temperature of the forehead position is read.

[0013] Furthermore, the temperature at the forehead position of the face image is processed, and the temperatures of all pixels at the forehead position are sorted in descending order. The first X1 temperature values ​​in the sequence are selected for calibration, and then the calibrated temperatures are sorted in descending order. The first X2 temperature values ​​in the sequence are selected to take the average, and the average is the body temperature value. X1 and X2 are both positive integers, and X1≥X2.

[0014] Furthermore, the temperature measurement result is displayed or alarmed through a human-computer interaction device. The temperature of the person being measured, whether to wear a mask, and other information obtained by combining the data collected by the visible light camera with the algorithm are displayed through the human-computer interaction device, and real-time alarms are issued for people who do not wear masks or have abnormal body temperatures in public places as required, or for people with abnormal travel history according to big data. The present invention uses a visible light camera and a thermal camera to collect data at the same time, performs face detection and matching on the data collected by the visible light camera and the thermal camera, obtains the forehead temperature of the person being measured as the measured temperature, and the measurement method is simple and does not require close contact. Using the measurement method of the present invention to measure within a range of 0.5 meters to 5 meters, the absolute value of the difference between the detected temperature and the marked temperature is 0.208, the absolute value of the difference between the detected temperature and the marked temperature is less than 0.3, accounting for 73.5%, the absolute value of the difference between the detected temperature and the marked temperature is less than 0.5, accounting for 89%, and the average value of the difference between the detected temperature and the marked temperature is -0.0054, and the standard deviation is 0.258.

[0015] A non-contact body temperature measurement system uses the non-contact body temperature measurement method. The measurement system includes a visible light camera, a thermal camera, an embedded processor and a human-computer interaction device. The visible light camera and the thermal camera are connected to the embedded processor through a connector. The visible light camera and the thermal camera both send collected information to the embedded processor, and the embedded processor sends processed data to the human-computer interaction device.

[0016] Furthermore, the relative positions of the visible light camera and the thermal camera are fixed. The visible light camera and the thermal camera are relatively fixed. The measurement system of the present invention does not need to consider the distance between the cameras when calculating face matching, and has a fast calculation speed, high matching efficiency, and stronger system reliability.

[0017] The measurement system of the present invention has a simple structure, uses a high-performance embedded processor for data processing, has a fast measurement speed, high measurement accuracy, and simple later maintenance. It can not only realize non-contact mobile temperature measurement, but also calibrate the temperature measured by the thermal camera in real time, has high measurement accuracy, and is suitable for application in public places such as stations, airports, and schools.

[0018] 3. Beneficial effects

[0019] Compared with the prior art, the advantages of the present invention are:

[0020] The temperature measurement system of the present invention realizes non-contact measurement. Unlike traditional infrared cameras, it does not require people to stand in a fixed position. As long as the face appears in the imaging of the camera, the temperature can be measured. The present invention can realize the simultaneous measurement of multiple faces. In theory, the faces within the field of view of the thermal image can measure the body temperature. The present invention has a fast measurement speed and is particularly suitable for temperature measurement in public places, which can greatly improve the efficiency of measuring body temperature. The present invention does not require a high measurement distance, and the preparation measurement can be realized between 0.5 meters and 5 meters, effectively widening the distance between people during measurement, realizing mobile body temperature measurement, avoiding close contact during temperature measurement, and avoiding the spread of viruses.

[0021] The measurement method of the present invention has very strong robustness. For common infrared thermometers, if the hair covers the forehead, it is necessary to use hands to push the hair aside, otherwise it will affect the measurement result. However, the temperature measurement system of the present invention uses thermistor temperature measurement. As long as the forehead part is not completely blocked, the gap part can be accurately found to measure the body temperature. In most cases, the measurement can be achieved by simply exposing the subject in front of the camera of the measurement system, which is very convenient.

[0022] The present invention adopts a deep learning method, and whether it is a visible light image or a thermal image, it can accurately find the position of the face in various complex situations. Unlike the prior art that requires close-range temperature measurement within a range of 0.5 meters to 1 meter, the present invention can achieve temperature measurement from 0.5 meters to 5 meters. After a limited number of test experiments, the absolute value of the difference between the detected temperature and the marked temperature is 0.208 degrees Celsius, the absolute value of the difference between the detected temperature and the marked temperature is less than 0.3 degrees Celsius, accounting for 73.5%, and the absolute value of the difference between the detected temperature and the marked temperature is less than 0.5 degrees Celsius, accounting for 89%, and the average value of the difference between the detected temperature and the marked temperature is -0.0054, and the standard deviation is 0.258, with high measurement accuracy and strong reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a block diagram of the body temperature measurement system of the present invention;

[0024] Figure 2 Schematic diagram of the epipolar geometry algorithm for face pairing according to the present invention. DETAILED DESCRIPTION

[0025] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0026] Example 1

[0027] A non-contact body temperature measurement system Figure 1As shown, the measurement system includes a visible light camera, a thermal camera, an embedded processor and a human-computer interaction device, and also includes other auxiliary equipment; the embedded processor is connected to the visible light camera, the thermal camera and the human-computer interaction device, and the visible light camera and the thermal camera are connected to the embedded processor through a FAKRA connector; the visible light camera is used to collect visible light image information, the thermal camera is used to collect thermal image information and temperature information, the embedded processor uses a face detection algorithm, a face matching algorithm and a forehead temperature measurement algorithm to process the camera collected data, and the human-computer interaction device displays and alarms the processed data.

[0028] The embedded processor uses a high-performance deep convolutional neural network algorithm to detect the facial information collected by the visible light camera and the thermal camera, and detects whether the face in the face data collected by the visible light camera is wearing a mask. The corresponding faces in the images collected by the visible light camera and the thermal camera are paired through the stereo vision algorithm and the epipolar geometry algorithm. In the image collected by the thermal camera, the forehead position is estimated according to the face ratio and the temperature is collected. The image collected by the visible light camera, whether a mask is worn, the forehead temperature, and other relevant information obtained by the big data algorithm based on the face information are output to the human-computer interaction device to realize information display or alarm.

[0029] Example 2

[0030] This embodiment is based on the non-contact body temperature measurement system described in Embodiment 1, and specifically describes the measurement method of the system.

[0031] When measuring body temperature, the system first collects data through a visible light camera and a thermal camera respectively, and performs face detection on the collected data. The visible light camera collects RGB images, that is, visible light images, and the thermal camera collects thermal images. Through the deep face detection algorithm, faces are detected in visible light images and thermal images. This system uses NVIDIA Jetson-TX2 embedded processor for face detection. It takes about 75 milliseconds to detect faces in visible light images with a resolution of 480x640, and it takes about 45ms to detect faces in thermal images with a resolution of 320x240. Compared with the prior art face detection that takes about 100 milliseconds, this measurement system has shorter detection time and higher efficiency.

[0032] Specifically, this embodiment adopts an object detection algorithm of a deep convolutional neural network. The specific method is to transmit the image to the network, first extract the general features of the image through the backbone network framework (mobilenet_v2), then extract the C3, C4, and C5 layers in the general features through the feature pyramid, and pass the upper layer information to the bottom from top to bottom through the feature fusion method of C3, C4, and C5 layers, and finally connect the head layer to output the classification and positioning information of the object. When training the face detection model, multi-task loss is adopted, which mainly includes sample classification loss and sample positioning loss. The specific formula is: L = a1* Lcls + a2*Lbox, where L represents the loss and, a1 and a2 represent the sample classification loss coefficient and the sample positioning loss coefficient respectively, Lcls is the sample classification loss, and Lbox is the sample positioning loss. The face detection algorithm used in this embodiment can quickly and reliably detect faces in images collected by visible light cameras and thermal cameras.

[0033] According to the face data collected by the thermal camera, the position of the face in the data collected by the thermal camera is marked, and the thermal face detection model based on the deep convolutional neural network is trained with the marked thermal camera data set. The thermal face detection model is continuously optimized. The model is used to detect the position of the face in the thermal image.

[0034] According to the face data collected by the visible light camera, the position of the face in the visible light camera data and whether the face is wearing a mask are marked. The visible light face detection model based on the deep convolutional neural network is trained with the marked visible light camera data set to optimize the visible light face detection model. The visible light face detection model is used to detect the position of the face in the visible light image and determine whether the face is wearing a mask. The face information detected by the visible light face detection model is sent to the human-computer interaction device for display. For those who do not wear masks as required in public places and have abnormal body temperature, the human-computer interaction device will issue an alarm.

[0035] The face image collected by the visible light camera after face detection and the face image collected by the thermal camera are matched, and the temperature data collected by the thermal camera is matched with the face image data to obtain the temperature data corresponding to each face.

[0036] When matching faces, first find the face center XL of the visible light face image, such as Figure 2 The epipolar geometry theory diagram shown in the figure shows that the face center XL must correspond to an epipolar line on the thermal face image. Figure 2 Thermal image on X R and e RIf there is a face detected near the epipolar line in the thermal image, then the face closest to the epipolar line is the thermal face image corresponding to the visible light face image, thus completing the face pairing between the visible light image and the thermal image. Generally speaking, in an image with a resolution of 320*240, the epipolar line refers to a distance of 0-40 pixels from the epipolar line.

[0037] The correspondence between the visible light image and the thermal image can be represented by the basic matrix in stereo vision, which describes the internal parameters of the two cameras, such as focal length, and the spatial correspondence between the two cameras. Since the positions of the visible light camera and the thermal camera are relatively fixed, the basic matrix can be calculated in advance by the stereo camera calibration method.

[0038] This embodiment uses the traditional chessboard method to perform stereo camera calibration (Stereo Camera Calibration) on the visible light camera and the thermal camera. Stereo camera calibration is to calculate the basic matrix based on the correspondence between the midpoints of the chessboards taken by the two cameras. That is, within the field of view of the two cameras, a large number of chessboards with different postures and distances are taken, and multiple photos are taken. Then, the camera calibration algorithm is used to calculate the camera's internal and external parameters, and then the basic matrix is ​​combined based on the camera's internal and external parameters.

[0039] Since thermal cameras form images based on temperature, the chessboard needs to be heated during the calibration of the stereo camera. Because thermal cameras can only present a clear picture when the temperature difference is relatively large, the chessboard needs to be specially heated to produce a clear temperature difference between the white and black grids. This is based on the characteristics that black has a high absorption temperature and white has a high reflection temperature. When sunlight is used to illuminate the chessboard for a long time, a clear temperature difference will appear. The temperature difference will bring different display images to the thermal camera, and the basic matrix is ​​calculated using the chessboard calibration algorithm.

[0040] The specific calculation formula of the stereo vision algorithm is Z = P*X, where Z represents the coordinates of the object in the image coordinate system, X represents the coordinates of the object in the world coordinate system, and P represents the projection matrix. When the coordinates X of the object in the world coordinate system and the coordinates Z of the object in the image coordinate system are known, the projection matrix P can be calculated. The projection matrix P includes the rotation matrix R, the parallel matrix T and the camera parameter K. E represents the essential matrix, F represents the basic matrix, and the expression formula of the essential matrix E is:

[0041] E = T^R or ,

[0042] According to the rotation matrix R and parallel matrix T in the projection matrix P, the essential matrix E can be obtained, and then the basic matrix F can be obtained. For a pair of points of the same name in the stereo pair, their homogeneous image coordinates are P and , the formula is expressed as Therefore, when the basic matrix F and one of the homogenized coordinates are known, a straight line passing through the point in another image coordinate system can be obtained. This straight line is the epipolar line in the epipolar geometry algorithm.

[0043] The embedded processor calculates the corresponding epipolar line of the center point of the face detected by the visible light camera in the thermal camera based on the basic matrix, and searches for the nearest face image detected by the thermal camera near the epipolar line to achieve face matching. When matching faces, the relative positions of the visible light camera and the thermal camera to be measured must be fixed, that is, after the two cameras are calibrated and the basic matrix is ​​calculated, the relative positions cannot change significantly. In fact, when leaving the factory, the two cameras in the measurement system are fixed in position by the outer shell, so in principle, there will be no positional movement during use.

[0044] After the visible light image and the thermal image are paired for the face, the forehead temperature measurement algorithm is used to obtain the forehead temperature data in the face image and record it as body temperature data. In the forehead temperature measurement algorithm, a large amount of data is first collected through the visible light camera, and the ratio of the upper and lower boundaries of the forehead of the face in the detected face rectangular frame is counted. The face image is detected in combination with the thermal image, and the forehead position of the thermal face image is estimated based on the ratio of the upper and lower boundaries of the forehead in the rectangular frame counted by the visible light face image, and the temperature of the forehead position of the thermal face image is read. Since there are a lot of pixels collected at the forehead position, and each person has a different hairstyle, the coverage of the hair will also cause an offset to the temperature information collected by the thermal camera, resulting in a larger temperature range.

[0045] For images with a large range of forehead temperature data, first read the temperature of a single pixel according to the image coordinates collected by the thermal camera, then sort all the pixel temperatures in descending order, select the temperature values ​​in the first 1%-10% interval, and then compensate and calibrate the read temperature; then select the first 1%-50% of the data by descending order, and take the average of the selected data as the final temperature. According to the results of a limited number of experiments, the absolute value of the difference between the temperature detected and the marked temperature using the non-contact measurement method of this embodiment is 0.208 degrees Celsius, the absolute value of the difference between the detected temperature and the marked temperature is less than 0.3 degrees Celsius, accounting for 73.5%, the absolute value of the difference between the detected temperature and the marked temperature is less than 0.5 degrees Celsius, accounting for 89%, and the average value of the difference between the detected temperature and the marked temperature is -0.0054, and the standard deviation is 0.258.

[0046] This embodiment can be used for non-contact mobile temperature measurement. The person being measured can measure body temperature without standing in a fixed position. The applicable measurement distance is 0.5 meters to 5 meters. Due to the different distances between the face and the camera, the radiation attenuation is different. In addition, factors such as ambient temperature and humidity cause the original temperature obtained from the thermal image to fluctuate greatly.

[0047] The measured temperature of the thermal camera is compensated in real time. The thermal camera is connected to the temperature sensor in front, and the offset of the thermal camera is calibrated according to the measured temperature of the thermal camera and the actual temperature of the temperature sensor. A temperature compensation function is constructed by setting a black body in front of the thermal camera to calibrate the measurement errors caused by the non-uniformity of the thermal camera, the refractive index of the temperature sensor, the ambient temperature and the measuring distance, which greatly reduces the error of non-contact measurement and has higher accuracy.

[0048] The temperature compensation calibration of a thermal camera consists of the following steps.

[0049] Step 1: Calibrate the drift of the thermal camera itself:

[0050] The thermal camera is connected to a temperature sensor, which is arranged in front of the thermal camera. The temperature sensor is coated with black body paint, and the thermal camera uses the temperature sensor to calibrate the drift of the camera itself.

[0051] The drift of the thermal camera itself is between ±2 degrees Celsius and ±5 degrees Celsius. Since the drift of the camera itself is the drift of the entire image, there will not be an error in the measured temperature of a certain part or a point in the image while the measurement elsewhere is normal. Therefore, if the accurate temperature of a part of the image is known, the temperature of the entire picture can be compensated.

[0052] Use a thermal camera to measure a part or a point of the temperature sensor multiple times at different times and temperatures to obtain i A thermal camera measures the temperature Tx 1i , i is a natural number greater than 1. Get the actual temperature Ty of the temperature sensor when the thermal camera measures the temperature 1i 。 For discrete data Tx 1i and Ty 1i , use linear fitting to get the function between the temperature measured by the thermal camera and the actual temperature of the temperature sensor f1 , f1 The expression is Ty 1i =n 1* Tx 1i ，n 1 is the thermal camera drift compensation coefficient, using the function f1 The calibration camera itself drifts causing errors.

[0053] Most of the existing technologies calibrate the thermal camera by setting up an additional black body, but the black body is expensive and has high requirements for the ambient temperature. This embodiment takes the economical issue into consideration and connects the thermal camera with a high-precision temperature sensor, and coats the temperature sensor with high-reflectivity black body paint. Through experiments with multiple calibration algorithms, the temperature sensor coated with black body paint is equivalent to a black body, which can achieve an effect similar to that of a black body, greatly reducing the cost while ensuring accuracy.

[0054] The high reflectivity black body paint uses oily 95 black body paint with a reflectivity of 0.95±0.02. The thermal camera reads the temperature of the high-precision temperature sensor through the I2C bus. At the same time, the thermal camera also measures the temperature of the temperature sensor. The linear fitting function is updated in real time according to the actual temperature of the temperature sensor and the temperature measured by the thermal camera. f1 Parameters n 1. Through this parameter n 1. Real-time calibration of the measured temperature of the thermal camera. The high-precision temperature sensor is located on the upper left of the thermal camera, about 20 cm away from the thermal camera. When in use, the temperature sensor does not need to maintain a fixed position with the thermal camera. Even if the position of the temperature sensor shifts during use, as long as the reference area of ​​the temperature sensor for the thermal camera is corrected, it will not affect the temperature measurement and data acquisition.

[0055] Step 2: Calibrate the non-uniformity of the thermal camera:

[0056] Since the thermal camera is composed of thermal imaging dot matrix, the values ​​obtained for objects with the same temperature at different positions on the screen will be inconsistent. The non-uniformity of the thermal camera refers to the inconsistent values ​​obtained for objects with the same temperature at different positions on the screen. Step 2 is to set a thermally uniform black body at a fixed position in front of the thermal camera. The black body covers the entire screen of the thermal camera, that is, the entire screen captured by the thermal camera is the reflective surface of the black body. Set different black body temperatures, save the corresponding data image of the thermal camera, and obtain the coordinates (x, y) of each pixel in the data image to obtain the function between the temperature measured by the thermal camera and the black body temperature. f2 , through the function f2 Calibration camera non-uniformity causes errors.

[0057] During calibration, the distance between the black body and the thermal camera is about 10 cm, and the temperature fluctuation of the entire black body screen is within 0.3°C. When the black body temperature is set to T1, a data image of the thermal camera is saved. Img T1 ; Then set the black body temperature to T2 and save a data image of the thermal camera Img T2 .

[0058] First, set the low temperature interval temperature T1, the low temperature interval temperature range is 26 to 30 ° C, wait for the temperature to stabilize, that is, when the temperature fluctuation of the black body's reflective surface is less than 0.3 ° C, collect several pictures, and take the temperature average value of all pixels in the image to obtain the data image Img T1 Then set the black body temperature to the high temperature range T2, the high temperature range is 40 to 42 ° C, wait for the temperature to stabilize, that is, when the temperature fluctuation of the black body's reflective surface is less than 0.3 ° C, collect several pictures, and take the temperature average value for all pixels in the image to obtain the data image Img T2 The slope of the thermal camera temperature non-uniformity compensation function is calculated by measuring the average value of the image data twice. gain and offset offset , get the compensated temperature. gain and offset The calculation formula is as follows:

[0059] (1)

[0060] (2)

[0061] In the above formula, Img T 1 [x, y] Representing data images Img T1 exist (x, y) The temperature of the pixel at the coordinate point, Img T 2 [x, y] Representing data images Img T2 exist (x, y) The temperature of the pixel at the coordinate point, gain[x, y] express (x, y) The slope of the pixel at the coordinate point gain , offset[x, y] express (x, y) Coordinate point pixel offset offset .pass gain and offset Constructing functions f2 ,Right now T[x, y] = gain[x, y] * Img[x, y] + offset[x, y] ,in Img[x, y] Measured for thermal cameras (x, y) The temperature of the pixel, T[x, y] is the temperature after correction for inhomogeneity. Use the function f2 The original measured temperature is linearly corrected to obtain the temperature of each coordinate point after calibration, that is, the thermal camera measurement value after non-uniformity calibration is obtained.

[0062] Step 3: Calibration of temperature sensor reflectivity:

[0063] Steps 1 and 2 calibrate the temperature of the thermal camera itself. Step 1 reads the temperature of the high-precision temperature sensor in the image and performs a differential operation with the actual temperature of the high-precision temperature sensor for calibration. The non-uniformity calibration in step 2 does not take into account the difference between the reflectivity of the temperature sensor and the black body reflectivity. The temperature sensor is very critical in the thermal imaging system of the thermal camera. Since the high-precision temperature sensor still has a certain error, after the camera drift calibration in step 1 and the non-uniformity calibration in step 2, the black body temperature scanning calibration is used in step 3 to further calibrate the reflectivity of the temperature sensor.

[0064] The blackbody temperature scanning calibration process is simply to fit the relationship between the temperature of a certain point read by the thermal camera after the first two steps of calibration and the actual temperature of the point, and the actual temperature of the point is obtained by reading the blackbody temperature. Since temperature is volatile and is a range, the relationship between the temperature of a certain point read by the thermal camera after calibration and the actual temperature of the blackbody is calculated by dynamically setting the temperature of the blackbody.

[0065] During calibration, place the black body in front of the thermal camera and connect the black body to the serial port of the TX2 box. The TX2 box is an embedded small computer of NVIDIA, and its serial port is a common USB interface. After the two are connected, the TX2 box can read the temperature of the black body. Set the temperature of the black body reflective surface, which is the plane of the temperature read by the thermal camera. After directly connecting to the TX2 box, click the center of the black body with the mouse to select the black body temperature scanning range. The scanning range starts from the low temperature range temperature to the high temperature range temperature. The low temperature range temperature here is generally 26 to 30°C, and the high temperature range temperature is generally 40 to 42°C.

[0066] After each temperature measurement point stabilizes, the thermal camera reads the temperature of the selected area, selects the temperature read by multiple pictures, and takes the average temperature, which is the final reading of the thermal camera. f3 The independent variable is the reading of the thermal camera after the uniformity calibration in step 2 and the high-precision sensor calibration in step 1; the function f3 The dependent variable is the temperature of the black body, which is our target temperature. Linearly fit the independent variable, the temperature measured by the thermal camera, and the dependent variable, the actual temperature of the black body, to obtain the function f3 .

[0067] Specifically, a black body is set in front of the thermal camera, and different temperatures Ty are set on the black body reflective surface. 3i , read the measured temperature Tx of the thermal camera after calibration in steps 1 and 2 3i , we get the function between the temperature measured by the thermal camera and the temperature of the blackbody reflective surface f3 ,function f3 The formula is Ty3i =n 3* Tx 3i ,in n 3 is the temperature sensor reflectivity compensation coefficient; Tx 3i Measuring values ​​Tx with thermal camera 3i0 The relationship is Tx 3i = (gain × Tx 3i0 + offset) * n 1 ， Tx 3i [x, y] Represents the measured temperature of the pixel at the (x, y) coordinate point after calibration in steps 1 and 2, through the function f3 Calibration introduces errors due to the reflectivity of the temperature sensor.

[0068] The thermal camera is not actually connected to the black body. The black body is used here only to calculate the fitting function. f3 The user reads the temperature after the camera itself has been calibrated for drift and non-uniformity, and then uses the linear transformation function calibrated by the temperature sensor f3 Further corrections have been made to make the temperature read by the thermal camera more accurate.

[0069] Step 4: Calibration for ambient temperature:

[0070] Different ambient temperatures will affect the measurement of thermal cameras. Since thermal radiation will reach the lens and heat it, and the camera itself will also heat up, it is easy for the lens to radiate heat to the thermal imaging chip, affecting the temperature reading. Although the thermal camera itself has ambient temperature calibration, after actual measurement, the ambient temperature calibration standard of the thermal camera itself is relatively low. In addition, the thermal imaging system of this embodiment needs to read a high-precision temperature sensor to calibrate the camera drift, so that the system needs to achieve more accurate ambient temperature calibration.

[0071] When calibrating the ambient temperature, first set different ambient temperatures, such as setting the ambient temperature to T3 and T4, read the camera readings V1 and V2 after calibration in step 3, draw the correlation curve between the readings V1 and V2 and the ambient temperatures T3 and T4, and calculate the linear ambient temperature calibration coefficient n 4. n 4 = (V2 – V1) / (T3 – T4).

[0072] The ambient temperature calibration process requires a temperature-controlled test environment, or a test environment where the temperature rises or falls slowly. When calibrating, place the black body in front of the thermal camera and set the black body temperature. Then run the software, set the relevant parameters, click the center of the black body with the mouse and collect data. At this time, the function f4The independent variable is the ambient temperature, and the dependent variable is the temperature of the black body. Observe the effect of ambient temperature on the final reading, fit and calculate the functional relationship between the two, and obtain the function between the temperature measured by the thermal camera and the ambient temperature. f4 , that is, Ty 4i =Tx 4i * n 4. Tx 4i is the temperature measured by the thermal camera after calibration in step 3 at the current different ambient temperatures, Ty 4i Represents the temperature compensation value after ambient temperature calibration, through the function f4 Ambient temperature compensation for thermal cameras.

[0073] Step 5: Calibration for test distance:

[0074] Thermal cameras measure temperature by reading the energy of far-infrared wavelengths. During the transmission process, the energy of infrared light will be absorbed by the air and attenuated, so the farther the distance, the lower the measured temperature. Step 5 Measure the temperature of the same black body at different distances and calculate the distance coefficient. n 5. Calibrate the thermal camera at the test distance.

[0075] The calibration process for the test distance is as follows: first, place the black body in front of the thermal camera and set the black body temperature. Then, by moving the black body, read the temperature of the black body at the corresponding distance. f5 The independent variable is the distance between the black body and the camera, and the dependent variable is the temperature of the black body read by the camera. By fitting the data, the relationship between the independent variable and the dependent variable is obtained, that is, the function f5 .function f5 is a polynomial formula, and the specific calculation formula is Ty 5i = n 51 *Tx 5i m-1 + n 52 *Tx 5i m-2 +…+ n 5(m-1) *Tx 5i 1 + n 5m , distance coefficient n5=[ n 51, n 52, … ， n 5(m-1), n 5m ], where m is an integer greater than 1, Tx 5i is the measured temperature of the thermal camera calibrated in step 4, Ty 5iIndicates the temperature compensation value after distance calibration. When the distance between the object and the camera is known, the distance compensation coefficient can be used to calculate the temperature compensation value. n 5 and the polynomial degree m, the temperature after calibration is obtained. In this embodiment, the polynomial degree m=7.

[0076] The temperature fluctuation read by the thermal camera in the prior art is about 2-5 degrees Celsius. According to limited experimental data, the absolute value of the difference between the temperature of a single point read and the actual temperature of the black body on the data set collected from 0.5 meters to 5 meters using the thermal camera calibrated by the present invention is 0.133 degrees Celsius, the average value of the error is 0.0026, the standard deviation (std) of the error is 0.1718, the proportion of pixels with an absolute error value less than 0.3°C is about 92%, and the proportion of pixels with an absolute error value less than 0.5 degrees Celsius can reach 99.44%, and the accuracy and precision are greatly improved.

[0077] The human-computer interaction device of the present invention uploads the facial data, body temperature, whether a mask is worn, and other related information collected by the visible light camera to the cloud and the display and alarm device to realize non-contact long-distance human body temperature measurement. When collecting human body temperature, the present invention can realize non-contact body temperature measurement in an area of ​​0.5 meters to 5 meters, and has a wide range of applications. During measurement, the thermal camera also calibrates the read data through a temperature compensation method to obtain higher accuracy and precision. The measurement system of the present invention is used in security inspections in public places with a large flow of people. It can realize accurate measurement of body temperature, and timely alarm prompts for those who do not wear masks or have abnormal body temperatures. It has high accuracy and reliability, and has good practicality.

[0078] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A non-contact body temperature measurement method, characterized in that: A visible light camera is used to collect visible light images, and a thermal camera is used to collect thermal images and temperatures. Face detection is performed on the collected visible light images and thermal images respectively. The visible light face images and thermal face images obtained after face detection are matched with faces. The temperature collected by the thermal camera is matched with the matched face images. The forehead temperature of the face images is obtained and processed to obtain body temperature data. The thermal camera is calibrated by a temperature compensation method, which includes: connecting the thermal camera to a temperature sensor coated with black body paint, the temperature sensor is arranged in front of the thermal camera, and the offset of the thermal camera is calibrated according to the temperature of the temperature sensor measured by the thermal camera and the actual temperature fitting function of the temperature sensor; a black body is arranged in front of the thermal camera, and the measurement error of the thermal camera caused by the non-uniformity of the thermal camera, the refractive index of the temperature sensor, the ambient temperature and the measurement distance is calibrated according to the temperature measured by the thermal camera and the black body temperature fitting function; the temperature compensation method specifically includes the following steps: Step 1: The thermal camera is connected to the temperature sensor coated with black body paint. The temperature sensor is set in front of the thermal camera. The thermal camera measures multiple temperatures Tx of the temperature sensor at different temperatures. 1i , and the actual temperature Ty of the temperature sensor when the thermal camera measures the temperature 1i , linear fitting is used to obtain the function f1 between the temperature measured by the thermal camera and the actual temperature of the temperature sensor, where i is a natural number greater than 1, and the expression of function f1 is Ty 1i =n1*Tx 1i , n1 is the drift compensation coefficient of the thermal camera, and the error caused by the drift of the thermal camera itself is calibrated by function f1; Step 2: Set a black body with uniform heat in front of the thermal camera, so that the black body covers the entire screen. Set different black body temperatures, save the corresponding data image of the thermal camera, and obtain the function f2 between the temperature measured by the thermal camera and the black body temperature for each pixel in the data image. Use function f2 to calibrate the error caused by the unevenness of the camera. Step 3: Set different temperatures Ty on the blackbody reflective surface 3i , read the blackbody reflective surface temperature Tx measured by the thermal camera 3i0 , the measured temperature Tx 3i0 First, calibrate the function f1 and function f2 to get the calibrated temperature Tx 3i , get the temperature Tx of the thermal camera 3i and the blackbody reflector temperature Ty 3i The function f3 between the two is Ty 3i =n3*Tx 3i , n3 is the temperature sensor reflectivity compensation coefficient, and the error caused by the temperature sensor reflectivity is calibrated by function f3; Step 4: Set different ambient temperatures Ty 4i , read different ambient temperatures Ty 4i The temperature Tx measured by the thermal camera after calibration by function f3 4i The temperature Tx measured by the thermal camera is obtained by measuring the temperature difference between different ambient temperatures and the thermal camera. 4i and ambient temperature Ty 4i The ambient temperature calibration coefficient n4 between the two obtains the function f4, namely Ty 4i =Tx 4i *n4, calibrate the error caused by ambient temperature through function f4; Step 5: Read the temperature Ty of the black body at different distances from the thermal camera 5i , get the temperature Tx measured by the thermal camera after calibration by function f4 5i Function f5 of the measured distance, function f5 is a polynomial formula, and the specific calculation formula is Ty 5i =n 51 *Tx 5i m-1 +n 52 *Tx 5i m-2 +…+n 5(m-1) *Tx 5i 1 +n 5m , distance coefficient n5=[n 51 ,n 52 ,…,n 5(m-1) ,n 5m ], where m is an integer greater than 1, and the error caused by the measured distance is calibrated by function f5.

2. A non-contact body temperature measurement method according to claim 1, characterized in that: The face detection model used in face detection is built using a deep convolutional neural network detection algorithm.

3. A non-contact body temperature measurement method according to claim 2, characterized in that: The collected visible light face image is input to the visible light face detection model for detecting the position of the face in the visible light image and determining whether the face is wearing a mask; the collected thermal face image is input to the thermal face detection model for detecting the position of the face in the thermal image.

4. A non-contact body temperature measurement method according to claim 1, characterized in that: The epipolar geometry algorithm is used for face matching. The thermal face image is matched with the visible light face image based on the face center of the visible light face image and the epipolar line of the corresponding thermal face image.

5. A non-contact body temperature measurement method according to claim 4, characterized in that: When matching faces, the spatial correspondence between the visible light camera and the thermal camera reconstructed by the stereo vision algorithm is used to calculate the epipolar line. The correspondence is represented by a basic matrix, which is calculated using a checkerboard calibration method. During the checkerboard calibration, the thermal camera is irradiated with sunlight to form a temperature difference.

6. A non-contact body temperature measurement method according to claim 1, characterized in that: According to the ratio of the upper boundary line and the lower boundary line of the forehead in the visible light face image, the temperature in the corresponding ratio area of ​​the thermal face image is read, that is, the forehead temperature of the face image.

7. A non-contact body temperature measurement method according to claim 6, characterized in that: The forehead temperature of the face image is processed, and the obtained temperatures of all pixels at the forehead are sorted in descending order. The first X1 temperature values ​​in the sequence are selected for calibration. The calibrated temperatures are then sorted in descending order, and the first X2 temperature values ​​in the sequence are selected to take the average. The average is the body temperature value. X1 and X2 are both positive integers, and X1≥X2.

8. A non-contact body temperature measurement method according to claim 1, characterized in that: The temperature measurement results are displayed or alarmed through the human-computer interaction device.

9. A non-contact body temperature measurement method according to claim 1, characterized in that: The thermal camera in step 1 is connected to the temperature sensor via a bus, and the thermal camera obtains the actual temperature of the temperature sensor via the bus.

10. A non-contact body temperature measurement method according to claim 1, characterized in that: In step 2, the blackbody temperature T is first set, several pictures are collected, the temperature average value is taken for each pixel in the picture, the blackbody temperature is set repeatedly, the image data of the blackbody reflection surface at different temperatures are saved, and the temperature of each coordinate point of the thermal camera after calibration is obtained through linear correction by function f2.

11. A non-contact body temperature measurement method according to claim 10, characterized in that: Set the black body temperature to T1 and save the thermal data image Img T1. Set the black body temperature to T2 and save the thermal data image Img T2. For each pixel at the position (x, y) of the image, calculate the slope gain and offset offset. The formula is as follows: offset[x,y]=T1-gain[x,y]*ImgT1[x,y] (2) In the above formula, Img T1[x, y] represents the temperature of the pixel at the (x, y) coordinate point of the data image Img T1, and Img T2[x, y] represents the temperature of the pixel at the (x, y) coordinate point of the data image Img T2. The function f2 is constructed by the slope gain and the offset offset, that is, T[x, y] = gain[x, y]*Img[x, y]+offset[x, y], where Img[x, y] is the temperature of the (x, y) pixel measured by the thermal camera, and T[x, y] is the temperature after non-uniformity calibration. The error caused by camera non-uniformity is calibrated by function f2.

12. A non-contact body temperature measurement method according to claim 1, characterized in that: In step three, the temperature of the blackbody reflective surface is first set. After the temperature stabilizes, the temperature of the blackbody reflective surface is measured multiple times and then the average value is taken. The average value is the temperature measured by the thermal camera; the actual temperature of the blackbody reflective surface is read through the calculation module connected to the blackbody.

13. A non-contact body temperature measurement system, characterized in that: A non-contact body temperature measurement method as described in any one of claims 1 to 12 is used, and the measurement system includes a visible light camera, a thermal camera, an embedded processor and a human-computer interaction device. The visible light camera and the thermal camera are connected to the embedded processor through a connector. The visible light camera and the thermal camera both send collected information to the embedded processor, and the embedded processor sends processed data to the human-computer interaction device.

14. A non-contact body temperature measurement system according to claim 13, characterized in that: The relative positions of the visible light camera and the thermal camera are fixed.

Images