Temperature detection method, device and equipment

Abstract

The present invention provides a temperature detection method, apparatus, and device, relating to the field of communications technology. The method comprises: obtaining first and second key point information of a target object; determining target key point information of the target object based on offset information of relevant key points in the first and second key point information; and measuring the temperature of the target object based on the target key point information. The present invention addresses the low efficiency and poor accuracy of existing temperature detection methods.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a temperature detection method, device and equipment. Background Art

[0002] Existing facial temperature measurement technologies primarily utilize handheld temperature measurement and infrared thermal imaging. Handheld temperature measurement requires manually aiming a handheld thermometer at a close distance, such as the forehead, to measure temperature. Infrared thermal imaging uses infrared imaging equipment to measure the temperature distribution within a specific range.

[0003] However, handheld temperature measurement requires manual operation to align with the part to be measured, which is inefficient; infrared thermal imaging measurement is easily interfered by high-temperature objects within the range and cannot accurately measure the required facial temperature. Summary of the Invention

[0004] The object of the present invention is to provide a temperature detection method, device and equipment to achieve faster and more accurate temperature measurement.

[0005] To achieve the above objectives, an embodiment of the present invention provides a temperature detection method, comprising:

[0006] Obtain first key point information and second key point information of the target object;

[0007] Determining target key point information of the target object according to offset information of relevant key points in the first key point information and the second key point information;

[0008] The temperature of the target object is measured according to the target key point information.

[0009] Optionally, obtaining the first key point information of the target object includes:

[0010] Processing the image data of the target object using a first detection model to obtain first key point information of the target object;

[0011] The image data is a fusion feature of the depth image feature and the RGB image feature of the target object;

[0012] The first detection model is a constructed neural network model that determines key point information based on image data.

[0013] Optionally, before processing the image data of the target object using the first detection model to obtain the first key point information of the target object, the method further includes:

[0014] Acquire a depth image and an RGB image of the target object;

[0015] Extracting depth image features of the depth image and RGB image features of the RGB image respectively;

[0016] Determining target weights for the depth image and the RGB image;

[0017] The depth image feature and the RGB image feature are combined according to the target weight to obtain the fusion feature.

[0018] Optionally, the target weight is expressed as Calculated, where w is the target weight, μ intra is the mean of the inter-class distances between the depth image and the RGB image, σ intra is the variance of the inter-class distance between the depth image and the RGB image, μ inter is the mean of the intra-class distances between the depth image and the RGB image, σ inter is the variance of the intra-class distance between the depth image and the RGB image.

[0019] Optionally, obtaining the second key point information of the target object includes:

[0020] Processing the reference image of the target object using a second detection model to obtain two-dimensional coordinate information of key points of the target object;

[0021] Obtaining second key point information of the target object according to the two-dimensional coordinate information and the depth information of the key point;

[0022] Wherein, the reference image is an RGB image;

[0023] The second detection model is a constructed neural network module that determines the coordinates of key points based on RGB images.

[0024] Optionally, the offset information is an offset distance;

[0025] The determining the target key point information of the target object according to the offset information of the relevant key points in the first key point information and the second key point information includes:

[0026] Obtaining an offset distance of a related key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information;

[0027] Selecting target key points according to the offset distance;

[0028] The target key point information is determined according to the three-dimensional coordinate information of the target key point.

[0029] Optionally, obtaining the offset distance of the relevant key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information includes:

[0030] In the same coordinate system, the offset distance is calculated based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information.

[0031] Optionally, selecting a target key point according to the offset distance includes:

[0032] Based on a preset value N, N key points are selected as target key points, the offset distances of the N key points are smaller than the offset distances of the remaining key points, and N is an integer greater than or equal to 1.

[0033] Optionally, obtaining the offset distance of the relevant key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information includes:

[0034] In different coordinate systems, an offset distance is calculated based on three-dimensional coordinate information of relevant key points in the first key point information and the second key point information, wherein the origins of the different coordinate systems are different key points.

[0035] Optionally, selecting a target key point according to the offset distance includes:

[0036] Calculate the average offset distance of each key point;

[0037] Based on a preset value M, M key points are selected as target key points, the average offset distance of the M key points is smaller than the average offset distance of the remaining key points, and M is an integer greater than or equal to 1.

[0038] Optionally, determining the target key point information according to the three-dimensional coordinate information of the target key point includes:

[0039] Correcting the three-dimensional coordinate information of the target key point according to the correction parameter and the offset distance of the target key point to obtain corrected three-dimensional coordinate information;

[0040] The corrected three-dimensional coordinate information is used as the target key point information.

[0041] Optionally, after measuring the temperature of the target object according to the target key point information, the method further includes:

[0042] The measured temperature is corrected by a temperature correction model to obtain a target temperature;

[0043] The loss function of the temperature correction model is set based on the standard temperature difference, the ambient temperature and the depth information of the target object.

[0044] To achieve the above objectives, an embodiment of the present invention provides a temperature detection device, comprising:

[0045] An acquisition module, configured to acquire first key point information and second key point information of a target object;

[0046] a processing module, configured to determine target key point information of the target object based on offset information of relevant key points in the first key point information and the second key point information;

[0047] The measuring module is used to measure the temperature of the target object according to the target key point information.

[0048] Optionally, the acquisition module includes:

[0049] A first processing submodule is configured to process the image data of the target object using a first detection model to obtain first key point information of the target object;

[0050] The image data is a fusion feature of the depth image feature and the RGB image feature of the target object;

[0051] The first detection model is a constructed neural network model that determines key point information based on image data.

[0052] Optionally, the acquisition module further includes:

[0053] An acquisition submodule, configured to acquire a depth image and an RGB image of the target object;

[0054] An extraction submodule, configured to extract depth image features of the depth image and RGB image features of the RGB image respectively;

[0055] A determination submodule, configured to determine target weights of the depth image and the RGB image;

[0056] A feature fusion submodule is used to combine the depth image feature and the RGB image feature according to the target weight to obtain the fused feature.

[0057] Optionally, the target weight is expressed as Calculated, where w is the target weight, μ intra is the mean of the inter-class distances between the depth image and the RGB image, σ intra is the variance of the inter-class distance between the depth image and the RGB image, μ interis the mean of the intra-class distances between the depth image and the RGB image, σ inter is the variance of the intra-class distance between the depth image and the RGB image.

[0058] Optionally, the acquisition module further includes:

[0059] A second processing submodule is configured to process the reference image of the target object using a second detection model to obtain two-dimensional coordinate information of key points of the target object;

[0060] a third processing submodule, configured to obtain second key point information of the target object according to the two-dimensional coordinate information and the depth information of the key point;

[0061] Wherein, the reference image is an RGB image;

[0062] The second detection model is a constructed neural network module that determines the coordinates of key points based on RGB images.

[0063] Optionally, the offset information is an offset distance;

[0064] The processing module includes:

[0065] a fourth processing submodule, configured to obtain an offset distance of a related key point based on the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information;

[0066] A selection submodule, for selecting target key points according to the offset distance;

[0067] The fifth processing submodule is configured to determine the target key point information according to the three-dimensional coordinate information of the target key point.

[0068] Optionally, the fourth processing submodule is further configured to:

[0069] In the same coordinate system, the offset distance is calculated based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information.

[0070] Optionally, the selection submodule is further configured to:

[0071] Based on a preset value N, N key points are selected as target key points, the offset distances of the N key points are smaller than the offset distances of the remaining key points, and N is an integer greater than or equal to 1.

[0072] Optionally, the fourth processing submodule is further configured to:

[0073] In different coordinate systems, an offset distance is calculated based on three-dimensional coordinate information of relevant key points in the first key point information and the second key point information, wherein the origins of the different coordinate systems are different key points.

[0074] Optionally, the selection submodule is further configured to:

[0075] Calculate the average offset distance of each key point;

[0076] Based on a preset value M, M key points are selected as target key points, the average offset distance of the M key points is smaller than the average offset distance of the remaining key points, and M is an integer greater than or equal to 1.

[0077] Optionally, the fifth processing submodule includes:

[0078] a correction unit, configured to correct the three-dimensional coordinate information of the target key point according to the correction parameter and the offset distance of the target key point to obtain corrected three-dimensional coordinate information;

[0079] A processing unit is configured to use the corrected three-dimensional coordinate information as the target key point information.

[0080] Optionally, the device further comprises:

[0081] A correction module is used to correct the measured temperature through a temperature correction model to obtain a target temperature;

[0082] The loss function of the temperature correction model is set based on the standard temperature difference, the ambient temperature and the depth information of the target object.

[0083] To achieve the above objectives, an embodiment of the present invention provides a temperature detection device, including a processor, wherein the processor is configured to:

[0084] Obtain first key point information and second key point information of the target object;

[0085] Determining target key point information of the target object according to offset information of relevant key points in the first key point information and the second key point information;

[0086] The temperature of the target object is measured according to the target key point information.

[0087] Optionally, the processor is further configured to:

[0088] Processing the image data of the target object using a first detection model to obtain first key point information of the target object;

[0089] The image data is a fusion feature of the depth image feature and the RGB image feature of the target object;

[0090] The first detection model is a constructed neural network model that determines key point information based on image data.

[0091] Optionally, the processor is further configured to:

[0092] Acquire a depth image and an RGB image of the target object;

[0093] Extracting depth image features of the depth image and RGB image features of the RGB image respectively;

[0094] Determining target weights for the depth image and the RGB image;

[0095] The depth image feature and the RGB image feature are combined according to the target weight to obtain the fusion feature.

[0096] Optionally, the target weight is expressed as Calculated, where w is the target weight, μ intra is the mean of the inter-class distances between the depth image and the RGB image, σ intra is the variance of the inter-class distance between the depth image and the RGB image, μ inter is the mean of the intra-class distances between the depth image and the RGB image, σ inter is the variance of the intra-class distance between the depth image and the RGB image.

[0097] Optionally, the processor is further configured to:

[0098] Processing the reference image of the target object using a second detection model to obtain two-dimensional coordinate information of key points of the target object;

[0099] Obtaining second key point information of the target object according to the two-dimensional coordinate information and the depth information of the key point;

[0100] Wherein, the reference image is an RGB image;

[0101] The second detection model is a constructed neural network module that determines the coordinates of key points based on RGB images.

[0102] Optionally, the offset information is an offset distance;

[0103] The processor is further configured to:

[0104] Obtaining an offset distance of a related key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information;

[0105] Selecting target key points according to the offset distance;

[0106] The target key point information is determined according to the three-dimensional coordinate information of the target key point.

[0107] Optionally, the processor is further configured to:

[0108] In the same coordinate system, the offset distance is calculated based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information.

[0109] Optionally, the processor is further configured to:

[0110] Based on a preset value N, N key points are selected as target key points, the offset distances of the N key points are smaller than the offset distances of the remaining key points, and N is an integer greater than or equal to 1.

[0111] Optionally, the processor is further configured to:

[0112] In different coordinate systems, an offset distance is calculated based on three-dimensional coordinate information of relevant key points in the first key point information and the second key point information, wherein the origins of the different coordinate systems are different key points.

[0113] Optionally, the processor is further configured to:

[0114] Calculate the average offset distance of each key point;

[0115] Based on a preset value M, M key points are selected as target key points, the average offset distance of the M key points is smaller than the average offset distance of the remaining key points, and M is an integer greater than or equal to 1.

[0116] Optionally, the processor is further configured to:

[0117] Correcting the three-dimensional coordinate information of the target key point according to the correction parameter and the offset distance of the target key point to obtain corrected three-dimensional coordinate information;

[0118] The corrected three-dimensional coordinate information is used as the target key point information.

[0119] Optionally, the processor is further configured to:

[0120] The measured temperature is corrected by a temperature correction model to obtain a target temperature;

[0121] The loss function of the temperature correction model is set based on the standard temperature difference, the ambient temperature and the depth information of the target object.

[0122] To achieve the above-mentioned purpose, an embodiment of the present invention provides a temperature detection device, including a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; when the processor executes the program or instruction, the temperature detection method described above is implemented.

[0123] To achieve the above objectives, an embodiment of the present invention provides a readable storage medium having a program or instruction stored thereon, which implements the steps in the above-mentioned temperature detection method when executed by a processor.

[0124] The beneficial effects of the above technical solution of the present invention are as follows:

[0125] The method of the embodiment of the present invention, for the target object of temperature measurement, after obtaining two sets of key point information of the target object, will further determine more accurate target key point information for measuring the target object based on the offset information of the relevant key points in the two sets of key point information, and then use the target key point information to complete the temperature measurement of the target object. Here, because the target key point information can more accurately indicate the effective temperature measurement position of the target object, more accurate and efficient temperature measurement can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Figure 1 is a flow chart of a temperature detection method according to an embodiment of the present invention;

[0127] Figure 2 Schematic diagram of the positions of key points in an embodiment of the present invention;

[0128] Figure 3 is a structural diagram of a temperature detection device according to an embodiment of the present invention;

[0129] Figure 4 This is a structural diagram of a temperature detection device according to an embodiment of the present invention;

[0130] Figure 5 A structural diagram of a temperature detection device according to another embodiment of the present invention;

[0131] Figure 6 Schematic diagram of the application of the temperature detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0132] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0133] It should be understood that references throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present invention. Therefore, the appearances of "in one embodiment" or "in an embodiment" throughout this specification do not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0134] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0135] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0136] In the embodiments provided herein, it should be understood that "B corresponding to A" means that B is associated with A and B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B based solely on A; B can also be determined based on A and / or other information.

[0137] like Figure 1 As shown, a temperature detection method according to an embodiment of the present invention includes:

[0138] Step 101: Acquire first key point information and second key point information of a target object;

[0139] Step 102: determining target key point information of the target object based on offset information of relevant key points in the first key point information and the second key point information;

[0140] Step 103: Measure the temperature of the target object according to the target key point information.

[0141] According to the above steps, the method of the embodiment of the present invention, for the target object for temperature measurement, after obtaining two sets of key point information of the target object, will further determine more accurate target key point information for measuring the target object based on the offset information of the relevant key points in the two sets of key point information, and then use the target key point information to complete the temperature measurement of the target object. Here, because the target key point information can more accurately indicate the effective temperature measurement position of the target object, more accurate and efficient temperature measurement can be achieved.

[0142] Among them, the temperature detection method of an embodiment of the present invention is applied to the scenario of measuring the temperature of a human face, that is, when the target object is a human face, after determining the target key point information by the offset information of the relevant key points in the two sets of key point information, accurate temperature measurement can be performed at the effective temperature measurement position of the human face with higher efficiency.

[0143] It should be noted that in this embodiment, the key points are the temperature measurement location points of the target object. For example, when the target object is a face, the key points include key points in the forehead area, key points in the eye area, key points in the neck area, and so on. The key point information is the location information of the key points. For the same target object, the first key point information and the second key point information include the location information of related key points. The related key points are key points in the same area, or the same key point. For example, the first key point information includes the information of key point A, and key point A is the pupil location. The second key point information includes key point B, and key point B is the pupil location. Then key point A and key point B are related key points. Of course, if the second key point information does not include the key point of the pupil location, but there is a key point C in the eye area, then key point A and key point C can also be called related key points.

[0144] Optionally, in this embodiment, obtaining first key point information of the target object includes:

[0145] Processing the image data of the target object using a first detection model to obtain first key point information of the target object;

[0146] The image data is a fusion feature of the depth image feature and the RGB image feature of the target object;

[0147] The first detection model is a constructed neural network model that determines key point information based on image data.

[0148] Here, the first detection model is constructed by training a neural network model, which is used to detect key temperature measurement points on the target object and output the three-dimensional coordinate information of the key points. The fusion features of the target object's depth image features and RGB image features serve as image data and are the input of the first detection model. Training is to fine-tune the weights and parameters of the neural network model to obtain the first detection model.

[0149] In order to obtain image data as input to the first detection model, optionally, in this embodiment, before processing the image data of the target object using the first detection model to obtain the first key point information of the target object, the method further includes:

[0150] Acquire a depth image and an RGB image of the target object;

[0151] Extracting depth image features of the depth image and RGB image features of the RGB image respectively;

[0152] Determining target weights for the depth image and the RGB image;

[0153] The depth image feature and the RGB image feature are combined according to the target weight to obtain the fusion feature.

[0154] Here, considering that different features contain their own unique influence information, after obtaining the depth image and RGB image of the target object and extracting their respective image features, the applicable target weight will be further determined, and then the depth image features and the RGB image features will be combined according to the target weight to obtain the fused features, i.e., image data.

[0155] Among them, the target weight is expressed by the formula Calculated, where w is the target weight, μ intra is the mean of the inter-class distances between the depth image and the RGB image, σ intra is the variance of the inter-class distance between the depth image and the RGB image, μ inter is the mean of the intra-class distances between the depth image and the RGB image, σ inter is the variance of the intra-class distance between the depth image and the RGB image.

[0156] Here, μ intra With μ inter The larger the difference, the greater the intra +σ inter The smaller it is, the greater the weight of the target is.

[0157] It should be noted that in this embodiment, a depth camera (such as a structured light depth camera) can be used to capture a depth image of the target object, and an RGB camera (such as a visible light camera) can be used to capture an RGB image of the target object. That is, to obtain a depth image and an RGB image of the target object, specifically: to obtain a depth image of the target object captured by the depth camera, and to obtain an RGB image of the target object captured by the RGB camera. Since the depth image may have problems such as depth discontinuity, before extracting the depth image features of the depth image, the depth image will also be pre-processed by asymmetric Gaussian filtering and pixel blocking to eliminate holes and noise in the depth image and enhance the quality of the image.

[0158] Then extract the features of the corresponding image. For depth images, since spherical coordinates describe curves, it is easier to express feature changes. The depth information of the depth image will be converted to a spherical coordinate system. The coordinates are expressed as To express, the depth image is divided into non-overlapping n×n parts according to a certain number, and each part is divided into n×n parts according to the pixel points. The gradient value is divided into m parts, resulting in m histograms. Each part is then represented by a feature vector of n×n×m×2 dimensions, converting the depth information into a multi-dimensional feature vector and completing the conversion and storage of depth information. For RGB images, the SUFT descriptor is used. Each 4×4 region unit has four relative gradient histograms, resulting in a 64-dimensional feature vector.

[0159] Furthermore, optionally, in this embodiment, obtaining the second key point information of the target object includes:

[0160] Processing the reference image of the target object using a second detection model to obtain two-dimensional coordinate information of key points of the target object;

[0161] Obtaining second key point information of the target object according to the two-dimensional coordinate information and the depth information of the key point;

[0162] Wherein, the reference image is an RGB image;

[0163] The second detection model is a constructed neural network module that determines the coordinates of key points based on RGB images.

[0164] Here, the second detection model is also constructed by training a neural network model and is used to detect key temperature points on the target object. However, because the reference image is an RGB image, the output is the two-dimensional coordinate information of the key points. The RGB image serves as the input to this second detection model. Of course, the RGB image input to this second detection model is preferably an image obtained by performing a rough extraction of the target object region based on the original RGB image of the target object to improve detection efficiency.

[0165] Taking the target object as a human face, for example, for the RGB image of the face, the second detection model uses MobileNet-V2. The second detection model can be obtained by training the neural network model based on the training set for a predetermined number of times. The output layer obtains the coordinate information of the key points of the face through the fully connected layer and the softmax layer, such as the two-dimensional coordinate information of 25 key points of the eyes and forehead. Figure 2 The image area 201 is shown. Of course, if the neck position can be detected, 5 more key points at the corresponding position are added. The training set is obtained by preprocessing the data set of dense key points of the face, and the key points of the eyes, forehead, and neck are retained here.

[0166] The two-dimensional coordinate information of the key points obtained by the second detection model is further combined with the depth information of the key points to obtain the second key point information of the target object and convert it into three-dimensional coordinate information. Here, the depth information of the key points used can be collected by the camera shooting the target object.

[0167] Optionally, in this embodiment, the offset information is an offset distance; step 102 includes:

[0168] Obtaining an offset distance of a related key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information;

[0169] Selecting target key points according to the offset distance;

[0170] The target key point information is determined according to the three-dimensional coordinate information of the target key point.

[0171] In this way, after obtaining the offset distance of the relevant key points from the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information, the target key point can be further selected, and the required target key point information can be determined from the three-dimensional coordinate information of the target key point.

[0172] Specifically, the two-dimensional coordinate information of the key points output by the second detection model can be converted into XYZ three-dimensional coordinates in combination with the depth information of the key points. If the first detection model outputs three-dimensional spherical coordinate information, it will also be mapped to XYZ coordinates to facilitate the processing between the two sets of key point information. In this way, two point sets will be obtained, each of which contains the three-dimensional coordinates of multiple key points in different parts: M RGB =(X 1i ,Y li ,Z li ,…,X ni ,Y ni ,Z ni ) T , represents the key point coordinates of the second key point information; M D =(X li ′,Y li ′,Z li ′,…,X ni ′,Y ni ′,Z ni ′) T , representing the key point coordinates of the first key point information. Wherein, n is the number of key points. In this embodiment, it is preferred that the first detection model and the second detection model output the same number of key points and have mutually related key points, that is, there are n groups of related key points.

[0173] Due to the position difference of the cameras, the coordinates of the two sets of key points need to be aligned to the unified coordinate system according to the following calculation formula, where θ and S represent the rotation and displacement relationship:

[0174]

[0175] Among them, (X D , Y D , Z D ) is the coordinate of the key point in the first key point information under XYZ; (X RGB , Y RGB , Z RGB ) is the XYZ coordinate of the key point in the second key point information.

[0176] Optionally, in this embodiment, obtaining the offset distance of the relevant key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information includes:

[0177] In the same coordinate system, the offset distance is calculated based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information.

[0178] Accordingly, selecting target key points according to the offset distance includes:

[0179] Based on a preset value N, N key points are selected as target key points, the offset distances of the N key points are smaller than the offset distances of the remaining key points, and N is an integer greater than or equal to 1.

[0180] In this way, in the same coordinate system, based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information, the offset distance can be calculated by the offset coordinate difference. If the relevant key point is key point A in the first key point information and key point B in the second key point information, the coordinates of key point A (X i , Y i , Z i ) and the coordinates of key point B (X i ′,Y i ′,Z i ′), the offset coordinate difference is Δx=|X i -X i ′|,Δy=|Y i -Y i ′|,Δz=|Z i -Z i ′|, then the offset distance ΔL is calculated by the formula calculate.

[0181] After calculating the offset distances of each group of related key points, N key points can be selected as target key points based on a preset value N, and the offset distances of the N key points are smaller than the offset distances of the remaining key points.

[0182] Alternatively, optionally, obtaining the offset distance of the relevant key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information includes:

[0183] In different coordinate systems, an offset distance is calculated based on three-dimensional coordinate information of relevant key points in the first key point information and the second key point information, wherein the origins of the different coordinate systems are different key points.

[0184] Accordingly, selecting target key points according to the offset distance includes:

[0185] Calculate the average offset distance of each key point;

[0186] Based on a preset value M, M key points are selected as target key points, the average offset distance of the M key points is smaller than the average offset distance of the remaining key points, and M is an integer greater than or equal to 1.

[0187] Thus, for multiple groups of related keypoints in the first and second keypoint information, different coordinate systems can be constructed with each keypoint as the origin. The offset distances for the other groups of related keypoints can be calculated using the same offset distance calculation method as described above. This results in multiple offset distances for each group of related keypoints. Therefore, after further calculating the average offset distance for each keypoint, M keypoints can be selected as target keypoints based on a preset value M, with the average offset distance of these M keypoints being less than the average offset distance of the remaining keypoints.

[0188] In addition, for the selected target key points, there are two sets of three-dimensional coordinates with offsets, and the offsets need to be corrected. Optionally, in this embodiment, determining the target key point information based on the three-dimensional coordinate information of the target key point includes:

[0189] Correcting the three-dimensional coordinate information of the target key point according to the correction parameter and the offset distance of the target key point to obtain corrected three-dimensional coordinate information;

[0190] The corrected three-dimensional coordinate information is used as the target key point information.

[0191] That is, after determining the target key point, the three-dimensional coordinates of the target key point are corrected using the correction parameter based on the offset distance between the target key point and the corresponding related key point. For example, if the correction parameter is σ, the correction displacement is S x =σ·Δx, S x =σ·Δy, S z=σ·Δz, where 0<σ<1, the displacement direction is the opposite direction of the corresponding points, and the corrected coordinates are used as the target key point coordinates, that is, the target key point information.

[0192] Among them, the correction parameters are determined according to different influencing factors, including but not limited to light intensity, face angle, face occlusion, depth map noise, etc., because the errors in RGB image recognition are caused by light intensity, face angle, and face occlusion, while the errors in depth cameras are caused by narrow field of view, noise, and inability to measure transmissive materials. By sampling multiple key points in different parts for temperature measurement, the inaccuracy that is easily caused by single-point measurement is avoided.

[0193] After determining the coordinates of the target key points, the coordinate information can be transmitted to the high-precision infrared temperature sensor to measure the temperature. However, the measured temperature may be affected by external factors and may have errors. Therefore, in this embodiment, after step 103, the following steps are further included:

[0194] The measured temperature is corrected by a temperature correction model to obtain a target temperature;

[0195] The loss function of the temperature correction model is set based on the standard temperature difference, the ambient temperature and the depth information of the target object.

[0196] That is, the loss function is set based on the relative template standard temperature difference, ambient temperature, and target depth data of different key points measured by the infrared camera, and the temperature correction model is trained. The model outputs the corrected temperature as the final measured temperature, that is, the target temperature, which is more accurate. The loss function is as follows f(ε 2 x) is the temperature difference function, T(x) is the environmental influence unit, and S(x) is the distance influence unit.

[0197] Next, combine Figure 6 The specific application of the embodiment of the present invention is described as follows:

[0198] First, the structured light depth camera captures the face (target object) to obtain a depth image, and the visible light camera (RGB camera) captures the face to obtain an RGB image.

[0199] After that, the depth image is preprocessed and feature-fused with the RGB image. After the fused features are input into the first detection model, the first key point information is output, that is, the 3D coordinate information of a set of key points.

[0200] After the RGB image is input into the second detection model, the output is a set of two-dimensional coordinate information of key points. This two-dimensional coordinate information of the key points is further combined with the depth information of the key points to obtain the second key point information, that is, the three-dimensional coordinate information of another set of key points.

[0201] By processing the three-dimensional coordinate information of the two sets of key points and obtaining the offset information of the relevant key points, the target key point information can be obtained, that is, the three-dimensional coordinate information of the target key point used for temperature measurement. For example, an infrared temperature sensor measures the temperature of the target key point.

[0202] Taking into account the influence of external factors, the measured temperature will also be input into the temperature correction model for correction to output the corrected temperature to achieve more accurate measurement.

[0203] In this embodiment, the device applying the method of the embodiment of the present invention includes a computer processor for storing and running a computer program to complete the above method tasks; the device also includes: a structured light depth camera and a visible light camera for acquiring depth images and RGB images; and a high-precision infrared temperature sensor for measuring temperature.

[0204] like Figure 3 As shown, a temperature detection device according to an embodiment of the present invention includes:

[0205] An acquisition module 310 is configured to acquire first key point information and second key point information of a target object;

[0206] A processing module 320 is configured to determine target key point information of the target object based on the offset information of the relevant key points in the first key point information and the second key point information;

[0207] The measuring module 330 is configured to measure the temperature of the target object according to the target key point information.

[0208] Optionally, the acquisition module includes:

[0209] A first processing submodule is configured to process the image data of the target object using a first detection model to obtain first key point information of the target object;

[0210] The image data is a fusion feature of the depth image feature and the RGB image feature of the target object;

[0211] The first detection model is a constructed neural network model that determines key point information based on image data.

[0212] Optionally, the acquisition module further includes:

[0213] An acquisition submodule, configured to acquire a depth image and an RGB image of the target object;

[0214] An extraction submodule, configured to extract depth image features of the depth image and RGB image features of the RGB image respectively;

[0215] A determination submodule, configured to determine target weights of the depth image and the RGB image;

[0216] A feature fusion submodule is used to combine the depth image feature and the RGB image feature according to the target weight to obtain the fused feature.

[0217] Optionally, the target weight is expressed as Calculated, where w is the target weight, μ intra is the mean of the inter-class distances between the depth image and the RGB image, σ intra is the variance of the inter-class distance between the depth image and the RGB image, μ inter is the mean of the intra-class distances between the depth image and the RGB image, σ inter is the variance of the intra-class distance between the depth image and the RGB image.

[0218] Optionally, the acquisition module further includes:

[0219] A second processing submodule is configured to process the reference image of the target object using a second detection model to obtain two-dimensional coordinate information of key points of the target object;

[0220] a third processing submodule, configured to obtain second key point information of the target object according to the two-dimensional coordinate information and the depth information of the key point;

[0221] Wherein, the reference image is an RGB image;

[0222] The second detection model is a constructed neural network module that determines the coordinates of key points based on RGB images.

[0223] Optionally, the offset information is an offset distance;

[0224] The processing module includes:

[0225] a fourth processing submodule, configured to obtain an offset distance of a related key point based on the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information;

[0226] A selection submodule, for selecting target key points according to the offset distance;

[0227] The fifth processing submodule is configured to determine the target key point information according to the three-dimensional coordinate information of the target key point.

[0228] Optionally, the fourth processing submodule is further configured to:

[0229] In the same coordinate system, the offset distance is calculated based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information.

[0230] Optionally, the selection submodule is further configured to:

[0231] Based on a preset value N, N key points are selected as target key points, the offset distances of the N key points are smaller than the offset distances of the remaining key points, and N is an integer greater than or equal to 1.

[0232] Optionally, the fourth processing submodule is further configured to:

[0233] In different coordinate systems, an offset distance is calculated based on three-dimensional coordinate information of relevant key points in the first key point information and the second key point information, wherein the origins of the different coordinate systems are different key points.

[0234] Optionally, the selection submodule is further configured to:

[0235] Calculate the average offset distance of each key point;

[0236] Based on a preset value M, M key points are selected as target key points, the average offset distance of the M key points is smaller than the average offset distance of the remaining key points, and M is an integer greater than or equal to 1.

[0237] Optionally, the fifth processing submodule includes:

[0238] a correction unit, configured to correct the three-dimensional coordinate information of the target key point according to the correction parameter and the offset distance of the target key point to obtain corrected three-dimensional coordinate information;

[0239] A processing unit is configured to use the corrected three-dimensional coordinate information as the target key point information.

[0240] Optionally, the device further comprises:

[0241] A correction module is used to correct the measured temperature through a temperature correction model to obtain a target temperature;

[0242] The loss function of the temperature correction model is set based on the standard temperature difference, the ambient temperature and the depth information of the target object.

[0243] For the target object of temperature measurement, after obtaining two sets of key point information of the target object, the device will further determine more accurate target key point information for measuring the target object based on the offset information of the relevant key points in the two sets of key point information, and then use the target key point information to complete the temperature measurement of the target object. Here, because the target key point information can more accurately indicate the effective temperature measurement position of the target object, more accurate and efficient temperature measurement can be achieved.

[0244] It should be noted that this device is a device that applies the above-mentioned temperature detection method. The implementation method of the above-mentioned method embodiment is applicable to this device and can also achieve the same technical effect.

[0245] like Figure 4 As shown, a temperature detection device 400 according to an embodiment of the present invention includes a processor 410, wherein the processor 410 is configured to:

[0246] Obtain first key point information and second key point information of the target object;

[0247] Determining target key point information of the target object according to offset information of relevant key points in the first key point information and the second key point information;

[0248] The temperature of the target object is measured according to the target key point information.

[0249] Optionally, the processor is further configured to:

[0250] Processing the image data of the target object using a first detection model to obtain first key point information of the target object;

[0251] The image data is a fusion feature of the depth image feature and the RGB image feature of the target object;

[0252] The first detection model is a constructed neural network model that determines key point information based on image data.

[0253] Optionally, the processor is further configured to:

[0254] Acquire a depth image and an RGB image of the target object;

[0255] Extracting depth image features of the depth image and RGB image features of the RGB image respectively;

[0256] Determining target weights for the depth image and the RGB image;

[0257] The depth image feature and the RGB image feature are combined according to the target weight to obtain the fusion feature.

[0258] Optionally, the target weight is expressed as Calculated, where w is the target weight, μ intra is the mean of the inter-class distances between the depth image and the RGB image, σ intra is the variance of the inter-class distance between the depth image and the RGB image, μ inter is the mean of the intra-class distances between the depth image and the RGB image, σ inter is the variance of the intra-class distance between the depth image and the RGB image.

[0259] Optionally, the processor is further configured to:

[0260] Processing the reference image of the target object using a second detection model to obtain two-dimensional coordinate information of key points of the target object;

[0261] Obtaining second key point information of the target object according to the two-dimensional coordinate information and the depth information of the key point;

[0262] Wherein, the reference image is an RGB image;

[0263] The second detection model is a constructed neural network module that determines the coordinates of key points based on RGB images.

[0264] Optionally, the offset information is an offset distance;

[0265] The processor is further configured to:

[0266] Obtaining an offset distance of a related key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information;

[0267] Selecting target key points according to the offset distance;

[0268] The target key point information is determined according to the three-dimensional coordinate information of the target key point.

[0269] Optionally, the processor is further configured to:

[0270] In the same coordinate system, the offset distance is calculated based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information.

[0271] Optionally, the processor is further configured to:

[0272] Based on a preset value N, N key points are selected as target key points, the offset distances of the N key points are smaller than the offset distances of the remaining key points, and N is an integer greater than or equal to 1.

[0273] Optionally, the processor is further configured to:

[0274] In different coordinate systems, an offset distance is calculated based on three-dimensional coordinate information of relevant key points in the first key point information and the second key point information, wherein the origins of the different coordinate systems are different key points.

[0275] Optionally, the processor is further configured to:

[0276] Calculate the average offset distance of each key point;

[0277] Based on a preset value M, M key points are selected as target key points, the average offset distance of the M key points is smaller than the average offset distance of the remaining key points, and M is an integer greater than or equal to 1.

[0278] Optionally, the processor is further configured to:

[0279] Correcting the three-dimensional coordinate information of the target key point according to the correction parameter and the offset distance of the target key point to obtain corrected three-dimensional coordinate information;

[0280] The corrected three-dimensional coordinate information is used as the target key point information.

[0281] Optionally, the processor is further configured to:

[0282] The measured temperature is corrected by a temperature correction model to obtain a target temperature;

[0283] The loss function of the temperature correction model is set based on the standard temperature difference, the ambient temperature and the depth information of the target object.

[0284] Optionally, the temperature detection device 400 further includes a transceiver 420 that transmits and receives data under the control of the processor 410. The temperature detection device of this embodiment, for a target object for temperature measurement, after obtaining two sets of key point information of the target object, further determines more accurate target key point information for measuring the target object based on the offset information of the relevant key points in the two sets of key point information, and then uses the target key point information to complete the temperature measurement of the target object. Here, because the target key point information can more accurately indicate the effective temperature measurement position of the target object, more accurate and efficient temperature measurement can be achieved.

[0285] A temperature detection device according to another embodiment of the present invention, such as Figure 5As shown, it includes a transceiver 510, a processor 500, a memory 520, and a program or instruction stored in the memory 520 and executable on the processor 500; when the processor 500 executes the program or instruction, the above-mentioned temperature detection method is implemented.

[0286] The transceiver 510 is configured to receive and send data under the control of the processor 500 .

[0287] Among them, Figure 5 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically various circuits of one or more processors represented by processor 500 and memory represented by memory 520, which are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not described further herein. The bus interface provides an interface. The transceiver 510 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different devices, the interface 530 may also be an interface capable of externally or internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, etc.

[0288] The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 can store data used by the processor 500 when performing operations.

[0289] A readable storage medium according to an embodiment of the present invention stores a program or instruction thereon. When the program or instruction is executed by a processor, the steps in the temperature detection method described above are implemented and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0290] The processor is the processor in the temperature detection device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0291] It should be further noted that many functional components described in this specification are referred to as modules in order to more particularly emphasize the independence of their implementation methods.

[0292] In embodiments of the present invention, modules can be implemented in software so that they can be executed by various types of processors. For example, an identified executable code module can include one or more physical or logical blocks of computer instructions, for example, which can be constructed as objects, procedures, or functions. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different locations, which, when logically combined together, constitute the module and achieve the specified purpose of the module.

[0293] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed in the middle of different programs, and distributed across a plurality of memory devices.Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form.Described operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0294] When a module can be implemented using software, given the current state of hardware technology, those skilled in the art can build corresponding hardware circuits to implement the corresponding functions of the module, regardless of cost. The hardware circuits may include conventional very large scale integration (VLSI) circuits or gate arrays, as well as existing semiconductors such as logic chips and transistors, or other discrete components. Modules may also be implemented using programmable hardware devices, such as field programmable gate arrays, programmable array logic, or programmable logic devices.

[0295] The above exemplary embodiments are described with reference to the accompanying drawings. Many different forms and embodiments are possible without departing from the spirit and teachings of the present invention. Therefore, the present invention should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be complete and perfect and will convey the scope of the invention to those skilled in the art. In the drawings, component sizes and relative sizes may be exaggerated for clarity. The terminology used herein is for purposes of describing specific exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "comprising" and / or "including," when used in this specification, indicate the presence of stated features, integers, steps, operations, components, and / or elements, but do not preclude the presence or addition of one or more other features, integers, steps, operations, components, elements, and / or groups thereof. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of that range and any subranges therebetween.

[0296] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A temperature detection method, characterized in that: include: Acquire first key point information and second key point information of a target object, where the first key point information is position information of a first temperature measurement point of the target object, and the second key point information is position information of a second temperature measurement point of the target object; Determining target key point information of the target object based on offset information of relevant key points in the first key point information and the second key point information, where the relevant key point is a key point in the same area or the same key point in the first key point information and the second key point information, and the key point is a temperature measurement location point of the target object; Measuring the temperature of the target object according to the target key point information; The offset information is an offset distance; The determining the target key point information of the target object according to the offset information of the relevant key points in the first key point information and the second key point information includes: Obtaining an offset distance of a related key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information; Selecting target key points according to the offset distance; The target key point information is determined according to the three-dimensional coordinate information of the target key point.

2. The method according to claim 1, characterized in that Get the first key point information of the target object, including: Processing the image data of the target object using a first detection model to obtain first key point information of the target object; The image data is a fusion feature of the depth image feature and the RGB image feature of the target object; The first detection model is a constructed neural network model that determines key point information based on image data.

3. The method according to claim 2, characterized in that Before the image data of the target object is processed by the first detection model to obtain the first key point information of the target object, the method further includes: Acquire a depth image and an RGB image of the target object; Extracting depth image features of the depth image and RGB image features of the RGB image respectively; Determining target weights for the depth image and the RGB image; The depth image feature and the RGB image feature are combined according to the target weight to obtain the fusion feature.

4. The method according to claim 3, characterized in that The target weight is expressed by the formula Calculated, where w is the target weight, μ intra is the mean of the inter-class distances between the depth image and the RGB image, σ intra is the variance of the inter-class distance between the depth image and the RGB image, μ inter is the mean of the intra-class distances between the depth image and the RGB image, σ inter is the variance of the intra-class distance between the depth image and the RGB image.

5. The method according to claim 1, wherein Get the second key point information of the target object, including: Processing the reference image of the target object using a second detection model to obtain two-dimensional coordinate information of key points of the target object; Obtaining second key point information of the target object according to the two-dimensional coordinate information and the depth information of the key point; Wherein, the reference image is an RGB image; The second detection model is a constructed neural network module that determines the coordinates of key points based on RGB images.

6. The method according to claim 1, characterized in that The obtaining, based on the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information, an offset distance of a related key point includes: In the same coordinate system, the offset distance is calculated based on the three-dimensional coordinate information of the relevant key points in the first key point information and the second key point information.

7. The method according to claim 6, characterized in that The step of selecting a target key point according to the offset distance includes: Based on a preset value N, N key points are selected as target key points, the offset distances of the N key points are smaller than the offset distances of the remaining key points, and N is an integer greater than or equal to 1.

8. The method according to claim 1, characterized in that The obtaining, based on the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information, an offset distance of a related key point includes: In different coordinate systems, an offset distance is calculated based on three-dimensional coordinate information of relevant key points in the first key point information and the second key point information, wherein the origins of the different coordinate systems are different key points.

9. The method according to claim 8, characterized in that The step of selecting a target key point according to the offset distance includes: Calculate the average offset distance of each key point; Based on a preset value M, M key points are selected as target key points, the average offset distance of the M key points is smaller than the average offset distance of the remaining key points, and M is an integer greater than or equal to 1.

10. The method according to claim 1, characterized in that The determining of the target key point information according to the three-dimensional coordinate information of the target key point includes: Correcting the three-dimensional coordinate information of the target key point according to the correction parameter and the offset distance of the target key point to obtain corrected three-dimensional coordinate information; The corrected three-dimensional coordinate information is used as the target key point information.

11. The method according to claim 1, wherein After measuring the temperature of the target object according to the target key point information, the method further includes: The measured temperature is corrected by a temperature correction model to obtain a target temperature; The loss function of the temperature correction model is set based on the standard temperature difference, the ambient temperature and the depth information of the target object.

12. A temperature detection device, characterized in that: include: an acquisition module, configured to acquire first key point information and second key point information of a target object, wherein the first key point information is position information of a first temperature measurement point of the target object, and the second key point information is position information of a second temperature measurement point of the target object; a processing module, configured to determine target key point information of the target object based on offset information of relevant key points in the first key point information and the second key point information, where the relevant key points are key points in the same area or the same key points in the first key point information and the second key point information, and the key points are temperature measurement locations of the target object; A measuring module, configured to measure the temperature of the target object based on the target key point information; The offset information is an offset distance; the processing module includes: a fourth processing submodule, configured to obtain an offset distance of a related key point based on the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information; A selection submodule, for selecting target key points according to the offset distance; The fifth processing submodule is configured to determine the target key point information according to the three-dimensional coordinate information of the target key point.

13. A temperature detection device, characterized in that: comprising a processor configured to: Acquire first key point information and second key point information of a target object, where the first key point information is position information of a first temperature measurement point of the target object, and the second key point information is position information of a second temperature measurement point of the target object; Determining target key point information of the target object based on offset information of relevant key points in the first key point information and the second key point information, where the relevant key point is a key point in the same area or the same key point in the first key point information and the second key point information, and the key point is a temperature measurement location point of the target object; Measuring the temperature of the target object according to the target key point information; The offset information is an offset distance; and the processor is further configured to: Obtaining an offset distance of a related key point according to the three-dimensional coordinate information of each key point in the first key point information and the three-dimensional coordinate information of each key point in the second key point information; Selecting target key points according to the offset distance; The target key point information is determined according to the three-dimensional coordinate information of the target key point.

14. A temperature detection device comprising: A transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; wherein the processor implements the temperature detection method according to any one of claims 1 to 11 when executing the program or instruction.

15. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps in the temperature detection method according to any one of claims 1 to 11 are implemented.

Images