Contactless sublingual body temperature measurement device and method

Abstract

A non-contact sublingual body temperature measuring device and a method for measuring sublingual body temperature are provided, which detect whether the tongue of a subject is open or closed and measure the temperature under the tongue of the subject in a non-contact manner. By using a measuring device comprising a recognition unit that recognizes the state of the subject, a control unit that determines the state of the subject received through the recognition unit and determines the body temperature of the subject, and an output unit that outputs the body temperature of the subject determined through the control unit, the body temperature of the subject can be measured in a non-contact manner through the steps of: analyzing face image data of the subject; measuring the temperature under the tongue of the subject when it is determined that the tongue of the subject is open; determining the body temperature of the subject; and outputting the body temperature of the subject.

Description

Technology Field

[0001] The present application relates to a non-contact sublingual body temperature measuring device and a sublingual body temperature measuring method that detect whether the tongue of a subject is open or closed and non-contact the temperature under the tongue of the subject. More specifically, the invention relates to a method for non-contact measuring the body temperature of a subject by using a measuring device comprising: a recognition unit that recognizes the state of the subject; a control unit that determines the state of the subject received through the recognition unit and determines the body temperature of the subject; and an output unit that outputs the body temperature of the subject determined through the control unit, through the steps of: analyzing face image data of the subject; measuring the temperature under the tongue of the subject when it is determined that the tongue of the subject is open; determining the body temperature of the subject; and outputting the body temperature of the subject. Background Technology

[0003] Generally, when body temperature measurement is required, the temperature of the outer parts of the body, such as the underarm (axilla), ear (eardrum), or forehead, which are easy to measure—that is, the skin temperature—is measured. However, skin temperature measured in this way is easily influenced by temperature changes in the environment in which the subject is measured, so it is unsuitable when accurate body temperature data of the subject is required.

[0004] Meanwhile, in cases such as surgical procedures, body temperature data must be acquired as a means of continuously monitoring changes in a patient's body over extended periods. For instance, it is common to measure rectal temperature using thermometers. Furthermore, in physiological research studies, accurate measurement of body temperature is also required to monitor changes in subjects over long durations, similar to surgical procedures.

[0005] Therefore, in cases requiring accurate body temperature data, such as surgical procedures or physiological research tests, core body temperatures—such as those found in the mouth or rectum—are measured rather than the conventional skin temperature. In other words, to obtain accurate body temperature data, core temperatures are measured, as they do not fluctuate significantly due to changes in the ambient temperature of the subject.

[0006] Accordingly, while the measurement of core rectal temperature is considered in cases such as surgical procedures or physiological research tests, unlike surgery, rectal temperature measurement causes discomfort to the subject, leading to the problem that the subject's psychological stress affects the test results. Furthermore, in environments where the subject is placed, such as measuring changes in body temperature during bathing, there were issues where the rectal temperature measurement itself could be dangerous or difficult. The problem to be solved

[0008] The technical problem that this application aims to solve is to provide a non-contact sublingual body temperature measuring device and a non-contact sublingual body temperature measuring method that are easy to measure.

[0009] Another technical problem that this application aims to solve is to provide a non-contact sublingual body temperature measuring device and a non-contact sublingual body temperature measuring method with improved accuracy.

[0010] Another technical problem that this application aims to solve is to provide a non-contact sublingual body temperature measuring device and a non-contact sublingual body temperature measuring method with a low risk of infection.

[0011] Another technical problem that this application aims to solve is to provide a non-contact sublingual body temperature measuring device and a non-contact sublingual body temperature measuring method in which discomfort to the measurement subject is reduced.

[0012] Another technical problem that this application aims to solve is to provide a highly safe non-contact sublingual body temperature measuring device and a non-contact sublingual body temperature measuring method.

[0013] Another technical problem that this application aims to solve is to provide a non-contact sublingual body temperature measuring device and a non-contact sublingual body temperature measuring method with improved convenience for the measurer and the subject being measured.

[0014] The technical problems that this application aims to solve are not limited to those described above. means of solving the problem

[0016] To solve the above technical problem, the present application provides a non-contact sublingual body temperature measuring device that detects whether the tongue of a subject to measurement is open or closed and measures the temperature under the tongue of the subject to measurement in a non-contact manner.

[0017] According to one embodiment, the non-contact sublingual body temperature measuring device may include a recognition unit that recognizes the state of the measurement target, a control unit that determines the state of the measurement target received through the recognition unit and determines the body temperature of the measurement target, and an output unit that outputs the body temperature of the measurement target determined through the control unit.

[0018] According to one embodiment, the recognition unit may include an optical camera that analyzes face image data of the measurement target, and an infrared sensor that measures the body temperature of the measurement target.

[0019] To solve the above technical problem, the present application provides a method for measuring sublingual body temperature by measuring the temperature under the tongue of a subject in a non-contact manner.

[0020] According to one embodiment, a method for measuring sublingual body temperature may include the steps of: analyzing facial image data of a subject to measurement; measuring the temperature under the tongue of a subject to measurement when it is determined that the tongue of the subject to measurement is open; determining the body temperature of the subject to measurement; and outputting the body temperature of the subject to measurement.

[0021] According to one embodiment, the step of analyzing the face image data of the measurement target may include the step of determining whether the measurement target is wearing a mask through an optical camera, the step of determining whether the mouth of the measurement target is open or closed through an optical camera, the step of determining whether the tongue of the measurement target is open or closed through an optical camera, the step of specifying the sublingual region of the measurement target, the step of acquiring sublingual image data of the measurement target, and the step of extracting feature points within the sublingual image data.

[0022] According to one embodiment, the step of measuring the temperature under the tongue of the subject to measurement may include the step of acquiring sublingual temperature data of the subject to measurement through an infrared sensor, and the step of extracting feature points within the sublingual temperature data. Effects of the invention

[0024] According to an embodiment of the present application, a non-contact sublingual body temperature measuring device comprises a recognition unit that recognizes the state of a measurement target, a control unit that determines the state of the measurement target received through the recognition unit and determines the body temperature of the measurement target, and an output unit that outputs the body temperature of the measurement target determined through the control unit. The recognition unit may include an optical camera that analyzes face image data of the measurement target and an infrared sensor that measures the body temperature of the measurement target. Accordingly, the body temperature of the measurement target can be easily measured, and the accuracy of the body temperature measurement can be improved.

[0025] According to a non-contact method for measuring sublingual body temperature according to an embodiment of the present application, the method comprises the steps of: analyzing facial image data of a subject to measurement; recognizing the sublingual region of the subject to measurement when it is determined that the tongue of the subject to measurement is open; measuring the temperature under the tongue of the subject to measurement; determining the body temperature of the subject to measurement; and outputting the body temperature of the subject to measurement. The step of analyzing facial image data of the subject to measurement may include the steps of: determining whether the subject to measurement is wearing a mask through an optical camera; determining whether the mouth of the subject to measurement is open or closed through an optical camera; determining whether the tongue of the subject to measurement is open or closed through an optical camera; specifying the sublingual region within the facial image data; acquiring sublingual image data of the subject to measurement; and extracting feature points within the sublingual image data. Accordingly, the risk of infection by contact is reduced, and discomfort of the subject to measurement may be reduced. Furthermore, safety may be enhanced as the body temperature is measured non-contact. Brief explanation of the drawing

[0027] FIG. 1 is a drawing for explaining a non-contact sublingual body temperature measurement method according to an embodiment of the present application. FIG. 2 is an image of the tongue of a measurement target to explain a non-contact method for measuring sublingual body temperature according to an embodiment of the present application. FIG. 3 is a drawing for explaining a non-contact sublingual body temperature measuring device according to an embodiment of the present application. FIGS. 4 to 8 are drawings for explaining the step of analyzing face image data of a measurement target in a non-contact sublingual body temperature measurement method according to an embodiment of the present application. FIG. 9 is a drawing illustrating the step of measuring the temperature under the tongue of a subject to measurement in a non-contact sublingual body temperature measurement method according to an embodiment of the present application. FIG. 10 is a drawing for explaining the step of determining the body temperature of a subject to measurement in a non-contact sublingual body temperature measurement method according to an embodiment of the present application. FIG. 11 is a signal flow diagram illustrating a non-contact sublingual body temperature measurement method according to an embodiment of the present application. FIG. 12 is a signal flow diagram for explaining a non-contact sublingual body temperature measurement method according to a first variation of the present application. FIG. 13 is a signal flow diagram for explaining a non-contact sublingual body temperature measurement method according to a second variation of the present application. Specific details for implementing the invention

[0028] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. However, the technical concept of the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided to ensure that the disclosed content is thorough and complete and to ensure that the concept of the present invention is sufficiently conveyed to those skilled in the art.

[0029] Additionally, although terms such as first, second, third, etc., have been used to describe various components in the various embodiments of this specification, these components should not be limited by such terms. These terms are used merely to distinguish one component from another. Accordingly, what is referred to as the first component in one embodiment may be referred to as the second component in another embodiment. Each embodiment described and illustrated herein also includes its complementary embodiment. Furthermore, in this specification, "and / or" is used to mean including at least one of the components listed before and after it.

[0030] In the specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, terms such as "comprising" or "having" are intended to specify the existence of the features, numbers, steps, components, or combinations thereof described in the specification, and should not be understood as excluding the existence or addition of one or more other features, numbers, steps, components, or combinations thereof. Additionally, in the following description of the invention, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the invention, such detailed description will be omitted.

[0032] FIG. 1 is a drawing for explaining a non-contact sublingual body temperature measurement method according to an embodiment of the present application, FIG. 2 is a sublingual image of a measurement target for explaining a non-contact sublingual body temperature measurement method according to an embodiment of the present application, FIG. 3 is a drawing for explaining a non-contact sublingual body temperature measurement device according to an embodiment of the present application, FIG. 4 to FIG. 8 are drawings for explaining a step of analyzing face image data of a measurement target among a non-contact sublingual body temperature measurement method according to an embodiment of the present application, FIG. 9 is a drawing for explaining a step of measuring the temperature under the tongue of a measurement target among a non-contact sublingual body temperature measurement method according to an embodiment of the present application, and FIG. 10 is a drawing for explaining a step of determining the body temperature of a measurement target among a non-contact sublingual body temperature measurement method according to an embodiment of the present application.

[0033] Referring to FIGS. 1 to 10, a non-contact sublingual body temperature measuring device according to an embodiment of the present application detects whether the tongue of a subject being measured is open or closed, and can non-contact measure the temperature of the sublingual (1), which is the temperature under the tongue of the subject being measured.

[0034] The above non-contact sublingual body temperature measuring device includes a recognition unit (100), a control unit (200), and an output unit (300), and can measure the temperature of the sublingual (1) of the subject to measurement non-contact.

[0035] As described above, the sublingual area (1) can be defined as the area under the tongue of the subject being measured. The temperature of the sublingual area (1) can be measured to obtain accurate body temperature data of the subject being measured. As illustrated in FIG. 2, the sublingual area (1) includes a lingual frenulum (2) and a pair of sublingual veins (3).

[0036] The lingual frenulum (2) can be defined as a tendon formed in the shape of a vertical band connecting from the underside of the tongue to the lower part of the mouth. The lingual frenulum (2) serves to limit the range of motion of the tongue to a certain level and can also assist in swallowing.

[0037] The above-mentioned sublingual vein (3) is a pair of veins symmetrically formed around the lingual frenulum (2) and may have the characteristic of being blue, contrasting with the red tongue. Accordingly, the above-mentioned sublingual vein (3) is known as the vein that can be most clearly observed among the veins that can generally be observed. The temperature of the above-mentioned sublingual vein (3) corresponds to the core body temperature, so accurate body temperature data of the subject to measurement can be obtained through the temperature of the above-mentioned sublingual vein (3).

[0038] The face image data of the above measurement target is analyzed (S100).

[0039] The recognition unit (100) can recognize the state of the measurement target. The recognition unit (100) can recognize the face image of the measurement target and measure the surface temperature of the measurement target. More specifically, the recognition unit (100) can determine whether a mask is worn, whether the mouth of the measurement target is open or closed, and whether the tongue of the measurement target is open or closed through the face image of the measurement target, and measure the surface temperature of the measurement target. The recognition unit (100) may include an optical camera (110) and an infrared sensor (120).

[0040] Whether the measurement target is wearing a mask is determined through an optical camera (S110).

[0041] The optical camera (110) can capture a face image of the measurement target. The face image data of the measurement target obtained through the optical camera (110) can be transmitted to the control unit (200). Using the face image data obtained through the optical camera (110), it can be determined whether the measurement target is wearing a mask. Through the optical camera (110), it can be determined not only whether the measurement target is wearing a mask, but also whether there is a means covering the mouth of the measurement target, such as a niqab, burka, or chador. If it is determined through the optical camera (110) that the measurement target is wearing a mask, a notification to remove the mask can be transmitted to the measurement target.

[0042] The opening or closing of the mouth of the above-mentioned measurement target is determined (S120).

[0043] Using the image obtained through the optical camera (110), if it is determined that the subject being measured is not wearing a mask, after confirming whether the face is recognized, it can be determined whether the mouth of the subject being measured is open or closed. If it is determined through the optical camera (110) that the mouth of the subject being measured is closed, a notification requesting the subject to open their mouth can be sent.

[0044] The control unit (200) can determine whether the mouth of the measurement target is open or closed by using the face image data of the measurement target obtained through the optical camera (110). The control unit (200) can identify the upper lip and the lower lip in the face image data by utilizing a face recognition library. For example, the control unit (200) can calculate the distance between the upper lip and the lower lip identified through the face image data, but is not limited thereto.

[0045] The opening or closing of the tongue of the above-mentioned measurement target is determined (S130).

[0046] When it is determined that the mouth of the measurement target is open using the image data obtained through the optical camera (110), the openness or closure of the tongue of the measurement target can be determined. When it is determined that the tongue of the measurement target is closed through the optical camera (110), a notification to open the tongue can be sent to the measurement target.

[0047] The control unit (200) can determine whether the tongue of the measurement target is open or closed by using face image data obtained through the optical camera (110). For example, the control unit (200) can determine that the tongue of the measurement target is open when the tongue of the measurement target is lifted. Conversely, the control unit (200) can determine that the tongue of the measurement target is closed when the tongue of the measurement target is lowered rather than lifted. The control unit (200) can determine whether the tongue is open or closed by training a dataset regarding the opening or closing of the tongue with artificial intelligence, or the control unit (200) can determine through image analysis whether a pair of the sublingual veins (3) are symmetrical with respect to the lingual frenulum (2). The control unit (200) can determine that the tongue is open if it has an error within a reference range and determines that a pair of the sublingual veins (3) are symmetrical.

[0048] The sublingual region is specified within the above face image data (S135).

[0049] The control unit (200) can identify a sublingual region (30a) including the lingual frenulum (2) and a pair of sublingual veins (3) of the measurement target within the face image data obtained through the optical camera (110). For example, the control unit (200) can identify the sublingual region (30a) by training a sublingual image data set (21) for the sublingual region (30a) with artificial intelligence. As another example, the control unit (200) can identify the sublingual region (30a) through image analysis to determine whether the pair of sublingual veins (3) are symmetrical with respect to the lingual frenulum (2).

[0050] Sublingual image data of the above measurement target is obtained (S140).

[0051] The control unit (200) can determine that the tongue of the measurement target is open by using the face image data obtained through the optical camera (110), and can specify the sublingual region (30a) by utilizing the sublingual image data set (21) and the U-Net model (23). The control unit (200) can obtain the sublingual image processing data (30) in which the sublingual region (30a) of the measurement target is specified.

[0052] The above sublingual image data (10) may include an image of the sublingual area (1) under the tongue of the measurement target. The above sublingual image data (10) may include image data of the lingual frenulum (2) of the measurement target, image data of a pair of sublingual veins (3), image data of teeth, etc.

[0053] Feature points are extracted from the above sublingual image data (S150).

[0054] Using the sublingual image data (10) obtained through the optical camera (110), feature points can be extracted from the sublingual image data (10) of the measurement target. One or more feature points may be extracted.

[0055] As described above, the control unit (200) can receive face image data of the measurement target obtained through the optical camera (110). The control unit (200) can determine whether the measurement target is wearing a mask, whether the mouth of the measurement target is open or closed, and whether the tongue of the measurement target is open or closed by using the face image data of the measurement target obtained through the optical camera (110). More specifically, the control unit (200) can determine whether the measurement target is wearing a mask, whether the mouth of the measurement target is open or closed, and whether the tongue of the measurement target is open or closed by applying the face image data obtained through the optical camera (110) to a deep learning model (20). When the control unit (200) determines that the measurement target is not wearing a mask, that the measurement target has its mouth open, and that the tongue of the measurement target is open, it can obtain the sublingual image data (10) from the face image data of the measurement target through the optical camera (110).

[0056] The control unit (200) can generate sublingual image processing data (30) by applying the sublingual image data (10) obtained through the optical camera (110) to a deep learning model (20). The control unit (200) can extract one or more first feature points (31, 32, 33) from the sublingual image processing data (30).

[0057] The deep learning model (20) can specify the area of ​​the sublingual (1) of the measurement target by utilizing medical image segmentation (medical image segmentation function). The deep learning model (20) may include a convolutional neural network model. For example, the deep learning model (20) may include the application of a U-net model.

[0058] The above sublingual image processing data (30) may include a simplified version for analyzing the above sublingual image data (10). The above sublingual image processing data (30) may include one or more of the above first feature points (31, 32, 33).

[0059] The first feature points (31, 32, 33) may include a certain region having certain features within the sublingual image processing data (30). For example, the first feature points (31, 32, 33) may include a first-1 feature point (31), a first-2 feature point (32), and a first-3 feature point (33).

[0060] The above 1-1 feature point (31) may include a feature point determined to be the lingual frenulum (2) within the sublingual image processing data (30). For example, the above 1-1 feature point (31) may include a feature point determined to be located in the center of the area determined to be the tooth of the measurement target by applying the sublingual image processing data (30) to the deep learning model (20). As another example, the above 1-1 feature point (31) may include a feature point determined to be formed in a vertical band shape by applying the sublingual image processing data (30) to the deep learning model (20).

[0061] The first-2 feature point (32) and the first-3 feature point (33) may include feature points determined to be a pair of the sublingual veins (3). For example, the first-2 feature point (32) and the first-3 feature point (33) may include features determined to be blue, formed symmetrically around the first-1 feature point (31) by applying the sublingual image processing data (30) to the deep learning model (20).

[0062] In other words, the measurement target can be photographed through the optical camera (110) to obtain the sublingual image data (10) and the sublingual image processing data (30), and by applying the sublingual image data (10) and the sublingual image processing data (30) to the deep learning model (20), the condition of the measurement target can be recognized non-contactually to determine whether the measurement target is wearing a mask, whether the measurement target's mouth is open or closed, and whether the measurement target's tongue is open or closed, thereby making it easy to measure body temperature, significantly lowering the risk of infection, and preventing discomfort in the measurement target.

[0063] The temperature under the tongue of the above-mentioned measurement target is measured (S200).

[0064] The infrared sensor (120) can measure the surface temperature of the measurement target. The infrared sensor (120) can measure the temperature of the sublingual (1) of the measurement target when the measurement target removes the mask, opens the mouth, and opens the tongue so that the sublingual (1) is exposed to the outside.

[0065] The sublingual temperature data of the measurement target is obtained through an infrared sensor (S210).

[0066] Through the infrared sensor (120), the temperature data (40) including the temperature distribution of the underside (1) of the measurement target can be obtained.

[0067] The above-mentioned sublingual temperature data (40) may include one or more feature points, such as second feature points (41, 42, 43).

[0068] Feature points are extracted from the above sublingual temperature data (S220).

[0069] The control unit (200) can apply the under-tongue temperature data (40) obtained through the infrared sensor (120) to the deep learning model (20) to extract the second feature points (41, 42, 43).

[0070] The second feature points (41, 42, 43) may include a certain region having a certain feature within the sublingual temperature data (40). For example, the second feature points (41, 42, 43) may include a second-1 feature point (41), a second-2 feature point (42), and a second-3 feature point (43).

[0071] The above 2-1 feature point (41) may include a feature point determined to be the lingual frenulum (2) within the above sublingual temperature data (40). For example, the above 2-1 feature point (41) may include a feature point determined to be located at the center of the area determined to be the tooth of the measurement target by applying the above sublingual temperature data (40) to the deep learning model (20). As another example, the above 2-1 feature point (41) may include a feature point determined to be formed in the shape of a vertical band by applying the above sublingual temperature data (40) to the deep learning model (20).

[0072] The above 2-2 feature point (42) and the above 2-3 feature point (43) may include feature points determined to be a pair of the above sublingual veins (3). For example, the above 2-2 feature point (42) and the above 2-3 feature point (43) may include being formed symmetrically around the above 2-1 feature point (41) by applying the above sublingual temperature data (40) to the above deep learning model (20) and being determined to have a blue color.

[0073] In other words, the surface temperature of the measurement target can be measured through the infrared sensor (120) to obtain the sublingual temperature data (40). By applying the sublingual temperature data (40) to the deep learning model (20), the partial temperature of each of the second feature points (41, 42, 43) can be measured, thereby allowing the body temperature of the measurement target to be easily measured and improving the accuracy of the body temperature.

[0074] The body temperature of the above-mentioned measurement target is determined (S300).

[0075] The control unit (200) can determine the body temperature of the measurement target by comparing the sublingual image processing data (30) and the sublingual temperature data (40). More specifically, the control unit (200) can match the first feature point (31, 32, 33) and the second feature point (41, 42, 43). As shown in FIG. 10, the control unit (200) can match the first-1 feature point (31) with the second-1 feature point (41), match the first-2 feature point (32) with the second-2 feature point (42), and match the first-3 feature point (33) with the second-3 feature point (43). The control unit (200) can determine the body temperature of the measurement target by matching the first feature point (31, 32, 33) and the second feature point (41, 42, 43). The control unit (200) can transmit the determined body temperature of the measurement target to the output unit (300).

[0076] In other words, the control unit (200) can match the first feature points (31, 32, 33) of the sublingual image processing data (30) with the second feature points (41, 42, 43) of the sublingual temperature data (40) to match the surface temperature for each of the areas determined by the first feature points (31, 32, 33). As a result, the control unit (200) can measure the partial temperature of the lingual frenulum (2), the partial temperature of a pair of the sublingual veins (3), or the partial temperature of an area excluding the lingual frenulum (2) and the sublingual veins (3). The control unit (200) can determine the body temperature of the measurement target by utilizing a pair of the sublingual veins (3). Alternatively, the control unit (200) may determine the body temperature of the subject to measurement by selectively utilizing the partial temperature of the lingual frenulum (2), the partial temperature of a pair of the sublingual veins (3), or the partial temperature of a part excluding the lingual frenulum (2) and the sublingual veins (3). Accordingly, the body temperature of the subject to measurement can be accurately determined, and the safety of the subject to measurement can be guaranteed.

[0077] The body temperature of the above-mentioned measurement target is output (S400).

[0078] The output unit (300) can output the body temperature of the measurement target received through the control unit (200). The output unit (300) may include a display (310) and a voice output unit (320).

[0079] The display (310) can display the received body temperature of the subject being measured as a visual signal. For example, the display (310) can display the body temperature of the subject being measured as a number, or display it in colors such as low temperature (blue), average (green), and high temperature (red). Additionally, the display (310) can display the aforementioned notifications as visual signals. For example, it can visually display notifications such as "Please remove your mask," "Please open your mouth wide," or "Please lift your tongue up," but is not limited thereto.

[0080] The voice output unit (320) can transmit the received body temperature of the measurement target as an auditory signal. For example, the voice output unit (320) can notify the body temperature of the measurement target through sound, or provide a low temperature (slow warning sound), average temperature (silent or cheerful sound), high temperature (fast warning sound), etc. Additionally, the voice output unit (320) can transmit the aforementioned notifications as auditory signals. For example, it can transmit auditory notifications such as "Please remove your mask," "Please open your mouth wide," or "Please lift your tongue up," but is not limited thereto.

[0081] FIG. 11 is a signal flow diagram illustrating a non-contact sublingual body temperature measurement method according to an embodiment of the present application.

[0082] Referring to FIG. 11, a non-contact sublingual body temperature measurement method according to an embodiment of the present application can be measured through a non-contact sublingual body temperature measurement device comprising the recognition unit (100), the control unit (200), and the output unit (300).

[0083] The control unit (200) may request face image data from the recognition unit (100). After receiving the request for face image data from the control unit (200), the recognition unit (100) may acquire the face image data of the measurement target. The face image data acquired from the recognition unit (100) may be transmitted to the control unit (200). The control unit (200) may check whether a face is recognized through the face image data received from the recognition unit (100). The control unit (200) may determine whether the measurement target is wearing a mask. Although not illustrated, the control unit (200) may determine whether the target is wearing a means of covering the mouth, such as a niqab, burka, or chador, in addition to a mask. The control unit (200) may transmit the mask wearing information of the measurement target to the output unit (300). If the measurement subject is wearing a mask, the output unit (300) may output a notification such as "Please remove the mask." Alternatively, if the measurement subject is not wearing a mask, the process may proceed to the next step.

[0084] The output unit (300) can transmit a signal to the control unit (200) indicating completion of mask wearing information output when the measurement target is not wearing a mask. The control unit (200) can determine whether the mouth of the measurement target is open or closed after confirming whether the face is recognized through the face image data received from the recognition unit (100). The control unit (200) can transmit the status of whether the mouth of the measurement target is open or closed to the output unit (300). If the mouth of the measurement target is closed, the output unit (300) can output a notification such as "Please open your mouth wide." Alternatively, if the mouth of the measurement target is open, the process can proceed to the next step.

[0085] The output unit (300) can transmit a signal indicating the completion of outputting information on the opening and closing of the mouth to the control unit (200) when the measurement target has its mouth open. The control unit (200) can determine whether the tongue of the measurement target is open or closed through the face image data received from the recognition unit (100). The control unit (200) can transmit whether the tongue of the measurement target is open or closed to the output unit (300). If the tongue of the measurement target is closed, the output unit (300) can output a notification such as "Please lift your tongue up." Alternatively, if the tongue of the measurement target is open, the process can proceed to the next step.

[0086] The output unit (300) can transmit a signal indicating the completion of outputting information on whether the tongue is open or closed to the control unit (200) when the tongue of the measurement target is open. The control unit (200) can obtain sublingual image data (10) of the measurement target through the face image data received from the recognition unit (100). The control unit (200) can obtain sublingual image processing data (30) by applying the obtained sublingual image data (10) to the deep learning model (20). The control unit (200) can extract the first feature points (31, 32, 33) by applying the obtained sublingual image processing data (30) to the deep learning model (20).

[0087] After extracting the first feature points (31, 32, 33), the control unit (200) may request the sublingual temperature data (40) from the recognition unit (100). After receiving the request for the sublingual temperature data (40) from the control unit (200), the recognition unit (100) may acquire the sublingual temperature data (40) through the infrared sensor (120). The recognition unit (100) may transmit the acquired sublingual temperature data (40) to the control unit (200). The control unit (200) may apply the sublingual temperature data (40) to the deep learning model (20) to extract the second feature points (41, 42, 43). The control unit (200) can determine the body temperature of the measurement target by comparing the first feature points (31, 32, 33) of the sublingual image processing data (30) with the second feature points (41, 42, 43) of the sublingual temperature data (40). The control unit (200) can transmit the determined body temperature information of the measurement target to the output unit (300). The output unit (300) can output the body temperature information of the measurement target.

[0088] FIG. 12 is a signal flow diagram for explaining a non-contact sublingual body temperature measurement method according to a first variation of the present application.

[0089] Referring to FIG. 12, a non-contact sublingual body temperature measurement method according to a first variant of the present application can be measured through a non-contact sublingual body temperature measurement device comprising the recognition unit (100), the control unit (200), and the output unit (300).

[0090] The control unit (200) may request face image data from the recognition unit (100). After receiving the request for face image data from the control unit (200), the recognition unit (100) may acquire the face image data of the measurement target. The face image data acquired from the recognition unit (100) may be transmitted to the control unit (200). The control unit (200) may determine whether the measurement target is wearing a mask through the face image data received from the recognition unit (100). Although not illustrated, the control unit (200) may determine whether the measurement target is wearing a means of covering the mouth, such as a niqab, burka, or chador, in addition to a mask. The control unit (200) may transmit information on the mask wearing of the measurement target to the output unit (300). If the measurement target is wearing a mask, the output unit (300) may output a notification such as "Please remove your mask." Alternatively, if the subject being measured is not wearing a mask, the process may proceed to the next step.

[0091] The output unit (300) can transmit a signal to the control unit (200) indicating completion of mask wearing information output when the measurement target is not wearing a mask. The control unit (200) can determine whether the mouth of the measurement target is open or closed after confirming whether the face is recognized through the face image data received from the recognition unit (100). Additionally, the control unit (200) can measure an opening time defined as the time during which the mouth of the measurement target remains open. The control unit (200) can transmit whether the mouth of the measurement target is open or closed to the output unit (300). If the mouth of the measurement target is closed, the output unit (300) can output a notification such as "Please open your mouth wide." Alternatively, if the measurement subject has its mouth open but the opening time exceeds the reference time, the control unit (200) may request face image data again from the recognition unit (100) and transmit information regarding the mouth opening time exceeding to the output unit (300). The output unit (300) may output information regarding the mouth opening time exceeding when the measurement subject has its mouth open but the opening time exceeds the reference time. For example, the output unit (300) may output a notification such as "Please close your mouth and wait." If the measurement subject has its mouth open and the opening time is within the reference time, the process may proceed to the next step.

[0092] For example, the above reference time can be set to 3 seconds. When the mouth of the subject being measured is open and 3 seconds have passed, the temperature inside the mouth may decrease rapidly as saliva (moisture) inside the mouth evaporates. In other words, the control unit (200) may determine that the reliability of the measured temperature is low when the mouth of the subject being measured is open and 3 seconds have passed. Consequently, the control unit (200) may request face image data from the recognition unit (100) again after the subject being measured closes and a certain amount of time has elapsed. Accordingly, the accuracy of the body temperature measurement can be improved.

[0093] The output unit (300) can transmit a signal indicating completion of the output of mouth opening / closing information to the control unit (200) when the measurement target has its mouth open and the opening time is within a reference time. The output unit (300) can acquire sublingual image processing data (30) and apply it to the deep learning model (20) to extract the first feature points (31, 32, 33). The control unit (200) can apply the sublingual temperature data (40) to the deep learning model (20) to extract the second feature points (41, 42, 43). The control unit (200) can determine the body temperature of the measurement target by comparing the first feature points (31, 32, 33) of the sublingual image processing data (30) with the second feature points (41, 42, 43) of the sublingual temperature data (40).

[0094] FIG. 13 is a signal flow diagram for explaining a non-contact sublingual body temperature measurement method according to a second variation of the present application.

[0095] Referring to FIG. 13, a non-contact sublingual body temperature measurement method according to a second variant of the present application can be measured through a non-contact sublingual body temperature measurement device comprising the recognition unit (100), the control unit (200), and the output unit (300).

[0096] The control unit (200) may request face image data from the recognition unit (100). After receiving the request for face image data from the control unit (200), the recognition unit (100) may acquire the face image data of the measurement target. The face image data acquired from the recognition unit (100) may be transmitted to the control unit (200). The control unit (200) may check whether a face is recognized through the face image data received from the recognition unit (100). The control unit (200) may determine whether the measurement target is wearing a mask. Although not illustrated, the control unit (200) may determine whether the target is wearing a means of covering the mouth, such as a niqab, burka, or chador, in addition to a mask. The control unit (200) may transmit the mask wearing information of the measurement target to the output unit (300). If the measurement subject is wearing a mask, the output unit (300) may output a notification such as "Please remove the mask." Alternatively, if the measurement subject is not wearing a mask, the process may proceed to the next step.

[0097] The output unit (300) can transmit a signal to the control unit (200) indicating completion of mask wearing information output when the measurement target is not wearing a mask. The control unit (200) can determine whether the mouth of the measurement target is open or closed after confirming whether the face is recognized through the face image data received from the recognition unit (100). The control unit (200) can transmit the status of whether the mouth of the measurement target is open or closed to the output unit (300). If the mouth of the measurement target is closed, the output unit (300) can output a notification such as "Please open your mouth wide." Alternatively, if the mouth of the measurement target is open, the process can proceed to the next step.

[0098] The output unit (300) can transmit a signal to the control unit (200) that the output of the mouth opening / closing information is complete when the measurement target has its mouth open. The control unit (200) can determine whether the tongue of the measurement target is open or closed through the face image data received from the recognition unit (100).

[0099] The output unit (300) can transmit a signal indicating the completion of outputting information on whether the tongue is open or closed to the control unit (200) when the tongue of the measurement target is open. The output unit (300) can determine whether the tongue of the measurement target is open or closed. Additionally, the control unit (200) can measure the tongue openness maintenance time, which is defined as the time during which the tongue remains open after the tongue of the measurement target is lifted upward. The control unit (200) can transmit whether the tongue of the measurement target is open or closed to the output unit (300). When the tongue of the measurement target is closed, the output unit (300) can output a notification such as "Please lift your tongue upward." Alternatively, if the tongue of the measurement target is open but the time the tongue is kept open exceeds a reference time, the control unit (200) may request face image data again from the recognition unit (100) and transmit information regarding the time the tongue is kept open / closed exceeds to the output unit (300). The output unit (300) may output information regarding the time the tongue is kept open exceeds a reference time if the tongue of the measurement target is open but the time the tongue is kept open exceeds a reference time. For example, the output unit (300) may output a notification such as "Please close your mouth and wait." If the tongue of the measurement target is open and the time the tongue is kept open is within the reference time, the process may proceed to the next step.

[0100] For example, the above reference time can be set to within 3 seconds. If 3 seconds pass after the tongue of the measurement subject is opened, the temperature of the sublingual area (1) can decrease rapidly as the saliva (moisture) under the tongue evaporates. In other words, the control unit (200) can determine that the reliability of the measured temperature is low if 3 seconds pass after the tongue of the measurement subject is opened. Consequently, the control unit (200) can request face image data from the recognition unit (100) again after the mouth of the measurement subject is closed and a certain amount of time has elapsed. Accordingly, the accuracy of the body temperature measurement can be improved.

[0101] The output unit (300) can transmit a signal indicating completion of output of information on whether the tongue is open or closed to the control unit (200) when the tongue of the measurement target is open and the time of maintaining the open state of the tongue is within a reference time. The output unit (300) can acquire sublingual image processing data (30) and apply it to the deep learning model (20) to extract the first feature points (31, 32, 33). The control unit (200) can apply the sublingual temperature data (40) to the deep learning model (20) to extract the second feature points (41, 42, 43). The control unit (200) can determine the body temperature of the measurement target by comparing the first feature points (31, 32, 33) of the sublingual image processing data (30) with the second feature points (41, 42, 43) of the sublingual temperature data (40).

[0103] Although the present invention has been described in detail using preferred embodiments, the scope of the invention is not limited to specific embodiments and should be interpreted by the appended claims. Furthermore, those skilled in the art will understand that many modifications and variations are possible without departing from the scope of the invention. Explanation of the symbols

[0105] 1: Seolha 2: Tongue frenulum 3: Sublingual vein 10: Sublingual image data 20: Deep learning models 21: Sublingual image dataset 23: U-Net Model 30: Sublingual image processing data 30a: Sublingual region 40: Sublingual temperature data 100: Recognition unit 110: Optical camera 120: Infrared sensor 200: Control unit 300: Output section 310: Display 320: Audio output section

Claims

Claim 1 A non-contact sublingual body temperature measuring device for detecting whether the tongue of a subject is open or closed and for measuring the temperature under the tongue of the subject non-contactually, comprising: a recognition unit for recognizing the state of the subject; a control unit for determining the state of the subject received through the recognition unit and determining the body temperature of the subject; and an output unit for outputting the body temperature of the subject determined through the control unit, wherein the recognition unit comprises an optical camera for analyzing face image data of the subject; A non-contact sublingual body temperature measuring device comprising an infrared sensor for measuring the surface temperature of the measurement target, wherein the control unit acquires the face image data through the optical camera, applies the acquired face image data to a U-Net model to specify a sublingual region including a lingual frenulum and a pair of sublingual veins, applies the sublingual image data including the specified sublingual region to a deep learning model to extract a first feature point within the sublingual region, acquires the sublingual temperature data of the measurement target through the infrared sensor, applies the acquired sublingual temperature data to a deep learning model to extract a second feature point, and matches the first feature point and the second feature point to determine the body temperature of the measurement target. Claim 2 A non-contact sublingual body temperature measuring device according to claim 1, wherein the output unit outputs information on the mouth opening time exceeding when the measurement subject opens its mouth and the opening time exceeds a reference time, and outputs information on the tongue opening time exceeding when the opening time of the tongue maintained after the tongue of the measurement subject is lifted upward exceeds the reference time, and the control unit requests the face image data from the recognition unit again after the mouth of the measurement subject is closed and a certain amount of time has elapsed when the opening time or the opening time exceeds the reference time. Claim 3 A method for measuring sublingual body temperature by non-contact measuring the temperature under the tongue of a subject, comprising: a step of analyzing facial image data of a subject; a step of measuring the temperature under the tongue of a subject when it is determined that the tongue of the subject is open; a step of determining the body temperature of the subject; and a step of outputting the body temperature of the subject, wherein the step of analyzing facial image data of the subject includes: a step of determining whether the subject is wearing a mask through an optical camera; a step of determining whether the mouth of the subject is open or closed through an optical camera; a step of determining whether the tongue of the subject is open or closed through an optical camera; a step of specifying a sublingual region including a lingual frenulum and a pair of sublingual veins within the facial image data; a step of acquiring sublingual image data of the subject; and a step of extracting a first feature point within the sublingual image data, and the step of measuring the temperature under the tongue of the subject includes: a step of acquiring sublingual temperature data of the subject through an infrared sensor. A non-contact method for measuring sublingual body temperature, comprising the step of extracting a second feature point within the sublingual temperature data, and determining the body temperature of the measurement target by matching the first feature point and the second feature point. Claim 4 delete Claim 5 delete

Images