Traditional Chinese medicine constitution recognition method and device, electronic equipment, storage medium and program

Abstract

The Chinese medicine constitution recognition method, device, electronic equipment, storage medium and program provided by the present disclosure belong to the technical field of computers. The method comprises the following steps: acquiring an infrared human body image of a target user; inputting the infrared human body image into an infrared key point recognition model to obtain Chinese medicine human body key points in the infrared human body image; determining a temperature type of the Chinese medicine human body key points in the infrared human body image according to a temperature distribution in the infrared human body image; and recognizing a Chinese medicine constitution of the target user according to the temperature type.

Description

Technical Field

[0001] This disclosure belongs to the field of computer technology, and specifically relates to a method, device, electronic device, storage medium and program for identifying traditional Chinese medicine constitution. Background Technology

[0002] Infrared imaging, as an extension of the "observation" aspect of the four diagnostic methods in Traditional Chinese Medicine (TCM), involves TCM doctors manually processing infrared human images using color tomography after training. They then analyze the cold and heat signs of different body parts by observing the processed images. Based on a comprehensive analysis of the cold and heat signs in multiple key areas such as the hands, feet, head, neck, abdomen, and stomach, they derive TCM constitution classifications such as cold stagnation and heat stagnation. However, TCM doctors who are not trained or have limited experience cannot accurately determine the patient's physical signs. Summary of the Invention

[0003] This disclosure provides a method, device, electronic device, storage medium, and program for identifying traditional Chinese medicine constitution.

[0004] This disclosure provides a method for identifying traditional Chinese medicine constitution through various embodiments, the method comprising:

[0005] Acquire infrared human images of the target user;

[0006] The infrared human body image is input into the infrared key point recognition model to obtain the TCM human body key points in the infrared human body image.

[0007] Based on the temperature distribution in the infrared human body image, determine the temperature type of the key points of the TCM human body in the infrared human body image;

[0008] The target user's TCM constitution is identified based on the temperature type.

[0009] Optionally, the infrared key point recognition model is obtained through the following steps:

[0010] Acquire natural light key point recognition model and sample infrared human body images;

[0011] Mark key points of the human body in traditional Chinese medicine in the infrared human body images of the samples;

[0012] The natural light key point recognition model is transferred to the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0013] Optionally, the step of using labeled sample infrared human images to perform transfer learning on the natural light keypoint recognition model to obtain an infrared keypoint recognition model includes:

[0014] The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine.

[0015] While keeping the model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0016] Optionally, the original human body key points of the natural light key point recognition model include the traditional Chinese medicine human body key points, and the number of the original human body key points is greater than the number of traditional Chinese medicine human body key points.

[0017] The adjustment of the fully connected layer nodes of the natural light key point recognition model based on the key points of the human body in traditional Chinese medicine includes:

[0018] In the fully connected layer nodes of the natural light key point recognition model, the human key point corresponding to the deleted node label is not a fully connected layer node of the TCM human key point.

[0019] Optionally, the step of using labeled sample infrared human images to perform transfer learning on the natural light keypoint recognition model to obtain an infrared keypoint recognition model includes:

[0020] The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine.

[0021] The natural light keypoint recognition model with adjusted nodes is trained using the labeled sample infrared human body images to obtain the infrared keypoint recognition model.

[0022] Optionally, the feature extraction network in the natural light keypoint recognition model is a lightweight feature extraction network, and the amount of parameter data of the lightweight feature extraction network is less than the amount of parameter data of the original feature extraction network of the natural light keypoint recognition model.

[0023] Optionally, the key points of the human body in traditional Chinese medicine include at least one of the following: key points of the head, key points of the neck, key points of the shoulder, key points of the elbow, key points of the hand, key points of the abdomen, key points of the hip, key points of the leg and elbow, and key points of the foot.

[0024] Optionally, the temperature type includes at least one of a high-temperature type and a low-temperature type;

[0025] The step of determining the temperature type of the key points of the TCM human body in the infrared human body image based on the temperature distribution in the infrared human body image includes:

[0026] Determine a first number of temperature maxima locations and / or a second number of temperature minima locations in the infrared human body image;

[0027] The temperature type of the key point of the human body in traditional Chinese medicine corresponding to the image region where the temperature maximum value is located is determined to be high temperature type, and / or the temperature type of the key point of the human body in traditional Chinese medicine corresponding to the image region where the temperature minimum value is located is determined to be low temperature type.

[0028] Optionally, determining the first number of temperature maxima locations and / or the second number of temperature minima locations in the infrared human body image includes:

[0029] The image location of the first number of temperature maxima in the infrared human body image is obtained by using a maximum value filter and is taken as the temperature maxima location.

[0030] And / or use the average of the maximum and minimum temperature values ​​in the infrared human body image as the flip plane to invert the temperature values ​​in the infrared human body image;

[0031] The location of the second maximum temperature value in the inverted infrared human body image obtained by the maximum value filter is used as the location of the minimum temperature value.

[0032] Optionally, identifying the target user's TCM constitution based on the temperature type includes:

[0033] In the TCM constitution mapping relationship, search for TCM constitutions that match the temperature type of each of the aforementioned TCM human body key points.

[0034] Optionally, acquiring the infrared human image of the target user includes:

[0035] Acquire the initial infrared image of the user captured by an infrared camera;

[0036] A global threshold search is performed on the initial infrared image using a human body temperature range threshold. Pixels in the initial infrared image that are within the human body temperature range threshold are set to 1, and pixels that are outside the human body temperature range threshold are set to 0, thus obtaining a binarized initial infrared image.

[0037] Multiply the initial infrared image and the binarized initial infrared image to obtain an infrared human body image.

[0038] Optionally, the step of using labeled sample infrared human images to perform transfer learning on the natural light keypoint recognition model to obtain an infrared keypoint recognition model includes:

[0039] The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine.

[0040] While keeping some preset model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0041] Optionally, the key points of the human body in traditional Chinese medicine include at least one of the following: key points of the head, key points of the neck, key points of the shoulder, key points of the elbow, key points of the hand, key points of the abdomen, key points of the hip, key points of the leg and elbow, and key points of the foot.

[0042] This disclosure provides a traditional Chinese medicine constitution identification device according to some embodiments, the device comprising:

[0043] The receiving module is configured to acquire infrared human images of the target user.

[0044] The model prediction module is configured to input the infrared human body image into the infrared key point recognition model to obtain the TCM human body key points in the infrared human body image.

[0045] The recognition module is configured to determine the temperature type of the key points of the TCM human body in the infrared human body image based on the temperature distribution in the infrared human body image;

[0046] The target user's TCM constitution is identified based on the temperature type.

[0047] Optionally, the device further includes: a training module configured to:

[0048] Acquire natural light key point recognition model and sample infrared human body images;

[0049] Mark key points of the human body in traditional Chinese medicine in the infrared human body images of the samples;

[0050] The natural light key point recognition model is transferred to the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0051] Optionally, the training module is further configured to:

[0052] The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine.

[0053] While keeping the model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0054] Optionally, the original human body key points of the natural light key point recognition model include the traditional Chinese medicine human body key points, and the number of the original human body key points is greater than the number of traditional Chinese medicine human body key points.

[0055] Optionally, the training module is further configured to:

[0056] In the fully connected layer nodes of the natural light key point recognition model, the human key point corresponding to the deleted node label is not a fully connected layer node of the TCM human key point.

[0057] Optionally, the feature extraction network in the natural light keypoint recognition model is a lightweight feature extraction network, and the amount of parameter data of the lightweight feature extraction network is less than the amount of parameter data of the original feature extraction network of the natural light keypoint recognition model.

[0058] Optionally, the temperature type includes at least one of a high-temperature type and a low-temperature type;

[0059] The identification module is further configured to:

[0060] Determine a first number of temperature maxima locations and / or a second number of temperature minima locations in the infrared human body image;

[0061] The temperature type of the key points in the human body corresponding to the image region where the temperature maximum value is located is determined to be high temperature type, and / or the temperature type of the key points in the human body corresponding to the image region where the temperature minimum value is located is determined to be low temperature type.

[0062] Optionally, the identification module is further configured to:

[0063] The image location of the first number of temperature maxima in the infrared human body image is obtained by using a maximum value filter and is taken as the temperature maxima location.

[0064] And / or use the average of the maximum and minimum temperature values ​​in the infrared human body image as the flip plane to invert the temperature values ​​in the infrared human body image;

[0065] The location of the second maximum temperature value in the inverted infrared human body image obtained by the maximum value filter is used as the location of the minimum temperature value.

[0066] Optionally, the identification module is further configured to:

[0067] In the TCM constitution mapping relationship, search for TCM constitutions that match the temperature type of each of the aforementioned TCM human body key points.

[0068] Optionally, the receiving module is further configured to:

[0069] Acquire the initial infrared image of the user captured by an infrared camera;

[0070] A global threshold search is performed on the initial infrared image using a human body temperature range threshold. Pixels in the initial infrared image that are within the human body temperature range threshold are set to 1, and pixels that are outside the human body temperature range threshold are set to 0, thus obtaining a binarized initial infrared image.

[0071] Multiply the initial infrared image and the binarized initial infrared image to obtain an infrared human body image.

[0072] Optionally, the training module is also configured as follows:

[0073] The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine.

[0074] While keeping some preset model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0075] Optionally, the key points of the human body in traditional Chinese medicine include at least one of the following: key points of the head, key points of the neck, key points of the shoulder, key points of the elbow, key points of the hand, key points of the abdomen, key points of the hip, key points of the leg and elbow, and key points of the foot.

[0076] Some embodiments of this disclosure provide a computing processing device, including:

[0077] Memory containing computer-readable code;

[0078] One or more processors, when the computer-readable code is executed by the one or more processors, the computing processing device performs the TCM constitution identification method as described above.

[0079] This disclosure provides a computer program, including computer-readable code, which, when run on a computing processing device, causes the computing processing device to perform the TCM constitution identification method described above.

[0080] This disclosure provides a computer-readable medium storing, in some embodiments, the traditional Chinese medicine constitution identification method described above.

[0081] This disclosure provides a method, device, electronic device, storage medium, and program for identifying traditional Chinese medicine constitution. It uses an infrared key point recognition model to identify the temperature type of various parts of a user's body to determine the user's traditional Chinese medicine constitution, avoiding errors caused by human negligence in identifying temperature types and improving the accuracy of traditional Chinese medicine constitution identification.

[0082] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0084] Figure 1 The illustration schematically shows one of the flowcharts of a traditional Chinese medicine constitution identification method provided in some embodiments of this disclosure;

[0085] Figure 2 The schematic diagram illustrates a flowchart of a model training method provided in some embodiments of this disclosure;

[0086] Figure 3 This illustration shows one of the effect diagrams of a sample annotation method provided by some embodiments of the present disclosure;

[0087] Figure 4 The second schematic diagram illustrates a flowchart of a model training method provided in some embodiments of this disclosure;

[0088] Figure 5 The second schematic diagram illustrates a flowchart of a method for identifying traditional Chinese medicine constitutions provided in some embodiments of this disclosure.

[0089] Figure 6 The third schematic diagram illustrates a flowchart of a method for identifying traditional Chinese medicine constitutions provided in some embodiments of this disclosure;

[0090] Figure 7 The illustration schematically shows a schematic diagram of the effect of a human infrared image provided by some embodiments of the present disclosure;

[0091] Figure 8 This illustration schematically shows another effect diagram of human infrared image provided by some embodiments of the present disclosure;

[0092] Figure 9 The fourth schematic diagram illustrates a flowchart of a method for identifying traditional Chinese medicine constitutions provided in some embodiments of this disclosure;

[0093] Figure 10The schematic diagram illustrates the principle of a traditional Chinese medicine constitution identification method provided in some embodiments of this disclosure;

[0094] Figure 11 The schematic diagram illustrates the structure of a traditional Chinese medicine constitution identification device provided in some embodiments of this disclosure;

[0095] Figure 12 A block diagram schematically illustrates a computing processing apparatus for performing methods according to some embodiments of the present disclosure;

[0096] Figure 13 A storage unit for holding or carrying program code implementing methods according to some embodiments of the present disclosure is illustrated schematically. Detailed Implementation

[0097] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0098] Figure 1 The schematic diagram illustrates a flowchart of a traditional Chinese medicine constitution identification method provided in this disclosure. The method can be executed by any electronic device, such as an application with functions like information display, information transmission, and data processing. The method can be executed by the application's server or terminal device; optionally, it can be executed by a terminal device. The method includes:

[0099] Step 101: Obtain the infrared human body image of the target user.

[0100] In this embodiment, the target user is the user who needs to undergo TCM constitution identification. The infrared human body image is an infrared image obtained by capturing the target user using an infrared camera or an image acquisition device equipped with an infrared camera. Therefore, the value of each pixel in the infrared human body image can reflect the body temperature of different parts of the user's body. The image can be captured each time the user needs to undergo TCM constitution identification. In some embodiments, the infrared human body image of the target user can also be obtained by acquiring an infrared human body image already stored in memory or by receiving an infrared human body image from another device; this is not limited to this method. For example, the infrared image captured by the infrared camera or an image acquisition device equipped with an infrared camera can be stored for later retrieval.

[0101] In some embodiments of this disclosure, a terminal device equipped with an infrared camera, such as a computer connected to an infrared camera, can be used to take pictures of the target user to obtain an infrared human body image. Alternatively, an infrared camera can be used to take pictures of the target user to obtain an infrared human body image, which can then be sent to the user's mobile phone, personal computer, or tablet computer, etc., so that the terminal device can perform subsequent steps to identify the TCM constitution.

[0102] In some embodiments of this disclosure, the image can be taken from the front of the human body or from other angles; no limitation is made here.

[0103] Step 102: Input the infrared human body image into the infrared key point recognition model to obtain the TCM human body key points in the infrared human body image.

[0104] In this embodiment, TCM human body key points refer to the human body locations required for TCM constitution identification. These may include at least one of the following: head key points, neck key points, shoulder key points, elbow key points, hand key points, abdominal key points, hip key points, leg-elbow key points, and foot key points. Some key points, such as eye key points and nose key points, may be discarded. Key points with little effect on TCM constitution identification can be discarded to reduce computational load, while key points with significant effect, such as head key points, can be retained to ensure the accuracy of the identification results. It is understood that other human body key points can also be used as TCM human body key points, and the specific criteria can be determined according to actual needs; no limitation is made here. The infrared key point recognition model is a machine model used to identify TCM human body key points on the user's body in infrared human body images.

[0105] In some embodiments of this disclosure, a client application with an infrared key point recognition model is installed on the terminal device. Upon receiving an infrared image of a human body for which TCM constitution identification is required, the client invokes the infrared key point recognition model to identify key TCM points in the infrared human body image. Alternatively, the infrared key point recognition model can also be deployed on a server, allowing the client to submit the infrared human body image to the server for TCM constitution identification before returning it to the client for display by the user.

[0106] Step 103: Determine the temperature type of the key points of the TCM human body in the infrared human body image based on the temperature distribution in the infrared human body image.

[0107] In this embodiment of the disclosure, the temperature type can be a type used to describe the cold and hot conditions of the human body in traditional Chinese medicine theory. Since the cold or hot condition of a part of the human body in traditional Chinese medicine theory is based on the relative temperature of different parts of the body, it cannot be measured based on a certain standard temperature threshold. For example, if the temperature of most parts of a target user's body is 36.9°C, and the temperature of the feet is 36.5°C, then the user's feet belong to the cold condition with a lower temperature. If the temperature of the head is 37.1°C, then the user's head belongs to the hot condition with a higher temperature. Therefore, when identifying a user's TCM constitution, it is necessary to measure the temperature type of each key point of the human body in TCM based on the temperature distribution in the infrared human body image.

[0108] In some embodiments of this disclosure, the temperature type of the image region where each key point of the human body in traditional Chinese medicine is located can be measured based on the extreme temperature values ​​of each pixel in the infrared human body image. For example, the key point of the human body in traditional Chinese medicine located in the image region with the maximum value is hot, and the key point of the human body in traditional Chinese medicine located in the image region with the minimum value is cold. Of course, the temperature type can also be measured based on the overall average temperature of each pixel. For example, the key point of the human body in traditional Chinese medicine located in the image region with the average temperature greater than the overall average temperature is hot, and the key point of the human body in traditional Chinese medicine located in the image region with the average temperature less than the overall average temperature is cold. The overall average temperature can be a single value or a range of values, which can be set according to actual needs and is not limited here.

[0109] Step 104: Identify the target user's TCM constitution based on the temperature type.

[0110] In some implementations, the user's TCM constitution is determined based on the temperature type of the body part where the key point is located. For example, if the temperature type of the key point in the stomach is cold, then the user's TCM constitution is determined to be stomach cold.

[0111] In this embodiment, a knowledge base can be set up using the TCM constitution identification method from TCM academic theory. The knowledge base can then be queried based on the temperature type of key points on the human body in TCM to obtain the TCM constitution of the target user. For example, if the key point on the head is hot and the key point on the limbs is cold, the target user's TCM constitution can be determined to be Yang deficiency. Or, if the key points are neither hot nor cold, the target user's TCM constitution can be determined to be balanced, and so on. Specific settings can be configured according to actual needs and are not limited here.

[0112] The terminal device can display the identified TCM manifestations on a screen for users to view. Alternatively, it can provide information to users through voice broadcasts, but the specific format is not limited here.

[0113] This embodiment of the disclosure uses an infrared key point recognition model to identify the temperature type of various parts of the user's body in order to determine the user's TCM constitution. This avoids errors caused by human negligence in identifying temperature types and improves the accuracy of TCM constitution identification.

[0114] Optionally, refer to Figure 2 The infrared key point recognition model is obtained through the following steps:

[0115] Step 201: Obtain the natural light key point recognition model and sample infrared human body images.

[0116] In this embodiment of the disclosure, the sample infrared human body image can be an infrared human body image obtained by taking pictures of the sample user using an infrared camera or the like. The natural light key point recognition model is a pre-trained image recognition model that can recognize human key points in natural light images. For details, please refer to the human key point recognition model in related technologies, which will not be elaborated here.

[0117] Step 202: Mark key points of the human body in traditional Chinese medicine in the infrared human body image of the sample.

[0118] In this embodiment of the disclosure, reference is made to Figure 3 For TCM (Traditional Chinese Medicine) human body key points, at least one of the following can be selected: head key points, neck key points, shoulder key points, elbow key points, hand key points, abdominal key points, hip key points, leg-elbow key points, and foot key points. Among these, the shoulder, elbow, hand, hip, leg-elbow, and foot key points are arranged according to the left-right symmetry of the human body, including key points on both sides. Therefore, a total of 15 TCM human body key points can be set. Furthermore, key points such as eye key points and nose key points, which are not considered in TCM academic theory, are discarded. Since there is a large-scale labeled dataset of natural light human body key points in related technologies, using a portion of the natural light human body key points can overcome the drawback of the extremely small number of key point labeled datasets in infrared human body images in related technologies. Only key point type needs to be selected from the human body key points in related technologies, reducing the resources required for sample labeling.

[0119] Optionally, the feature extraction network in the natural light keypoint recognition model is a lightweight feature extraction network, and the amount of parameter data of the lightweight feature extraction network is less than the amount of parameter data of the original feature extraction network of the natural light keypoint recognition model.

[0120] In some implementations, the original feature extraction network of the natural light keypoint recognition model can be replaced with a lightweight feature extraction network, wherein the amount of parameter data of the lightweight feature extraction network is less than that of the original feature extraction network.

[0121] In some embodiments of this disclosure, the natural light keypoint recognition model can be the OpenPose human keypoint detection model. The original model's feature extraction network has approximately 200M model parameters, resulting in a large amount of data and computation. Therefore, to reduce the number of parameters and computation, Mobilennet (a lightweight feature extraction network) can be used to replace the original feature extraction network of the original model, thereby reducing the model parameters to 7M and significantly reducing the computation required for model training. The natural light keypoint recognition model with the lightweight feature extraction network is used as a pre-training module, and the model parameters of the pre-trained model are used as parameters for initialization. The initial learning rate is 3e-3, and the training is performed for 300 epochs (one epoch is the completion of one round of training dataset training). The learning rate is reduced by 1 / 10 every 100 epochs.

[0122] Step 203: Use the labeled sample infrared human body images to perform transfer learning on the natural light key point recognition model to obtain the infrared key point recognition model.

[0123] In this embodiment, the natural light keypoint recognition model is a machine model used to identify human keypoints in natural light human images. Its difference from the infrared keypoint recognition model lies not only in the different model inputs but also in the fact that the natural light keypoint recognition model is trained based on natural light human images. Therefore, the classifier in the natural light keypoint recognition model can only identify human keypoints in natural light human images and cannot identify key points related to traditional Chinese medicine in infrared human images. However, thanks to the clear boundaries in natural light human images, the feature extraction layer in the natural light keypoint recognition model can accurately extract image features from human images. Therefore, this embodiment uses the model parameters of the feature extraction layer in the pre-trained natural light keypoint recognition model and retrains it using sample infrared human images labeled with key points related to traditional Chinese medicine. By replacing the input of the natural light keypoint recognition model with labeled sample infrared human images and changing the training target from the original keypoints to key points related to traditional Chinese medicine, transfer learning is performed on the classifier in the natural light keypoint recognition model to obtain the infrared keypoint recognition model. Infrared key point recognition models can be obtained through transfer learning, which have good recognition capabilities for image textures and improve the accuracy of key point recognition in traditional Chinese medicine.

[0124] Optionally, refer to Figure 4 Step 203 may include:

[0125] Step 2031: Adjust the fully connected layer nodes of the natural light key point recognition model based on the TCM human body key points.

[0126] Step 2032: While keeping the model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0127] In some embodiments of this disclosure, the transfer learning specifically uses the model parameters of the feature extraction layer in the natural light keypoint recognition model as the initial training parameters. In some implementations, the parameters of the feature extraction layer can be changed or kept constant, depending on the training effect, while the parameters of other layers, such as fully connected layers, are continuously adjusted during training until training is complete. In some implementations, not changing the parameters of the feature extraction layer yields better results.

[0128] Optionally, step 2032 can be replaced by: while keeping some preset model parameters of the feature extraction layer unchanged, using the labeled sample infrared human body image to train the natural light key point recognition model after adjusting the nodes, so as to obtain the infrared key point recognition model.

[0129] In some embodiments of this disclosure, during the transfer learning process, the parameters of the entire network or a portion of the feature extraction network can be adjusted. The parameters of the feature extraction network can be changed as a whole or partially, while the parameters of other layers will be continuously adjusted during training. The specific settings can be made according to actual needs and requirements, and are not limited here.

[0130] Optionally, the original human body key points of the natural light key point recognition model include the traditional Chinese medicine human body key points, and the number of the original human body key points is greater than the number of traditional Chinese medicine human body key points. Step 2031 may include: in the fully connected layer nodes of the natural light key point recognition model, deleting the fully connected layer nodes whose corresponding human body key points are not traditional Chinese medicine human body key points.

[0131] In some implementations, the number of key points required for TCM constitution recognition based on natural light images differs from the number required based on infrared images. Therefore, retraining necessitates adjusting the number of key points, which in turn requires adjusting the number of nodes in the fully connected layers of the natural light keypoint recognition model. Generally, the number of nodes in the fully connected layers is adjusted and added or removed according to actual needs. For example, eye key points are more important in natural light images but less important in infrared images, so the label of the node is removed from the fully connected layer. The label includes the coordinates of each key node and the name of the body part it belongs to. An infrared keypoint recognition model capable of recognizing TCM human body key points in infrared human images is obtained by using transfer learning that retains at least some model parameters. This allows the model to maintain the accurate feature extraction capability of the natural light keypoint recognition model while also recognizing TCM human body key points in infrared human images.

[0132] Optionally, the temperature type includes at least one of a high-temperature type and a low-temperature type, as shown in the reference. Figure 5 Step 103 may include:

[0133] Step 1031: Determine the first number of temperature maxima locations and / or the second number of temperature minima locations in the infrared human body image.

[0134] In this embodiment, the local extrema in the infrared human body image can be sorted. For example, the image location of the first number of local extrema in the descending order is taken as the temperature maximum location, and the image location of the second number of local extrema in the ascending order is taken as the temperature minimum location. Whether the order is ascending or descending can be adjusted according to the actual situation; this is only an example and not a limitation. The first number and the second number are the number of temperature minimum and temperature maximum locations to be selected, respectively. These first and second numbers are less than the total number of key points in the infrared human body image (TCM). The first and second numbers can be the same or different; for example, both the first and second numbers can be 3, or the first number can be 3 and the second number can be 2. The specific settings can be made according to actual needs and are not limited here.

[0135] Step 1032: Determine the temperature type of the key point of the human body in traditional Chinese medicine corresponding to the image region where the temperature maximum value is located as high temperature type, and / or determine the temperature type of the key point of the human body in traditional Chinese medicine corresponding to the image region where the temperature minimum value is located as low temperature type.

[0136] In this embodiment of the disclosure, since traditional Chinese medicine theory usually determines the constitution of a person with a medium body type based on the highest and lowest temperatures of several body parts, the location of the body with a predominance of cold or heat can be identified by summarizing the minimum and maximum temperatures in the infrared human body image and sorting them by size.

[0137] Optionally, refer to Figure 6 Step 1031 may include:

[0138] Step 10311: Obtain the image location of the first number of temperature maxima in the infrared human body image through the maximum value filter as the temperature maxima location.

[0139] In some embodiments of this disclosure, the maximum value filter can be the Peak_local_max local maximum algorithm from the Skimage library in OpenCV, or other filters with a maximum value filtering process to extract the temperature maxima of the local image region where each key point of the human body in traditional Chinese medicine is located. For example, the step size of the algorithm is set to 10.

[0140] Figure 7 A frontal image obtained by color transillumination of a human infrared image, where different colors represent temperature, while a reference... Figure 8 To use the Peak_local_max local maximum algorithm to... Figure 7 The processed human infrared image's side temperature map, with the vertical axis representing the infrared image pixel values, i.e., temperature; the higher the value, the higher the temperature. However, due to the unclear boundary texture of infrared images, the resulting frontal image cannot accurately represent the human body's temperature. The side image obtained through the Peak_local_max local maxima algorithm only needs to consider the pixel values ​​in the human infrared image, avoiding the impact of unclear infrared image boundary texture on the accuracy of identifying temperature maxima and minima.

[0141] Step 10312: Using the average of the maximum and minimum temperature values ​​in the infrared human body image as the flip plane, the temperature values ​​in the infrared human body image are reversed.

[0142] In this embodiment, since the method of judging the cold and heat of human body parts in traditional Chinese medicine theory is based on the overall temperature distribution of the human body, and in order to ensure that the data space does not shift, the average of the maximum and minimum temperature values ​​of the pixel values ​​of the infrared human body image can be used as a reversal plane to reverse the temperature values ​​of the infrared human body image. This avoids the situation where the temperature value becomes negative after reversal due to the shift in the data space. For example, if the maximum temperature is x and the minimum temperature is y, then the reversal plane is (x+y) / 2. At this time, for any temperature n in the infrared human body image, the reversed temperature is (x+y) / 2*2-n.

[0143] Step 10313: The location of the second number of temperature maxima in the inverted infrared human body image is obtained through the maximum value filter and used as the location of the temperature minimum.

[0144] In this embodiment of the disclosure, since the temperature values ​​in the inverted infrared human body image are reversed, the maximum value can be re-extracted through the maximum value filter, and the maximum value can be used to characterize the minimum temperature value in the infrared human body image.

[0145] Optionally, step 104 may include: querying the TCM constitution mapping relationship to find the TCM constitution that matches the temperature type of each of the TCM key points of the human body.

[0146] In this embodiment of the disclosure, a TCM constitution mapping relationship can be set with reference to TCM academic theories. This TCM constitution mapping relationship includes TCM constitutions corresponding to various temperature types for each key point of the human body in TCM, thereby facilitating the identification of the TCM constitution of the target user.

[0147] Optionally, step 101 refers to Figure 9 It can include:

[0148] Step 1011: Obtain the initial infrared image of the user captured by an infrared camera.

[0149] In this embodiment of the disclosure, since the medical infrared image acquisition scenario is relatively simple, users usually take infrared photos in a dedicated shooting space in a hospital. Therefore, there is less interference from irrelevant personnel. However, there is still some irrelevant background interference content, which needs to be removed from the initial infrared image.

[0150] Step 1012: Perform a global threshold search on the initial infrared image using the human body temperature range threshold, so that pixels in the initial infrared image within the human body temperature range threshold are set to 1, and pixels outside the human body temperature range threshold are set to 0, to obtain a binarized initial infrared image.

[0151] In this embodiment of the disclosure, the initial infrared image is binarized using a human body temperature range threshold. For example, refer to... Figure 10 If the initial infrared image shows a distinct "bimodal" pattern in its histogram, OTSU (Otsu's algorithm) can be used for global threshold search. Finally, the obtained threshold is used for threshold segmentation to obtain a mask image where pixels in the human image region have a value of 1, and pixels in the background image region have a value of 0. Multiplying the mask image by the initial infrared image yields the infrared human image with the background removed.

[0152] Step 1013: Multiply the initial infrared image and the binarized initial infrared image to obtain an infrared human body image.

[0153] This embodiment of the invention removes irrelevant background image areas from the initial infrared image after binarization, thereby reducing the interference of irrelevant image content on subsequent TCM key point recognition and improving the accuracy of TCM key point recognition.

[0154] Figure 11 A schematic diagram of the structure of a traditional Chinese medicine constitution identification device 30 provided in this disclosure is shown. The device includes:

[0155] The receiving module 301 is configured to acquire infrared human images of the target user;

[0156] The model prediction module 302 is configured to input the infrared human body image into the infrared key point recognition model to obtain the TCM human body key points in the infrared human body image.

[0157] The identification module 303 is configured to determine the temperature type of the key points of the TCM human body in the infrared human body image based on the temperature distribution in the infrared human body image.

[0158] The target user's TCM constitution is identified based on the temperature type.

[0159] Optionally, the device further includes: a training module configured to:

[0160] Acquire natural light key point recognition model and sample infrared human body images;

[0161] Mark key points of the human body in traditional Chinese medicine in the infrared human body images of the samples;

[0162] The natural light key point recognition model is transferred to the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0163] Optionally, the training module is further configured to:

[0164] The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine.

[0165] While keeping the model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0166] Optionally, the original human body key points of the natural light key point recognition model include the traditional Chinese medicine human body key points, and the number of the original human body key points is greater than the number of traditional Chinese medicine human body key points.

[0167] Optionally, the training module is further configured to:

[0168] In the fully connected layer nodes of the natural light key point recognition model, the human key point corresponding to the deleted node label is not a fully connected layer node of the TCM human key point.

[0169] Optionally, the feature extraction network in the natural light keypoint recognition model is a lightweight feature extraction network, and the amount of parameter data of the lightweight feature extraction network is less than the amount of parameter data of the original feature extraction network of the natural light keypoint recognition model.

[0170] Optionally, the temperature type includes at least one of a high-temperature type and a low-temperature type;

[0171] The identification module 303 is further configured to:

[0172] Determine a first number of temperature maxima locations and / or a second number of temperature minima locations in the infrared human body image;

[0173] The temperature type of the key points in the human body corresponding to the image region where the temperature maximum value is located is determined to be high temperature type, and / or the temperature type of the key points in the human body corresponding to the image region where the temperature minimum value is located is determined to be low temperature type.

[0174] Optionally, the identification module 303 is further configured to:

[0175] The image location of the first number of temperature maxima in the infrared human body image is obtained by using a maximum value filter and is taken as the temperature maxima location.

[0176] And / or use the average of the maximum and minimum temperature values ​​in the infrared human body image as the flip plane to invert the temperature values ​​in the infrared human body image;

[0177] The location of the second maximum temperature value in the inverted infrared human body image obtained by the maximum value filter is used as the location of the minimum temperature value.

[0178] Optionally, the identification module 303 is further configured to:

[0179] In the TCM constitution mapping relationship, search for TCM constitutions that match the temperature type of each of the aforementioned TCM human body key points.

[0180] Optionally, the receiving module 301 is further configured to:

[0181] Acquire the initial infrared image of the user captured by an infrared camera;

[0182] A global threshold search is performed on the initial infrared image using a human body temperature range threshold. Pixels in the initial infrared image that are within the human body temperature range threshold are set to 1, and pixels that are outside the human body temperature range threshold are set to 0, thus obtaining a binarized initial infrared image.

[0183] Multiply the initial infrared image and the binarized initial infrared image to obtain an infrared human body image.

[0184] Optionally, the training module is also configured as follows:

[0185] The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine.

[0186] While keeping some preset model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

[0187] Optionally, the key points of the human body in traditional Chinese medicine include at least one of the following: key points of the head, key points of the neck, key points of the shoulder, key points of the elbow, key points of the hand, key points of the abdomen, key points of the hip, key points of the leg and elbow, and key points of the foot.

[0188] This embodiment of the disclosure uses an infrared key point recognition model to identify the temperature type of various parts of the user's body in order to determine the user's TCM constitution. This avoids errors caused by human negligence in identifying temperature types and improves the accuracy of TCM constitution identification.

[0189] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0190] The various component embodiments of this disclosure can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the computing processing device according to embodiments of this disclosure. This disclosure can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such an implementation of this disclosure can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0191] For example, Figure 12 A computing processing apparatus is shown that can implement the methods according to this disclosure. The computing processing apparatus conventionally includes a processor 410 and a computer program product or computer-readable medium in the form of a memory 420. The memory 420 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. The memory 420 has a storage space 430 for program code 431 for performing any of the method steps described above. For example, the storage space 430 for program code may include various program codes 431 respectively for implementing the various steps in the methods described above. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, CDs, memory cards, or floppy disks. Such computer program products are typically as described in the references. Figure 13 The portable or fixed storage unit. This storage unit may have the same characteristics as... Figure 12The memory 420 in the computing processing device is similarly arranged as storage segments, storage spaces, etc. Program code can be compressed, for example, in an appropriate form. Typically, the storage unit includes computer-readable code 431', that is, code that can be read by a processor such as 410, which, when run by the computing processing device, causes the computing processing device to perform the various steps in the methods described above.

[0192] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0193] The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of this disclosure. Furthermore, please note that the examples of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0194] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0195] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This disclosure can be implemented by means of hardware comprising a plurality of different elements and by means of a suitably programmed computer. In a unit claim enumerating a plurality of means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for identifying constitution in Traditional Chinese Medicine, characterized in that, The method includes: Acquire infrared human images of the target user; The infrared human body image is input into the infrared key point recognition model to obtain the TCM human body key points in the infrared human body image. Based on the temperature distribution in the infrared human body image, determine the temperature type of the key points of the TCM human body in the infrared human body image; Identify the target user's TCM constitution based on the temperature type; The infrared key point recognition model is obtained through the following steps: Acquire natural light key point recognition model and sample infrared human body images; Mark key points of the human body in traditional Chinese medicine in the infrared human body images of the samples; The natural light key point recognition model is transferred to the labeled sample infrared human body images to obtain the infrared key point recognition model.

2. The method according to claim 1, characterized in that, The step of using labeled sample infrared human images to perform transfer learning on the natural light keypoint recognition model to obtain an infrared keypoint recognition model includes: The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine. While keeping the model parameters of the feature extraction layer unchanged, the natural light key point recognition model after adjusting the nodes is trained using the labeled sample infrared human body images to obtain the infrared key point recognition model.

3. The method according to claim 2, characterized in that, The original human body key points of the natural light key point recognition model include the traditional Chinese medicine human body key points, and the number of the original human body key points is greater than the number of traditional Chinese medicine human body key points. The adjustment of the fully connected layer nodes of the natural light key point recognition model based on the key points of the human body in traditional Chinese medicine includes: In the fully connected layer nodes of the natural light key point recognition model, the human key point corresponding to the deleted node label is not a fully connected layer node of the TCM human key point.

4. The method according to claim 1, characterized in that, The step of using labeled sample infrared human images to perform transfer learning on the natural light keypoint recognition model to obtain an infrared keypoint recognition model includes: The fully connected layer nodes of the natural light key point recognition model are adjusted based on the key points of the human body in traditional Chinese medicine. The natural light keypoint recognition model with adjusted nodes is trained using the labeled sample infrared human body images to obtain the infrared keypoint recognition model.

5. The method according to claim 1, characterized in that, The feature extraction network in the natural light keypoint recognition model is a lightweight feature extraction network, and the amount of parameter data of the lightweight feature extraction network is less than the amount of parameter data of the original feature extraction network of the natural light keypoint recognition model.

6. The method according to claim 1, characterized in that, The key points of the human body in Traditional Chinese Medicine include at least one of the following: key points of the head, key points of the neck, key points of the shoulders, key points of the elbows, key points of the hands, key points of the abdomen, key points of the hips, key points of the legs and elbows, and key points of the feet.

7. The method according to claim 1, characterized in that, The temperature type includes at least one of high temperature type and low temperature type; The step of determining the temperature type of the key points of the TCM human body in the infrared human body image based on the temperature distribution in the infrared human body image includes: Determine a first number of temperature maxima locations and / or a second number of temperature minima locations in the infrared human body image; The temperature type of the key point of the human body in traditional Chinese medicine corresponding to the image region where the temperature maximum value is located is determined to be high temperature type, and / or the temperature type of the key point of the human body in traditional Chinese medicine corresponding to the image region where the temperature minimum value is located is determined to be low temperature type.

8. The method according to claim 7, characterized in that, Determining the first number of temperature maxima locations and / or the second number of temperature minima locations in the infrared human body image includes: The image location of the first number of temperature maxima in the infrared human body image is obtained by using a maximum value filter and is taken as the temperature maxima location. Using the average of the maximum and minimum temperature values ​​in the infrared human body image as a flip plane, the temperature values ​​in the infrared human body image are reversed. The location of the second maximum temperature value in the inverted infrared human body image is obtained by using the maximum value filter and is taken as the location of the minimum temperature value.

9. The method according to any one of claims 1-8, characterized in that, The step of identifying the target user's TCM constitution based on the temperature type includes: In the TCM constitution mapping relationship, search for TCM constitutions that match the temperature type of each of the aforementioned TCM human body key points.

10. A traditional Chinese medicine constitution identification device, characterized in that, The device includes: The receiving module is configured to acquire infrared human images of the target user. The model prediction module is configured to input the infrared human body image into the infrared key point recognition model to obtain the TCM human body key points in the infrared human body image. The recognition module is configured to determine the temperature type of the key points of the TCM human body in the infrared human body image based on the temperature distribution in the infrared human body image; The target user's TCM constitution is identified based on the temperature type.

11. A computing processing device, characterized in that, include: Memory containing computer-readable code; One or more processors, when the computer-readable code is executed by the one or more processors, the computing processing device performs the traditional Chinese medicine constitution identification method as described in any one of claims 1-8.

12. A computer program, characterized in that, It includes computer-readable code that, when run on a computing processing device, causes the computing processing device to perform the traditional Chinese medicine constitution identification method as described in any one of claims 1-9.

13. A computer-readable medium, characterized in that, It contains a computer program for the TCM constitution identification method as described in any one of claims 1-9.

Images