Method, device, electronic equipment and medium for identifying tongue image

Abstract

Embodiments of the present disclosure disclose a method, device, electronic equipment and medium for identifying tongue image. A specific embodiment of the method comprises: obtaining a tongue image to be identified; extracting a first image feature of the tongue image to be identified, wherein the first image feature belongs to a traditional image feature; extracting a second image feature of the tongue image to be identified by using a pre-trained neural network; and generating health status information prompted by the tongue image to be identified based on matching between the first image feature, the second image feature and a preset tongue image database, wherein the tongue image database comprises a corresponding relationship between an image feature and a description label for indicating a health status. The embodiment fully utilizes the advantages of local features and global features, and improves the matching degree of the generated health status information and the tongue image to be identified.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer technology, and particularly relate to a method and apparatus for identifying tongue appearance, an electronic device and a medium. BACKGROUND

[0002] With the development of artificial intelligence technology, the application of computer vision technology in identifying and analyzing medical images is also increasing.

[0003] In the prior art, the identification of tongue appearance mainly adopts computer vision to analyze the manually designed features (such as color, texture, shape, etc.) of tongue fur, and then obtains the corresponding symptoms according to relevant experience. However, the method of extracting simple features and then associating symptoms according to relevant experience often results in inaccurate results due to the non-one-to-one relationship of the association. If the existing image classification method is directly applied, it is difficult to achieve accurate classification due to the high similarity of the tongue. SUMMARY

[0004] Embodiments of the present disclosure provide a method and apparatus for identifying tongue appearance, an electronic device and a medium.

[0005] In a first aspect, embodiments of the present disclosure provide a method for identifying tongue appearance, the method comprising: obtaining a tongue image to be identified; extracting a first image feature of the tongue image to be identified, wherein the first image feature belongs to a traditional image feature; extracting a second image feature of the tongue image to be identified using a pre-trained neural network; generating health status information prompted by the tongue image to be identified based on matching of the first image feature, the second image feature and a preset tongue image database, wherein the tongue image database comprises a corresponding relationship between an image feature and a description label indicating a health status.

[0006] In a second aspect, embodiments of the present disclosure provide a method for identifying tongue appearance, the method comprising: obtaining a tongue image to be identified; sending the tongue image to be identified to a target server; receiving health status information sent by the target server; and displaying a tongue appearance identification result based on the health status information.

[0007] In a third aspect, embodiments of the present disclosure provide a device for identifying tongue image, the device comprising: a first acquisition unit configured to acquire a tongue image to be identified; a first extraction unit configured to extract a first image feature of the tongue image to be identified, wherein the first image feature belongs to a traditional image feature; a second extraction unit configured to extract a second image feature of the tongue image to be identified by using a pre-trained neural network; and a generation unit configured to generate health status information prompted by the tongue image to be identified based on matching of the first image feature, the second image feature and a preset tongue image database, wherein the tongue image database comprises a corresponding relationship between an image feature and a description label for indicating a health status.

[0008] In a fourth aspect, embodiments of the present disclosure provide a device for identifying tongue image, the device comprising: a second acquisition unit configured to acquire a tongue image to be identified; a third sending unit configured to send the tongue image to be identified to a target server; a second receiving unit configured to receive health status information sent by the target server; and a display unit configured to display a tongue image identification result based on the health status information.

[0009] In a fifth aspect, embodiments of the present disclosure provide an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation manner of the first aspect and the second aspect.

[0010] In a sixth aspect, embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, which is executed by a processor to implement the method as described in any implementation manner of the first aspect and the second aspect.

[0011] The method, device, electronic device and medium for identifying tongue image provided by the embodiments of the present disclosure fully utilize the advantages of local features and global features by fusing the artificial design features of the image and the features extracted by the neural network as the basis for image matching, thereby improving the matching degree of the generated health status information and the tongue image to be identified. Moreover, the accuracy of tongue image identification is improved by corresponding processing from aspects such as image acquisition and result display. BRIEF DESCRIPTION OF DRAWINGS

[0012] Other features, objects and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments made with reference to the attached drawings:

[0013] Figure 1 is an example system architecture diagram to which an embodiment of the present disclosure can be applied;

[0014] Figure 2 is a flowchart of one embodiment of a method for identifying tongue appearance according to the present disclosure;

[0015] Figure 3 is a schematic diagram of one application scenario of a method for identifying tongue appearance according to an embodiment of the present disclosure;

[0016] Figure 4 is a flowchart of yet another embodiment of a method for identifying tongue appearance according to the present disclosure;

[0017] Figure 5 is a structural schematic diagram of one embodiment of an apparatus for identifying tongue appearance according to the present disclosure;

[0018] Figure 6 is a structural schematic diagram of one embodiment of an apparatus for identifying tongue appearance according to the present disclosure;

[0019] Figure 7 is a structural schematic diagram of an electronic device suitable for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The present disclosure will be further described below in conjunction with the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only parts related to the application are shown in the drawings for ease of description.

[0021] It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0022] Figure 1 An exemplary architecture 100 of a method for identifying tongue appearance or an apparatus for identifying tongue appearance to which the present disclosure can be applied is shown.

[0023] As shown in Figure 1 , the system architecture 100 can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a communication link medium between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0024] The terminal devices 101, 102, 103 interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, image recognition applications, etc.

[0025] The terminal device 101, 102, 103 can be hardware or software. When the terminal device 101, 102, 103 is hardware, it can be various electronic devices with a display screen and supporting human-computer interaction, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like. When the terminal device 101, 102, 103 is software, it can be installed in the above-mentioned electronic devices. It can be implemented as multiple software or software modules (for example, software or software modules for providing distributed services) or as a single software or software module. No specific limitation is made herein.

[0026] The server 105 can be a server providing various services, for example, a background server supporting image recognition type applications on the terminal device 101, 102, 103. The background server can analyze and process a received tongue image to be recognized, and generate a processing result (for example, health status information prompted by the tongue image to be recognized), and can also feed back the generated processing result to the terminal device.

[0027] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When the server is software, it can be implemented as multiple software or software modules (for example, software or software modules for providing distributed services) or as a single software or software module. No specific limitation is made herein.

[0028] It should be noted that the method for identifying tongue appearance as described in the foregoing first aspect is generally executed by the server 105, and correspondingly, the apparatus for identifying tongue appearance is generally arranged in the server 105. The method for identifying tongue appearance as described in the foregoing second aspect is generally executed by the terminal device 101, 102, 103, and correspondingly, the apparatus for identifying tongue appearance is generally arranged in the terminal device 101, 102, 103. It should be understood that the method for identifying tongue appearance and the apparatus for identifying tongue appearance can be executed by or arranged in any one of the terminal device 101, 102, 103, the network, and the server. Figure 1 The number of terminal devices, networks, and servers in

[0029] With reference to Figure 2 , a flow 200 of one embodiment of the method for identifying tongue appearance according to the present disclosure is shown. The method for identifying tongue appearance includes the following steps:

[0030] Step 201, obtaining a tongue image to be recognized.

[0031] In the present embodiment, the execution subject of the method for identifying tongue appearance (for example, the terminal device 101, 102, 103, the network, or the server 105) can be a terminal device, a network, or a server. Figure 1The server 105 shown can obtain the tongue image to be identified through wired or wireless connection. The tongue image to be identified can be an original image of the tongue taken by the user, or an image highlighting the tongue body after processing the original image. As an example, the execution subject can obtain the tongue image to be identified stored locally, or from an electronic device (e.g. Figure 1 The terminal device 101, 102, 103 shown obtains the tongue image to be identified.

[0032] In some optional implementations of the embodiment, the execution subject can obtain the tongue image to be identified by the following steps:

[0033] Step 1: Obtain the initial tongue image collected.

[0034] In these implementations, the execution subject can first obtain the initial tongue image collected by various means. The initial tongue image can be an unprocessed image taken by a camera of the tongue. In practice, the initial tongue image can be distorted in color by the light used to take the image, and can include non-tongue regions.

[0035] Step 2: Pre-process the initial tongue image to generate the tongue image to be identified.

[0036] In these implementations, the execution subject can pre-process the initial tongue image obtained in the first step to generate the tongue image to be identified. The pre-processing can include at least one of the following: tongue color correction, tongue segmentation.

[0037] Based on the optional implementations, the scheme can reduce the color deviation caused by light through a color correction algorithm, and remove irrelevant regions through an image segmentation algorithm to reduce the adverse effects of extracting features of non-tongue regions on subsequent matching.

[0038] Step 202: Extract first image features of the tongue image to be identified.

[0039] In this embodiment, the execution subject can extract features of the tongue image to be identified obtained in step 201 as first image features by various means. The first image features generally belong to traditional image features. The traditional image features can include but are not limited to at least one of the following: color features, texture features, shape features.

[0040] Step 203: Extract second image features of the tongue image to be identified using a pre-trained neural network.

[0041] In the embodiment, the execution subject can extract the second image feature of the tongue image to be identified by using a pre-trained neural network. The pre-trained neural network can include various convolutional neural networks for extracting image features. As an example, the convolutional neural network can be trained by using training samples composed of sample tongue images and corresponding sample description information to adjust network parameters of the convolutional neural network. Thus, the second image feature can include an image feature extracted by the neural network, which is distinguished from a conventional manually constructed image feature.

[0042] In step 204, based on the matching of the first image feature, the second image feature, and the preset tongue image database, the health status information prompted by the tongue image to be identified is generated.

[0043] In the embodiment, based on the matching of the first image feature extracted in step 202 and the second image feature extracted in step 203 with the preset tongue image database, the execution subject can generate the health status information prompted by the tongue image to be identified in various ways. The tongue image database can include a corresponding relationship between image features and description labels for indicating health status. The description labels can include constitution judgment labels such as “hot and humid constitution” and “cold constitution”, disease labels such as “excessive liver fire” and “heavy dampness”, and health condition description labels such as “good constitution” and “healthy”. As an example, the execution subject can first match the first image feature extracted in step 202 and the second image feature extracted in step 203 with the image features in the preset tongue image database. Then, based on the description labels corresponding to the matched features, the execution subject can generate health status information consistent with the corresponding description labels.

[0044] In some optional implementations of the embodiment, the tongue image database can include a traditional image feature comparison sub-database and a network extracted feature comparison sub-database. The traditional image feature comparison sub-database can include a corresponding relationship between traditional image features and description labels for indicating health status. The network extracted feature comparison sub-database can include a corresponding relationship between network extracted features and description labels for indicating health status. The traditional image features and network extracted features can be consistent with the forms of the first image feature and the second image feature respectively. Based on this, the execution subject can generate the health status information prompted by the tongue image to be identified by the following steps:

[0045] Firstly, the first image feature is compared with the traditional image feature comparison sub-database to generate a first matching result vector.

[0046] In these implementations, the execution subject can compare the first image feature extracted in step 202 with the traditional image feature comparison sub-library in various manners, thereby generating a first matching result vector. The elements in the first matching result vector can be used to represent the similarity between the first image feature and the features included in the traditional image feature comparison sub-library.

[0047] As an example, the image feature can be in the form of a feature vector, and the execution subject can determine the cosine similarity between the extracted first image feature and the traditional image features in the traditional image feature comparison sub-library. Then, the execution subject can combine the determined cosine similarities into a first matching result vector. The first element in the first matching result vector indicates the cosine similarity between the first image feature extracted in step 202 and the first traditional image feature compared in the traditional image feature comparison sub-library, and so on.

[0048] Secondly, compare the second image feature with the network extracted feature comparison sub-library to generate a second matching result vector.

[0049] In these implementations, the execution subject can compare the second image feature extracted in step 203 with the network extracted feature comparison sub-library in various manners, thereby generating a second matching result vector. The elements in the second matching result vector can be used to represent the similarity between the second image feature and the features included in the network extracted feature comparison sub-library. The process of feature comparison and generation of the second matching result vector can be similar to the description of the first step, which will not be repeated here.

[0050] Thirdly, select a first target image from the tongue image database based on the combination of the generated first matching result vector and second matching result vector.

[0051] In these implementations, based on the combination of the generated first matching result vector and second matching result vector, the execution subject can select a first target image from the tongue image database in various manners.

[0052] Optionally, the execution subject can select a first target image from the tongue image database by the following steps:

[0053] S1, weighted average the first matching result vector and the second matching result vector to generate a target matching result vector.

[0054] In these implementations, the execution subject can perform a weighted average of the first matching result vector generated in the first step and the second matching result vector generated in the second step to generate a target matching result vector. Specifically, the execution subject can multiply the first matching result vector and the second matching result vector by preset weights respectively and then sum them to generate the target matching result vector. The sum of the preset weights multiplied respectively is usually 1.

[0055] S2, selecting an image corresponding to an element representing the greatest similarity from the target matching result vector as the first target image from the tongue image database.

[0056] In these implementations, the execution subject can select an image corresponding to an element representing the greatest similarity from the target matching result vector generated in step S1 as the first target image from the tongue image database. As an example, the dimension of the target matching result vector can be 10,000. The 128th element (e.g., cosine similarity is 0.89) in the target matching result vector indicates the greatest similarity. The execution subject can select an image corresponding to the 128th element from the tongue image database as the first target image.

[0057] Based on the optional implementation, the matching results of the extracted traditional image features and the matching results of the network extracted image features can be fused to determine the matched image more comprehensively.

[0058] Optionally, the execution subject can also select images corresponding to elements with a similarity greater than a preset threshold from the first matching result vector and the second matching result vector generated from the tongue image database as at least one first target image. As an example, the preset threshold can be 0.7. The execution subject can select images corresponding to elements greater than 0.7 from the first matching result vector and the second matching result vector as the first target image from the tongue image database. As another example, the preset threshold can be set to 0.65 and 0.72 respectively. The execution subject can select images corresponding to elements greater than 0.65 from the first matching result vector and images corresponding to elements greater than 0.72 from the second matching result vector as the first target image from the tongue image database.

[0059] Step 4, generating health status information according to the description label corresponding to the first target image.

[0060] In these implementations, according to the description label corresponding to the first target image selected in the third step described above, the execution subject can generate health status information consistent with the corresponding description label in various ways. As an example, when the number of first target images is 1, the execution subject can generate health status information (such as "healthy", "sub-healthy", etc.) matching the description label corresponding to the first target image.

[0061] Alternatively, based on the selected at least one first target image, the execution subject can generate health status information according to the fusion of the description labels corresponding to the at least one first target image in various ways. As an example, the execution subject can generate health status information according to whether the description labels corresponding to the at least one first target image are consistent. If the description labels are consistent, the execution subject can generate health status information according to the health status indicated by the corresponding description labels. If the description labels are inconsistent, the execution subject can generate health status information consistent with the description label corresponding to the image indicated by the element representing the greatest similarity in the first matching result vector and the second matching result vector. As another example, the execution subject can generate health status information according to the number of health statuses indicated by the description labels corresponding to the at least one first target image. For example, when the health statuses indicated by the description labels corresponding to the at least one first target image are 3 "healthy" and 1 "sub-healthy", the execution subject can generate health status information representing health.

[0062] Based on the optional implementation described above, alternatively, according to the fusion of the description labels corresponding to the at least one first target image, the execution subject can also generate health status information by the following steps:

[0063] S1, in response to determining that the health statuses indicated by the description labels corresponding to the at least one first target image are inconsistent, obtaining a preset traditional image feature matching weight and a network extracted feature matching weight.

[0064] In these implementations, the preset traditional image feature matching weight and the network extracted feature matching weight can be used to adjust the weights of traditional image features and network extracted features. For example, when the quality of traditional image features is considered to be high, the preset traditional image feature matching weight can be appropriately increased; when the quality of the network used for feature extraction is considered to be high, the network extracted feature matching weight can be appropriately increased.

[0065] S2, multiplying the elements corresponding to the at least one first target image by the obtained preset traditional image feature matching weight or network extracted feature matching weight to obtain an adjusted result.

[0066] In these implementations, as an example, the elements corresponding to the at least one first target image described above can include elements (e.g., 0.67, 0.75) greater than a preset threshold (e.g., 0.6) selected from the first matching result vector and elements (e.g., 0.82, 0.78) greater than a preset threshold (e.g., 0.7) selected from the second matching result vector. Assuming that the preset conventional image feature matching weight and the network extracted feature matching weight obtained can be 0.9 and 0.8 respectively, then for the element 0.67, the adjusted result can be 0.67x0.9=0.603. For the element 0.78, the adjusted result can be 0.78x0.8=0.624.

[0067] S3, determining the image corresponding to the adjusted result representing the greatest similarity in the obtained adjusted results as the first reference image.

[0068] S4, generating health status information consistent with the description label corresponding to the first reference image.

[0069] Based on the above optional implementation, the scheme improves the flexibility of matching by introducing matching weights corresponding to different features.

[0070] Based on the above optional implementation, optionally, according to the fusion of the description label corresponding to the at least one first target image, the execution subject can also generate health status information by the following steps:

[0071] S1, in response to determining that the health status indicated by the description label corresponding to the at least one first target image is inconsistent, selecting a matching additional question from a preset additional question library.

[0072] In these implementations, the additional question library usually stores questions associated with the description label. As an example, in response to determining that the description label corresponding to the at least one first target image includes "yang deficiency constitution", "yang deficiency constitution", the execution subject can select questions related to the "yang deficiency constitution" and "yang deficiency constitution" from the preset additional question library as the matching additional question, respectively. Yin deficiency constitution Yin deficiency constitution ”, the execution subject can select questions related to the "yang deficiency constitution" and "yang deficiency constitution" from the preset additional question library as the matching additional question, respectively. Figure 3

[0073] S2, sending the matching additional question to the target device.

[0074] In these implementations, the target device can usually be a device that sends the tongue image to be identified.

[0075] S3, receiving answer information corresponding to the matching additional question.

[0076] ​S4, generating the health state information according to a matching degree of the description label corresponding to the at least one first target image in the description label corresponding to the at least one first target image.

[0077] Based on the above optional implementation, the tongue image recognition accuracy can be improved by comprehensively considering the image recognition result and the additional information related to the interaction with the user terminal.

[0078] In some optional implementations of the embodiment, the tongue image database can include a correspondence between the image features and the description label indicating the health state. Based on this, the execution subject can generate the health state information prompted by the tongue image to be recognized by the following steps:

[0079] Firstly, the first image feature and the second image feature are fused to generate a target image feature.

[0080] In these implementations, the execution subject can fuse the first image feature extracted in step 202 and the second image feature extracted in step 203 in various ways to generate a target image feature. The feature fusion can be performed in various ways. For example, for features with the same dimension, a weighted average method can be used. For another example, a splicing and dimension compression method can be used. Details are not described herein.

[0081] Secondly, the target image feature is compared with the image features in the tongue image database to generate a third matching result vector.

[0082] In these implementations, the execution subject can generate the third matching result vector in a manner similar to the comparison of the first image feature with the traditional image feature in the sub-database.

[0083] Thirdly, an image corresponding to an element with the greatest similarity in the third matching result vector is selected from the tongue image database as a second target image.

[0084] Fourthly, the health state information is generated according to the description label corresponding to the second target image.

[0085] In these implementations, the method of selecting the second target image and generating the health state information can be consistent with the description of the corresponding part described above, and details are not described herein.

[0086] Based on the above optional implementation, the tongue image recognition accuracy can be improved by comprehensively considering the image recognition result and the additional information related to the interaction with the user terminal.

[0087] In some optional implementations of the embodiment, the execution subject can further continue to perform the following steps:

[0088] In the first step, in response to determining that the generated health status information indicates an unhealthy state, the execution subject selects a matching question from a preconfigured question bank according to the health status information.

[0089] In these implementations, in response to determining that the generated health status information indicates an unhealthy state, the execution subject selects a matching question from a preconfigured question bank according to the health status information. For example, the health status information can be "weak constitution". Then the execution subject can select "Have you felt cold recently?" and "Have you felt weak recently?" as the matching questions from the preconfigured question bank.

[0090] In the second step, the target device is sent the matching question.

[0091] In these implementations, the execution subject can send the matching question selected in the first step to the target device. The target device can be the device that sent the tongue image to be identified.

[0092] In the third step, the answer information corresponding to the matching question is received.

[0093] In these implementations, the execution subject can receive the answer information corresponding to the matching question sent by the target device. For example, the answer information can include "yes" or "no". Alternatively, the answer information can also include content not included in the matching question. For example, the answer information can be "yes, and recently it is easy to feel cold hands and cold feet".

[0094] In the fourth step, the corresponding disease information is selected from a preconfigured disease information bank as disease prompt information according to the answer information and the health status information.

[0095] In these implementations, according to the answer information received in the third step and the generated health status information, the execution subject can select the corresponding disease information from the preconfigured disease information bank as the disease prompt information in various ways. The disease information bank can include a corresponding relationship between disease description information and disease information. For example, the disease information bank can include "headache, dry throat, runny nose - typhoid".

[0096] In the fifth step, the target device is sent the disease prompt information.

[0097] Based on the above optional implementation manner, when it is determined that the tongue image indicates an unhealthy state, additional information related to the user end can be obtained through interaction, and disease prompt information for prompting the disease can be generated according to the obtained additional information and the health state information determined based on the image features, so as to further improve the accuracy of tongue image recognition.

[0098] With reference to the above Figure 3 , Figure 3 is one of the application scenarios of the method for identifying tongue image according to the embodiments of the present disclosure. In Figure 4 , the user 301 uses the terminal device 302 to capture the tongue image 303. Then, the user 301 clicks the “upload” button, and the terminal device 302 sends the tongue image 303 to the background server 304. The background server 304 extracts the first image feature 305 and the second image feature 306 of the tongue image 303. Based on the matching between the extracted first image feature 305 and the second image feature 306 and the preset tongue image database 307, the background server 304 can generate the health state information 308 (for example, “healthy”) prompted by the tongue image 303. Optionally, the background server 304 can further send the generated health state information 308 to the terminal device 302.

[0099] At present, one of the existing technologies is usually to identify the tongue image only according to the artificially designed features or the existing image classification method, which leads to inaccurate matching results. The method provided in the above embodiments of the present disclosure uses the artificially designed features and the features extracted by the neural network as the basis for image matching, so as to fully utilize the advantages of local features and global features and improve the matching degree of the generated health state information and the tongue image to be identified.

[0100] With reference to the above Figure 1 , one embodiment of the method for identifying tongue image is shown. The flow 400 of the method for identifying tongue image includes the following steps:

[0101] Step 401, obtaining a tongue image to be identified.

[0102] In the present embodiment, the execution subject (for example, the terminal device 101, 102 or 103 in Figure 2 ) of the method for identifying tongue image can obtain the tongue image to be identified in various ways. The tongue image to be identified can be an original image captured by the user for the tongue, or can be an image highlighting the tongue subject after processing the original image.

[0103] In some optional implementation manners of the present embodiment, the execution subject can obtain the tongue image to be identified by the following steps:

[0104] In a first step, in response to detecting the tongue image capturing operation, light condition information of an environment for capturing the tongue image is acquired.

[0105] In these implementations, in response to detecting the tongue image capturing operation, the execution subject can acquire the light condition information of the environment for capturing the tongue image in various ways. As an example, the tongue image capturing operation can be, for example, a user clicking a “capture tongue photo” button. The execution subject can acquire the light condition information from a light sensor installed therein.

[0106] In a second step, it is determined whether the ambient light indicated by the light condition information meets a preset tongue image capturing condition.

[0107] In these implementations, the preset tongue image capturing condition can include, for example, that the light intensity belongs to a first preset interval, the light color temperature belongs to a second preset interval, and the like.

[0108] In a third step, in response to determining that it does not meet, information for prompting to capture the tongue image under suitable lighting environment is displayed.

[0109] In these implementations, the information for prompting to capture the tongue image under suitable lighting environment can be, for example, “please capture the tongue image under suitable lighting conditions”.

[0110] In a fourth step, in response to determining that it meets, the tongue image is captured as a tongue image to be identified.

[0111] Based on the optional implementations described above, the present scheme can improve the quality of the acquired tongue image to be identified from the image capturing stage through light condition detection, providing a basis for accurate identification of the tongue image.

[0112] Step 402, sending the tongue image to be identified to a target server.

[0113] In this embodiment, the execution subject can send the tongue image to be identified acquired in step 401 to a target server. The target server can include various servers capable of implementing tongue image identification. Alternatively, the target server can also include a server for executing the foregoing Figure 2 The execution subject of the method for identifying tongue images described in the embodiments shown.

[0114] Step 403, receiving health status information sent by the target server.

[0115] In this embodiment, the execution subject can receive the health status information sent by the target server. The health status information can be consistent with the corresponding description in the foregoing embodiments, which will not be described here.

[0116] In some optional implementations of the embodiment, the health status information can be based on the tongue image recognition method described in the foregoing embodiments. Figure 4 The method for identifying tongue image described in the embodiment generates.

[0117] Step 404, display the tongue image recognition result based on the health status information.

[0118] In the embodiment, based on the health status information received in step 403, the execution subject can display the tongue image recognition result in various ways. As an example, the execution subject can directly display the health status information received in step 403 as the tongue image recognition result. As another example, the execution subject can also send the health status information received in step 403 to an artificial review terminal, and in response to receiving confirmation information sent by the artificial review terminal, the execution subject can display the health status information as the tongue image recognition result.

[0119] In some optional implementations of the embodiment, based on the health status information, the execution subject can display the tongue image recognition result in the following steps:

[0120] First, in response to determining that the received health status information indicates unhealthy, send a matching question acquisition request to a target server.

[0121] In these implementations, in response to determining that the received health status information indicates unhealthy, the execution subject can send a matching question acquisition request to a target server. The matching question acquisition request is used to acquire a question associated with the received health status. Optionally, the question associated with the received health status can be consistent with the question selected from the pre-set question library in the foregoing embodiments, which will not be described here.

[0122] Second, in response to receiving a matching question corresponding to the matching question acquisition request, display the matching question.

[0123] Third, receive user inputted answer information corresponding to the matching question.

[0124] In these implementations, the execution subject can receive user inputted answer information corresponding to the matching question. The answer information can include various forms. For example, the answer information can include information representing approval of the question generated by the user clicking the "Yes" button or the camera capturing the user's nodding behavior. For another example, the answer information can also include the user's answer voice.

[0125] Fourth, send the answer information to the target server.

[0126] In the fifth step, the disease prompt information sent by the target server is received.

[0127] In the implementations, the disease prompt information can be determined according to the answer information and the health status information. As an example, the disease prompt information is generally consistent with at least one of the answer information and the health status information. Alternatively, the disease prompt information can also be consistent with the corresponding disease information selected from the preset disease information library as described in the foregoing embodiments, which will not be repeated here.

[0128] In the sixth step, the disease prompt information is displayed.

[0129] Based on the optional implementation, when it is determined that the tongue image indicates an unhealthy state, the related additional information can be obtained through the interaction with the user, and the disease prompt information for prompting the disease is generated according to the obtained additional information and the health status information determined based on the image features, so as to further improve the accuracy of tongue image recognition.

[0130] As can be seen from Figure 5 , the flow 400 of the method for identifying the tongue image in the embodiment embodies the steps of obtaining the tongue image to be identified and displaying the tongue image recognition result based on the received health status information. Therefore, the scheme described in the embodiment can perform corresponding processing from the aspects of image acquisition and result display, so as to improve the accuracy of tongue image recognition.

[0131] Further referring to Figure 2 , as an implementation of the method shown in the foregoing figures, the disclosure provides one embodiment of an apparatus for identifying the tongue image, which corresponds to the method embodiment shown in Figure 5 , and the apparatus can be applied in various electronic devices.

[0132] As shown in Figure 2 , the apparatus 500 for identifying the tongue image provided in the embodiment includes a first obtaining unit 501, a first extracting unit 502, a second extracting unit 503, and a generating unit 504. The first obtaining unit 501 is configured to obtain a tongue image to be identified. The first extracting unit 502 is configured to extract a first image feature of the tongue image to be identified, wherein the first image feature belongs to a traditional image feature. The second extracting unit 503 is configured to extract a second image feature of the tongue image to be identified by using a pre-trained neural network. The generating unit 504 is configured to generate health status information prompted by the tongue image to be identified based on matching between the first image feature, the second image feature, and a preset tongue image database, wherein the tongue image database includes a corresponding relationship between an image feature and a description label for indicating a health status.

[0133] In the embodiment, the specific processing of the first obtaining unit 501, the first extracting unit 502, the second extracting unit 503, and the generating unit 504 in the device 500 for identifying tongue appearance and the technical effects brought by the same can be referred to the corresponding descriptions of the first obtaining unit 501, the first extracting unit 502, the second extracting unit 503, and the generating unit 504 in the device 500 for identifying tongue appearance respectively. Figure 6 The related descriptions of the step 201, the step 202, the step 203, and the step 204 in the corresponding embodiment will not be repeated here.

[0134] In some optional implementation of the embodiment, the tongue image database can include a traditional image feature comparison sub-database and a network extracted feature comparison sub-database. The generating unit 504 can include a first comparison sub-unit (not shown in the figure), a second comparison sub-unit (not shown in the figure), a first selection sub-unit (not shown in the figure), and a first generation sub-unit (not shown in the figure). The first comparison sub-unit can be configured to compare the first image feature with the traditional image feature comparison sub-database to generate a first matching result vector. The elements in the first matching result vector can be used to represent the similarity between the first image feature and the features included in the traditional image feature comparison sub-database. The second comparison sub-unit can be configured to compare the second image feature with the network extracted feature comparison sub-database to generate a second matching result vector. The elements in the second matching result vector can be used to represent the similarity between the second image feature and the features included in the network extracted feature comparison sub-database. The first selection sub-unit can be configured to select the first target image from the tongue image database based on the combination of the generated first matching result vector and the second matching result vector. The first generation sub-unit can be configured to generate the health status information according to the description label corresponding to the first target image.

[0135] In some optional implementation of the embodiment, the first selection sub-unit can include a generation module (not shown in the figure) and a selection module (not shown in the figure). The generation module can be configured to perform weighted average on the first matching result vector and the second matching result vector to generate a target matching result vector. The selection module can be configured to select, as the first target image, the image corresponding to the element representing the greatest similarity in the target matching result vector from the tongue image database.

[0136] In some optional implementation of the embodiment, the first selection sub-unit can be further configured to select, as at least one first target image, the image corresponding to the element with a similarity greater than a preset threshold from the generated first matching result vector and the second matching result vector from the tongue image database. The first generation sub-unit can be further configured to generate the health status information according to the fusion of the description labels corresponding to the at least one first target image.

[0137] In some optional implementations of the present embodiment, the first generation sub-unit can be further configured to: in response to determining that the health status indicated by the description label corresponding to the at least one first target image is inconsistent, obtain a preset traditional image feature matching weight and a network extracted feature matching weight; multiply the elements corresponding to the at least one first target image with the obtained preset traditional image feature matching weight or network extracted feature matching weight to obtain an adjusted result; determine the image corresponding to the adjusted result representing the greatest similarity degree in the obtained adjusted result as the first reference image; and generate the health status information consistent with the description label corresponding to the first reference image. In some optional implementations of the present embodiment, the first generation sub-unit can be further configured to: in response to determining that the health status indicated by the description label corresponding to the at least one first target image is inconsistent, select a matching additional question from a preset additional question library; send the matching additional question to the target device; receive the answer information corresponding to the matching additional question; and generate the health status information according to the matching degree of the description label corresponding to the at least one first target image.

[0138] In some optional implementations of the present embodiment, the tongue image database can include a corresponding relationship between the image total feature and the description label used to indicate the health status. The generation unit 504 can include a fusion sub-unit (not shown in the figure), a third comparison sub-unit (not shown in the figure), a second selection sub-unit (not shown in the figure), and a second generation sub-unit (not shown in the figure). The fusion sub-unit can be configured to fuse the first image feature and the second image feature to generate a target image feature. The third comparison sub-unit can be configured to compare the target image feature with the image total feature in the tongue image database to generate a third matching result vector. The second selection sub-unit can be configured to select, from the tongue image database, an image corresponding to an element representing the greatest similarity degree in the third matching result vector as a second target image. The second generation sub-unit can be configured to generate the health status information according to the description label corresponding to the second target image.

[0139] In some optional implementations of the present embodiment, the acquisition unit 501 can include an acquisition sub-unit (not shown in the figure) and a preprocessing sub-unit (not shown in the figure). The acquisition sub-unit can be configured to acquire the collected initial tongue image. The preprocessing sub-unit can be configured to pre-process the initial tongue image to generate a tongue image to be recognized. The preprocessing can include at least one of the following: tongue color correction, tongue segmentation.

[0140] In some optional implementations of the embodiment, the device 500 for identifying tongue image can further include a first selecting unit (not shown in the figure), a first sending unit (not shown in the figure), a first receiving unit (not shown in the figure), a second selecting unit (not shown in the figure), and a second sending unit (not shown in the figure). The first selecting unit can be further configured to select a matching question from a pre-set question library according to the health status information in response to determining that the generated health status information prompt is unhealthy. The first sending unit can be further configured to send the matching question to the target device. The first receiving unit can be further configured to receive answer information corresponding to the matching question. The second selecting unit can be further configured to select corresponding disease information from a pre-set disease information library as disease prompt information according to the answer information and the health status information. The second sending unit can be further configured to send the disease prompt information to the target device.

[0141] The device provided by the above embodiment of the present disclosure can fuse the artificial design features extracted by the first extracting unit 502 and the neural network extracted features extracted by the second extracting unit 503 as the basis for image matching by the generating unit 504, so as to fully utilize the advantages of local features and global features, and improve the matching degree of the generated health status information and the tongue image to be identified.

[0142] Further reference Figure 4 , as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of a device for identifying tongue image, which corresponds to the method embodiment shown in Figure 6 , and the device can be applied in various electronic devices.

[0143] As shown in Figure 4 , the device 600 for identifying tongue image provided by the embodiment includes a second acquiring unit 601, a third sending unit 602, a second receiving unit 603, and a display unit 604. The second acquiring unit 601 is configured to acquire a tongue image to be identified. The third sending unit 602 is configured to send the tongue image to be identified to a target server. The second receiving unit 603 is configured to receive health status information sent by the target server. The display unit 604 is configured to display a tongue image identification result based on the health status information.

[0144] In the embodiment, the second acquiring unit 601, the third sending unit 602, the second receiving unit 603, and the display unit 604 in the device 600 for identifying tongue image, and the specific processing thereof and the technical effects brought by the specific processing can refer to the related descriptions of the steps 401, 402, 403, and 404 in the corresponding embodiments, which will not be repeated here. Figure 2 The corresponding embodiments of the steps 401, 402, 403, and 404 in the corresponding embodiments, which will not be repeated here.

[0145] In some optional implementation of the embodiment, the second obtaining unit 601 can include an obtaining subunit (not shown in the figure), a determining subunit (not shown in the figure), a first displaying subunit (not shown in the figure), and a collecting subunit (not shown in the figure). The obtaining subunit can be configured to, in response to detecting the tongue image collecting operation, obtain the light condition information of the environment for shooting the tongue image. The determining subunit can be configured to determine whether the ambient light indicated by the light condition information meets the preset tongue image collecting condition. The first displaying subunit can be configured to, in response to determining that the condition is not met, display information for prompting to collect the tongue image in a suitable light environment. The collecting subunit can be configured to, in response to determining that the condition is met, collect the tongue image as the tongue image to be recognized.

[0146] In some optional implementation of the embodiment, the display unit 604 can include a first sending subunit (not shown in the figure), a second displaying subunit (not shown in the figure), a first receiving subunit (not shown in the figure), a second sending subunit (not shown in the figure), a second receiving subunit (not shown in the figure), and a third displaying subunit (not shown in the figure). The first sending subunit can be configured to, in response to determining that the received health status information prompts an unhealthy state, send a matching question obtaining request to a target server. The second displaying subunit can be configured to, in response to receiving a matching question corresponding to the matching question obtaining request, display the matching question. The first receiving subunit can be configured to receive answer information input by a user and corresponding to the matching question. The second sending subunit can be configured to send the answer information to the target server. The second receiving subunit can be configured to receive disease prompt information sent by the target server. The disease prompt information can be determined according to the answer information and the health status information. The third displaying subunit can be configured to display the disease prompt information.

[0147] In some optional implementation of the embodiment, the health status information can be generated based on the method described in the foregoing Figure 7 In some optional implementation of the embodiment, the health status information can be generated based on the method described in the foregoing

[0148] The apparatus provided by the foregoing embodiments of the present disclosure improves the accuracy of tongue image recognition through the respective processing of the second obtaining unit 601 and the display unit 604 in terms of image obtaining and result displaying.

[0149] Reference is made below to Figure 1 which shows an electronic device (e.g., a mobile phone) suitable for implementing embodiments of the present disclosure. Figure 7The diagram shows the structure of the server (700) in this disclosure. The terminal devices in the embodiments of this disclosure may include, but are not limited to, mobile terminals such as mobile phones and laptops, and fixed terminals such as digital TVs and desktop computers. Figure 7 The server shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0150] like Figure 7 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0151] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. ​ Each box shown can represent a device or multiple devices as needed.

[0152] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0153] It should be noted that the computer readable medium described in the embodiments of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of computer readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the embodiments of the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, an optical cable, an RF (Radio Frequency, radio frequency) or the like, or any suitable combination of the above.

[0154] The computer readable medium described above can be contained in the above-mentioned electronic device (server or terminal device); or can exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the server to: acquire a tongue image to be identified; extract a first image feature of the tongue image to be identified, wherein the first image feature belongs to a traditional image feature; extract a second image feature of the tongue image to be identified using a pre-trained neural network; generate health state information prompted by the tongue image to be identified based on matching of the first image feature, the second image feature and a preset tongue image database, wherein the tongue image database includes a corresponding relationship between an image feature and a description label indicating a health state; or cause the terminal device to: acquire a tongue image to be identified; send the tongue image to be identified to a target server; receive health state information sent by the target server; and display a tongue image recognition result based on the health state information.

[0155] Computer program code for carrying out operations of embodiments of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0156] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by a dedicated hardware-based system that carries out specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0157] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. The described units can also be implemented by a processor, for example, can be described as: a processor comprising a first acquisition unit, a first extraction unit, a second extraction unit, a generation unit; or a processor comprising a second acquisition unit, a third sending unit, a second receiving unit, a display unit. Among them, the name of these units does not constitute a limitation to the units themselves in some cases, for example, the first acquisition unit can also be described as: a unit for acquiring the tongue part image to be recognized.

[0158] The above description is merely that of the preferred embodiments of the present disclosure and a description of the technical principles of the present disclosure. It should be understood by those skilled in the art that the inventive scope of the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or equivalent features without departing from the above inventive concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) form the technical solutions.

Claims

1. A method for identifying tongue appearance, comprising: Acquire an image of the tongue to be identified; Extract the first image feature of the tongue image to be identified, wherein the first image feature belongs to traditional image features; The second image features of the tongue image to be identified are extracted using a pre-trained neural network; Based on the matching of the first image features, the second image features, and a preset tongue image database, the health status information indicated by the tongue image to be identified is generated, including: comparing the first image features with a traditional image feature comparison sub-library to generate a first matching result vector; comparing the second image features with a network-extracted feature comparison sub-library to generate a second matching result vector; selecting a first target image from the tongue image database based on the combination of the generated first and second matching result vectors; and generating the health status information according to the description tag corresponding to the first target image. The step of selecting a first target image from the tongue image database based on the combination of the generated first matching result vector and the second matching result vector includes: selecting images corresponding to elements in the generated first matching result vector and the second matching result vector with a similarity greater than a preset threshold as at least one first target image from the tongue image database. The tongue image database includes the traditional image feature comparison sub-database and the network extracted feature comparison sub-database. The step of generating the health status information based on the description label corresponding to the first target image includes: in response to determining that the health status indicated by the description label corresponding to the at least one first target image is inconsistent, obtaining preset traditional image feature matching weights and network extracted feature matching weights; multiplying the elements corresponding to the at least one first target image with the obtained preset traditional image feature matching weights or network extracted feature matching weights to obtain an adjusted result; determining the image corresponding to the adjusted result with the highest degree of similarity among the obtained adjusted results as the first reference image; and generating health status information consistent with the description label corresponding to the first reference image.

2. The method according to claim 1, wherein, The elements in the first matching result vector are used to characterize the similarity between the first image feature and the features included in the traditional image feature comparison sub-library, and the elements in the second matching result vector are used to characterize the similarity between the second image feature and the features included in the network extracted feature comparison sub-library.

3. The method according to claim 2, wherein, The step of selecting a first target image from the tongue image database based on the combination of the generated first matching result vector and the second matching result vector includes: The first matching result vector and the second matching result vector are weighted and averaged to generate the target matching result vector; The image corresponding to the element with the highest similarity in the target matching result vector is selected from the tongue image database as the first target image.

4. The method according to claim 1, wherein, Also includes: In response to determining that the health status indicated by the description tag corresponding to the at least one first target image is inconsistent, a matching additional question is selected from a preset additional question library; Send the matching additional questions to the target device; Receive the answer information corresponding to the matched additional question; The health status information is generated based on the degree of matching between the description tags corresponding to the at least one first target image and the received at least one first target image.

5. The method according to claim 1, wherein, The tongue image database includes a correspondence between total image features and descriptive labels used to indicate health status; and The step of generating health status information indicated by the tongue image to be identified based on the matching of the first image features, the second image features, and a preset tongue image database includes: The first image features and the second image features are fused to generate target image features; The target image features are compared with the total image features in the tongue image database to generate a third matching result vector; The image corresponding to the element with the highest similarity to the representation in the third matching result vector is selected from the tongue image database as the second target image; The health status information is generated based on the description label corresponding to the second target image.

6. The method according to claim 1, wherein, The process of acquiring the image of the tongue to be identified includes: Acquire the initial image of the tongue; The initial tongue image is preprocessed to generate the tongue image to be identified, wherein the preprocessing includes at least one of the following: tongue color correction and tongue segmentation.

7. The method according to any one of claims 1-6, wherein, The method further includes: In response to the determination that the generated health status information indicates an unhealthy condition, a matching question is selected from a preset question database based on the health status information; Send the matching question to the target device; Receive the answer information corresponding to the matched question; Based on the answer information and the health status information, select the corresponding disease information from the preset disease information database as disease prompt information; Send the symptom alert information to the target device.

8. A method for identifying tongue appearance, comprising: Acquire an image of the tongue to be identified; The image of the tongue to be identified is sent to the target server. The system receives health status information sent by the target server, wherein the health status information is determined based on the method for identifying tongue appearance as described in any one of claims 1-7. Based on the health status information, the tongue image recognition result is displayed.

9. The method according to claim 8, wherein, The process of acquiring the tongue image to be identified includes: In response to the detection of a tongue image acquisition operation, obtain the ambient lighting conditions information for capturing the tongue image; Determine whether the ambient light indicated by the illumination condition information meets the preset tongue image acquisition conditions; In response to the determination that the conditions are not met, information is displayed to prompt that an image of the tongue should be acquired under appropriate lighting conditions; In response to the determination that the condition is met, a tongue image is acquired as the tongue image to be identified.

10. The method according to claim 8, wherein, The process of displaying tongue image recognition results based on the health status information includes: In response to the determination that the received health status information indicates an unhealthy condition, a matching problem retrieval request is sent to the target server; Upon receiving a matching question corresponding to the matching question retrieval request, the matching question is displayed; Receive user input of answer information corresponding to the matching question; Send the response information to the target server; Receive symptom alert information sent by the target server, wherein the symptom alert information is determined based on the response information and the health status information; Display the symptoms information.

11. A device for identifying tongue images, comprising: The first acquisition unit is configured to acquire an image of the tongue to be identified; The first extraction unit is configured to extract a first image feature from the tongue image to be identified, wherein the first image feature is a traditional image feature; The second extraction unit is configured to extract second image features from the tongue image to be identified using a pre-trained neural network; The generation unit is configured to generate health status information indicated by the tongue image to be identified based on the matching of the first image features, the second image features, and a preset tongue image database. This includes: comparing the first image features with a traditional image feature comparison sub-library to generate a first matching result vector; comparing the second image features with a network-extracted feature comparison sub-library to generate a second matching result vector; selecting a first target image from the tongue image database based on the combination of the generated first and second matching result vectors; and generating the health status information according to the description tag corresponding to the first target image. The step of selecting a first target image from the tongue image database based on the combination of the generated first matching result vector and the second matching result vector includes: selecting images corresponding to elements in the generated first matching result vector and the second matching result vector with a similarity greater than a preset threshold as at least one first target image from the tongue image database. The tongue image database includes the traditional image feature comparison sub-database and the network extracted feature comparison sub-database. The step of generating the health status information based on the description label corresponding to the first target image includes: in response to determining that the health status indicated by the description label corresponding to the at least one first target image is inconsistent, obtaining preset traditional image feature matching weights and network extracted feature matching weights; multiplying the elements corresponding to the at least one first target image with the obtained preset traditional image feature matching weights or network extracted feature matching weights to obtain an adjusted result; determining the image corresponding to the adjusted result with the highest degree of similarity among the obtained adjusted results as the first reference image; and generating health status information consistent with the description label corresponding to the first reference image.

12. A device for identifying tongue images, comprising: The second acquisition unit is configured to acquire an image of the tongue to be identified; The third sending unit is configured to send the tongue image to be identified to the target server. The second receiving unit is configured to receive health status information sent by the target server, wherein the health status information is determined based on the device for identifying tongue image as described in claim 11; The display unit is configured to display the tongue image recognition result based on the health status information.

13. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-10.

14. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-10.

Images