Tongue multispectral image generation method, device, equipment and storage medium

Abstract

The present invention discloses a tongue image multi-spectral image generation method, device, equipment and storage medium, the method comprising: acquiring a tongue image detection image through a photographing device; calculating the ambient light and the spectral response of the photographing device according to a preset standard colorimetric card; extracting a tongue image from the tongue image detection image according to a preset neural network model; and spectrally reconstructing the tongue image according to the ambient light and the spectral response of the photographing device, thereby generating a tongue image multi-spectral image. The present invention solves the technical problems in the prior art that the photographing device and the surrounding ambient light interfere greatly with the acquisition of tongue images and the accuracy of the tongue image multi-spectral image is low, and facilitates users to acquire tongue image multi-spectral images anytime and anywhere to conduct tongue image consultation, reduces the user's requirements for the surrounding environmental conditions and the performance of the photographing device, thereby improving the user's experience, and provides accurate and professional image information for traditional Chinese medicine tongue diagnosis, and provides auxiliary clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of medical detection technology, and in particular to a tongue multispectral image generation method, device, equipment and storage medium. Background Art

[0002] Tongue diagnosis is an important part of TCM visual diagnosis. For example, the thickness of the tongue coating and the color of the tongue can reflect the physical condition of the human body and provide important evaluation indicators for clinical medical diagnosis. In clinical diagnosis of tongue images, the tongue images can be roughly divided into 27 types according to the different colors of the tongue coating, and different tongue images correspond to different clinical diagnosis schemes. In addition, the nature, severity, trend, and urgency of the disease can be inferred based on the pattern of tongue color changes.

[0003] At present, TCM tongue diagnosis still relies on visual observation and qualitative description. Due to the differences in color perception among different physicians, TCM tongue diagnosis is greatly affected by subjective factors and lacks objective and scientific evaluation indicators. It is difficult to integrate clinical observation materials from various medical institutions into a system, which greatly affects the in-depth research and clinical application of this work. The existing tongue image acquisition is mainly done by the user taking pictures of the tongue through imaging equipment, and using artificial intelligence technology or image analysis technology to judge the tongue image. This type of tongue diagnosis system does not eliminate the differences in shooting equipment and the influence of ambient light, and is greatly affected by external interference, making it difficult to serve as an accurate indicator for clinical tongue diagnosis.

[0004] Therefore, there is an urgent need for a method that can eliminate the influence of shooting equipment and ambient light and accurately and quickly generate multi-spectral images of the tongue. Summary of the invention

[0005] The present invention provides a tongue multispectral image generation method, device, equipment and storage medium to solve the technical problems in the prior art that the shooting equipment and the surrounding environment light have great interference on the acquisition of tongue images and the accuracy of the tongue multispectral image is low.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a method for generating a tongue multispectral image, comprising:

[0007] Obtain tongue image detection images through a camera;

[0008] Calculating the spectral response of the ambient light and the camera device according to a preset standard colorimetric card;

[0009] Extracting a tongue image from the tongue image detection image according to a preset neural network model;

[0010] The tongue image is spectrally reconstructed according to the ambient light and the spectral response of the camera device, thereby generating a tongue image multispectral image.

[0011] As a preferred solution, the tongue image detection image is obtained by photographing equipment, specifically:

[0012] The tongue and a preset standard colorimetric card aligned with the tongue are photographed by means of a photographing device to obtain a tongue image detection image; wherein the tongue image detection image includes an image of the preset standard colorimetric card area and a tongue image of the tongue.

[0013] As a preferred solution, the spectral response of the ambient light and the camera device is calculated according to a preset standard colorimetric card, specifically:

[0014] Extracting the preset standard colorimetric card area image in the tongue image detection image;

[0015] Converting the preset standard colorimetric card area image into an HSV image format, and performing color partitioning on the converted preset standard colorimetric card area image, thereby respectively extracting the corresponding standard colors in different color partitions;

[0016] The RGB color gamut value of each of the standard colors is calculated, and the spectral response of the ambient light and the shooting device is calculated through an optimization algorithm according to the reflection spectrum curve of each of the standard colors and the RGB color gamut value of each of the standard colors.

[0017] As a preferred solution, the tongue image is extracted from the tongue image detection image according to the preset neural network model, specifically:

[0018] According to a preset neural network model, the tongue image detection image is segmented to generate a mask image of the tongue image;

[0019] According to the mask image of the tongue image, tongue image region information is extracted from the tongue image detection image, thereby obtaining the tongue image.

[0020] As a preferred solution, the method for constructing the preset neural network model includes:

[0021] Build an initial neural network model;

[0022] The loss function and optimizer in the model training process are set, and the initial neural network model is trained using a preset tongue image data set to obtain a trained neural network model.

[0023] As a preferred solution, the tongue image is spectrally reconstructed according to the ambient light and the spectral response of the camera device to generate a tongue multispectral image, specifically:

[0024] Performing spectral reconstruction on each pixel of the tongue image, in the process of spectral reconstruction on any pixel, obtaining the color gamut value of the pixel in the tongue image, combining the color gamut value with the ambient light and the spectral response of the camera, calculating the optimal characteristic coefficient through an optimization algorithm, and calculating the pixel value of the pixel in the multispectral image through a preset characteristic vector based on the characteristic coefficient;

[0025] After the corresponding multi-spectral image pixel value is calculated for each pixel point in the tongue image, tongue spectral images in different bands are generated according to the spectral image pixel value corresponding to each pixel point in different band ranges.

[0026] As a preferred solution, after generating the tongue multispectral image, the method further includes:

[0027] The tongue multispectral image is uploaded to a data server for storage.

[0028] Accordingly, the present invention provides a tongue image multi-spectral image generation device, comprising: an image acquisition module, a standard colorimetric card module, a tongue image module and a spectrum reconstruction module;

[0029] The image acquisition module is used to acquire tongue image detection images through a camera device;

[0030] The standard colorimetric card module is used to calculate the spectral response of the ambient light and the camera device according to a preset standard colorimetric card;

[0031] The tongue image module is used to extract the tongue image from the tongue image detection image according to a preset neural network model;

[0032] The spectral reconstruction module is used to perform spectral reconstruction on the tongue image according to the ambient light and the spectral response of the camera device, so as to generate a tongue multi-spectral image.

[0033] As a preferred solution, the tongue image detection image is obtained by photographing equipment, specifically:

[0034] The tongue and a preset standard colorimetric card aligned with the tongue are photographed by means of a photographing device to obtain a tongue image detection image; wherein the tongue image detection image includes an image of the preset standard colorimetric card area and a tongue image of the tongue.

[0035] As a preferred solution, the spectral response of the ambient light and the camera device is calculated according to a preset standard colorimetric card, specifically:

[0036] Extracting the preset standard colorimetric card area image in the tongue image detection image;

[0037] Converting the preset standard colorimetric card area image into an HSV image format, and performing color partitioning on the converted preset standard colorimetric card area image, thereby respectively extracting the corresponding standard colors in different color partitions;

[0038] The RGB color gamut value of each of the standard colors is calculated, and the spectral response of the ambient light and the shooting device is calculated according to the reflection spectrum curve of each of the standard colors and the RGB color gamut value of each of the standard colors.

[0039] As a preferred solution, the tongue image is extracted from the tongue image detection image according to the preset neural network model, specifically:

[0040] According to a preset neural network model, the tongue image detection image is segmented to generate a mask image of the tongue image;

[0041] According to the mask image of the tongue image, tongue image region information is extracted from the tongue image detection image, thereby obtaining the tongue image.

[0042] As a preferred solution, the method for constructing the preset neural network model includes:

[0043] Build an initial neural network model;

[0044] The loss function and optimizer in the model training process are set, and the initial neural network model is trained using a preset tongue image data set to obtain a trained neural network model.

[0045] As a preferred solution, the tongue image is spectrally reconstructed according to the ambient light and the spectral response of the camera device to generate a tongue multispectral image, specifically:

[0046] Performing spectral reconstruction on each pixel of the tongue image, in the process of spectral reconstruction on any pixel, obtaining the color gamut value of the pixel in the tongue image, combining the color gamut value with the ambient light and the spectral response of the camera, calculating the optimal characteristic coefficient through an optimization algorithm, and calculating the pixel value of the pixel in the multispectral image through a preset characteristic vector based on the characteristic coefficient;

[0047] After the corresponding multi-spectral image pixel value is calculated for each pixel point in the tongue image, tongue spectral images in different bands are generated according to the spectral image pixel value corresponding to each pixel point in different band ranges.

[0048] As a preferred solution, after generating the tongue multispectral image, the method further includes:

[0049] The tongue multispectral image is uploaded to a data server for storage.

[0050] Accordingly, the present invention provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for generating a tongue multispectral image as described in any one of the above is implemented.

[0051] Accordingly, the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the tongue multispectral image generation method as described in any one of the above.

[0052] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0053] The technical solution of the present invention obtains tongue detection images and calculates the spectral response of ambient light and photographing equipment through a preset standard colorimetric card, thereby using the preset standard colorimetric card as a reference to avoid interference of the surrounding environment or photographing equipment on the precision and accuracy of tongue image acquisition, thereby reducing the error of external factors such as environment and equipment in the acquisition of tongue image data, and spectrally reconstructs the tongue image through the spectral response of ambient light and photographing equipment, thereby ensuring that the collected tongue image excludes the influence of the environment and equipment, thereby accurately generating a tongue multi-spectral image, and also facilitating users to obtain tongue multi-spectral images anytime and anywhere to conduct tongue diagnosis, thereby reducing users' requirements for the performance of surrounding environmental conditions and photographing equipment, thereby improving users' usage experience.

[0054] Furthermore, the multispectral imaging technology in the technical solution of the present invention can distinguish different areas of tongue color and fur color, obtain information that cannot be distinguished by the naked eye, and eliminate the influence of different personal perceptions of color. It provides accurate and professional imaging information for traditional Chinese medicine tongue diagnosis and provides auxiliary clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 : A flow chart of the steps of a method for generating a tongue multispectral image provided by an embodiment of the present invention;

[0056] Figure 2 : A schematic diagram of a standard colorimetric card provided in an embodiment of the present invention;

[0057] Figure 3 : A schematic diagram of generating a tongue multispectral image provided by an embodiment of the present invention;

[0058] Figure 4 : A structural diagram of a tongue multispectral image generating device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Embodiment 1

[0061] Please refer to Figure 1 , a tongue multispectral image generation method provided by an embodiment of the present invention includes the following steps S101-S104:

[0062] Step S101: Obtain a tongue detection image through a camera.

[0063] As a preferred solution of this embodiment, the tongue image detection image is obtained by using a camera, specifically:

[0064] The tongue and a preset standard colorimetric card aligned with the tongue are photographed by means of a photographing device to obtain a tongue image detection image; wherein the tongue image detection image includes an image of the preset standard colorimetric card area and a tongue image of the tongue.

[0065] In this embodiment, before taking a photo of the tongue and a preset standard colorimetric card aligned with the tongue, the standard colorimetric card needs to be placed on an appropriate area of ​​the face so that the tongue area in the standard colorimetric card is aligned with the extended tongue, so that a tongue spectral image can be generated directly through the single image taken, and then a diagnosis can be made directly. At the same time, the single image can also ensure that the standard colorimetric card and the tongue are in the same ambient light and shooting equipment conditions, thereby ensuring the accuracy of the tongue image detection image and reducing errors and interference in the tongue image detection caused by surrounding external factors.

[0066] It should be noted that the photographing equipment includes but is not limited to: cameras, camcorders, terminal devices with photographing functions, etc. Preferably, a smart phone can be used to take photos using a mobile phone camera, which can efficiently and accurately collect and acquire images of the tongue and standard colorimetric card. At the same time, the image can be directly processed after acquisition, and then the multi-spectral image of the tongue can be uploaded to the server for diagnosis or diagnosis, thereby improving the efficiency of tongue diagnosis. Multi-spectral images of the tongue can be obtained through a smart phone, providing accurate image information for clinical tongue diagnosis, assisting clinical diagnosis, and also facilitating user use, thereby improving user experience.

[0067] Furthermore, multispectral images of the tongue can be acquired through a single device, which has a simple structure, is easy to use, and is simple to operate. It does not require additional professional instruments, and is helpful to promote the popularization of intelligent tongue diagnosis systems based on multispectral images of the tongue.

[0068] Step S102: Calculate the spectral response of the ambient light and the camera device according to a preset standard colorimetric card.

[0069] As a preferred solution of this embodiment, the spectral response of the ambient light and the camera device is calculated according to a preset standard colorimetric card, specifically:

[0070] Extract the preset standard colorimetric card area image in the tongue detection image; convert the preset standard colorimetric card area image into an HSV image format, and perform color partitioning on the converted preset standard colorimetric card area image, so as to extract the corresponding standard colors in different color partitions respectively; calculate the RGB color gamut value of each of the standard colors, and calculate the spectral response of the ambient light and the shooting device according to the reflection spectrum curve of each of the standard colors and the RGB color gamut value of each of the standard colors.

[0071] Please note that Figure 2 , which is a standard colorimetric card provided by an exemplary embodiment of the present invention, and the rectangular frame in the standard colorimetric card is the standard color block in the colorimetric card. Preferably, the standard color blocks include orange standard color blocks, pink standard color blocks, yellow standard color blocks, green standard color blocks and blue standard color blocks, ensuring that the standard color blocks can cover a larger range of color frequencies, that is, visible light frequencies. Furthermore, for some tongue image recognition and acquisition that require high-precision, the color frequency intervals between the color blocks can be reduced, thereby reducing the range covered by the visible light frequency, as well as the adjustment step size of the color blocks corresponding to the visible light frequency, thereby improving the ability to identify subtle differences in ambient light or subtle performance of the camera equipment. Preferably, an area aligned with the tongue is provided in the standard colorimetric card, and the width of the area is slightly larger than the surface width of the tongue of a normal adult.

[0072] In this embodiment, for example, see Figure 3, the image is template matched by the cv2.matchTemplate function, so as to extract the standard colorimetric card area image in the tongue image detection image. The standard colorimetric card area is converted into the HSV image form, and the different color areas of the HSV image are partitioned by the cv2.inRange function, and the standard colors in the partitions are respectively extracted, so as to accurately obtain the hue (H), saturation (S) and brightness (V) of each standard color; the standard colorimetric card used in this exemplary embodiment is five standard colors. Then the RGB color gamut value of the standard color is calculated. In this exemplary embodiment, the collections.Counter function is used to count the occurrence frequency of the RGB color gamut value of the standard color area one by one, and the Counter.most_common function is used to find the RGB color gamut value with the highest occurrence frequency as the RGB color gamut value of the standard color, and according to the RGB color gamut value of the standard color and the known reflectance spectrum curve of the standard color, the spectral response of the ambient light and the camera device is calculated by the quadratic optimization algorithm. Wherein, preferably, the reflectance spectrum curve of the standard colorimetric card is obtained by measuring the spectrometer.

[0073] Step S103: extracting a tongue image from the tongue image detection image according to a preset neural network model.

[0074] As a preferred solution of this embodiment, the tongue image is extracted from the tongue image detection image according to the preset neural network model, specifically:

[0075] According to a preset neural network model, the tongue image detection image is segmented to generate a tongue image mask map; according to the mask map of the tongue image, tongue image area information is extracted from the tongue image detection image to obtain a tongue image.

[0076] It should be noted that, preferably, the preset neural network model is a Unet neural network model. Figure 3 , by loading the Unet neural network model to perform image segmentation on the tongue image detection image, a mask map of the tongue image is generated, and the tongue image area information of the detection image is extracted through the mask map to obtain the tongue image.

[0077] As a preferred solution of this embodiment, the method for constructing the preset neural network model includes:

[0078] Constructing an initial neural network model; setting a loss function and an optimizer in the model training process, and using a preset tongue image data set to train the initial neural network model to obtain a trained neural network model.

[0079] It should be noted that, preferably, the loss function in the training process is Dice Loss, and the optimizer is AdamOptimizer. The tongue image data set includes tongue images obtained under various ambient lights or different camera devices. The neural network model of this embodiment is mainly used to extract tongue image area information to obtain tongue image.

[0080] Step S104: performing spectral reconstruction on the tongue image according to the ambient light and the spectral response of the camera device, thereby generating a tongue image multispectral image.

[0081] As a preferred solution of this embodiment, the tongue image is spectrally reconstructed according to the ambient light and the spectral response of the camera device to generate a tongue multispectral image, specifically:

[0082] Perform spectral reconstruction on each pixel of the tongue image. In the process of spectral reconstruction of any pixel, obtain the color gamut value of the pixel in the tongue image, combine the color gamut value with the ambient light and the spectral response of the camera, calculate the optimal characteristic coefficient through an optimization algorithm, and calculate the pixel value of the pixel in the multispectral image through a preset characteristic vector based on the characteristic coefficient; until the corresponding multispectral image pixel value is calculated for each pixel in the tongue image, generate tongue spectral images in different bands according to the spectral image pixel value corresponding to each pixel in different band ranges.

[0083] In this example, see Figure 3 , obtain the color gamut value of a certain pixel in the tongue image, and combine the color gamut value and the spectral response of the ambient light and the camera device to obtain the optimal characteristic coefficient through the quadratic optimization algorithm, and then use the trained preset characteristic vector to obtain the multispectral image pixel value according to the characteristic coefficient. Traverse each pixel in the tongue image, and obtain the multispectral image pixel values ​​of all pixels through the above steps, so as to generate a tongue multispectral image.

[0084] As a preferred solution of this embodiment, after generating the tongue multispectral image, the method further includes:

[0085] The tongue multispectral image is uploaded to a data server for storage.

[0086] It can be understood that by uploading the acquired tongue multispectral images to the data server, it can facilitate users to check the tongue multispectral data. At the same time, through the server platform, etc., the tongue multispectral images can be diagnosed, thereby realizing online diagnosis and improving user convenience.

[0087] Furthermore, a multispectral image generation algorithm is used instead of the traditional multispectral system to generate multispectral images of the tongue, which reduces the cost of use and improves convenience. At the same time, the data of each test will be automatically recorded, and the long-term change trend of the tongue image can be viewed through the client platform, server platform, APP, etc., dynamic monitoring can be carried out, and the level of digital medical care can be improved.

[0088] It can be understood that this embodiment generates a multispectral image of the tongue based on the fact that tissues of different colors on the tongue have obvious differences in absorption and reflection of light of different spectra, and obtains specific information on the tongue that is difficult to obtain with the naked eye or simple RGB based on the multispectral image, and applies it to the clinical diagnosis of the tongue, thereby enabling multispectral imaging technology to be used as a standard scientific evaluation indicator for clinical diagnosis of the tongue.

[0089] Implementing the above embodiments has the following effects:

[0090] The technical solution of the present invention obtains tongue detection images and calculates the spectral response of ambient light and photographing equipment through a preset standard colorimetric card, thereby using the preset standard colorimetric card as a reference to avoid interference of the surrounding environment or photographing equipment on the precision and accuracy of tongue image acquisition, thereby reducing the error of external factors such as environment and equipment in the acquisition of tongue image data, and spectrally reconstructs the tongue image through the spectral response of ambient light and photographing equipment, thereby ensuring that the collected tongue image excludes the influence of the environment and equipment, thereby accurately generating a tongue multi-spectral image, and also facilitating users to obtain tongue multi-spectral images anytime and anywhere to conduct tongue diagnosis, thereby reducing users' requirements for the performance of surrounding environmental conditions and photographing equipment, thereby improving users' usage experience.

[0091] Furthermore, the multispectral imaging technology in the technical solution of the present invention can distinguish different areas of tongue color and fur color, obtain information that cannot be distinguished by the naked eye, and eliminate the influence of different personal perceptions of color. It provides accurate and professional imaging information for traditional Chinese medicine tongue diagnosis and provides auxiliary clinical diagnosis.

[0092] Embodiment 2

[0093] The present invention provides a tongue image multi-spectral image generating device, comprising: an image acquisition module 201, a standard colorimetric card module 202, a tongue image module 203 and a spectrum reconstruction module 204.

[0094] The image acquisition module 201 is used to acquire tongue image detection images through a camera.

[0095] The standard colorimetric card module 202 is used to calculate the spectral response of the ambient light and the camera device according to a preset standard colorimetric card.

[0096] The tongue image module 203 is used to extract the tongue image from the tongue image detection image according to a preset neural network model.

[0097] The spectral reconstruction module 204 is used to perform spectral reconstruction on the tongue image according to the ambient light and the spectral response of the camera device, so as to generate a tongue multi-spectral image.

[0098] As a preferred solution of this embodiment, the tongue image detection image is obtained by using a camera, specifically:

[0099] The tongue and a preset standard colorimetric card aligned with the tongue are photographed by means of a photographing device to obtain a tongue image detection image; wherein the tongue image detection image includes an image of the preset standard colorimetric card area and a tongue image of the tongue.

[0100] As a preferred solution of this embodiment, the spectral response of the ambient light and the camera device is calculated according to a preset standard colorimetric card, specifically:

[0101] Extract the preset standard colorimetric card area image in the tongue detection image; convert the preset standard colorimetric card area image into an HSV image format, and perform color partitioning on the converted preset standard colorimetric card area image, so as to extract the corresponding standard colors in different color partitions respectively; calculate the RGB color gamut value of each of the standard colors, and calculate the spectral response of the ambient light and the shooting device according to the reflection spectrum curve of each of the standard colors and the RGB color gamut value of each of the standard colors.

[0102] As a preferred solution of this embodiment, the tongue image is extracted from the tongue image detection image according to the preset neural network model, specifically:

[0103] According to a preset neural network model, the tongue image detection image is segmented to generate a mask image of the tongue image; according to the mask image of the tongue image, tongue image area information is extracted from the tongue image detection image to obtain a tongue image.

[0104] As a preferred solution of this embodiment, the method for constructing the preset neural network model includes:

[0105] Constructing an initial neural network model; setting a loss function and an optimizer in the model training process, and using a preset tongue image data set to train the initial neural network model to obtain a trained neural network model.

[0106] As a preferred solution of this embodiment, the tongue image is spectrally reconstructed according to the ambient light and the spectral response of the camera device to generate a tongue multispectral image, specifically:

[0107] Perform spectral reconstruction on each pixel of the tongue image. In the process of spectral reconstruction of a pixel, obtain the color gamut value of the pixel in the tongue image, combine the color gamut value with the ambient light and the spectral response of the camera, calculate the optimal characteristic coefficient through an optimization algorithm, and calculate the pixel value of the pixel in the multispectral image through a preset characteristic vector based on the characteristic coefficient; until the corresponding multispectral image pixel value is calculated for each pixel in the tongue image, generate tongue spectral images in different bands according to the multispectral image pixel value corresponding to each pixel in different band ranges.

[0108] As a preferred solution of this embodiment, after generating the tongue multispectral image, the method further includes:

[0109] The tongue multispectral image is uploaded to a data server for storage.

[0110] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0111] Implementing the embodiments of the present invention has the following effects:

[0112] The technical solution of the present invention obtains the tongue image detection image, and calculates the spectral response of the ambient light and the camera device through the preset standard colorimetric card, so as to avoid the interference of the surrounding environment or the camera device on the precision and accuracy of the tongue image acquisition by the preset standard colorimetric card as a reference, thereby reducing the error of the tongue image data acquisition caused by external factors such as the environment and the device, and reconstructs the tongue image spectrally through the spectral response of the ambient light and the camera device, thereby ensuring that the collected tongue image excludes the influence of the environment and the device, thereby accurately generating the tongue image multispectral image, and at the same time, it is convenient for users to obtain the tongue image multispectral image anytime and anywhere to conduct tongue image consultation, reducing the user's requirements for the surrounding environment conditions and the performance of the camera device, thereby improving the user's experience, and at the same time, through the multispectral imaging technology in the technical solution of the present invention, the areas of different tongue colors and moss colors are distinguished, and the information that cannot be distinguished by the naked eye is obtained, eliminating the influence of different personal perceptions of color. Provide accurate and professional image information for Chinese medicine tongue diagnosis and provide auxiliary clinical diagnosis.

[0113] Embodiment 3

[0114] Correspondingly, the present invention also provides a terminal device, comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for generating a tongue multispectral image as described in any one of the above embodiments is implemented.

[0115] The terminal device of this embodiment includes: a processor, a memory, and a computer program and a computer instruction stored in the memory and executable on the processor. When the processor executes the computer program, each step in the above-mentioned embodiment 1 is implemented, for example: Figure 1 Alternatively, when the processor executes the computer program, the functions of each module / unit in the above device embodiment, such as the tongue image module 203, are implemented.

[0116] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the terminal device. For example, the tongue image module 203 is used to extract a tongue image from the tongue image detection image according to a preset neural network model.

[0117] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. The terminal device may include more or fewer components than shown in the diagram, or may combine certain components, or different components. For example, the terminal device may also include an input / output device, a network access device, a bus, etc.

[0118] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0119] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile terminal, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0120] Wherein, if the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0121] Embodiment 4

[0122] Correspondingly, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the tongue multispectral image generation method as described in any one of the above embodiments.

[0123] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating a tongue multispectral image, characterized in that: include: By means of a photographing device, a tongue and a preset standard colorimetric card aligned with the tongue are photographed to obtain a tongue image detection image; wherein the tongue image detection image includes an image of the preset standard colorimetric card area and a tongue image of the tongue; Extracting the preset standard colorimetric card area image in the tongue image detection image; Converting the preset standard colorimetric card area image into an HSV image format, and performing color partitioning on the converted preset standard colorimetric card area image, thereby respectively extracting the corresponding standard colors in different color partitions; Calculating the RGB color gamut value of each of the standard colors, and calculating the spectral response of the ambient light and the camera device according to the reflection spectrum curve of each of the standard colors and the RGB color gamut value of each of the standard colors; extracting a tongue image from the tongue image detection image according to a preset neural network model; The tongue image is spectrally reconstructed according to the ambient light and the spectral response of the camera device, thereby generating a tongue image multispectral image.

2. The method for generating a tongue multispectral image according to claim 1, characterized in that: The step of extracting the tongue image from the tongue image detection image according to the preset neural network model is specifically as follows: According to a preset neural network model, the tongue image detection image is segmented to generate a mask image of the tongue image; According to the mask image of the tongue image, tongue image region information is extracted from the tongue image detection image, thereby obtaining the tongue image.

3. The method for generating a tongue multispectral image according to claim 2, characterized in that: The method for constructing the preset neural network model includes: Build an initial neural network model; The loss function and optimizer in the model training process are set, and the initial neural network model is trained using a preset tongue image data set to obtain a trained neural network model.

4. The method for generating a tongue multispectral image according to claim 3, characterized in that: The tongue image is spectrally reconstructed according to the ambient light and the spectral response of the camera device to generate a tongue image multispectral image, specifically: Performing spectral reconstruction on each pixel of the tongue image, in the process of spectral reconstruction on any pixel, obtaining the color gamut value of the pixel in the tongue image, combining the color gamut value with the ambient light and the spectral response of the camera, calculating the optimal characteristic coefficient through an optimization algorithm, and calculating the pixel value of the pixel in the multispectral image through a preset characteristic vector based on the characteristic coefficient; After the corresponding multi-spectral image pixel value is calculated for each pixel point in the tongue image, the tongue spectral images in different bands are generated according to the spectral image pixel value corresponding to each pixel point in different band ranges.

5. A tongue multispectral image generation method according to any one of claims 1 to 4, characterized in that: After generating the tongue multispectral image, it also includes: The tongue multispectral image is uploaded to a data server for storage.

6. A tongue multispectral image generating device, characterized in that: include: Image acquisition module, standard colorimetric card module, tongue image module and spectrum reconstruction module; The image acquisition module is used to take a photo of the tongue and a preset standard colorimetric card aligned with the tongue through a photographing device, so as to obtain a tongue image detection image; wherein the tongue image detection image includes an image of the preset standard colorimetric card area and a tongue image of the tongue; The standard colorimetric card module is used to extract the preset standard colorimetric card area image in the tongue image detection image, convert the preset standard colorimetric card area image into an HSV image format, and perform color partitioning on the converted preset standard colorimetric card area image, thereby respectively extracting the corresponding standard colors in different color partitions, calculating the RGB color gamut value of each of the standard colors, and calculating the spectral response of the ambient light and the camera device according to the reflection spectrum curve of each of the standard colors and the RGB color gamut value of each of the standard colors; The tongue image module is used to extract the tongue image from the tongue image detection image according to a preset neural network model; The spectral reconstruction module is used to perform spectral reconstruction on the tongue image according to the ambient light and the spectral response of the camera device, so as to generate a tongue multi-spectral image.

7. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for generating a tongue multispectral image as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for generating a tongue multispectral image as described in any one of claims 1 to 5.

Images