Collection device for hyperspectral and structured light images of traditional Chinese medicine lingual face diagnosis and image collection method thereof

Abstract

The embodiment of the invention provides an acquisition device for hyperspectral and structured light images of traditional Chinese medicine lingual face diagnosis and an image acquisition method thereof, and belongs to the technical field of image acquisition. The acquisition device comprises: a first image acquisition module configured to acquire a biomarker image in a tongue or a face, wherein the first image acquisition module comprises a light source part and a hyperspectral camera which are integrated together; the second image acquisition module is configured to acquire a three-dimensional contour image of the surface of the tongue or the face; the third image acquisition module is configured to acquire a tongue or face surface feature color image; the light source part emits light with different wavelengths to irradiate the tongue or the face and reflect the light, the hyperspectral camera receives the reflected light with different wavelengths to form biomarker images of the same tongue or the face under different spectral bands, and the biomarker images, the three-dimensional contour image and the color image jointly form a multi-feature fusion lingual surface image. The hyperspectral camera, the structured light camera and the visible light camera are integrated to obtain a multi-feature fusion lingual surface image at a time, and the efficiency and accuracy of traditional Chinese medicine inspection diagnosis are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image acquisition, and particularly relates to a device for acquiring hyperspectral and structured light images of traditional Chinese medicine tongue and face diagnosis and an image acquisition method thereof. Background Art

[0002] Traditional Chinese medicine inspection mainly relies on doctors' direct observation of the inquirer and doctors' diagnosis experience to judge the health status or disease condition of the inquirer. During the process of traditional Chinese medicine inspection, doctors mainly infer the functional state and pathological changes of the internal organs in the inquirer's body by observing the subtle changes in the overall face, local facial features, tongue and other parts of the inquirer. These subtle changes usually include, for example, the tone and texture of the skin on the face or tongue, the blood circulation condition under the skin, the pigment distribution of the skin, etc.

[0003] However, this traditional Chinese medicine inspection method, on the one hand, needs to rely on doctors' experience and diagnosis level, and on the other hand, the environmental conditions during the inspection process, such as light conditions, temperature changes, etc., are also important factors affecting doctors' judgment of the inquirer's situation. This leads to obvious subjectivity and limitations in the traditional Chinese medicine inspection method.

[0004] Currently, with the development of optical imaging technology and its extensive application in the medical field, traditional Chinese medicine inspection technology is gradually developing towards digitalization and intelligence through integration with optical imaging technology. For example, high-resolution images are obtained through visible light technology to replace doctors' subjective observation, and surface features such as the tone and texture of the target skin can be accurately captured through image data.

[0005] However, the research on the intelligence and digitalization of traditional Chinese medicine inspection is still in the initial basic research stage. In particular, there is a lack of feasible technical solutions and in-depth research on how to diversely combine optical imaging technology and equipment with traditional Chinese medicine inspection to obtain multi-dimensional biological information about the skin of the inquirer's face and tongue.

[0006] In view of this, a new device for acquiring hyperspectral and structured light images of traditional Chinese medicine tongue and face diagnosis and an image acquisition method thereof need to be proposed to fully or partially solve the above problems. Summary of the Invention

[0007] To address at least one of the above problems and deficiencies in the prior art, embodiments of the present invention provide an acquisition device and an image acquisition method for hyperspectral and structured light images of traditional Chinese medicine tongue and face diagnosis. By integrating a hyperspectral camera, a structured light camera, and a visible light camera on the same acquisition device, images of the face and tongue of the person being examined are collected at one time, and simultaneously, a multi-spectral imaging biomarker image, a three-dimensional contour image obtained by structured light imaging, and a color image taken by the visible light camera are obtained, providing objective, detailed, and multi-dimensional tongue and face image data for traditional Chinese medicine inspection to improve the accuracy and efficiency of traditional Chinese medicine inspection. The technical solutions are as follows:

[0008] According to one aspect of the present invention, there is provided an acquisition device for hyperspectral and structured light images of traditional Chinese medicine tongue and face diagnosis. The acquisition device includes:

[0009] A first image acquisition module configured to acquire biomarker images inside the tongue or face. Among them, the first image acquisition module includes a light source part and a hyperspectral camera integrated together, and the hyperspectral camera is arranged on the backlight side of the light source part;

[0010] A second image acquisition module configured to acquire three-dimensional contour images of the surface of the tongue or face. Among them, the second image acquisition module includes a projector and at least one structured light camera integrated together;

[0011] A third image acquisition module configured to acquire color images of the surface features of the tongue or face. Among them, the third image acquisition module includes at least one visible light camera;

[0012] When the light source part emits light of different wavelengths to irradiate the tongue or face and reflection occurs, the hyperspectral camera receives the light of different wavelengths reflected by it to form biomarker images of the same tongue or the same face in different spectral bands. The biomarker images, three-dimensional contour images, and color images together form a multi-feature fusion face and tongue image.

[0013] In some embodiments, specifically, the first image acquisition module is further configured as follows: the light source part includes an annular light source plate and at least one group of lamp bead groups arranged on the annular light source plate. Each group of lamp bead groups in the at least one group of lamp bead groups includes a first lamp bead group for providing white light and a second lamp bead group for providing different wavelengths. The lamp beads in each group of lamp bead groups are arranged in a ring around the center of the annular light source plate.

[0014] In some embodiments, further, the lens of the hyperspectral camera is coaxially arranged with the annular light source plate. The lens direction of the hyperspectral camera is the same as the light emission direction of the light source part.

[0015] In some embodiments, specifically, the second image acquisition module is further configured such that at least one structured light camera is disposed on the backlight side of the light source unit, and the lens direction of each structured light camera in the at least one structured light camera is the same as the light emission direction of the light source unit.

[0016] In some embodiments, further, the at least one structured light camera includes a first structured light camera and a second structured light camera, and a preset distance is provided between the first structured light camera and the second structured light camera.

[0017] In some embodiments, specifically, the first light bead group includes at least one light bead for providing white light. At least one preset wavelength value is set in the second light bead group, and at least one light bead of a preset wavelength is provided corresponding to each preset wavelength value in the at least one preset wavelength value. Wherein, the range of the preset wavelength value is 390 nm to 940 nm. The light beads in each light bead group are arranged in a clockwise or counterclockwise order from the light bead providing white light in ascending order of the preset wavelength value.

[0018] In some embodiments, preferably, the number of light beads in the first light bead group is equal to the number of light beads in each preset wavelength value in the second light bead group. At least one light bead group is at least two light bead groups, and each light bead group in the at least two light bead groups is arranged on the side away from the center of the annular light source board after the previous light bead group is arranged on the annular light source board.

[0019] In some embodiments, alternatively, the light source unit further includes an annular reflector disposed on the annular light source board. Wherein, the annular reflector covers the surface of the annular light source board on which the light beads are disposed. A coating film for improving the diffuse reflectivity of the light source is provided on the surface of the annular reflector. The coating film is a micro-nano material with a high reflectivity.

[0020] In some embodiments, specifically, the at least one structured light camera acquires depth information at different positions on the surface of the tongue or face.

[0021] In some embodiments, specifically, the acquisition device further includes a control module, and the control module is configured to respectively control the startup and stop of the first image acquisition module, the second image acquisition module, and the third image acquisition module, and control the light beads in at least one light bead group of the light source unit to emit light and go out according to a preset light source emission instruction.

[0022] In some embodiments, preferably, the acquisition device further includes a housing for fixedly mounting the first image acquisition module, the second image acquisition module, the third image acquisition module, and the control module. An open end is provided at one end of the housing in the light emission direction of the light source unit, and a head bracket for fixing the position of the tongue or face is provided on the open end.

[0023] According to another aspect of the present invention, there is provided an image acquisition method for high - spectrum and structured - light image acquisition of tongue and face diagnosis in traditional Chinese medicine. This image acquisition method uses the acquisition device described in the above - mentioned aspect to obtain a tongue - surface image with multi - feature fusion. The image acquisition method includes the following specific steps:

[0024] Generate a preset light - source emission instruction according to the requirements of tongue - surface inspection and input the preset light - source emission instruction into the control module;

[0025] Initialize the first image acquisition module, the second image acquisition module, and the third image acquisition module through the control module;

[0026] Generate a hyperspectral camera control thread, a structured - light camera control thread, and a visible - light camera control thread corresponding to the first image acquisition module, the second image acquisition module, and the third image acquisition module through the control module;

[0027] The control module controls the lamp beads of the light - source part to emit light and go out within one light - source emission cycle according to the preset light - source emission instruction;

[0028] When the light source emits light, start the first image acquisition module through the hyperspectral camera control thread to acquire the biometric image inside the tongue or face, and start the third image acquisition module through the visible - light camera control thread to acquire the color image of the surface features of the tongue or face;

[0029] When the light source stops emitting light, start the second image acquisition module through the structured - light camera control thread to acquire the three - dimensional contour image of the surface of the tongue or face.

[0030] In some embodiments, specifically, when all the lamp beads of the light - source part sequentially include the lamp beads of the first wavelength to the Nth wavelength according to their arrangement order in the light - source part, where N is an integer greater than or equal to 1, the preset light - source emission instruction includes:

[0031] Set the completion of light emission of all lamp beads from the first wavelength to the Nth wavelength in sequence as one light - source emission cycle;

[0032] Within one light - source emission cycle, all lamp beads emit light in sequence according to the order of the first wavelength to the Nth wavelength;

[0033] Set the preset exposure time corresponding to the lamp beads of each wavelength from the first wavelength to the Nth wavelength when they emit light;

[0034] Set the preset light - emission interval time between the lamp beads of two adjacent wavelengths.

[0035] The image acquisition device and its image acquisition method for high - spectrum and structured - light images of tongue diagnosis in traditional Chinese medicine provided by the embodiments of the present invention have at least one or a part of at least one of the following advantages:

[0036] (1) By integrating a hyperspectral camera, a structured light camera, and a visible light camera on the same acquisition device, the face and tongue of the person being acquired are imaged at one time, and a multi-spectral imaging biomarker image, a three-dimensional contour image, and a color image are obtained simultaneously, providing objective, detailed, and multi-dimensional tongue and face image data for traditional Chinese medicine inspection by inspection to improve the efficiency and accuracy of traditional Chinese medicine inspection by inspection;

[0037] (2) By combining a hyperspectral camera, a structured light camera, and a visible light camera, the disadvantage that a visible light camera can only capture the surface image information of the tongue and / or face of the person being acquired can be effectively compensated for, and deep capture of the physiological changes of the tongue and / or face can be achieved based on the surface image information of the tongue and / or face, so as to obtain multi-dimensional tongue and face image information data;

[0038] (3) Through the first image acquisition module integrated with a hyperspectral camera and a multi-wavelength light source unit, biomarker images corresponding to different spectral bands can be obtained within a large wavelength range by adjusting the emission wavelength of the light source unit, so as to realize in-depth exploration of physiological characteristics such as blood vessels and lymph nodes under the skin of the face and tongue, which has significant guiding value for the discovery of potential diseases and the early judgment of diseases;

[0039] (4) By setting and adjusting the wavelength of the lamp beads and the arrangement layout of the lamp bead groups in the light source unit, the light source can be adaptively adjusted according to various tongue and face inspection requirements in the same acquisition device, and multi-dimensional tongue and face image data corresponding to the tongue and face inspection requirements can be obtained, thereby improving the imaging stability and image acquisition efficiency of the acquisition device;

[0040] (5) By adjusting the number of lamp beads and the arrangement layout of the lamp bead groups, it can generally adapt to most traditional Chinese medicine inspection scenarios and tongue and face inspection requirements, while obtaining more objective and comprehensive image information, reducing the acquisition difficulty and equipment investment cost, and having a wide application prospect;

[0041] (6) By performing three-dimensional contour imaging on the face and tongue through a structured light camera, precise three-dimensional modeling of the face and tongue of the person being acquired can be carried out, providing rich micro-physiological characteristic changes from the structural information level, and improving the image information acquisition sensitivity and accuracy of the acquisition device;

[0042] (7) Through the control module and by using the image data processing modules configured inside the hyperspectral camera, the structured light camera, and the visible light camera, the processes of the light source and image acquisition can be integratedly controlled, effectively shortening the interrogation time of traditional Chinese medicine inspection by inspection and the acquisition time of obtaining image data, greatly improving the interrogation efficiency, shortening the interrogation time of the person being acquired, and improving the quality of medical services. Description of the Drawings

[0043] These and / or other aspects and advantages of the present invention will become apparent and readily understood from the following description of the preferred embodiments in conjunction with the accompanying drawings, wherein:

[0044] Figure 1 is a flowchart of an image acquisition method according to an embodiment of the present invention;

[0045] Figure 2 is a schematic diagram of the control logic inside an acquisition device according to an embodiment of the present invention;

[0046] Figure 3 is a schematic diagram of a biometric marker image of the face of a person to be acquired obtained by using the first image acquisition module of the acquisition device according to various embodiments of the present invention;

[0047] Figure 4 is a schematic diagram of a three-dimensional contour image of the tongue of a person to be acquired obtained by using the second image acquisition module of the acquisition device according to various embodiments of the present invention;

[0048] Figure 5 is a schematic diagram of the structure of an acquisition device according to an embodiment of the present invention;

[0049] Figure 6 is Figure 5 an exploded view of the structure of the acquisition device shown;

[0050] Figure 7 is a schematic diagram of an arrangement layout of lamp beads of a light source part of an acquisition device according to an embodiment of the present invention;

[0051] Figure 8 is a schematic diagram of the working process of a hyperspectral camera of an acquisition device according to an embodiment of the present invention;

[0052] Figure 9 is a schematic diagram of the structure of a second image acquisition module of an acquisition device according to an embodiment of the present invention. Detailed Embodiments

[0053] Hereinafter, through embodiments and in conjunction with the accompanying drawings, the technical solutions of the present invention will be further specifically described. In the specification, the same or similar reference numerals indicate the same or similar components. The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be construed as a limitation to the present invention.

[0054] In the process of traditional Chinese medicine inspection by observation, in order to obtain objective, detailed and multi-dimensional tongue surface image data to improve the efficiency and accuracy of traditional Chinese medicine inspection by observation, each embodiment of the present invention integrates a hyperspectral camera, a structured light camera and a visible light camera on the same acquisition device, and controls and adjusts the light-emitting mode of the light source through a control module, so as to achieve simultaneous acquisition of multi-spectral imaging biomarker images, three-dimensional contour images and color images of the face and tongue of the person to be acquired at one time.

[0055] See Figure 1 , which shows the specific steps of the image acquisition method according to an embodiment of the present invention. See Figure 2 , which shows the control logic of the control module 40 of the acquisition device 100 (the specific structure of the acquisition device 100 is as shown in Figure 5 and Figure 6 ). Combining Figure 1 and Figure 2 shown, the image acquisition method includes the following specific steps:

[0056] Step S100: Generate a preset light source emission instruction according to the tongue surface inspection requirements and input the preset light source emission instruction into the control module 40 of the acquisition device 100;

[0057] Step S200: Initialize the first image acquisition module 10, the second image acquisition module 20 and the third image acquisition module 30 through the control module 40;

[0058] Step S300: Generate a hyperspectral camera control thread L1, a structured light camera control thread L2 and a visible light camera control thread L3 corresponding to the first image acquisition module 10, the second image acquisition module 20 and the third image acquisition module 30 through the control module 40;

[0059] Step S400: The control module 40 controls the lamp beads of the light source unit 11 to emit light and go out within a light source emission cycle according to the preset light source emission instruction;

[0060] Step S500: When the light source emits light, start the first image acquisition module 10 through the hyperspectral camera control thread L1 to acquire the biomarker image inside the tongue or face, and start the third image acquisition module 30 through the visible light camera control thread L3 to acquire the color image of the surface features of the tongue or face;

[0061] Step S600: When the light source stops emitting light, start the second image acquisition module 20 through the structured light camera control thread L2 to acquire the three-dimensional contour image of the surface of the tongue or face.

[0062] The complete image acquisition working process of the acquisition device 100 includes:

[0063] After starting the acquisition device 100, the first image acquisition module 10, the second image acquisition module 20, and the third image acquisition module 30 are first initialized, and the current acquisition parameters are set according to the current tongue inspection requirements and / or the specific requirements of the interrogation process.

[0064] During the current process of photographing and acquiring the face and / or tongue of the person being acquired, regardless of whether the light source is emitting light, the control module 40 controls the first image acquisition module 10 and the third image acquisition module 30 to always be in the acquisition working state and continuously acquire pictures at high speed. Among them, the first image acquisition module 10 obtains the biological marker image under the skin of the face and / or tongue of the person being acquired, and the third image acquisition module 30 obtains the surface feature color image of the face and / or tongue of the person being acquired.

[0065] The light-emitting light source provided by the light source unit 11 is actually realized by lighting several groups of lamp beads provided thereon. The lamp bead groups are controlled to be lit in sequence by the control switch in the control module 40, and there is a lighting interval time between every two adjacent groups of lamp beads. That is to say, the previous group of lamp beads is lit for a period of time and then extinguished first, and after waiting for a period of time, the next group of lamp beads is lit again, and so on.

[0066] During the waiting period of each group of lamp beads, the second image acquisition module 20 is turned on to take pictures to obtain the three-dimensional contour image of the surface of the face and / or tongue of the person being acquired.

[0067] In one example, according to steps S100 - S600, see Figure 3 , which shows the biological marker image of the face of the person being acquired obtained by the first image acquisition module 10 of the acquisition device 100. Among them, by centrally photographing and acquiring the face of the person being acquired at one time, a plurality of spectral images of different bands are obtained, and on the spectral images of different bands, the presentation of the texture, skin color, markings, etc. of the same part of the face can be seen to be different. By fusing the spectral images of different bands, more and more comprehensive biological marker information under the facial skin can be obtained.

[0068] In one example, according to steps S100 - S600, see Figure 4 , which shows the three-dimensional contour image of the tongue of the person being acquired obtained by the second image acquisition module 20 of the acquisition device 100. Through the fusion and reconstruction of point cloud data, the three-dimensional morphology, rich texture information, and depth information of the tongue of the person being acquired are formed.

[0069] The multi-spectral imaging technology, 3D structured light imaging technology, and visible light imaging technology are combined in the same acquisition device 100 for tongue and face inspection in traditional Chinese medicine. Through multi-level and multi-dimensional image information fusion, it can provide a more comprehensive, objective, and detailed basis for judgment in traditional Chinese medicine tongue and face inspection, enabling doctors to quickly and accurately obtain more potential information for judging the physical condition of the person being examined.

[0070] See Figure 5 , which shows the structure of the acquisition device 100 according to an embodiment of the present invention. The acquisition device 100 mainly consists of five main parts: a first image acquisition module 10, a second image acquisition module 20, a third image acquisition module 30, a control module 40, and a housing 50. Among them, the housing 50 has an open end, and the first image acquisition module 10, the second image acquisition module 20, the third image acquisition module 30, and the control module 40 are integrated inside the housing 50. When the person being examined places their head on the head bracket 51 at the open end of the housing 50, the first image acquisition module 10, the second image acquisition module 20, and the third image acquisition module 30 sequentially take pictures of their face and / or tongue and record the taken pictures and image data, thus obtaining tongue and face image data containing multi-dimensional information at one time.

[0071] Specifically, the first image acquisition module 10 is configured to acquire the internal biological marker images of the tongue or face, especially under the skin of the tongue or face. Among them, the first image acquisition module 10 further includes a light source unit 11 and a hyperspectral camera 12 integrated together, and the hyperspectral camera 12 is arranged on the backlight side of the light source unit 11.

[0072] The second image acquisition module 20 is configured to acquire the three-dimensional contour images of the surface of the tongue or face. Among them, the second image acquisition module 20 further includes a projector 21 and at least one structured light camera 22 integrated together.

[0073] The third image acquisition module 30 is configured to acquire the color images of the surface features of the tongue or face. Among them, the third image acquisition module 30 includes at least one visible light camera 31.

[0074] When the light source unit 11 emits light of different wavelengths to irradiate the tongue or face and reflection occurs, the hyperspectral camera 12 receives the light of different wavelengths reflected by it to form biological marker images of the same tongue or the same face under different spectral bands. The biological marker images, three-dimensional contour images, and color images together form a multi-feature fused tongue and face image.

[0075] See Figure 6 , which shows Figure 5 the exploded view of the structure of the acquisition device 100 as shown in Figure 5 and Figure 6As shown below, the first image acquisition module 10, the second image acquisition module 20, the third image acquisition module 30, and the control module 40 will be described in more detail respectively.

[0076] Hyperspectral imaging technology is an optoelectronic imaging technology that can simultaneously acquire multi-band spectral features and spatial image information. The wavelength range of the spectral features it can acquire covers a wider spectral band beyond the visible light range of the human eye. Therefore, through the comprehensive spectral information formed by these extensive multi-band spectral features and spatial images, biological marker information such as the blood circulation status, pigment distribution, and texture changes under the skin of the face and / or tongue of the person being collected can be analyzed in depth.

[0077] In one example, in combination with Figure 5 and Figure 6 As shown, the first image acquisition module 10 includes a light source unit 11 and a hyperspectral camera 12, and is integrated with the control module 40. It adjusts the light of different wavelengths emitted by the light source unit 11, controls the light source unit 11 to turn on or off, and controls the interval time of each emission of the light source unit 11 through circuit connection and / or signal connection. Among them, the light source unit 11 is used to emit light of different wavelengths to the position where the tongue or face (including the whole face and local face) of the person being collected is located. The hyperspectral camera 12 is arranged behind the light source unit 11 and is used to receive the light reflected from the tongue or face (including the whole face and local face) of the person being collected by the light emitted by the light source unit 11 and obtain spectral feature information and spatial spectral images through the reflected light.

[0078] In one example, specifically, the light source unit 11 includes an annular light source board 111 and at least one set of lamp bead groups 112 arranged on the annular light source board 111. Refer to Figure 7 , which shows the structure of an embodiment in which the light source board 111 and at least one set of lamp bead groups 112 are assembled together. Among them, each set of lamp bead groups in the at least one set of lamp bead groups includes a first lamp bead group 1121 for providing white light (i.e., mixed visible light with a wavelength range of 390nm - 780nm) and a second lamp bead group 1122 for providing different wavelengths. The lamp beads in each set of lamp bead groups are arranged in a ring around the center of the annular light source board 111.

[0079] In one example, further, the first light bead group 1121 includes at least one light bead for providing white light. At least one preset wavelength value is set in the second light bead group 1122, and at least one light bead of this preset wavelength is set for each preset wavelength value in the at least one preset wavelength value. The range of the preset wavelength value is 390nm to 940nm. The light beads in each light bead group are arranged evenly in the order of increasing preset wavelength value from the light bead providing white light counterclockwise or clockwise. The specific number of light beads set in any one light bead group can be custom-designed according to the specific requirements of actual tongue surface inspection. In any one light bead group, 1, 2, 3 or more light beads can be set. Usually, at least 2 light beads are set. On the one hand, it can effectively prevent a light bead of a certain wavelength from being damaged during the use of the acquisition device 100 without a substitute light bead of the same wavelength. On the other hand, when processing the acquired image data later, multiple light beads of the same wavelength work simultaneously, which can also ensure the reliability of the acquired data and contribute to the analysis of the image data. The setting of the number of light beads only provides some illustrative examples here, and those skilled in the art should not understand it as a limitation of the present invention.

[0080] In one example, preferably, the number of light beads in the first light bead group 1121 is equal to the number of light beads in each preset wavelength value in the second light bead group 1122. That is to say, if N light beads (N is an integer greater than or equal to 1) are set in the first light bead group 1121, then the number of light beads set in each light bead group corresponding to each wavelength value in the second light bead group 1122 is also N. The purpose of such a setting is to evenly arrange all the light beads on the annular light source board 111. When the light beads of each wavelength are sequentially started to be lit, the number of lit light beads each time is the same, so the parameters such as the uniformity and intensity of each emission are the same.

[0081] In one example, preferably, the number of groups of at least one light bead group 112 is at least two groups. Among them, each light bead group 112 in the at least two light bead groups is arranged on the side away from the center of the annular light source board 111 of the previous light bead group after the previous light bead group is arranged on the annular light source board 111.

[0082] In one example, specifically, as Figure 7As shown, the specific structure of an embodiment of the lamp bead arrangement is exemplarily presented. The lamp beads can be, for example, LED lamp beads. Among them, the first lamp bead group 1121 is used to provide white light. The second lamp bead group 1122 sets 14 preset wavelength values according to the range of preset wavelength values, and sets a group of lamp beads for each of the 14 preset wavelength values, successively including a 390nm lamp bead group a, a 420nm lamp bead group b, a 450nm lamp bead group c, a 525nm lamp bead group d, a 570nm lamp bead group e, a 630nm lamp bead group f, a 660nm lamp bead group g, a 730nm lamp bead group h, a 780nm lamp bead group i, an 850nm lamp bead group j, an 870nm lamp bead group k, an 890nm lamp bead group l, a 910nm lamp bead group m, and a 940nm lamp bead group n.

[0083] In one example, further, two lamp bead groups 112, namely the first lamp bead group 112a and the second lamp bead group 112b, are arranged on the annular light source plate 111. First, the first lamp bead group 112a is arranged, and then the second lamp bead group 112b is arranged around the outer periphery of the first lamp bead group 112a.

[0084] Specifically, taking the first lamp bead group 112a as an example, starting from the starting point A of the annular light source plate 111, the 390nm lamp bead group a, the 420nm lamp bead group b, the 450nm lamp bead group c, the 525nm lamp bead group d, the 570nm lamp bead group e, the 630nm lamp bead group f, the 660nm lamp bead group g, the 730nm lamp bead group h, the 780nm lamp bead group i, the 850nm lamp bead group j, the 870nm lamp bead group k, the 890nm lamp bead group l, the 910nm lamp bead group m, and the 940nm lamp bead group n in the first lamp bead group 1121 and the second lamp bead group 1122 are arranged in sequence clockwise. 12 lamp beads are arranged in each of the above groups. That is to say, a total of 180 lamp beads are arranged in one round. The arrangement method of the second lamp bead group 112b is arranged in sequence around its outer periphery according to the arrangement method of the first lamp bead group 112a, which will not be elaborated here.

[0085] When arranging each lamp bead, on the one hand, it is necessary to ensure that there is a certain gap between each lamp bead. On the other hand, it is also necessary to ensure that the overall arrangement of the lamp beads in the first lamp bead group 112a and the overall arrangement of the lamp beads in the second lamp bead group 112b are arranged in a staggered position. That is to say, it is necessary to avoid mutual occlusion between the lamp beads, so as to ensure that the state of each light emission is uniform, and improve the accuracy and reliability of image information data acquisition.

[0086] Those skilled in the art can understand that the specific number of lamp beads in each group of lamp beads in the same cycle, the number of preset wavelength values, and the number of cycles of lamp bead groups set on the same annular light source plate 111 can be set according to actual tongue inspection requirements and image data processing requirements. The embodiments of the present invention only provide some illustrative examples, so those skilled in the art should not be understood as a limitation of the present invention.

[0087] In one example, specifically, the lens of the hyperspectral camera 12 is coaxially arranged with the annular light source plate 111. The lens direction of the hyperspectral camera 12 is in the same direction as the light emission direction of the light source unit 11. The working principle of the hyperspectral camera 12 is based on spectral decomposition and spectral reconstruction, that is, the hyperspectral camera 12 divides the light signals of each band (generally a wide band) of the incident light reflected after the light source unit 11 irradiates the face and / or tongue of the subject into a plurality of narrow bands of light beams, and then images these light beams on their corresponding detectors to obtain images of different spectral bands. Therefore, by switching the lamp bead group of the light source unit 11 to illuminate the face and / or tongue of the subject and then reflecting it, incident light of different wavelengths can be provided to enter the hyperspectral camera 12. Furthermore, after the hyperspectral camera 12 obtains spectral images of different wavelengths, it can also combine a group of spectral images of different wavelengths through image fusion processing to form a multi-spectral image of the same tongue or the same face.

[0088] In one example, alternatively, see Figure 8 , exemplarily showing an embodiment of the working principle of a hyperspectral camera 12 and its main parameter settings. The hyperspectral camera 12 can use common spectral camera products. The higher the spectral resolution accuracy of its detector system, the higher the accuracy of the images of different spectral bands that can be obtained. At the same time, in the hyperspectral camera 12, shooting parameters can also be set through its data acquisition module, such as the width, height, magnification, etc. of the shooting target, and the horizontal viewing angle, vertical viewing angle, working distance, etc. of the lens of the hyperspectral camera 12. Only some illustrative examples are provided here, and those skilled in the art should not be understood as a limitation of the present invention.

[0089] In one example, alternatively, the hyperspectral camera 12 receives image data through an image sensor and then processes the image data through, for example, an ISP image processing algorithm, and finally transmits the image data to the USB interface 60 (see Figure 6 As shown) completes high-speed transmission of image data.

[0090] Combination Figures 5 - 7 As shown, when the collection device 100 is used to photograph the face and / or tongue of the person being collected, the light source unit 11, especially the groups of lamp beads thereon, works as follows:

[0091] After starting the acquisition device 100, first, the control module 40 controls the activation of a total of 24 first lamp bead groups 1121 for two weeks. Meanwhile, the hyperspectral camera 12 receives the reflected light after it irradiates the tongue or face (including the entire face and local face) and forms a spectral image corresponding to the first lamp bead group 1121. Then, all the first lamp bead groups 1121 are turned off. When the instruction to light the lamp beads next time is issued, the 390 nm lamp bead group a in the second lamp bead group 1122, which totals 24 for two weeks, is activated. Meanwhile, the hyperspectral camera 12 receives the reflected light after it irradiates the tongue or face (including the entire face and local face) and forms a spectral image corresponding to the 390 nm lamp bead group a. And so on, until the 940 nm lamp bead group n in the last of the second lamp bead groups 1122, which totals 24 for two weeks, is lit. Meanwhile, the hyperspectral camera 12 receives the reflected light after it irradiates the tongue or face (including the entire face and local face) and forms a spectral image corresponding to the 940 nm lamp bead group n. Thus, a complete multi-spectral shooting process ends (in this process, the shooting processes of the second image acquisition module 20 and the third image acquisition module 30 introduced below are also included).

[0092] In one example, alternatively, the light source unit 11 further includes an annular reflector 113 disposed on the annular light source plate 111 and a light source cover plate 114 for protecting the lamp beads, electrical circuits, control chips, etc. inside the light source unit 11. Among them, the annular reflector 113 is covered on the surface of the annular light source plate 111 where the lamp beads are disposed, and a coating film for improving the diffuse reflectivity of the light source is provided on the surface of the annular reflector 113. The light source cover plate 114 fixes the annular light source plate 111, at least one group of lamp bead groups 112, and the annular reflector 113 together.

[0093] In one example, preferably, a micro-nano material, namely nano 97% high diffuse reflection coating (HDRC-V001), is used to spray the surface of the annular reflector 113 to form a coating film. This nano 97% high diffuse reflection coating is a white emulsion coating refined by compounding high reflectivity micro-nano materials with water as the medium. Its total reflectivity and diffuse reflectivity are as high as 98.2% and 97% respectively. This coating film can improve the luminous efficiency, make the reflected light uniform and soft, reduce glare with wide-angle light output, reduce chromatic aberration or deviation of light, and at the same time can prevent dust, prevent UV, and extend the service life.

[0094] The 3D structured light imaging technology and its structured light camera can provide three-dimensional contour data of the target of interest. For tongue surface inspection, the three-dimensional contour data of the tongue or face (including the entire face and local face) of the person being collected obtained through the 3D structured light imaging technology can digitalize the contour data of the skin surface, which helps to further obtain detailed information on the minute changes occurring on the skin surface through precise measurement to assist in judgment.

[0095] In one example, referring to Figure 9 , the structure of a second image acquisition module 20 is exemplarily shown. Combining Figure 6 and Figure 9 as shown, the second image acquisition module 20 is disposed on the backlight side of the light source unit 11. Specifically, the second image acquisition module 20 includes a projector 21 and at least one structured light camera 22 integrated with the projector 21. The lens direction of each structured light camera 22 in the at least one structured light camera 22 is the same as the light emission direction of the light source unit 11.

[0096] In one example, alternatively, the second image acquisition module 20 can be assembled using a common projector device and a common industrial-grade camera. For example, it can be assembled using 1 high-definition Blu-ray industrial-grade DLP projector above 720P and 2 fixed-focus high-definition industrial cameras.

[0097] Specifically, the working process of the second image acquisition module 20 is as follows: The second image acquisition module 20 is built-in with a grating encoder, and the projector 21 projects a set of spatial orientation coding patterns onto the target to be measured (the face and / or tongue of the person being collected). At the same time, at least one structured light camera 22 is triggered to collect pictures of the face and / or tongue of the person being collected with the orientation coding patterns superimposed thereon. Then, the high-speed computing unit inside the second image acquisition module 20 performs parallel decoding on the captured pictures, and the three-dimensional topography and texture information of the surface of the face and / or tongue of the person being collected can be reconstructed by combining the pre-calibrated parameters of the projector 21 and the at least one structured light camera 22.

[0098] In particular, when using 2 structured light cameras 22, it is possible to obtain two sets of point cloud data of the face and / or tongue of the person being collected from the 2 structured light cameras 22 respectively through one-shot acquisition. After fusing the key information in the two sets of point cloud data, a three-dimensional contour image and its image data of the face and / or tongue of the person being collected with richer texture details can be obtained. At the same time, the depth information at different positions on the surface of the tongue or face of the person being collected is obtained after collecting the point cloud data and performing three-dimensional structure reconstruction.

[0099] Visible light imaging technology and visible light cameras can provide high-resolution color images. Through the high-resolution color images, surface features such as the skin tone and texture of the face and / or tongue of the person being collected can be accurately captured.

[0100] In one example, specifically, as Figure 6As shown in the figure, the third image acquisition module 30 includes at least one visible light camera 31, and a fixing bracket 32, a lens envelope 33, and a lens bracket 34 for fixing the at least one visible light camera 31. The at least one visible light camera 31 is fixedly arranged on the backlight side of the light source part 11 through the fixing bracket 32, the lens envelope 33, and the lens bracket 34, and the lens direction of each visible light camera 31 of the at least one visible light camera 31 is the same as the light emission direction of the light source part 11.

[0101] In one example, specifically, the acquisition device 100 further includes a control module 40. The start and stop of the first image acquisition module 10, the second image acquisition module 20, and the third image acquisition module 30 are respectively controlled through the control chip and its control switch inside the control module 40, and the light beads in at least one group of light bead groups of the light source part 11 are controlled to emit light and go out according to a preset light source emission instruction.

[0102] In one example, alternatively, the control module 40 is integrated with a serial port for connecting to a PC. The PC controls the switching of the light bead groups in the light source part 11 to emit light or go out, and the light emission and extinction of each group of light beads through the serial port. Data can also be sent and received through the serial port with other devices via a communication protocol.

[0103] In one example, further, when all the light beads of the light source part 11 include light beads of the first wavelength to light beads of the Nth wavelength in sequence according to their arrangement order in the light source part (for example, Figure 7 an embodiment of the arrangement of a light bead combination shown), where N is an integer greater than or equal to 1, the preset light source emission instruction includes:

[0104] Setting that all the light beads complete light emission from the first wavelength to the Nth wavelength in sequence as a light source emission cycle; within a light source emission cycle, all the light beads emit light in sequence according to the order of the first wavelength to the Nth wavelength; setting the preset exposure time corresponding to the light beads of each wavelength among the first wavelength to the Nth wavelength when emitting light; setting the preset light emission interval time between the light beads of two adjacent wavelengths.

[0105] Those skilled in the art can understand that the specific preset exposure time and preset light emission interval time can be set in the control module 40 according to the actual acquisition device 100 in combination with the actual tongue surface inspection requirements.

[0106] In one example, alternatively, relative to the open end of the housing 50 of the acquisition device 100, that is, behind the backlight side of the light source part 11, a split-assembled cover plate 52 is further provided. When the acquisition device 100 needs to replace its internal components such as the first image acquisition module 10, the second image acquisition module 20, and the third image acquisition module 30, by removing the cover plate 52, there is enough operating space inside the housing 50, which is convenient for equipment maintenance and debugging.

[0107] In one example, alternatively, a rack 53 may also be provided inside the housing 50 for fixing the first image acquisition module 10, the second image acquisition module 20, and the third image acquisition module 30 to the rack 53 respectively through fasteners.

[0108] In one example, alternatively, a power supply module 60 is also provided inside the acquisition device 100 for supplying power to the first image acquisition module 10, the second image acquisition module 20, the third image acquisition module 30, and the control module 40. According to the voltage requirements of these modules for power consumption, the supply voltage can be adjusted by integrating a transformer inside the power supply module 60. For example, the 220V voltage is converted into two paths of 24V and 5V voltages through the transformer. The 24V voltage supplies power to the structured light camera 22 of the second image acquisition module 20, and the 5V voltage supplies power to the light source unit 11 of the first image acquisition module 10 and the control module 40.

[0109] The acquisition device for hyperspectral and structured light images of traditional Chinese medicine tongue and face diagnosis and its image acquisition method provided by the embodiments of the present invention have at least one or a part of at least one of the following advantages:

[0110] (1) By integrating a hyperspectral camera, a structured light camera, and a visible light camera on the same acquisition device, the face and tongue of the person to be acquired are imaged at one time, and multi-spectral imaging biomarker images, three-dimensional contour images, and color images are obtained simultaneously, providing objective, detailed, and multi-dimensional tongue and face image data for traditional Chinese medicine inspection by inspection to improve the efficiency and accuracy of traditional Chinese medicine inspection by inspection;

[0111] (2) By combining a hyperspectral camera, a structured light camera, and a visible light camera, the disadvantage that the visible light camera can only capture the surface image information of the tongue and / or face of the person to be acquired can be effectively compensated, and it is possible to deeply capture the physiological changes of the tongue and / or face on the basis of the surface image information of the tongue and / or face, so as to obtain multi-dimensional tongue and face image information data;

[0112] (3) Through the first image acquisition module integrated with a hyperspectral camera and a multi-wavelength light source unit, biomarker images corresponding to different spectral bands can be obtained in a large wavelength range by adjusting the emission wavelength of the light source unit, so as to realize in-depth exploration of physiological characteristics such as blood vessels and lymph nodes under the skin of the face and tongue, which has significant guiding value for the discovery of potential diseases and the early judgment of diseases;

[0113] (4) By setting and adjusting the wavelengths of the lamp beads and arranging the lamp bead groups within the light source unit, it is possible to complete the adaptive adjustment of the light source according to various tongue surface inspection requirements within the same acquisition device and obtain multi-dimensional tongue surface image data corresponding to the tongue surface inspection requirements, thereby improving the imaging stability and image acquisition efficiency of the acquisition device;

[0114] (5) By adjusting the number of lamp beads and the arrangement of the lamp bead groups, it is possible to generally adapt to most traditional Chinese medicine inspection scenarios and tongue surface inspection requirements. While obtaining more objective and comprehensive image information, it also reduces the acquisition difficulty and equipment investment cost, and has a wide range of application prospects;

[0115] (6) By using a structured light camera to perform three-dimensional contour imaging on the face and tongue, it is possible to accurately perform three-dimensional modeling on the face and tongue of the person being examined, providing rich microscopic physiological feature changes at the structural information level and improving the image information acquisition sensitivity and accuracy of the acquisition device;

[0116] (7) Through the control module and by using the image data processing modules configured inside the hyperspectral camera, structured light camera, and visible light camera, it is possible to integrally control the process of the light source and image acquisition, effectively shortening the interrogation time of traditional Chinese medicine inspection and the acquisition time for obtaining image data, greatly improving the interrogation efficiency, shortening the interrogation time of the person being examined, and improving the quality of medical services.

[0117] Although some embodiments of the general inventive concept have been shown and described, those of ordinary skill in the art will understand that changes can be made to these embodiments without departing from the principles and spirit of the general inventive concept. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. An acquisition device for hyperspectral and structured light images of traditional Chinese medicine tongue diagnosis, characterized in that, The acquisition device includes: A first image acquisition module configured to acquire a biomarker image inside the tongue or face. The first image acquisition module includes a light source unit and a hyperspectral camera integrated together, and the hyperspectral camera is disposed on the backlight side of the light source unit; A second image acquisition module configured to acquire a three-dimensional contour image of the surface of the tongue or face. The second image acquisition module includes a projector and at least one structured light camera integrated together; A third image acquisition module configured to acquire a color image of the surface features of the tongue or face. The third image acquisition module includes at least one visible light camera; Wherein, the light source unit emits lights of different wavelengths to irradiate the tongue or face and reflection occurs, and the hyperspectral camera receives the reflected lights of different wavelengths to form a biomarker image of the same tongue or the same face under different spectral bands; the biomarker image, the three-dimensional contour image, and the color image jointly form a multi-feature fusion tongue surface image.

2. The acquisition device according to claim 1, wherein: The first image acquisition module is further configured to: The light source unit includes an annular light source plate and at least one set of lamp bead groups disposed on the annular light source plate; Each set of lamp bead groups in the at least one set of lamp bead groups includes a first lamp bead group for providing white light and a second lamp bead group for providing different wavelengths; The lamp beads in each set of lamp bead groups are arranged in a ring around the center of the annular light source plate; The lens of the hyperspectral camera is coaxially disposed with the annular light source plate; The lens direction of the hyperspectral camera is the same as the light emission direction of the light source unit.

3. The acquisition device according to claim 1, wherein: The second image acquisition module is further configured to: The at least one structured light camera is disposed on the backlight side of the light source unit, and the lens direction of each structured light camera in the at least one structured light camera is the same as the light emission direction of the light source unit; The at least one structured light camera includes a first structured light camera and a second structured light camera, and there is a preset distance between the first structured light camera and the second structured light camera.

4. The acquisition device according to claim 2, wherein: The first lamp bead group includes at least one lamp bead for providing white light; At least one preset wavelength value is set in the second lamp bead group, and at least one lamp bead of a preset wavelength is set corresponding to each preset wavelength value in the at least one preset wavelength value; The range of the preset wavelength value is 390nm to 940nm; The lamp beads in each set of lamp bead groups are uniformly arranged in a counterclockwise or clockwise order from the lamp bead providing white light in ascending order of the preset wavelength value.

5. The acquisition device according to claim 4, wherein: The number of lamp beads in the first lamp bead group is equal to the number of lamp beads in each preset wavelength value in the second lamp bead group; The at least one set of lamp bead groups is at least two sets of lamp bead groups; Each set of lamp bead groups in the at least two sets of lamp bead groups is arranged on the annular light source plate after the previous set of lamp bead groups, and then arranged on the side away from the center of the annular light source plate of the previous lamp bead group.

6. The acquisition device according to claim 2, wherein the light source unit further includes an annular reflector disposed on the annular light source board; the annular reflector is covered on the surface of the annular light source board where the lamp beads are disposed; a coating film for improving the diffuse reflectivity of the light source is disposed on the surface of the annular reflector; the coating film is a micro-nano material with a high reflectivity.

7. The acquisition device according to any one of claims 1-3, wherein the at least one structured light camera acquires depth information at different positions on the surface of the tongue or face; the acquisition device further includes a control module, and the control module is configured to respectively control the startup and stop of the first image acquisition module, the second image acquisition module, and the third image acquisition module, and control the lamp beads in at least one set of lamp bead groups of the light source unit to emit light and go out according to a preset light source emission instruction.

8. The acquisition device according to claim 7, wherein the acquisition device further includes a housing for fixedly mounting the first image acquisition module, the second image acquisition module, the third image acquisition module, and the control module; one end of the housing in the light emission direction of the light source unit is provided with an open end, and a head bracket for fixing the position of the tongue or face is provided on the open end.

9. An image acquisition method for acquiring hyperspectral and structured light images of the tongue surface in traditional Chinese medicine tongue diagnosis. The image acquisition method uses the acquisition device according to any one of claims 1-8 to obtain a tongue surface image with multi-feature fusion, characterized in that The image acquisition method includes the following steps: generating a preset light source emission instruction according to the requirements of tongue surface inspection and inputting the preset light source emission instruction into the control module; initializing the first image acquisition module, the second image acquisition module, and the third image acquisition module through the control module; generating a multi-spectral camera control thread, a structured light camera control thread, and a visible light camera control thread corresponding to the first image acquisition module, the second image acquisition module, and the third image acquisition module through the control module; the control module controls the lamp beads of the light source unit to emit light and go out within a light source emission cycle according to the preset light source emission instruction; when the light source emits light, the first image acquisition module is started through the multi-spectral camera control thread to acquire a biological marker image inside the tongue or face, and the third image acquisition module is started through the visible light camera control thread to acquire a color image of the surface features of the tongue or face; when the light source stops emitting light, the second image acquisition module is started through the structured light camera control thread to acquire a three-dimensional contour image of the surface of the tongue or face.

10. The image acquisition method according to claim 9, wherein when all the lamp beads of the light source unit sequentially include lamp beads of the first wavelength to the Nth wavelength in the arrangement order in the light source unit, where N is an integer greater than or equal to 1, the preset light source emission instruction includes: setting that the complete emission of all the lamp beads from the first wavelength to the Nth wavelength is one light source emission cycle; within one light source emission cycle, all the lamp beads emit light in sequence according to the order of the first wavelength to the Nth wavelength; setting the preset exposure time corresponding to the lamp beads of each wavelength from the first wavelength to the Nth wavelength when emitting light; setting the preset light emission interval time between the lamp beads of two adjacent wavelengths.

Images