Spectroscopy-based medical image processing method

Abstract

The present application relates to a kind of based on spectrum medical image processing method and device.The based on spectrum medical image processing method includes: obtaining the spectral image at focus;According to the color space numerical value of each pixel point restored from the spectral image, to obtain the first color image;The color characteristic of the first color image is carried out, and image color is converted to large color domain range to obtain the second color image;And, using high fidelity display shows the second color image.In this way, high-resolution spectral information and color information can be presented to the user.

Description

Technical Field

[0001] This application relates to the field of medical imaging and display technology, and more specifically, to a spectral-based medical image processing method. Background Technology

[0002] With the continuous improvement of social informatization, information technology has permeated all aspects of social life. Telemedicine is a product of the combination of information technology and traditional medical technology, bringing about a completely new transformation in the medical field. Telemedicine utilizes information technologies such as digitalization, computer automation, long-distance network communication, and multimedia to make it possible to seek medical treatment without leaving home, while also providing a more intelligent, efficient, and convenient model for daily family healthcare. Current telemedicine technologies cover remote diagnosis, remote emergency care, remote consultation, and remote monitoring, and are continuously developing and gradually becoming more widespread.

[0003] The concept of telemedicine originated in the 1960s. The University of Nebraska was one of the first places to apply telemedicine technology, using two-way closed-loop microwave television for psychiatric consultations. NASA, under the guidance of the National Library of Medicine, also supported a telemedicine project in the 1960s using satellite communications, operating telemedicine services between the Appalacian Rocky Mountain and Alaska regions.

[0004] In recent years, thanks to the rapid development of microelectronics, network communication and computer technologies, telemedicine technology has developed rapidly, enabling people to remotely transmit and monitor medical information between themselves and medical institutions through modern communication networks, realizing functions such as remote monitoring, medical care, home-based care, and remote consultation.

[0005] In the field of biomedical applications, spectral imaging is still a relatively new technology, but it has broad potential applications as a non-invasive method for diagnosis and treatment assessment. At specific wavelengths, the chemical composition and physical characteristics of tissues in different pathological states exhibit varying reflectance, absorbance, and electromagnetic energy, manifested as differences in characteristic spectral peaks. By analyzing these spectral signals, qualitative or quantitative detection of tissue state information can be achieved, and based on the spatial distribution information provided by hyperspectral images, the different pathological states of tissues can be visualized, thereby enabling the diagnosis of tissue disease states. This capability of spectral imaging technology is increasingly being applied in the medical field for disease detection and surgical guidance. As a novel, non-contact optical diagnostic technology, spectral imaging provides an effective auxiliary diagnostic tool for clinical medicine through spectral image information, possessing enormous development potential.

[0006] Spectral imaging technology is a comprehensive technology integrating detector technology, precision optics and mechanics, weak signal detection, computer technology, and information processing technology. Spectral images contain rich spatial, radiometric, and spectral information, such as... Figure 1 As shown. Here, Figure 1 The illustration shows a schematic diagram of the information contained in a spectral image.

[0007] In medical imaging, doctors can diagnose and treat patients by the color of lesions. Commercial color cameras currently mainly use red, green, and blue (R, G, B) three-primary-color sensors. During image acquisition and transmission, a large amount of image information is lost, most typically the color information of the original image. Moreover, most existing color cameras use the sRGB format for image transmission and encoding. Since the sRGB color gamut is small, it is limited to a small triangle. Colors outside this range cannot be distinguished, resulting in a large loss of color information.

[0008] Therefore, in the medical field, there is a need to provide improved spectral-based medical imaging and display solutions. Summary of the Invention

[0009] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a spectral-based medical image processing method and apparatus, which, by recovering color images from spectral images and displaying them on a high-fidelity display, can present high-resolution spectral and color information to the user.

[0010] According to one aspect of this application, a spectral-based medical image processing method is provided, comprising: acquiring a spectral image of a lesion; reconstructing the color space value of each pixel from the spectral image to obtain a first color image; performing color characterization on the first color image to convert the image color to a large color gamut to obtain a second color image; and displaying the second color image using a high-fidelity display.

[0011] In the above-mentioned spectral-based medical image processing method, obtaining the spectral image of the lesion includes: performing spectral imaging of the lesion using a spectral camera; and reconstructing the spectral curve of each pixel based on the original signal from the image sensor of the spectral camera to obtain the spectral image.

[0012] In the above-mentioned spectral-based medical image processing method, acquiring the spectral image of the lesion further includes: storing the spectral image and archiving the data.

[0013] In the above-described spectral-based medical image processing method, obtaining the color space value of each pixel from the spectral image to obtain the first color image includes: obtaining the color value of each pixel in the XYZ color space using the following formula based on the spectral curve of each pixel in the spectral image and the CIE color standard:

[0014]

[0015]

[0016]

[0017] Where k is the normalization coefficient, used to limit the maximum value of Y to 100, and φ(λ) is the expression for the spectral curve. These are the color standards for the XYZ color space specified by CIE.

[0018] In the above-mentioned spectrum-based medical image processing method, the first color image is located in a color space that is independent of the color gamut, and there is no compression or loss of the color gamut during the color characterization process.

[0019] In the above-mentioned spectrum-based medical image processing method, the color space of the second color image is greater than the sRGB color space.

[0020] In the aforementioned spectral-based medical image processing method, displaying the second color image using a high-fidelity display includes screen characterization of the high-fidelity display based on the color-processed image data. The screen characterization includes: establishing characteristic samples, which are uniformly distributed according to brightness and chromaticity, covering various colors at each brightness level, and the colors are uniformly distributed according to hue and saturation; displaying the characteristic samples with known RGB values ​​on the display screen; measuring the chromaticity values ​​of the display screen using a screen colorimeter; and establishing a relationship between the known RGB values ​​and the measured chromaticity values.

[0021] In the above-described spectral-based medical image processing method, before displaying the second color image using a high-fidelity display, the method further includes: performing digital image processing on the second color image, wherein the color data of the second color image is not compressed and the color space of the second color image is not changed during the digital image processing.

[0022] In the above-described spectral-based medical image processing method, before displaying the second color image using a high-fidelity display, the method further includes: encoding and compressing the second color image.

[0023] In the above-described spectral-based medical image processing method, after encoding and compressing the second color image, the method further includes: transmitting the encoded and compressed second color image to a high-fidelity display at a predetermined location.

[0024] In the above-described spectral-based medical image processing method, after encoding and compressing the second color image, the method further includes: decoding the second color image on the high-fidelity display.

[0025] In the above-mentioned spectral-based medical image processing method, imaging a patient's lesion using a spectral camera includes: using a first set of image detection chip module components comprising a spectral chip module, an imaging chip module, and a beam splitter for imaging, wherein the beam splitter is located on the path between the spectral chip and the imaging chip, so that a first portion of light is deflected and a second portion of light is transmitted and received by the spectral chip and the imaging chip respectively, thereby acquiring spectral information and image information.

[0026] In the above-mentioned spectral-based medical image processing method, the spectral chip assembly includes a light homogenizer disposed between the beam splitter and the spectral chip, which is used to homogenize the light before the spectral chip obtains spectral information.

[0027] In the above-mentioned spectral-based medical image processing method, imaging the lesion site of the patient using a spectral camera further includes: imaging healthy tissue outside the lesion using a second set of image detection chip module components, which includes at least an imaging chip module.

[0028] The above-described spectral-based medical image processing method further includes: comparing a first spectral image obtained by the first set of image detection chip module components with a pre-stored pathological spectral image; and, in response to the first spectral image being identical to the pre-stored pathological spectral image, saving and recording the first spectral image.

[0029] The above-described spectral-based medical image processing method further includes: comparing a first spectral image obtained by the first set of image detection chip module components with a second spectral image obtained by the second set of image detection chip module components; in response to the first spectral image being different from the second spectral image, comparing the first spectral image with a pre-stored pathological spectral image; and in response to the first spectral image being the same as the second spectral image, not comparing the first spectral image with the pre-stored pathological spectral image.

[0030] According to another aspect of this application, a spectral-based medical image processing apparatus is provided, which uses the spectral-based medical image processing method described above.

[0031] The spectral-based medical image processing method and apparatus provided in this application can present high-resolution spectral and color information to the user by recovering color images from spectral images and displaying them on a high-fidelity display. Attached Figure Description

[0032] Various other advantages and benefits of this application will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0033] Figure 1 The diagram illustrates the various information contained in a spectral image.

[0034] Figure 2 The illustration shows a schematic diagram of the imaging process in existing mobile devices;

[0035] Figure 3 The figure shows a schematic diagram of a spectral imaging device according to an embodiment of this application;

[0036] Figure 4 The illustration shows a schematic flowchart of a spectral-based medical image processing method according to an embodiment of this application;

[0037] Figure 5 The illustration shows a schematic diagram of an example of screen characterization in a spectral-based medical image processing method according to an embodiment of this application;

[0038] Figure 6 The illustration shows an application example of a spectral-based medical image processing method according to an embodiment of this application. Detailed Implementation

[0039] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0040] Application Overview

[0041] Mobile devices, such as smartphones and tablets, are widely used. A typical process from camera imaging to final display is as follows: Figure 2 As shown. Here, Figure 2 The illustration shows a schematic diagram of the imaging process in existing mobile devices.

[0042] like Figure 2 As shown, the image sensor acquires information from the scene and transmits it in Raw format. Raw format images can be displayed after being processed by a digital image processing system and then by RAM, or they can be stored directly or processed by the application layer for display.

[0043] Spectral imaging, with its unique advantages, can not only image the spatial features of a target but also acquire the spectral information of each pixel unit. In this way, through subsequent color restoration and digital image processing, accurate image color information can be obtained, enabling accurate color transmission between different digital media, such as from image acquisition to display on mobile phones and tablets, and from information acquisition to final display in the field of remote sensing.

[0044] Specifically, the spectral imaging device according to the embodiments of this application may have, for example: Figure 3 The configuration shown is shown here. Figure 3 The illustration shows a schematic diagram of a spectral imaging device according to an embodiment of this application. Figure 3 As shown, in the spectral imaging device according to the embodiments of this application, the optical system is optional and may be an optical system such as a lens assembly or a homogenizing assembly. The filter structure is a broadband filter structure in the frequency domain or wavelength domain. The pass spectra of different wavelengths of the filter structure are not completely the same at different locations. The filter structure can be a metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS (microelectromechanical systems), FP etalon, cavity layer, waveguide layer, diffraction element, or other structures or materials with filtering properties. For example, in the embodiments of this application, the filter structure can be the light modulation layer in Chinese Patent CN201921223201.2, and the image sensor (i.e., photodetector array) can be a CMOS image sensor (CIS), CCD, array photodetector, etc. In addition, the optional data processing unit can be a processing unit such as an MCU, CPU, GPU, FPGA, NPU, ASIC, etc., which can export the data generated by the image sensor to the outside for processing.

[0045] Furthermore, in this embodiment, the spectral image records the original information of the shooting scene, minimizing the loss of image information. Therefore, high-fidelity restoration can be achieved during color restoration. However, existing image processing processes involve image compression and color information compression, leading to information loss. Moreover, most existing displays use three primary colors and lack the ability to process spectral image information. Therefore, addressing the aforementioned color information loss during spectral image transmission and the data processing issues of the display during display, and considering the specific application scenarios of medical imaging, this application proposes a spectral-based medical imaging and display scheme. This scheme involves spectral image acquisition, data processing and transmission, and final spectral image display, achieving high color gamut and high-fidelity color transmission across the entire chain and realizing closed-loop color management.

[0046] Here, "end-to-end" refers to the entire data processing chain from image information acquisition, processing, transmission, encoding to final display. "High color gamut" means that a large color gamut is used for processing throughout the data transmission process. The concept of color gamut refers to the number of colors that can be represented in mathematical space. "High fidelity" means to reproduce the colors in the shooting scene to the greatest extent possible and to minimize color loss.

[0047] Exemplary methods

[0048] Figure 4 The illustration shows a schematic flowchart of a spectral-based medical image processing method according to an embodiment of this application.

[0049] like Figure 4 As shown, the spectral-based medical image processing method according to an embodiment of this application includes the following steps.

[0050] Step S110: Obtain the spectral image of the lesion. In this embodiment, a pre-stored spectral image of the lesion can be directly obtained, or a spectral imaging device, such as a spectral camera, can be used to obtain the spectral image.

[0051] For example, when using a spectroscopic camera to image a patient's lesion, the spectroscopic camera includes at least an image sensor and an optical system. The image sensor can be of various types, such as filter type, micro / nano structure type, or liquid crystal reconcilable type, and is designed to acquire information in the visible light band, specifically the 380nm-780nm wavelength range. Furthermore, the spectroscopic camera can be a fixed device, for example, where the area of ​​the patient requiring diagnosis is placed close to the imaging device; or it can be a non-fixed device, for example, using a probe device inserted deep into the lesion to image the patient's tissue using active illumination.

[0052] Furthermore, the optical system may include a light source assembly and a receiver assembly. The light source assembly emits light, which illuminates the lesion area or suspected lesion area on the human body. After reflection from the lesion area or suspected lesion area, the light is received by the receiver assembly. In this way, the subsequent imaging module assembly can acquire image and spectral information based on the received reflected light. The image information is then compared with physiological characteristics such as the lesion area or suspected lesion area to determine the pathological condition of the patient. The spectral information is used to determine the type of disease and the required surgical treatment, as well as the location and type of the lesion area or suspected lesion area. First, it is necessary to determine the degree of matching between the spectral image and the pre-stored analysis of various dissected lesions that have undergone pathological analysis. Only when the spectral image matches the pre-stored baseline lesion area or suspected lesion area spectral image (which includes at least one or a combination of spatial, radiometric, spectral, and color information) can it be entered into the database and sent to doctors and experts as a basis for diagnosis. The imaging module assembly will be described in more detail below.

[0053] After using a spectral camera to perform spectral imaging of the lesion, it is necessary to reconstruct the spectral curve of each pixel based on the raw signal from the spectral camera's image sensor to obtain a spectral image. Here, because the signal from the spectral camera's image sensor is mostly in DN values, preliminary data processing is required, such as normalization. Then, a spectral restoration algorithm is used to restore the spectral data. Spectral restoration methods include polynomial methods, deep learning methods, etc.

[0054] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, obtaining the spectral image of the lesion includes: performing spectral imaging on the lesion using a spectral camera; and reconstructing the spectral curve of each pixel based on the original signal from the image sensor of the spectral camera to obtain the spectral image.

[0055] In addition, in this embodiment of the application, considering the special nature of the medical industry, which requires the archiving of patient data and the tracking of changes in the patient's condition, the spectral images are stored and archived, for example, in a predetermined database, to establish a database of spectral information of the patient's lesions. This facilitates expert consultations and comparison of spectral information of similar patients, thereby seeking the optimal treatment plan.

[0056] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, acquiring the spectral image of the lesion further includes: storing the spectral image and archiving the data.

[0057] Step S120: The color space value of each pixel is reconstructed from the spectral image to obtain a first color image. For example, if the spectral data of the spectral image is to be converted to the XYZ color space, colorimetric methods can be used. Specifically, based on the obtained spectral curve of each pixel and the CIE color standard, the spectral data can be converted to the XYZ color space, specifically through the following formula:

[0058]

[0059]

[0060]

[0061] Where k is the normalization coefficient, used to limit the maximum value of Y to 100, and φ(λ) is the expression for the spectral curve. These are the color standards for the XYZ color space specified by CIE.

[0062] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, obtaining the color space value of each pixel from the spectral image to obtain a first color image includes: obtaining the color value of each pixel in the XYZ color space using the following formula based on the spectral curve of each pixel in the spectral image and the CIE color standard:

[0063]

[0064]

[0065]

[0066] Where k is the normalization coefficient, used to limit the maximum value of Y to 100, and φ(λ) is the expression for the spectral curve. These are the color standards for the XYZ color space specified by CIE.

[0067] Step S130 involves color characterizing the first color image by converting its colors to a wider color gamut to obtain a second color image. For example, as described above, the XYZ data of the first color image is equivalent to storing the image's color information in the XYZ color space, which is a device-independent space. A device-independent color space refers to a color space that does not change due to different devices. However, for other digital image processing and display, the image's XYZ data needs to be converted to another color space, such as the RGB color space, before other digital image processing operations and color manipulation can be performed. Specifically, in this embodiment, the image data of the first color image, such as XYZ data, is converted to the wider RGB color gamut. For example, the BT.2020 color space is preferably selected.

[0068] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, the first color image is located in a color space that is independent of the color gamut, and there is no compression or loss of the color gamut during the color characterization process.

[0069] Furthermore, in the spectral-based medical image processing method according to the embodiments of this application, the color space in which the second color image is located is a color space larger than sRGB.

[0070] For example, a color space larger than sRGB can be the aforementioned BT.2020 color space or the DCI-P3 color space, and those skilled in the art will understand that it also includes other larger color spaces that may emerge in the future.

[0071] Step S140: Display the second color image using a high-fidelity display. Several factors affect the display's performance, with key indicators including resolution, light emission mode, and color gamut. In this embodiment, high-fidelity color reproduction across the entire spectrum from acquisition and transmission to final display is required; therefore, a high-fidelity display is used. Specifically, a high-fidelity display refers to a display with a high color gamut and high color accuracy, meaning it can display a sufficient number of colors with high accuracy.

[0072] Furthermore, to improve the display effect of the high-fidelity display, in this embodiment, the display is preferably characterized. Here, screen characterization is to establish the relationship between the screen's driving values ​​and chromaticity values ​​using a colorimeter. For example, a three-dimensional lookup table can be used for screen characterization processing.

[0073] Figure 5 The illustration shows an example of screen characterization in a spectral-based medical image processing method according to an embodiment of this application.

[0074] like Figure 5 As shown, screen characterization mainly includes: establishing characterization samples, which are uniformly distributed according to brightness and chromaticity, covering various colors at each brightness level, and the colors are uniformly distributed according to hue and saturation; displaying the characterization samples on the display screen, where the RGB values ​​of the characterization samples are known; measuring their chromaticity values ​​using a screen colorimeter, and establishing the relationship between RGB information and chromaticity values. Here, screen characterization can be implemented by an independent module, that is, by characterizing the screen, the relationship between image driving values ​​and the final displayed chromaticity values ​​is established. In other words, through screen characterization, high-fidelity chromaticity information of the image can be displayed, for example, the BT.2020 color gamut can be selected for display, and through the display of a high color gamut, the color information of the image can be presented to the user in a lossless manner.

[0075] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, displaying the second color image using a high-fidelity display includes screen characterization of the high-fidelity display based on the color-processed image data, and the screen characterization includes: establishing a characteristic sample, the characteristic sample being uniformly distributed according to brightness and chromaticity, covering various colors at each brightness level, and the colors being uniformly distributed according to hue and saturation; displaying the characteristic sample with known RGB values ​​on the display screen; measuring the chromaticity value of the display screen using a screen colorimeter; and establishing a relationship between the known RGB values ​​and the measured chromaticity value.

[0076] In this way, when displaying image data on the screen, it can be displayed according to a predetermined large color gamut color space, such as the BT.2020 color gamut space, and the bit depth of the display can be set to a high number of bits, such as 10 bits, so as to show the color gradation.

[0077] Furthermore, in this embodiment of the application, before displaying the second color image using a high-fidelity display, the second color image may undergo digital image processing. This digital image processing includes at least one of image white balance processing, exposure processing, noise reduction, and resolution processing. It is noteworthy that during this digital image processing, the image's color data is not compressed, and the image's color space is not altered, in order to preserve the image's color data to the maximum extent possible.

[0078] In other words, during digital image processing, it is inevitable to convert the RGB information of an image to other color spaces, such as YUV, to facilitate image processing. However, in the embodiments of this application, the color space volume that the image format can represent is not changed during the image conversion process. That is, the color volume that the YUV color space can represent should still be maintained at the BT.2020 color gamut size, without color gamut compression or color gamut transformation.

[0079] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, before displaying the second color image using a high-fidelity display, the method further includes: performing digital image processing on the second color image, wherein the color data of the second color image is not compressed and the color space of the second color image is not changed during the digital image processing.

[0080] Furthermore, in this embodiment, the second color image can be encoded and compressed before being displayed on a high-fidelity display. That is, when data transmission is required, to reduce the amount of image data during transmission, the image needs to be stored and encoded according to usage requirements. Specifically, depending on the image format requirements, formats such as bmp, jpg (jpeg), and TIFF can be used to encode the data. When the usage scenario requires video, mp4, AVI, mkv, and mov can be selected for video storage and encoding.

[0081] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, before displaying the second color image using a high-fidelity display, the method further includes: encoding and compressing the second color image.

[0082] After encoding and compression, data can be transmitted wirelessly or via wired connections when necessary. Wireless methods include 5G cellular networks, local area networks, and Bluetooth, while wired methods include fiber optic cables and other methods. The data can then be transmitted to a display at a predetermined location, such as the display in the attending physician's area, to facilitate diagnosis and treatment based on the image data.

[0083] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, after encoding and compressing the second color image, the method further includes: transmitting the encoded and compressed second color image to a high-fidelity display at a predetermined location.

[0084] Furthermore, in order to minimize the loss of color information in the image, high-bit data transmission is performed in this embodiment. Here, in the art, data transmission of at least 8 bits is generally referred to as high-bit data transmission. For example, specifically, high-bit data transmission can be 10-bit, 12-bit, or 14-bit data transmission.

[0085] Furthermore, in this embodiment of the application, in order to display the second color image on the high-fidelity display, it is also necessary to decode the second color image after encoding and compression.

[0086] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, after encoding and compressing the second color image, it further includes: decoding the second color image on the high-fidelity display.

[0087] Application Examples

[0088] As described above, the spectral-based medical image processing method according to the embodiments of this application can be applied to doctors making diagnoses based on image data. Figure 6 The illustration shows an application example of a spectral-based medical image processing method according to an embodiment of this application.

[0089] like Figure 6 As shown, in scenarios such as telemedicine, the doctor is not physically present with the patient. Therefore, the doctor can diagnose the patient's lesions and condition based on the displayed high-fidelity image data, combined with their own clinical experience, and provide corresponding treatment plans.

[0090] For this application example, at least two sets of image detection chip module components can be used to obtain spectral images of specific lesion areas or suspected lesion areas.

[0091] The first set of image detection chip module components includes a spectral chip module, an imaging chip module, and a beam splitter. The beam splitter is located on the path between the spectral chip and the imaging chip. That is, after the incident light reaches the beam splitter, a first portion of the light is deflected, and a second portion is transmitted. These are then received by the spectral chip and the imaging chip, respectively, to acquire spectral information (e.g., a first spectral image) and image information. Preferably, the spectral chip component further includes a light homogenizer, which is disposed between the beam splitter and the spectral chip to homogenize the light. The spectral chip then obtains the spectral information for lesion identification. It should be noted that since the surface to be tested is often uneven, such as the lesion area or suspected lesion area having unevenness, changes in the corresponding area during testing will cause different spectral responses in different areas, increasing the difficulty of lesion identification. However, with light homogenization, even if the area changes during testing, the overall spectral information remains unchanged.

[0092] In addition, the second set of image detection chip module components also includes at least an imaging chip module. The imaging chip acquires spectral images of healthy tissue outside the lesion. Besides directly comparing the acquired first spectral image with pre-stored pathological spectral images as described above, the second set of image detection module components can also acquire a second spectral image corresponding to the detected healthy tissue. This second spectral image further determines whether the first and second spectral images are different. If the spectral images are determined to be different, a lesion is present. This is then compared with the pre-stored spectral image corresponding to the pathological analysis of the lesion. When they match, the image is saved as a diagnostic basis for the doctor. If the first and second spectral images are confirmed to be identical, no further comparison is performed. This method further confirms the condition of the lesion and provides a more intuitive confirmation of whether a lesion has occurred. Of course, if the healthy tissue portion cannot be confirmed, or if a large area of ​​lesion has occurred, further multiple measurements and acquisitions are required, and even further interventional case analysis may be necessary.

[0093] Therefore, in the spectral-based medical image processing method according to the embodiments of this application, imaging a patient's lesion using a spectral camera includes: using a first set of image detection chip module components comprising a spectral chip module, an imaging chip module, and a beam splitter for imaging, wherein the beam splitter is located on the path between the spectral chip and the imaging chip, such that a first portion of light is deflected and a second portion of light is transmitted and received by the spectral chip and the imaging chip respectively, thereby acquiring spectral information and image information.

[0094] Furthermore, in the spectral-based medical image processing method according to the embodiments of this application, the spectral chip assembly includes a light homogenizer disposed between the beam splitter and the spectral chip, for homogenizing the light before the spectral chip obtains spectral information.

[0095] Furthermore, in the spectral-based medical image processing method according to the embodiments of this application, imaging the lesion site of the patient using a spectral camera further includes: imaging healthy tissue outside the lesion using a second set of image detection chip module components that includes at least an imaging chip module.

[0096] Furthermore, the spectral-based medical image processing method according to the embodiments of this application further includes: comparing a first spectral image obtained by the first group of image detection chip module components with a pre-stored pathological spectral image; and saving and recording the first spectral image in response to the first spectral image being the same as the pre-stored pathological spectral image.

[0097] Furthermore, the spectral-based medical image processing method according to the embodiments of this application further includes: comparing a first spectral image obtained by the first group of image detection chip module components with a second spectral image obtained by the second group of image detection chip module components; in response to the first spectral image being different from the second spectral image, comparing the first spectral image with a pre-stored pathological spectral image; and in response to the first spectral image being the same as the second spectral image, not comparing the first spectral image with the pre-stored pathological spectral image.

[0098] Exemplary device

[0099] Furthermore, embodiments of this application further relate to a spectral-based medical image processing apparatus that uses the spectral-based medical image processing method described above to enable the presentation of high-resolution spectral and color information to the user.

[0100] For example, the spectral-based medical image processing device may include an image acquisition unit and an image display unit. The image acquisition unit may be a spectral camera as described above, which includes at least two sets of image detection chip module components, and the image display unit may be a high-fidelity display as described above. Furthermore, the image acquisition unit and the image display unit can communicate with each other via high-bit data transmission.

[0101] Here, those skilled in the art will understand that the specific functions and operations of each unit and module of the aforementioned spectral-based medical image processing device have been referenced above. Figures 1 to 6 The spectral-based medical image processing method is described in detail, and therefore, its repeated description will be omitted.

[0102] As described above, the spectral-based medical image processing apparatus according to embodiments of this application can be implemented in various terminal devices including an image acquisition unit and an image display unit, such as a spectral imaging device including a display. In one example, the spectral-based medical image processing apparatus according to embodiments of this application can be integrated as a hardware module into the terminal device.

[0103] Alternatively, in another example, the spectral-based medical image processing device and the terminal device can also be separate devices, and the spectral-based medical image processing device can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0104] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0105] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0106] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0107] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0108] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A spectral-based medical image processing method, characterized by, The method comprises: acquiring a spectral image of the lesion; reconstructing color space values of each pixel point from the spectral image to obtain a first color image; color characterizing the first color image to convert the image color into a large color gamut range to obtain a second color image; and displaying the second color image using a high-fidelity display; wherein acquiring the spectral image of the lesion comprises: spectral imaging the lesion using a spectral camera; and reconstructing a spectral curve of each pixel from a raw signal of an image sensor of the spectral camera to obtain the spectral image; wherein the spectral camera comprises a light filtering structure and an image sensor; and imaging the lesion of the patient using the spectral camera comprises: imaging using a first set of image detection chip module assembly comprising a spectral chip module, an imaging chip module and a light splitting member, wherein the light splitting member is located on a path of the spectral chip and the imaging chip to make a first part of light be reflected and a second part of light be transmitted to be received by the spectral chip and the imaging chip respectively, so as to acquire spectral information and image information; comparing the first spectral image obtained by the first set of image detection chip module assembly with a pre-stored pathological spectral image; and in response to the first spectral image being the same as the pre-stored pathological spectral image, saving and recording the first spectral image; wherein imaging the lesion of the patient using the spectral camera further comprises imaging healthy tissue outside the lesion using a second set of image detection chip module assembly comprising at least the imaging chip module; wherein the method further comprises: comparing the first spectral image obtained by the first set of image detection chip module assembly with a second spectral image obtained by the second set of image detection chip module assembly; in response to the first spectral image being different from the second spectral image, comparing the first spectral image with the pre-stored pathological spectral image; and in response to the first spectral image being the same as the second spectral image, not comparing the first spectral image with the pre-stored pathological spectral image. Acquiring the spectral image of the lesion further comprises:

2. The spectral-based medical image processing method of claim 1, wherein, storing the spectral image and archiving data. Reconstructing color space values of each pixel point from the spectral image to obtain a first color image comprises:

3. The spectral-based medical image processing method of claim 1, wherein, obtaining color values of each pixel in XYZ color space according to the spectral curve of each pixel of the spectral image and the color standard of CIE by the following formula: The first color image is located in a color space independent of color gamut, and there is no compression and loss of color gamut in the process of color characterizing. where k is a normalization coefficient to limit the maximum value of Y to 100, and φ(λ) is the expression of the spectral curve, are the color standards of the XYZ color space defined by CIE, respectively.

4. The spectral-based medical image processing method of claim 1, wherein, The color space where the second color image is located is a color space larger than sRGB.

5. The spectral-based medical image processing method of claim 4, wherein, Displaying the second color image using a high-fidelity display comprises screen characterizing the high-fidelity display based on image data that has been color processed, and the screen characterizing comprises:

6. The spectral-based medical image processing method of claim 1, wherein, establishing a characterization sample, which is uniformly distributed in terms of brightness and chroma, covers various colors at various brightness levels, and the colors are uniformly distributed in terms of hue and saturation; displaying the characterization sample with known RGB values on the display screen; ​ measuring a colorimetric value of the display screen by a screen colorimeter; and establishing a relationship between the known RGB values and the measured colorimetric value.

7. The spectral-based medical image processing method of claim 1, wherein, further comprising, before displaying the second color image using the high fidelity display: performing digital image processing on the second color image without compressing color data of the second color image and without changing a color space of the second color image during the digital image processing.

8. The spectral-based medical image processing method of claim 1, wherein, further comprising, before displaying the second color image using the high fidelity display: encoding compressing the second color image.

9. The spectral-based medical image processing method of claim 8, wherein, further comprising, after encoding compressing the second color image: transmitting the encoded compressed second color image to a high fidelity display at a predetermined location.

10. The spectral-based medical image processing method of claim 8, wherein, further comprising, after encoding compressing the second color image:

11. The spectral-based medical image processing method of claim 1, wherein, decoding processing the second color image at the high fidelity display.

12. A spectral-based medical image processing apparatus, characterized by comprising: The spectral chip assembly comprises a light homogenizing member arranged between the light splitting member and the spectral chip, for homogenizing light before the spectral chip obtains spectral information from the light. using a spectral-based medical image processing method as claimed in any one of claims 1 to 11.

Images