An arthroscopic-based cartilage spectral dispersion four-dimensional imaging system and grading method

Abstract

The application discloses a kind of based on arthroscope's cartilage spectrum dispersion four-dimensional imaging system and grading method.System, comprising: light source module;Illumination modulation module, including digital micro-mirror device (DMD) and optical guiding structure, for generating programmable dot matrix illumination beam;Dispersive confocal module, including chromatic aberration arthroscope, light splitting prism and first sleeve lens, wherein chromatic aberration arthroscope can focus light of different wavelengths at different depth positions of optical axis;High spectral light splitting module for spectral light splitting to return beam;Image acquisition module for acquiring monochromatic confocal image after light splitting;Control processing terminal is used for synchronous control dot matrix pattern, spectral band switching and image acquisition, and executes three-dimensional topography reconstruction and osteoarthritis grading algorithm.The application can provide objective basis for early detection, quantitative evaluation and surgical boundary confirmation of intraoperative cartilage degeneration, with the characteristics of non-destructive, multi-modal fusion and fast detection speed.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of biomedical optoelectronic detection, minimally invasive surgical navigation, and medical artificial intelligence, specifically relating to a non-destructive tissue detection device that combines hyperspectral imaging and dispersive confocal microscopy. More specifically, this invention relates to an arthroscopic-based cartilage spectral dispersive four-dimensional imaging system, and a method for rapidly and automatically grading the severity of intraoperative osteoarthritis using morphological-biochemical multimodal data acquired by this system. Background Technology

[0002] Osteoarthritis (KOA) is a degenerative joint disease that severely impacts human quality of life. Its core pathological change lies in the progressive wear and destruction of articular cartilage. As a cushioning structure on the joint surface, irreversible damage to cartilage leads to joint pain, stiffness, and even disability. Therefore, early diagnosis, accurate grading, and real-time intraoperative assessment of osteoarthritis are of extremely high clinical value for developing treatment plans and improving patient prognosis.

[0003] Currently, the clinical diagnosis of osteoarthritis mainly relies on non-invasive imaging techniques such as X-rays, CT scans, and MRI (magnetic resonance imaging). However, X-rays and CT scans are primarily sensitive to bone structures and are difficult to detect early cartilage lesions; while conventional MRI can display cartilage morphology, its spatial resolution is limited, making it difficult to observe micron-level micro-damage on the cartilage surface, and it cannot provide specific information reflecting changes in early biochemical components of cartilage (such as proteoglycan and collagen loss).

[0004] Arthroscopy is widely recognized as the "gold standard" for the diagnosis and treatment of osteoarthritis. Under arthroscopy, surgeons can directly observe the internal structure of the joint and assess the degree of cartilage damage according to the ICRS (International Cartilage Repair Society) or Outerbridge grading system. However, most existing commercial arthroscopy systems use wide-field white light illumination imaging, providing only two-dimensional color images. This traditional method has significant limitations: First, it lacks biochemical specificity: early cartilage degeneration often begins with changes in biochemical components (such as proteoglycans), followed by surface roughness or defects. White light arthroscopy cannot identify changes in biochemical components invisible to the human eye, easily missing early lesions (ICRS grade 0 or I). Second, it lacks quantitative depth information: traditional arthroscopy cannot quantitatively measure the microscopic roughness or defect depth of the cartilage surface; surgeons can only rely on experience for qualitative descriptions, leading to highly subjective grading results and inter-observer variability. Third, intraoperative decision-making relies on experience: When performing cartilage reshaping surgery, doctors often find it difficult to accurately define the boundary between diseased and healthy tissue, which often leads to incomplete or excessive resection.

[0005] To overcome these problems, researchers have attempted to introduce advanced optical technologies such as optical coherence tomography (OCT), Raman spectroscopy, and hyperspectral imaging into cartilage detection. OCT possesses excellent depth tomography capabilities and can construct three-dimensional tissue morphology, but it is insensitive to biochemical components and struggles to distinguish between cartilage degeneration and physical damage. While hyperspectral imaging (HSI) can acquire rich "map-spectrum integration" biochemical fingerprint information, conventional implementations often lack height / depth dimensions, failing to correct spectral scattering errors caused by cartilage surface curvature or unevenness. Furthermore, traditional push-broom hyperspectral imaging is slow and cannot meet the demands of real-time intraoperative clinical detection.

[0006] In summary, there is currently a lack of a detection system capable of simultaneously acquiring the microscopic three-dimensional morphology of cartilage (reflecting physical damage) and hyperspectral biochemical information (reflecting physiological degeneration), and capable of automated and objective grading based on this multimodal data. Developing an arthroscopic-based detection system with "four-dimensional imaging (3D morphology + 1D spectroscopy)" capabilities, and utilizing deep learning to mine multidimensional data features to assist surgeons in intraoperative decision-making, is a key technical challenge that urgently needs to be addressed in the field of biomedical optoelectronic detection. Summary of the Invention

[0007] The main objective of this invention is to provide an arthroscopic-based four-dimensional cartilage spectral dispersive imaging system and grading method to solve the technical problems of existing arthroscopic techniques that can only acquire two-dimensional surface images, lack biochemical specificity and depth quantitative information, and rely entirely on the doctor's subjective experience for grading, which can easily lead to missed or misdiagnosed early osteoarthritis.

[0008] To achieve the above objectives, the present invention provides the following technical solution: An arthroscopic-based four-dimensional cartilage spectral dispersive imaging system includes: The light source module is used to provide broadband illumination light; An illumination modulation module, including a digital micromirror device (DMD) and an optical guidance structure, is used to generate a programmable dot matrix illumination beam; The dispersive confocal module includes a chromatic arthroscope, a beam splitter prism, and a first sleeve lens. The chromatic arthroscope has axial chromatic aberration characteristics, which enables light of different wavelengths to be focused at different depth positions on the optical axis. The beam splitter prism is used to guide the illumination beam to the cartilage surface and guide the reflected beam to the detection branch. The hyperspectral beam splitting module includes a liquid crystal tunable filter (LCTF) and a second sleeve lens for spectral splitting of the returning beam; The image acquisition module includes a camera for acquiring a monochromatic confocal image after spectral dispersion; The control and processing terminal is electrically connected to the DMD, LCTF and camera to synchronously control the dot matrix pattern, spectral band switching and image acquisition, and to execute three-dimensional topography reconstruction and osteoarthritis grading algorithms.

[0009] The control processing terminal includes a processor, a memory, and a control program, wherein the control program is configured as follows: Establish the geometric mapping relationship between the DMD micromirror unit and the camera pixel unit, and define the virtual pinhole; Control the DMD to generate dot matrix patterns, control the LCTF to scan spectral bands, and synchronously trigger the camera to acquire images; Based on the principle of axial color difference encoding, the three-dimensional morphology of the cartilage surface is reconstructed according to the collected defocus spectral response data; Three-dimensional topographic data and hyperspectral data are fused and input into a pre-trained deep learning model for osteoarthritis grading.

[0010] A method for multidimensional data acquisition and three-dimensional morphology reconstruction of cartilage tissue using the system described above includes the following steps: Step S1 Virtual pinhole construction and calibration: Control the DMD projection calibration array, acquire camera images, calculate the geometric mapping relationship between the DMD micromirrors and camera pixels, and define a region of interest (ROI) as a virtual pinhole for each illumination point on the camera image; Step S2 Confocal Spectral Data Scanning: Control the DMD to generate a sparse scanning dot pattern and simultaneously control the LCTF to switch within a preset band. The camera acquires the confocal intensity signal filtered through a virtual pinhole to obtain defocus spectral response data. Step S3 Three-dimensional micro-morphology reconstruction: Based on the defocused spectral response data obtained in step S2, the height value of each point on the sample surface is calculated using the axial color difference encoding principle to construct a three-dimensional morphology map; Step S4: Acquisition of hyperspectral reflectance data: Control the DMD to perform full-field illumination, control the LCTF to perform step scanning across the entire band, and acquire a two-dimensional hyperspectral image cube.

[0011] The three-dimensional microstructure reconstruction in step S3 includes two optional modes: Mode 1: Full-spectrum peak localization method, which controls the LCTF to perform full-band subdivision scanning within the linear dispersion range of the color difference arthroscope, constructs a discrete spectral response sequence for each pixel, calculates the center wavelength corresponding to the spectral response peak through Gaussian fitting, and calculates the height value by combining a pre-calibrated wavelength-depth lookup table; Mode 2: Dual-wavelength linear region solution algorithm. Two specific wavelengths λ1 and λ2 within the monotonic range of color difference are selected. The LCTF is controlled to collect dot matrix images only at λ1 and λ2. The intensity ratio or normalized difference under the two wavelengths is calculated to construct a linear index. The height value is directly solved using this index.

[0012] In Mode 1, the calculation process for the height value Z is as follows: By fitting the discrete spectral intensity values ​​λ at the virtual pinhole using a Gaussian function, the center wavelength λ corresponding to the peak of the spectral response can be obtained. p : ; in These are discrete spectral intensity values ​​that vary with wavelength and are collected at a virtual pinhole. is the peak intensity constant of the spectral response curve; This refers to the current scan wavelength or variable of the liquid crystal tunable filter (LCTF). is the standard deviation of the Gaussian distribution, used to characterize the full width at half maximum (FWHM) of the spectral response curve; According to the pre-calibrated wavelength-depth mapping function F(λ) p ), calculate the height Z of the target point: ; in The order of the fitted polynomial; For the k-th order calibration coefficient; The center wavelength value calculated by the previous formula is used as the input variable here.

[0013] In Mode 2, the calculation process for the height value Z is as follows: Two wavelengths, λ1 and λ2, located in the linear region of the axial response curve of the chromatic aberration arthroscopy were selected to construct a normalized intensity difference index. : ; in , The confocal intensity values ​​collected by the camera within a virtual pinhole at two selected fixed wavelength locations; The height value Z is calculated directly using a pre-defined linear relationship: ; Where k is the linear sensitivity coefficient and d is the intercept constant.

[0014] A method for grading osteoarthritis cartilage based on the system includes the following steps: Step A: Use the system to acquire hyperspectral image data cubes of the cartilage sample to be tested. and three-dimensional topographic data ; Step B: Data preprocessing and fusion, spatially registering hyperspectral data with 3D topographic data to construct a four-dimensional multimodal data tensor. The tensor contains N spectral channels and 1 depth channel; Step C: Deep learning model inference, inputting the data tensor into a pre-trained convolutional neural network model to extract and fuse spectral biochemical features and morphological texture features; Step D: Automatic grading output. The severity grading results of osteoarthritis are output through the fully connected layers and classification activation function of the network model.

[0015] The convolutional neural network model in step C employs a dual-stream input or multi-channel stacking architecture; the data tensor It is an H×W×(N+1) dimensional tensor formed by stacking N bands of hyperspectral images with a grayscale depth map of 1 band in the channel dimension; the backbone network of the network model adopts the ResNet architecture to extract the biochemical composition change features and micro-roughness features of the cartilage surface, and outputs the probability distribution corresponding to the Outerbridge gradation through the Softmax function.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] 1. Multimodal four-dimensional data fusion enhances diagnostic dimensions: This invention innovatively integrates dispersive confocal microscopy and hyperspectral imaging technologies. Compared to OCT, which only provides morphological information, or traditional hyperspectral technology, which only provides spectral information, this system can simultaneously acquire micron-level three-dimensional morphology of cartilage surfaces (reflecting physical wear and roughness) and hyperspectral fingerprint information (reflecting the content of biochemical components such as collagen and proteoglycans), achieving "physical-biochemical" four-dimensional complementarity and significantly improving the detection rate of early, small lesions.

[0018] 2. DMD-based Virtual Pinhole Technology: No Moving Parts and High Speed: This invention utilizes a DMD to generate a programmable illumination array and, in conjunction with software algorithms, constructs a "virtual pinhole," replacing the mechanical scanning galvanometer and physical pinhole in traditional confocal microscopes. This design not only eliminates complex mechanical moving parts, improving system stability and lifespan, making it particularly suitable for scenarios with high vibration resistance requirements, such as arthroscopy, but also significantly increases imaging speed through parallel array scanning, meeting the needs of rapid intraoperative detection.

[0019] 3. Axial color difference encoding for axial-free 3D reconstruction: This invention utilizes the axial color difference characteristics of color difference arthroscopes to encode depth using wavelength, enabling 3D tomography without moving the probe or sample along the optical axis. This is particularly crucial for arthroscopic surgery, avoiding the mechanical extension and retraction of the probe within the narrow joint cavity, thus ensuring surgical safety and ease of operation.

[0020] 4. Deep Learning-Assisted Automated Objective Grading: This invention combines deep convolutional neural networks such as ResNet to construct a grading model specifically for morphological-spectral multimodal data. This method can automatically extract high-dimensional features that are difficult for the human eye to recognize, providing probability predictions for output ICRS or Outerbridge grading. It overcomes the problems of reliance on physician subjective experience and poor inter-observer consistency in traditional arthroscopic examinations, providing an objective basis for quantitative assessment and surgical decisions in osteoarthritis.

[0021] 5. Dual-mode reconstruction, balancing accuracy and speed: This invention provides two working modes: full-spectrum peak localization (high accuracy) and dual-wavelength linear solution (high speed). Doctors can choose the high-precision mode when detailed observation is needed and the high-speed mode when rapid browsing is needed, depending on the actual needs during the operation, which has good clinical adaptability. Attached Figure Description

[0022] Figure 1 : Overall optical path diagram of the system; wherein, knee joint 1, chromatic aberration arthroscope 2, beam splitter prism 3, first sleeve lens 4, total internal reflection prism 5, digital micromirror device 6, collimating lens 7, LED light source 8, second sleeve lens 9, liquid crystal tunable filter (LCTF) 10, camera 11.

[0023] Figure 2 Comparison of multimodal images of samples at different Outerbridge levels (top row: hyperspectral composite image; bottom row: 3D topography image).

[0024] Figure 3 The average reflectance spectrum curves of samples at each level (normalized to 620nm) show a "redshift" trend and a rebound at level IV.

[0025] Figure 4 The classification confusion matrix shows the model's ability to distinguish between Level I and Level IV. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0027] Example 1: System Construction and Data Acquisition for In Vitro Arthroscopy

[0028] This embodiment discloses a four-dimensional imaging detection system and data acquisition method for cartilage spectral dispersive imaging for in vivo arthroscopic examination. For example... Figure 1 As shown, the system adopts a split design in terms of physical architecture, mainly consisting of an optical control host placed on a mobile trolley and a handheld chromatic aberration arthroscope connected by optical fiber or relay lens.

[0029] The system includes: a chromatic aberration arthroscope 2, a beam splitter prism 3, a first sleeve lens 4, a total internal reflection prism 5, a digital micromirror device 6, a collimating lens 7, an LED light source 8, a second sleeve lens 9, a liquid crystal tunable filter 10, and a camera 11; the LED light source 8 is used to emit broadband illumination light. The collimating lens 7 is disposed in the output light path of the LED light source 8 to collimate the illumination light into a parallel beam. The total internal reflection prism 5 is disposed between the collimating lens 7 and the digital micromirror device 6, configured to guide the incident parallel beam to the surface of the digital micromirror device 6 through total internal reflection. The digital micromirror device 6 is configured to modulate the incident beam to generate an illumination dot matrix and reflect the illumination dot matrix back through the total internal reflection prism 5. The first sleeve lens 4, the beam splitter prism 3, and the colorimetric arthroscope 2 are sequentially disposed in the output light path of the digital micromirror device 6. After being relayed by the first sleeve lens 4 and transmitted through the beam splitter prism 3, the illumination dot matrix is ​​guided by the colorimetric arthroscope 2 into the joint cavity and imaged onto the cartilage surface inside the knee joint 1. Utilizing the axial chromatic aberration characteristics of the colorimetric arthroscope 2, light of different wavelengths is focused at different depth positions along the optical axis. The beam splitter prism 3 is also configured to reflect the return beam reflected from the cartilage surface and collected by the colorimetric arthroscope 2 to the detection branch. The second sleeve lens 9, the liquid crystal tunable filter 10, and the camera 11 are sequentially arranged in the detection branch; the liquid crystal tunable filter 10 is configured to perform spectral splitting on the returned beam, and the camera 11 is configured to acquire the monochromatic confocal image after spectral splitting.

[0030] The control and processing terminal is electrically connected to the digital micromirror device 6, the liquid crystal tunable filter 10 and the camera 11, respectively, and is used to synchronously control the dot matrix pattern switching, spectral band scanning and image acquisition, and execute three-dimensional morphology restoration and osteoarthritis grading algorithms.

[0031] The optical control unit integrates the system's core optoelectronic components, including a high-power broadband LED light source covering the 400nm to 700nm visible wavelength range, a digital micromirror device (DMD), a liquid crystal tunable filter (LCTF), and an sCMOS camera. The chromatic arthroscope, as an extension of the system, is designed with a specialized strong dispersive lens group, artificially introducing significant axial chromatic aberration. This allows light of different wavelengths to be focused at different depths along the optical axis, establishing a precise wavelength-depth coded mapping. This probe design eliminates the need for axial mechanical scanning components required by traditional confocal microscopes, making it suitable for the narrow operating space within the joint cavity. The system's optical path and imaging process are as follows: Broadband light emitted from the LED light source is collimated into a parallel beam by a collimating lens and guided to the DMD surface by a total internal reflection prism. The DMD, acting as a spatial light modulator, generates a programmable illumination dot pattern. This illumination dot pattern, relayed by the first sleeve lens and transmitted through a beam splitter prism, enters the chromatic arthroscope and is projected onto the cartilage sample surface within the knee joint. Due to the dispersive characteristics of the probe, the broadband illumination light forms a focused light field with axial chromatic aberration on the cartilage surface. The reflected light from the cartilage surface returns along the original path, is reflected by the beam splitter prism, and enters the detection branch. In the detection branch, the beam passes sequentially through the second sleeve lens and the LCTF, and is finally imaged onto the sCMOS camera target surface. During this process, the system employs "virtual pinhole" technology to suppress background scattering noise. Specifically, the conjugate geometric relationship between the DMD micromirror unit and the camera pixel unit is pre-calibrated using a software algorithm. A tiny region of interest is defined as a virtual pinhole for each DMD illumination point on the camera image, and only the light intensity signal within this region is extracted, thus achieving a confocal tomography effect similar to a physical pinhole. When using this system for cartilage detection, the LCTF is first controlled to perform a fine spectral scan across the entire wavelength range. For each pixel within the field of view, the camera records its spectral response curve as a function of wavelength. Due to the principle of chromatic aberration confocalization, only when the focal point of a certain wavelength precisely falls on the cartilage surface can the reflected light of that wavelength pass through the virtual pinhole most efficiently, thus forming a significant intensity peak on the spectral response curve. The system uses a Gaussian fitting algorithm to process the discrete spectral data and accurately locates the center wavelength at which the spectral response reaches its peak according to the formula. By combining a pre-calibrated wavelength-depth lookup table, the center wavelength is converted into the height value Z of that point, thus reconstructing the micron-level three-dimensional morphology of the cartilage surface without mechanical scanning. At the same time, the system simultaneously extracts the spectral reflectance data of each point, forming a multimodal data cube containing two-dimensional morphological information (reflecting physical wear and roughness) and three-dimensional hyperspectral information (reflecting subtle color changes in different regions).

[0032] To verify the clinical effectiveness of the system, this embodiment used the aforementioned device to collect four-dimensional datasets of cartilage samples from different Outerbridge grades. The test results revealed highly clinically challenging nonlinear multimodal evolution patterns: such as... Figure 2 As shown, in terms of microscopic morphology, Grade I cartilage and Grade IV exposed sclerotic surfaces exhibited relatively smooth characteristics (low RMS values), while Grade II and III samples showed significant fibrosis and "crab-like" fissures; Figure 3 As shown, in terms of spectral response, samples from Grade I to Grade III exhibit a "redshift" trend, with the reflectance in the short-wavelength band (430-500nm) gradually decreasing due to collagen oxidation and the accumulation of glycosylation products (AGEs). In contrast, Grade IV bone surfaces show a rebound in short-wavelength reflectance due to their pale color. This pattern indicates that relying solely on single-modal information is highly prone to misjudgment. To address this challenge, this embodiment constructs a deep convolutional neural network based on the ResNet architecture for feature fusion and automatic classification. Specifically, the system spatially registers and stacks channels of N bands (e.g., 51 bands) of hyperspectral images with a single band of grayscale depth topography image, constructing a four-dimensional input tensor. This tensor is input into a pre-trained ResNet-50 backbone network, which uses multi-layer convolutional kernels to simultaneously extract spatial texture features (cracks, roughness) and biochemical spectral features (yellowing, whitening fingerprints) of the cartilage surface. After deep features are subjected to global average pooling, the probability distribution of belonging to Outerbridge levels I-IV is output through a fully connected layer and a Softmax function. The classification results are as follows: Figure 4 As shown in the figure. This method effectively addresses the clinical challenge of easily confusing grade IV smooth bone surfaces with grade I early cartilage, achieving objective and accurate grading of the entire course of osteoarthritis.

[0033] The system adopts a split design: optical main unit (including LED, DMD, LCTF, and camera) + handheld color difference arthroscope.

[0034] Probe design: Built-in dispersive lens group, axial chromatic difference range ≥0.3μm / nm, covering the 400~700nm band.

[0035] Imaging speed: In dot matrix scanning mode, the imaging time for a single field of view (1280×720 pixels) is <3 seconds (dual wavelength mode); in full spectrum mode, it is <40 seconds.

[0036] 3D reconstruction accuracy: After calibration, the depth measurement resolution is <1.9μm.

[0037] Classification results: In a test of 100 samples, the ResNet model achieved a classification accuracy of 92.5% for Outerbridge I-IV levels with an F1-score of 0.91, which was significantly better than the subjective assessment of doctors (Kappa=0.63).

[0038] Example 2: High-precision calibration of benchtop in vitro detection and system

[0039] This embodiment discloses a desktop ex vivo cartilage detection configuration and system high-precision calibration method based on a microscope architecture. Unlike the handheld probe for in vivo arthroscopic detection in Embodiment 1, this embodiment aims to provide high-precision spectral-depth calibration parameters for the entire system and to construct a large-scale osteoarthritis pathology database for training deep learning models. In terms of hardware architecture, this embodiment integrates the aforementioned LED light source, DMD illumination modulation module, LCTF spectrophotometer module, and camera module into a vertical microscope body. However, at the imaging end, a high numerical aperture chromatic aberration microscope objective is used instead of the arthroscopic probe, and a high-precision motorized displacement stage with X, Y, and Z-axis motion capabilities is provided. The positioning accuracy of the Z-axis along the optical axis is better than 0.1 micrometers, which is used to carry ex vivo cartilage samples or standard plane mirror calibration components.

[0040] One of the core functions of this embodiment is to utilize the precise stepping capability of the motorized stage to calibrate the spectral-depth mapping relationship, which is a prerequisite for achieving the "in vivo scan-free 3D reconstruction" in Embodiment 1. The specific calibration process is as follows: a standard broadband high-reflectivity plane mirror is placed on the stage, and the motorized stage is controlled to perform axial layer-by-layer scanning within the working distance range of the chromatic aberration objective lens at a step size of 0.5 micrometers. At each Z-axis position, the system records the confocal spectral response at the DMD virtual pinhole, extracts the peak wavelength with the highest intensity, and thus fits a high-precision wavelength-depth mapping curve (LUT). Once this calibration curve is generated, it can be embedded into the control software of the handheld arthroscopy system in Embodiment 1. This allows the surgeon to use the probe during surgery without any mechanical movement, simply by reading the spectral peak value, and then use the calibration curve to inversely solve for the absolute depth information of the cartilage surface, thereby achieving a precision transfer from "stage calibration" to "in vivo measurement." This method is integrated into the acquisition control software.

[0041] Furthermore, this embodiment is also used to establish a high-standard multimodal database for osteoarthritis to support the training of the ResNet model described in Embodiment 2. Utilizing the stage's motorized scanning and stitching function along the X and Y axes, the system can perform panoramic imaging of large-diameter ex vivo tibial plateau or femoral condyle cartilage samples, acquiring micron-level four-dimensional data at a macro-centimeter scale. Subsequently, the scanned ex vivo samples are subjected to standard histological sections and Safranin O-Fast Green staining. Using the rigid coordinate system of the tabletop system, the four-dimensional photoelectric data and pathological section images are precisely registered at the pixel level. In this way, the pathologist's diagnostic results are used as the "gold standard" for fully supervised training of the deep learning model, ensuring that the final grading results output by the clinical in vivo detection system have high pathological reliability. In summary, this embodiment is not only an independent ex vivo pathological detection tool but also an indispensable "calibration station" and "data factory" within the entire technical system.

[0042] It adopts a microscope architecture and is equipped with a high-precision motorized stage (Z-axis accuracy 0.5μm).

[0043] Calibration procedure: Use a standard plane mirror with a step size of 0.5μm, scan the wavelength range of 430~680nm, and the interval is 1nm to generate a wavelength-depth lookup table (LUT).

[0044] Dataset construction: Four-dimensional data of isolated cartilage samples were collected, stained with safranin O-fast green, and registered pixel-level with pathological images as supervision labels.

[0045] Model training: ResNet-50 was trained using 5000 images segmented from 250 complete images, with a batch size of 128, a learning rate of 0.001, and 200 training epochs.

[0046] The embodiments described above can be further combined or replaced, and these embodiments are merely descriptions of preferred embodiments of the present invention, not limitations on the concept and scope of the present invention. Various changes and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the inventive concept are all within the protection scope of the present invention. The protection scope of the present invention is given by the appended claims and any equivalents.

Claims

1. A four-dimensional arthroscopic cartilage spectral dispersive imaging system, characterized in that, include: The light source module is used to provide broadband illumination light; An illumination modulation module, including a digital micromirror device (DMD) and an optical guidance structure, is used to generate a programmable dot matrix illumination beam; The dispersive confocal module includes a chromatic arthroscope, a beam splitter prism, and a first sleeve lens. The chromatic arthroscope has axial chromatic aberration characteristics, which enables light of different wavelengths to be focused at different depth positions on the optical axis. The beam splitter prism is used to guide the illumination beam to the cartilage surface and guide the reflected beam to the detection branch. The hyperspectral beam splitting module includes a liquid crystal tunable filter (LCTF) and a second sleeve lens for spectral splitting of the returning beam; The image acquisition module includes a camera for acquiring a monochromatic confocal image after spectral dispersion; The control and processing terminal is electrically connected to the DMD, LCTF and camera to synchronously control the dot matrix pattern, spectral band switching and image acquisition, and to execute three-dimensional topography reconstruction and osteoarthritis grading algorithms.

2. The detection system according to claim 1, characterized in that, The control processing terminal includes a processor, a memory, and a control program, wherein the control program is configured as follows: Establish the geometric mapping relationship between the DMD micromirror unit and the camera pixel unit, and define the virtual pinhole; Control the DMD to generate dot matrix patterns, control the LCTF to scan spectral bands, and synchronously trigger the camera to acquire images; Based on the principle of axial color difference encoding, the three-dimensional morphology of the cartilage surface is reconstructed according to the collected defocus spectral response data; Three-dimensional topographic data and hyperspectral data are fused and input into a pre-trained deep learning model for osteoarthritis grading.

3. A method for multidimensional data acquisition and three-dimensional morphology reconstruction of cartilage tissue using the system described in claim 1, characterized in that, Includes the following steps: Step S1 Virtual pinhole construction and calibration: Control the DMD projection calibration array, acquire camera images, calculate the geometric mapping relationship between the DMD micromirrors and camera pixels, and define a region of interest (ROI) as a virtual pinhole for each illumination point on the camera image; Step S2 Confocal Spectral Data Scanning: Control the DMD to generate a sparse scanning dot pattern and simultaneously control the LCTF to switch within a preset band. The camera acquires the confocal intensity signal filtered through a virtual pinhole to obtain defocus spectral response data. Step S3 Three-dimensional micro-morphology reconstruction: Based on the defocused spectral response data obtained in step S2, the height value of each point on the sample surface is calculated using the axial color difference encoding principle to construct a three-dimensional morphology map; Step S4: Acquisition of hyperspectral reflectance data: Control the DMD to perform full-field illumination, control the LCTF to perform step scanning across the entire band, and acquire a two-dimensional hyperspectral image cube.

4. The method according to claim 3, characterized in that, The three-dimensional microstructure reconstruction in step S3 includes two optional modes: Mode 1: Full-spectrum peak localization method, which controls the LCTF to perform full-band subdivision scanning within the linear dispersion range of the color difference arthroscope, constructs a discrete spectral response sequence for each pixel, calculates the center wavelength corresponding to the spectral response peak through Gaussian fitting, and calculates the height value by combining a pre-calibrated wavelength-depth lookup table; Mode 2: Dual-wavelength linear region solution algorithm. Two specific wavelengths λ1 and λ2 within the monotonic range of color difference are selected. The LCTF is controlled to collect dot matrix images only at λ1 and λ2. The intensity ratio or normalized difference under the two wavelengths is calculated to construct a linear index. The height value is directly solved using this index.

5. The method according to claim 4, characterized in that, In Mode 1, the calculation process for the height value Z is as follows: By fitting the discrete spectral intensity values ​​λ at the virtual pinhole using a Gaussian function, the center wavelength λ corresponding to the peak of the spectral response can be obtained. p : ； in These are discrete spectral intensity values ​​that vary with wavelength and are collected at a virtual pinhole. is the peak intensity constant of the spectral response curve; This refers to the current scan wavelength or variable of the liquid crystal tunable filter (LCTF). is the standard deviation of the Gaussian distribution, used to characterize the full width at half maximum (FWHM) of the spectral response curve; According to the pre-calibrated wavelength-depth mapping function F(λ) p ), calculate the height Z of the target point: ； in The order of the fitted polynomial; For the k-th order calibration coefficient; The center wavelength value calculated by the previous formula is used as the input variable here.

6. The method according to claim 4, characterized in that, In Mode 2, the calculation process for the height value Z is as follows: Two wavelengths, λ1 and λ2, located in the linear region of the axial response curve of the chromatic aberration arthroscopy were selected to construct a normalized intensity difference index. : ； in , The confocal intensity values ​​collected by the camera within a virtual pinhole at two selected fixed wavelength locations; The height value Z is calculated directly using a pre-defined linear relationship: ； Where k is the linear sensitivity coefficient and d is the intercept constant.

7. A method for grading osteoarthritis cartilage based on the system described in claim 1, characterized in that, Includes the following steps: Step A: Use the system to acquire hyperspectral image data cubes of the cartilage sample to be tested. and three-dimensional topographic data ; Step B: Data preprocessing and fusion, spatially registering hyperspectral data with 3D topographic data to construct a four-dimensional multimodal data tensor. The tensor contains N spectral channels and 1 depth channel; Step C: Deep learning model inference, inputting the data tensor into a pre-trained convolutional neural network model to extract and fuse spectral biochemical features and morphological texture features; Step D: Automatic grading output. The severity grading results of osteoarthritis are output through the fully connected layers and classification activation function of the network model.

8. The method according to claim 7, characterized in that, The convolutional neural network model in step C employs a dual-stream input or multi-channel stacking architecture; the data tensor It is an H×W×(N+1) dimensional tensor formed by stacking N bands of hyperspectral images with a grayscale depth map of 1 band in the channel dimension; the backbone network of the network model adopts the ResNet architecture to extract the biochemical composition change features and micro-roughness features of the cartilage surface, and outputs the probability distribution corresponding to the Outerbridge gradation through the Softmax function.

Images