In-situ projection system, method and device of non-common-optical-axis structure

Abstract

The embodiment of the invention provides an in-situ projection system, method and device of a non-common optical axis structure, and the system is characterized in that an indication light source is used for transmitting indication light of a visible light wave band to a sample tissue in a calibration stage, so as to form a first projection pattern on the sample tissue; the imaging assembly is used for shooting the first projection pattern in the calibration stage to obtain first image data; the projection assembly is used for projecting visible light to the sample tissue according to the first image data in the calibration stage so as to form a second projection pattern on the sample tissue; passively adjusting the own pose until the first projection pattern coincides with the second projection pattern; the imaging assembly is further used for collecting fluorescence emitted by the target tissue in the operation stage to obtain second image data; the projection assembly is further used for projecting visible light to the target tissue according to the second image data in the operation stage so as to form a third projection pattern on the target tissue. By applying the technical scheme provided by the embodiment of the invention, the non-optical-axis-sharing in-situ projection system can be realized.

Description

Technical Field

[0001] The present invention relates to the field of projection technology, and in particular to an in-situ projection system, method and device of a non-co-optical axis structure. Background Art

[0002] During surgery, some tissues emit light that is not in the visible light band and is invisible to the naked eye. Therefore, in related technologies, tissues are photographed using image acquisition devices that can capture this light, and then the captured images are projected to the original location of the tissue using projection equipment. This process is called in-situ projection, and the system that implements in-situ projection is called an in-situ projection system. Through in-situ projection, users can clearly see the tissues, making surgery easier.

[0003] In order to ensure that the projected position can return to its original position, current in-situ projection systems usually use a projection component and an imaging component that are co-axial. However, in the case of a co-axial projection, the imaging component and the projection component are easily blocked. Therefore, in some solutions, how to realize an in-situ projection system without co-axial projection has become a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide an in-situ projection system, method, and device with a non-co-optical axis structure to implement an in-situ projection system with a non-co-optical axis. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides an in-situ projection system with a non-co-optical axis structure, the system comprising:

[0006] An imaging component, an excitation light source, an indicator light source, and a projection component with adjustable posture, wherein the imaging component and the projection component do not share a common optical axis;

[0007] The indicator light source is used to emit indicator light in the visible light band to the sample tissue during the calibration phase to form a first projection pattern on the sample tissue;

[0008] The imaging component is configured to capture the first projection pattern during the calibration phase to obtain first image data;

[0009] The projection assembly is configured to project visible light onto the sample tissue according to the first image data during the calibration phase to form a second projection pattern on the sample tissue; and passively adjust its own position until the first projection pattern and the second projection pattern coincide with each other;

[0010] The excitation light source is used to emit excitation light to the target tissue during the operation phase to excite the target tissue to produce fluorescence;

[0011] The imaging component is further configured to collect fluorescence emitted by the target tissue during the operation phase to obtain second image data;

[0012] The projection component is further configured to project visible light onto the target tissue according to the second image data during the operation phase, so as to form a third projection pattern on the target tissue.

[0013] In some embodiments, the imaging assembly includes a focusing assembly, a first image sensor, a second image sensor, and a beam splitter;

[0014] The first image sensor and the second image sensor share a common optical axis, and the first image sensor, the second image sensor, and the projection assembly do not share a common optical axis;

[0015] The focusing assembly is used to focus the light reflected and emitted by the target tissue and the sample tissue, and guide the light to the beam splitter;

[0016] The beam splitter is configured to direct the incident fluorescence to the first image sensor and direct the incident indicator light to the second image sensor;

[0017] The first image sensor is used to capture the fluorescence generated by the target tissue during the operation phase to obtain second image data;

[0018] The second image sensor captures the first projection pattern during the calibration phase to obtain first image data.

[0019] In some embodiments, the beam splitter is further configured to direct the incident excitation light to the second image sensor;

[0020] The second image sensor is further configured to capture the excitation light reflected by the target tissue during the operation phase to obtain third image data;

[0021] The projection component is specifically configured to project visible light toward the target tissue according to the second image data and the third image data during the operation phase, so as to form a third projection pattern on the target tissue.

[0022] In some embodiments, the beam splitter is a dichroic mirror.

[0023] In some embodiments, the posture of the imaging component is adjustable, and the system further comprises a control component and a posture sensor;

[0024] The posture sensor is configured to detect, in the calibration phase, a first posture of the imaging assembly and a second posture of the projection assembly in response to the first projection pattern and the second projection pattern overlapping each other; and to detect, in the operation phase, a third posture of the imaging assembly;

[0025] The control component is configured to, during the operation phase, drive the projection module to move to the second posture in response to the third posture being equal to the first posture.

[0026] In some embodiments, the indicator light source is further configured to emit indicator light in the visible light band toward the sample tissue during the operation phase, so as to form a fourth projection pattern on the target tissue;

[0027] The imaging component is specifically configured to capture the fluorescence generated by the target tissue and the reflected indicator light during the operation phase to obtain second image data;

[0028] The projection assembly is further configured to passively adjust its own posture during the operation phase until the third projection pattern and the indicator light spot in the fourth projection pattern coincide with each other.

[0029] In some embodiments, the posture of the imaging component is adjustable, and the system further comprises a control component and a posture sensor;

[0030] The posture sensor is configured to detect, in response to the first projection pattern and the second projection pattern overlapping, a first posture of the imaging assembly and a second posture of the projection assembly during the calibration phase, and to detect, in the operation phase, a third posture of the imaging assembly;

[0031] The control component is configured to, during the operation phase, drive the projection component to move to the second posture in response to the third posture being equal to the first posture;

[0032] The indicator light source is further used to emit indicator light in the visible light band to the target tissue during the operation phase to form a fourth projection pattern on the target tissue;

[0033] The first image sensor is used to capture the fluorescence generated by the target tissue during the operation phase, and the second image sensor is also used to capture the reflected indicator light during the operation phase to obtain second image data;

[0034] The projection assembly is further configured to passively adjust its own posture during the operation phase until the third projection pattern coincides with the indicator light spot in the fourth projection pattern.

[0035] In a second aspect, an embodiment of the present application further provides an in-situ projection method for a non-co-optical axis structure, which is applied to an in-situ projection system for a non-co-optical axis structure, the system comprising: an imaging component, an excitation light source, an indicator light source, and a projection component with adjustable posture, wherein the imaging component and the projection component do not share a co-optical axis, and the method comprises:

[0036] During the calibration phase, the indicator light source is controlled to emit indicator light in the visible light band toward the sample tissue to form a first projection pattern on the sample tissue;

[0037] In the calibration phase, the imaging component is controlled to capture a first projection pattern to obtain first image data;

[0038] During the calibration phase, the projection assembly is controlled to project visible light onto the sample tissue according to the first image data to form a second projection pattern on the sample tissue; and the projection assembly passively adjusts its own posture until the first projection pattern coincides with the second projection pattern.

[0039] During the operation phase, the excitation light source is controlled to emit excitation light toward the target tissue to excite the target tissue to generate fluorescence;

[0040] During the operation phase, the imaging component is controlled to collect the fluorescence emitted by the target tissue to obtain second image data;

[0041] During the operation phase, the projection component is controlled to project visible light onto the target tissue according to the second image data, so as to form a third projection pattern on the target tissue.

[0042] In some embodiments, during the calibration phase, the method further includes, after the first projection pattern and the second projection pattern overlap, correspondingly recording the position of the imaging component and the position of the projection component, adjusting the position of the imaging component, and returning to the step of controlling the imaging component to capture the first projection pattern until a preset calibration end condition is met;

[0043] During the operation stage, the projection component is controlled to project visible light toward the target tissue according to the second image data, including, in response to the posture change of the imaging component, adjusting the posture of the projection component according to the corresponding recorded postures of the imaging component and the projection component, and controlling the projection component to project visible light toward the target tissue according to the second image data.

[0044] In some embodiments, the imaging assembly includes a focusing assembly, a first image sensor, a second image sensor, and a beam splitter; the first image sensor and the second image sensor share a common optical axis, and the first image sensor, the second image sensor, and the projection assembly do not share a common optical axis; the method further includes:

[0045] Controlling the focusing assembly to focus the light reflected and emitted by the target tissue and the sample tissue, and directing the light to the beam splitter;

[0046] controlling the beam splitter to direct the incident fluorescence to the first image sensor and to direct the incident indicator light to the second image sensor;

[0047] During the operation phase, controlling the first image sensor to capture the fluorescence generated by the target tissue to obtain second image data;

[0048] During the calibration phase, the second image sensor is controlled to capture the first projection pattern to obtain first image data.

[0049] In a third aspect, an embodiment of the present application further provides an in-situ projection device with a non-co-optical axis structure, which is applied to an in-situ projection system with a non-co-optical axis structure. The system includes: an imaging component, an excitation light source, an indicator light source, and a projection component with adjustable posture, wherein the imaging component and the projection component do not share the same optical axis. The device includes:

[0050] a light source control module, configured to control the indicator light source to emit indicator light in the visible light band toward the sample tissue during a calibration phase, so as to form a first projection pattern on the sample tissue;

[0051] a shooting module, configured to control the imaging component to shoot a first projection pattern to obtain first image data during a calibration phase;

[0052] The projection module is configured to control the projection assembly to project visible light onto the sample tissue according to the first image data to form a second projection pattern on the sample tissue during the calibration phase; and passively adjust its own position until the first projection pattern coincides with the second projection pattern;

[0053] The light source control module is further configured to control the excitation light source to emit excitation light toward the target tissue during operation, so as to excite the target tissue to generate fluorescence;

[0054] The shooting module is further configured to control the imaging component to collect the fluorescence emitted by the target tissue to obtain second image data during the operation phase;

[0055] The projection module is further configured to control the projection assembly to project visible light onto the target tissue according to the second image data during operation, so as to form a third projection pattern on the target tissue.

[0056] In some embodiments, during the calibration phase, the apparatus further comprises:

[0057] a loop module, configured to, after the first projection pattern and the second projection pattern overlap, correspondingly record the position of the imaging component and the position of the projection component, adjust the position of the imaging component, and return to the step of controlling the imaging component to capture the first projection pattern until a preset calibration end condition is met;

[0058] During the operation stage, the projection module is specifically used to respond to the posture change of the imaging component, adjust the posture of the projection component according to the corresponding recorded postures of the imaging component and the projection component, and control the projection component to project visible light toward the target tissue according to the second image data.

[0059] Beneficial effects of the embodiments of the present invention:

[0060] In the technical solution provided by the embodiment of the present invention, an in-situ projection system with a non-co-optical axis structure includes an imaging component, an excitation light source, an indicator light source, and a projection component with adjustable position, wherein the imaging component and the projection component do not share a co-optical axis and therefore do not cause obstruction. Specifically, in the in-situ projection system with a non-co-optical axis structure, the indicator light source is used to emit indicator light in the visible light band to the sample tissue during the calibration phase to form a first projection pattern on the sample tissue; the imaging tissue is used to project visible light to the sample tissue according to the first image data during the calibration phase to form a second projection pattern on the sample tissue, and then passively adjust its own position until the first projection pattern coincides with the second projection pattern, thereby achieving calibration of the optical axes of the projection component and the imaging component during the calibration phase. Based on this, during operation, the excitation light source emits excitation light to the target tissue during the operation phase to stimulate the target tissue to produce fluorescence, the imaging component collects the fluorescence emitted by the target tissue to obtain second image data, and then the projection component projects visible light to the target tissue according to the second image data, thereby achieving in-situ projection of the tissue without obstruction.

[0061] Of course, it is not necessary to achieve all of the advantages described above simultaneously in order to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0063] Figure 1 A schematic structural diagram of a non-co-optical axis in-situ projection system provided in an embodiment of the present application;

[0064] Figure 2A schematic diagram of a flow chart of the calibration phase of the non-co-optical axis in-situ projection method provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of a flow chart of the non-co-optical axis in-situ projection method during operation provided by an embodiment of the present application;

[0066] Figure 4 A schematic structural diagram of an imaging assembly provided in an embodiment of the present application;

[0067] Figure 5 A schematic structural diagram of an in-situ projection system with a non-co-optical axis structure provided in an embodiment of the present application;

[0068] Figure 6 A schematic flow chart of an in-situ projection method for a non-co-optical axis structure provided in an embodiment of the present application;

[0069] Figure 7 A schematic structural diagram of an in-situ projection device with a non-co-optical axis structure provided in an embodiment of the present application;

[0070] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of the present invention.

[0072] The following describes in detail the in-situ projection system with a non-co-optical axis structure provided by the embodiment of the present application with a specific example.

[0073] Figure 1 A structural diagram of a non-co-optical axis in-situ projection system provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the non-co-optical axis in-situ projection system includes an imaging component 11, an excitation light source 12, an indicator light source 13, and a projection component 14 with adjustable posture, wherein the imaging component 11 and the projection component 14 are not co-optical axis.

[0074] Corresponding to the non-co-optical axis in-situ projection system provided in the embodiment of the present application, the embodiment of the present application further provides a non-co-optical axis in-situ projection method, which is applied to the non-co-optical axis in-situ projection system provided in the present application. Figure 2 The calibration phase shown and Figure 3 The operation stages shown illustrate the non-co-optical axis in-situ projection method provided in the embodiment of the present application.

[0075] See also Figure 2 , Figure 2 A schematic flow chart of the calibration phase of the non-co-optical axis in-situ projection method provided in an embodiment of the present application includes the following steps:

[0076] Step S21: controlling the indicator light source to emit indicator light in the visible light band toward the sample tissue to form a first projection pattern on the sample tissue;

[0077] Step S22: controlling the imaging component to capture the first projection pattern to obtain first image data;

[0078] Step S23: controlling the projection component to project visible light onto the sample tissue according to the first image data to form a second projection pattern on the sample tissue; passively adjusting its own posture until the first projection pattern and the second projection pattern overlap.

[0079] In the technical solution provided in the embodiment of the present application, the indicator light source is controlled to emit indicator light in the visible light band to the sample tissue to form a first projection pattern on the sample tissue, and then the imaging component is controlled to capture the first projection pattern to obtain first image data, and then the projection component projects visible light to the sample tissue according to the first image data to form a second projection pattern on the sample tissue. By adjusting the posture of the projection component until the first projection pattern coincides with the second projection pattern, the mapping relationship between the projection component and the imaging component is calibrated.

[0080] In the above step S21 , the first projection pattern is a pattern formed on the sample tissue by the visible light band emitted by the indicator light source to the sample tissue during the calibration phase.

[0081] The indicator light source uses at least three identical lasers with wavelengths in the visible light band. It is understandable that the purpose of setting the indicator light source is to establish a corresponding relationship between the imaging component and the projection component by adjusting the overlap of the first projection pattern and the second projection pattern. Based on this, in the operation phase, in-situ projection can be achieved based on the corresponding relationship between the imaging component and the projection component. Therefore, in order to accurately determine the overlap of the first projection pattern and the second projection pattern, the indicator light source should be set on the same plane rather than on the same straight line. For example, the above Figure 1 middle, Figure 1 The indicator light source uses four identical lasers, and these four lasers are evenly distributed in an array, with wavelengths in the visible light band. However, it can be understood that the above Figure 1 The indicator light source shown is only an example and is not limiting.

[0082] In the above step S22, the first image data is image data obtained by the imaging component capturing the first projection pattern. In some embodiments, the first image data may only include the first projection pattern; in other embodiments, the first image data may also include sample tissue in addition to the first projection pattern.

[0083] In step S23, the second projection pattern is a pattern formed on the sample tissue after the projection assembly projects visible light onto the sample tissue according to the first image data. After the second projection image is formed, the position of the projection assembly is adjusted so that the first projection pattern and the second projection pattern overlap.

[0084] It is understood that since the second projection pattern is projected by the projection assembly, adjusting the position of the projection assembly can adjust the position of the second projection pattern. Based on this, the second projection pattern can be made to coincide with the first projection pattern by adjusting the position of the projection assembly. Here, the projection assembly passively adjusting its own position can be understood as the projection assembly receiving parameters input by the user for adjusting the position and then adjusting its own position based on the parameters input by the user; or the projection assembly receiving manual adjustment by the user.

[0085] See also Figure 3 , Figure 3 A schematic flow chart of the non-co-optical axis in-situ projection method provided in an embodiment of the present application during operation includes the following steps:

[0086] Step S31: controlling the excitation light source to emit excitation light toward the target tissue to excite the target tissue to generate fluorescence;

[0087] Step S32: controlling the imaging component to collect the fluorescence emitted by the target tissue to obtain second image data;

[0088] Step S33: controlling the projection component to project visible light toward the target tissue according to the second image data to form a third projection pattern on the target tissue.

[0089] In the technical solution provided by the embodiments of this application, an excitation light source is controlled to emit excitation light toward the target tissue, causing the target tissue to produce fluorescence. Furthermore, an imaging component is controlled to capture the fluorescence emitted by the target tissue to obtain second image data. Based on the mapping relationship determined in the calibration phase, a projection component is controlled to project visible light toward the target tissue according to the second image data, thereby forming a third projection pattern on the target tissue. In this way, in-situ projection is achieved while avoiding obstructions.

[0090] In step S31, the excitation light source is used to emit excitation light toward the target tissue during the operation phase to stimulate the target tissue to produce fluorescence. It is understood that many tissues and glands in the human body exhibit fluorescence properties, such as tissues in the cervix, gastrointestinal tract, nasopharynx, and oral cavity, as well as various glands such as the parathyroid gland, pancreas, and adrenal glands.

[0091] After being excited by the excitation light source, the target tissue can produce fluorescence with a wavelength different from the excitation light. The wavelength range of the excitation light source is between 350 nanometers (nm) and 1700nm. The specific wavelength range can be selected according to the type of target tissue in the actual application scenario. For different tissues, different excitation light sources with different wavelength ranges are used. For example, for thyroid tissue, an excitation light source with a wavelength range of 690nm-800nm ​​can be used; for neural tissue, an excitation light source with a wavelength range of 300nm-420nm can be used.

[0092] In the above step S32, the second image data is image data obtained by the imaging component collecting fluorescence emitted by the target tissue.

[0093] Here, using an excitation light source to excite the target tissue to produce fluorescence belongs to near-infrared fluorescence imaging technology (NTR-AFI). Near-infrared fluorescence imaging technology is a technology that uses the natural fluorescence characteristics of biological tissues in the near-infrared light band for imaging. Specifically, certain biological tissues can naturally emit fluorescence after being excited by light of a specific wavelength. Therefore, by using a specific near-infrared excitation light source to irradiate the biological tissue, a highly sensitive detector (such as a photomultiplier tube or a charge-coupled device (CCD)) is used to collect the fluorescence signal emitted by the tissue. The detector converts the fluorescence signal into an electrical signal or a digital signal, and after signal processing and image reconstruction algorithms, it finally forms an image that can reflect the fluorescence distribution inside the tissue. By analyzing the fluorescence image (i.e., the second image data), the state of the tissue can be evaluated, the lesion site can be identified, and surgical operations can be guided.

[0094] In the above step S33 , the third projection pattern is the third projection pattern formed on the target tissue after the projection component projects visible light onto the target tissue according to the second image data.

[0095] It is understandable that for the above Figure 2 The calibration process shown and the above Figure 3 The projection process shown in the figure can be pre-calibrated. That is, the calibration process is pre-calibrated for the in-situ projection system with a non-co-optical axis structure. Subsequently, the calibration process is only required to be performed based on the calibrated in-situ projection system with a non-co-optical axis structure. Figure 3 Just follow the running process shown.

[0096] In some embodiments, as Figure 4As shown, Figure 4 This is a schematic diagram of the structure of an imaging assembly provided in an embodiment of the present application, which includes a focusing assembly 111, a first image sensor 112, a second image sensor 113, and a beam splitter 114. The first image sensor 112 and the second image sensor 113 share a common optical axis, while the first image sensor 112 and the projection assembly do not share a common optical axis, and the second image sensor 113 and the projection assembly do not share a common optical axis.

[0097] Corresponding to the above Figure 4 The imaging assembly shown above Figure 2 and Figure 3 In the illustrated embodiment, the steps of acquiring the first image data and the second image data by the imaging assembly, namely, steps S22 and S32, are specifically, during the calibration phase, the focusing assembly 111 is used to focus light reflected and emitted by the sample tissue; during the operation phase, the focusing assembly 111 is used to focus light reflected and emitted by the target tissue. Here, the focusing assembly 111 can be understood as a lens assembly. Specifically, in some embodiments, the focusing assembly 111 can be a telecentric lens to adjust the optical path of the entire imaging assembly, achieving simultaneous zooming of the two image sensors in the imaging assembly. Furthermore, within the focal length range of the telecentric lens, even if the distance between the sample tissue, the target tissue, and the focusing assembly 111 changes, it will not affect the size of the captured image.

[0098] The focusing assembly 111 focuses the light reflected and emitted by the sample tissue and target tissue and directs it to the beam splitter 114. The beam splitter 114 then directs the incident fluorescence to the first image sensor 112 and the incident indicator light to the second image sensor 113. Thus, during the operation phase, the first sensor 112 can capture the fluorescence generated by the target tissue and obtain second image data, while the second sensor 113 can capture the first projection image and obtain first image data during the calibration phase. In some embodiments, the beam splitter can be a dichroic mirror. Furthermore, when the beam splitter is a dichroic mirror, in some embodiments, the optical axis of the dichroic mirror forms a 45° angle with the incident light from the image sensor.

[0099] It is understandable that during the calibration phase, since the calibration is only performed to determine the positional correspondence between the imaging assembly and the projection assembly by aligning the first projection pattern with the second projection pattern, in an embodiment of the present application, during the calibration phase, only the indicator light source may be turned on, and the excitation light source may not be turned on. Thus, during the calibration phase, the light received by the focusing assembly is the visible light reflected by the target tissue and the light from the indicator light source reflected by the target tissue. Based on this, after processing by the beam splitter, the first image sensor does not capture image data, and the second image sensor captures the first projection pattern of the indicator light source on the sample assembly, obtaining first image data. During the operation phase, since the excitation light source is required to excite the target tissue to produce fluorescence in order to determine the location of the target tissue, both the indicator light source and the excitation light source need to be turned on. Thus, the light received by the focusing assembly is the visible light reflected by the target tissue, the light from the indicator light source reflected by the target tissue, the light from the excitation light source reflected by the target tissue, and the fluorescence generated by the target tissue after the excitation light source irradiates the target tissue. Based on this, after processing by the beam splitter, the first image sensor can capture the fluorescence generated by the target tissue, obtaining second image data.

[0100] In other embodiments, during the calibration stage, the indicator light source and the excitation light source can also be turned on at the same time, so that the light received by the focusing component is the visible light reflected by the target tissue, the light of the indicator light source reflected by the target tissue, the light of the excitation light source reflected by the target tissue, and the fluorescence generated by the target tissue after the excitation light source is irradiated on the target tissue. Based on this, the first sensor captures the fluorescence generated by the target tissue to obtain the second image data, and the second image sensor captures the first projection pattern of the indicator light source on the sample component to obtain the first image data.

[0101] In some embodiments, the beam splitter can also direct the incident excitation light to the second image sensor, and the second image sensor can also capture the excitation light reflected by the target tissue during the operation phase to obtain third image data. It can be understood that during the operation phase, both the indicator light source and the excitation light source are turned on. Therefore, the light beam entering the beam splitter is visible light, indicator light, excitation light reflected by the target tissue, and fluorescence generated by the target tissue. Therefore, it is only processed by the beam splitter. The beam splitter can direct the fluorescence to the first image sensor, and the first image sensor obtains the second image data, and direct the visible light, indicator light, and excitation light to the second image sensor to generate the third image data. Based on this, during the operation phase, the projection component is specifically used to project visible light onto the target tissue according to the second image data and the third image data to form a third projection pattern on the target tissue.

[0102] Because the light beam generating the third image data includes excitation light reflected by the target tissue, and the laser speckle formed by the excitation light reflected by the target tissue can reflect blood flow information, the third image data can reflect blood flow information. In some embodiments, the characteristics of laser speckle can be used to assess blood flow on the tissue surface through laser speckle contrast imaging (LSCI). Specifically, a laser beam is uniformly irradiated onto the surface of the tissue to be measured. Due to the scattering of light by internal microstructures (such as red blood cells), a series of randomly distributed bright and dark areas, i.e., a speckle pattern, are formed on the detector. A high-sensitivity camera is used to capture the speckle pattern, and statistical analysis is performed on multiple frames of speckle images to calculate the speckle contrast at each pixel position. The speckle contrast is defined as: K = σ / μ, where σ is the standard deviation of the speckle intensity and μ is the mean speckle intensity. By changing the speckle contrast, blood flow velocity can be estimated, thereby dynamically displaying blood flow changes in real time.

[0103] In some embodiments, the posture of the imaged tissue can be adjusted, and the in-situ projection system with a non-co-optical axis structure further includes a control component and a posture sensor. Figure 2 In the calibration process shown, after adjusting the projection assembly until the first and second projection patterns overlap, the position of the imaging assembly and the projection assembly are recorded accordingly. The position of the imaging assembly is adjusted again, and the process returns to the step of controlling the imaging assembly to capture the first projection pattern until a preset calibration end condition is met. The preset calibration end condition can be a pre-set number of repetitions.

[0104] In an embodiment of the present application, during the calibration phase, the imaging component is controlled to capture a first projection pattern to obtain first image data; the projection component is controlled to project visible light onto the sample tissue according to the first image data to form a second projection pattern on the sample tissue, and then the position of the projection component is adjusted until the first projection pattern coincides with the second projection pattern. At this time, the position of the imaging component and the projection component is determined by the position sensor, and the position of the imaging component and the position of the projection component are recorded accordingly; then, it is determined whether the preset calibration end condition has been met. If the preset calibration end condition has been met, the calibration is terminated. If the preset calibration end condition is not met, the step of controlling the imaging component to capture the first projection pattern is returned to. Thus, after multiple calibrations, the positions of multiple imaging components and the positions of the projection components are recorded accordingly.

[0105] Therefore, in the above Figure 3In the illustrated operational process, in response to changes in the imaging assembly's posture, the position sensor determines the imaging assembly's posture. Furthermore, the control component determines the projection assembly's posture corresponding to the current imaging assembly's posture based on the correspondingly recorded postures of the imaging assembly and projection assembly. Furthermore, the control component controls the projection assembly to adjust to the determined posture and projects visible light onto the target tissue according to the second image data, thereby forming a third projection pattern on the target tissue. This improves projection accuracy.

[0106] In some embodiments, in the above Figure 1 Based on the disclosed in-situ projection system with a non-coaxial structure, the indicator light source can also be turned on during the operation phase. That is, during the operation phase, the indicator light source and the excitation light source are turned on simultaneously. As a result, during the operation phase, the indicator light source emits indicator light in the visible light band to the sample tissue, thereby forming a fourth projection pattern on the target tissue. Therefore, during the operation phase, the imaging component can capture the fluorescence generated by the target tissue and the reflected indicator light to obtain second image data. Then, the projection component projects the second image data onto the target tissue to form a third projection pattern. In this way, the user can adjust the position of the projection component so that the third projection pattern coincides with the indicator light spot in the fourth projection pattern, that is, the third projection image is projected to the correct position.

[0107] In some embodiments, in the above Figure 4 Based on the disclosed in-situ projection system with a non-coaxial structure, the imaging component is a component with adjustable position and posture, and the in-situ projection system with a non-coaxial structure also includes a control component and a posture sensor. Based on this, during the calibration phase, in response to the overlap of the first and second projection patterns, the posture sensor detects the first posture of the imaging component and the second posture of the projection component, thereby recording multiple sets of corresponding relationships between the first and second postures.

[0108] During the operation phase, the indicator light source and the excitation light source are turned on at the same time, and the light beam focused by the focusing component is the indicator light, the excitation light reflected by the target component, the fluorescence generated by the target component, and the visible light. Therefore, the beam splitter in the imaging component guides the incident excitation light, visible light, and indicator light to the second image sensor to generate the first image data; and guides the incident fluorescence to the first image sensor to generate the second image data. When it is detected that the posture of the imaging component has changed, the posture of the imaging component, that is, the third posture, is detected based on the posture detection component, and then according to the correspondence between the multiple sets of recorded first postures and the second postures, the second posture is obtained. The relationship determines the first posture corresponding to the third posture, and then determines the second posture corresponding to the first posture, and adjusts the projection component to the second posture, so that the projection component projects the first image data and the fluorescent area in the second image data to the target tissue in the second posture. At this time, if there is a position difference between the light spot formed by the indicator light source in the projected image and the light spot formed by the indicator light source in the first image data, the light spot formed by the indicator light source in the projected image and the light spot formed by the indicator light source in the first image data are adjusted to overlap by adjusting the projection component posture, thereby improving the accuracy of the projection.

[0109] To facilitate understanding of the above-mentioned in-situ projection method of the non-co-optical axis structure, the in-situ projection method of the non-co-optical axis structure and the in-situ projection system of the non-co-optical axis structure provided in the embodiments of the present application are described below with specific examples.

[0110] See also Figure 5 , Figure 5 A structural diagram of an in-situ projection system with a non-co-optical axis structure provided in an embodiment of the present application is shown as follows: Figure 5 As shown, the in-situ projection system with a non-co-optical axis structure includes a control component 17, a distance sensor 15, a posture sensor 16, an excitation light source 12, an imaging component 11, a projection component 14, and an indicator light source 13, wherein the imaging component 11 and the projection component 14 are not co-optical axis, and the indicator light source 13 is composed of four identical laser points. Figure 5 The “R” in represents the first projection pattern, and the “R′” represents the second projection pattern.

[0111] based on Figure 5 The in-situ projection system with non-co-optical axis structure shown and the above Figure 4 Imaging components shown, see Figure 6 , Figure 6 A schematic flow chart of an in-situ projection method for a non-co-optical axis structure provided in an embodiment of the present application may include the following steps:

[0112] Step S61: turning on the excitation light source and the indicator light source, and capturing a fluorescent image by the first image sensor in the imaging component, and capturing a standard image by the second image sensor in the imaging component.

[0113] It can be understood that after the excitation light source and the indicator light source are turned on, the focusing component in the imaging component can focus on light beams such as visible light, excitation light reflected by tissue, indicator light and fluorescence. Therefore, the beam splitter in the imaging component guides the fluorescence to the first image sensor, and guides the visible light, excitation light reflected by tissue and indicator light to the second image sensor. Thus, a fluorescence image is obtained based on the first image sensor, and a standard image is obtained based on the second image sensor.

[0114] Step S62: The imaging component sends the standard image and the fluorescence image to the control system, and the control system controls the projection component to project the standard image and the fluorescent area in the fluorescence image as one image onto the tissue to obtain a projection image.

[0115] Step S63: Adjust the posture of the projection component so that the projection image and the standard image coincide with each other, and determine the posture of the projection component and the imaging posture when the projection image coincides with the standard image through the distance sensor module and the posture sensor module, and record the posture of the projection component and the imaging component accordingly.

[0116] In this way, the posture of the imaging component is adjusted multiple times, the standard image and the projection image are repeatedly determined, and the projection image and the standard image are overlapped by adjusting the posture of the projection component, thereby obtaining multiple sets of posture correspondences between the projection component and the imaging component. In other words, a mapping model between the optical axis offset and the projection image offset is constructed;

[0117] Step S64: During the operation phase, the posture of the imaging component is detected, and based on the posture of the imaging component and the mapping model, the posture of the projection component during projection is determined, and the projection component is adjusted to the posture to achieve projection.

[0118] Step S65: Determine whether the field of view of the projected image coincides with the preset field of view. If so, proceed to step S66 below. If not, proceed to step S67 below. It will be appreciated that the projected image is derived from the standard image; therefore, the field of view of the projected image is the same as that of the standard image. A preset field of view is required to ensure that the projection area does not exceed the preset field of view.

[0119] Step S66: Lock the current position.

[0120] Step S67: filtering out the portion of the projected image that exceeds the preset field of view.

[0121] It can be understood that if the field of view of the projected image exceeds the preset field of view, it indicates that the field of view of the standard image becomes larger, that is, the distance between the imaging component and the tissue becomes farther. At this time, the projected image is corrected to filter out the portion of the projected image that exceeds the preset field of view.

[0122] In the technical solution provided in the embodiment of the present application, the projection component and the imaging component need to work synchronously to ensure the consistency of projection and imaging; a mapping model is constructed by correspondingly recording the posture of the projection component and the posture of the imaging component when the projected image coincides with the standard image, thereby determining the offset and angle difference of each optical axis in the non-co-optical axis system; and image correction algorithms (such as perspective transformation and distortion correction) are used to eliminate distortion and offset to ensure that the image can be accurately projected to the target position.

[0123] Corresponding to the above Figure 1 Provided is an in-situ projection system with a non-co-optical axis structure, wherein:

[0124] The indicator light source is used to emit indicator light in the visible light band to the sample tissue during the calibration phase to form a first projection pattern on the sample tissue;

[0125] The imaging component is configured to capture the first projection pattern during the calibration phase to obtain first image data;

[0126] The projection assembly is configured to project visible light onto the sample tissue according to the first image data during the calibration phase to form a second projection pattern on the sample tissue; and passively adjust its own position until the first projection pattern and the second projection pattern coincide with each other;

[0127] The excitation light source is used to emit excitation light to the target tissue during the operation phase to excite the target tissue to produce fluorescence;

[0128] The imaging component is further configured to collect fluorescence emitted by the target tissue during the operation phase to obtain second image data;

[0129] The projection component is further configured to project visible light onto the target tissue according to the second image data during the operation phase, so as to form a third projection pattern on the target tissue.

[0130] In some embodiments, the imaging assembly includes a focusing assembly, a first image sensor, a second image sensor, and a beam splitter;

[0131] The first image sensor and the second image sensor share a common optical axis, and the first image sensor, the second image sensor, and the projection assembly do not share a common optical axis;

[0132] The focusing assembly is used to focus the light reflected and emitted by the target tissue and the sample tissue, and guide the light to the beam splitter;

[0133] The beam splitter is configured to direct the incident fluorescence to the first image sensor and direct the incident indicator light to the second image sensor;

[0134] The first image sensor is used to capture the fluorescence generated by the target tissue during the operation phase to obtain second image data;

[0135] The second image sensor captures the first projection pattern during the calibration phase to obtain first image data.

[0136] In some embodiments, the beam splitter is further configured to direct the incident excitation light to the second image sensor;

[0137] The second image sensor is further configured to capture the excitation light reflected by the target tissue during the operation phase to obtain third image data;

[0138] The projection component is specifically configured to project visible light toward the target tissue according to the second image data and the third image data during the operation phase, so as to form a third projection pattern on the target tissue.

[0139] In some embodiments, the beam splitter is a dichroic mirror.

[0140] In some embodiments, the posture of the imaging component is adjustable, and the system further comprises a control component and a posture sensor;

[0141] The posture sensor is configured to detect, in the calibration phase, a first posture of the imaging assembly and a second posture of the projection assembly in response to the first projection pattern and the second projection pattern overlapping each other; and to detect, in the operation phase, a third posture of the imaging assembly;

[0142] The control component is configured to, during the operation phase, drive the projection module to move to the second posture in response to the third posture being equal to the first posture.

[0143] In some embodiments, the indicator light source is further used to emit indicator light in the visible light band to the sample tissue during the operation phase to form a fourth projection pattern on the target tissue.

[0144] The imaging component is specifically configured to capture the fluorescence generated by the target tissue and the reflected indicator light during the operation phase to obtain second image data;

[0145] The projection assembly is further configured to passively adjust its own posture during the operation phase until the third projection pattern and the indicator light spot in the fourth projection pattern coincide with each other.

[0146] In some embodiments, the posture of the imaging component is adjustable, and the system further comprises a control component and a posture sensor;

[0147] The posture sensor is configured to detect, in response to the first projection pattern and the second projection pattern overlapping, a first posture of the imaging assembly and a second posture of the projection assembly during the calibration phase, and to detect, in the operation phase, a third posture of the imaging assembly;

[0148] The control component is configured to, during the operation phase, drive the projection component to move to the second posture in response to the third posture being equal to the first posture;

[0149] The indicator light source is further used to emit indicator light in the visible light band to the target tissue during the operation phase to form a fourth projection pattern on the target tissue;

[0150] The first image sensor is used to capture the fluorescence generated by the target tissue during the operation phase, and the second image sensor is also used to capture the reflected indicator light during the operation phase to obtain second image data;

[0151] The projection assembly is further configured to passively adjust its own posture during the operation phase until the third projection pattern coincides with the indicator light spot in the fourth projection pattern.

[0152] Corresponding to the above-mentioned in-situ projection method of the non-co-optical axis structure, the embodiment of the present application also provides an in-situ projection device of the non-co-optical axis structure, which is applied to the in-situ projection system of the non-co-optical axis structure, the system includes: an imaging component, an excitation light source, an indicator light source, and a projection component with adjustable posture, wherein the imaging component and the projection component do not share the same optical axis, such as Figure 7 As shown, the device includes:

[0153] The light source control module 71 is used to control the indicator light source to emit indicator light in the visible light band to the sample tissue during the calibration phase, so as to form a first projection pattern on the sample tissue;

[0154] The shooting module 72 is used to control the imaging component to shoot the first projection pattern to obtain first image data during the calibration phase;

[0155] The projection module 73 is configured to control the projection assembly to project visible light onto the sample tissue according to the first image data to form a second projection pattern on the sample tissue during the calibration phase; and passively adjust its own position until the first projection pattern coincides with the second projection pattern.

[0156] The light source control module 71 is further configured to control the excitation light source to emit excitation light to the target tissue during operation, so as to excite the target tissue to generate fluorescence.

[0157] The shooting module 72 is further configured to control the imaging component to collect the fluorescence emitted by the target tissue to obtain second image data during the operation phase;

[0158] The projection module 73 is further configured to control the projection assembly to project visible light onto the target tissue according to the second image data during operation, so as to form a third projection pattern on the target tissue.

[0159] In some embodiments, during the calibration phase, the apparatus further comprises:

[0160] a loop module, configured to, after the first projection pattern and the second projection pattern overlap, correspondingly record the position of the imaging component and the position of the projection component, adjust the position of the imaging component, and return to the step of controlling the imaging component to capture the first projection pattern until a preset calibration end condition is met;

[0161] During the operation stage, the projection module is specifically used to respond to the posture change of the imaging component, adjust the posture of the projection component according to the corresponding recorded postures of the imaging component and the projection component, and control the projection component to project visible light toward the target tissue according to the second image data.

[0162] The embodiment of the present invention further provides an electronic device, such as Figure 8 As shown, it includes a processor 81 , a communication interface 82 , a memory 83 and a communication bus 84 , wherein the processor 81 , the communication interface 82 , and the memory 83 communicate with each other via the communication bus 84 .

[0163] Memory 83, for storing computer programs;

[0164] The processor 81 is configured to implement any of the above-mentioned in-situ projection methods for non-co-optical axis structures when executing the program stored in the memory 83 .

[0165] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0166] The communication interface is used for communication between the above electronic device and other devices.

[0167] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0168] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0169] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, the steps of the in-situ projection method of any non-co-optical axis structure described above are implemented.

[0170] In another embodiment provided by the present invention, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute the in-situ projection method of any non-co-optical axis structure in the above embodiments.

[0171] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0172] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0173] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, since the systems, devices, electronic devices, and storage media are generally similar to the method embodiments, their descriptions are relatively simple. For relevant portions, reference can be made to the descriptions of the method embodiments.

[0174] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. An in-situ projection system with a non-co-optical axis structure, characterized in that: The system comprises: An imaging component, an excitation light source, an indicator light source, and a projection component with adjustable posture, wherein the imaging component and the projection component do not share a common optical axis; The indicator light source is used to emit indicator light in the visible light band to the sample tissue during the calibration phase to form a first projection pattern on the sample tissue; The imaging component is configured to capture the first projection pattern during the calibration phase to obtain first image data; The projection assembly is configured to project visible light onto the sample tissue according to the first image data during the calibration phase to form a second projection pattern on the sample tissue; and passively adjust its own position until the first projection pattern and the second projection pattern coincide with each other; The excitation light source is used to emit excitation light to the target tissue during the operation phase to excite the target tissue to produce fluorescence; The imaging component is further configured to collect fluorescence emitted by the target tissue during the operation phase to obtain second image data; The projection component is further configured to project visible light onto the target tissue according to the second image data during the operation phase, so as to form a third projection pattern on the target tissue.

2. The system according to claim 1, wherein: The imaging assembly includes a focusing assembly, a first image sensor, a second image sensor, and a beam splitter; The first image sensor and the second image sensor share a common optical axis, and the first image sensor, the second image sensor, and the projection assembly do not share a common optical axis; The focusing assembly is used to focus the light reflected and emitted by the target tissue and the sample tissue, and guide the light to the beam splitter; The beam splitter is configured to direct the incident fluorescence to the first image sensor and direct the incident indicator light to the second image sensor; The first image sensor is used to capture the fluorescence generated by the target tissue during the operation phase to obtain second image data; The second image sensor captures the first projection pattern during the calibration phase to obtain first image data.

3. The system according to claim 2, characterized in that The beam splitter is further used to guide the incident excitation light to the second image sensor; The second image sensor is further configured to capture the excitation light reflected by the target tissue during the operation phase to obtain third image data; The projection component is specifically configured to project visible light toward the target tissue according to the second image data and the third image data during the operation phase, so as to form a third projection pattern on the target tissue.

4. The system according to any one of claims 2-3, characterized in that: The beam splitter is a dichroic mirror.

5. The system according to claim 1, wherein: The posture of the imaging component is adjustable, and the system further comprises a control component and a posture sensor; The posture sensor is configured to detect, during the calibration phase, a first posture of the imaging assembly and a second posture of the projection assembly in response to the first projection pattern and the second projection pattern overlapping; During the operation phase, a third posture of the imaging assembly is detected; The control component is configured to, during the operation phase, drive the projection module to move to the second posture in response to the third posture being equal to the first posture.

6. The system according to claim 1, wherein: The indicator light source is further used to emit indicator light in the visible light band to the sample tissue during the operation phase to form a fourth projection pattern on the target tissue The imaging component is specifically configured to capture the fluorescence generated by the target tissue and the reflected indicator light during the operation phase to obtain second image data; The projection assembly is further configured to passively adjust its own posture during the operation phase until the third projection pattern and the indicator light spot in the fourth projection pattern coincide with each other.

7. The system according to claim 3, wherein: The posture of the imaging component is adjustable, and the system further comprises a control component and a posture sensor; The posture sensor is configured to detect, in response to the first projection pattern and the second projection pattern overlapping, a first posture of the imaging assembly and a second posture of the projection assembly during the calibration phase, and to detect, in the operation phase, a third posture of the imaging assembly; The control component is configured to, during the operation phase, drive the projection component to move to the second posture in response to the third posture being equal to the first posture; The indicator light source is further used to emit indicator light in the visible light band to the target tissue during the operation phase to form a fourth projection pattern on the target tissue; The first image sensor is used to capture the fluorescence generated by the target tissue during the operation phase, and the second image sensor is also used to capture the reflected indicator light during the operation phase to obtain second image data; The projection assembly is further configured to passively adjust its own posture during the operation phase until the third projection pattern coincides with the indicator light spot in the fourth projection pattern.

8. The system according to claim 1, wherein: The indicator light source includes at least three identical lasers, and the lasers are arranged in the same plane and not on the same straight line.

9. The system according to claim 1, wherein: The wavelength range of the excitation light source is 350 nanometers to 1700 nanometers.

10. The system according to claim 1, wherein: The target tissue is thyroid tissue, and the wavelength range of the excitation light source is 690 nanometers to 800 nanometers.

11. The system according to claim 1, wherein: The target tissue is nerve tissue, and the wavelength range of the excitation light source is 300 nanometers to 420 nanometers.

12. An in-situ projection method for a non-co-optical axis structure, applied to an in-situ projection system for a non-co-optical axis structure, the system comprising: An imaging component, an excitation light source, an indicator light source, and a projection component with adjustable posture, wherein the imaging component and the projection component do not share a common optical axis, and the method includes: During the calibration phase, the indicator light source is controlled to emit indicator light in the visible light band toward the sample tissue to form a first projection pattern on the sample tissue; In the calibration phase, the imaging component is controlled to capture a first projection pattern to obtain first image data; During the calibration phase, the projection assembly is controlled to project visible light onto the sample tissue according to the first image data to form a second projection pattern on the sample tissue; and the projection assembly passively adjusts its own posture until the first projection pattern coincides with the second projection pattern. During the operation phase, the excitation light source is controlled to emit excitation light toward the target tissue to excite the target tissue to generate fluorescence; During the operation phase, the imaging component is controlled to collect the fluorescence emitted by the target tissue to obtain second image data; During the operation phase, the projection component is controlled to project visible light onto the target tissue according to the second image data, so as to form a third projection pattern on the target tissue.

13. The method according to claim 12, characterized in that In the calibration stage, the method further includes, after the first projection pattern and the second projection pattern overlap, correspondingly recording the position of the imaging component and the position of the projection component, adjusting the position of the imaging component, and returning to the step of controlling the imaging component to capture the first projection pattern until a preset calibration end condition is met; During the operation stage, the projection component is controlled to project visible light toward the target tissue according to the second image data, including, in response to the posture change of the imaging component, adjusting the posture of the projection component according to the corresponding recorded postures of the imaging component and the projection component, and controlling the projection component to project visible light toward the target tissue according to the second image data.

14. The method according to claim 12, characterized in that The imaging assembly includes a focusing assembly, a first image sensor, a second image sensor, and a beam splitter; the first image sensor and the second image sensor share a common optical axis, and the first image sensor, the second image sensor, and the projection assembly do not share a common optical axis; the method further includes: Controlling the focusing assembly to focus the light reflected and emitted by the target tissue and the sample tissue, and directing the light to the beam splitter; controlling the beam splitter to direct the incident fluorescence to the first image sensor and to direct the incident indicator light to the second image sensor; During the operation phase, controlling the first image sensor to capture the fluorescence generated by the target tissue to obtain second image data; During the calibration phase, the second image sensor is controlled to capture the first projection pattern to obtain first image data.

15. An in-situ projection device with a non-co-optical axis structure, applied to an in-situ projection system with a non-co-optical axis structure, the system comprising: An imaging component, an excitation light source, an indicator light source, and a projection component with adjustable posture, wherein the imaging component and the projection component are not co-axial, and the device comprises: a light source control module, configured to control the indicator light source to emit indicator light in the visible light band toward the sample tissue during a calibration phase, so as to form a first projection pattern on the sample tissue; a shooting module, configured to control the imaging component to shoot a first projection pattern to obtain first image data during a calibration phase; The projection module is configured to control the projection assembly to project visible light onto the sample tissue according to the first image data to form a second projection pattern on the sample tissue during the calibration phase; and passively adjust its own position until the first projection pattern coincides with the second projection pattern; The light source control module is further configured to control the excitation light source to emit excitation light toward the target tissue during operation, so as to excite the target tissue to generate fluorescence; The shooting module is further configured to control the imaging component to collect the fluorescence emitted by the target tissue to obtain second image data during the operation phase; The projection module is further configured to control the projection assembly to project visible light onto the target tissue according to the second image data during operation, so as to form a third projection pattern on the target tissue.

Images