Three-dimensional model construction system and method, equipment and medium

Abstract

The invention relates to the field of medicine, and discloses a three-dimensional model construction system and method, equipment and a medium, and the three-dimensional model construction system comprises an image collection module which is used for collecting color images of a target object at different visual angles and a fluorescence image of a target region in the target object; the control module is used for controlling the image acquisition module so as to enable the image acquisition module to normally complete acquisition of color images of the target object at different visual angles and fluorescence images of a target area in the target object; and the image processing module is used for obtaining a three-dimensional model corresponding to the target object based on the color images of the target object at different visual angles and the fluorescence image of the target area in the target object. When the three-dimensional model construction scheme is used for obtaining the image of the target area of the patient, the three-dimensional image of the target area of the patient can be obtained, and the three-dimensional image can provide more detailed structural information of the target area of the patient, so that a doctor is better assisted to obtain a more accurate disease diagnosis result.

Description

Technical Field

[0001] The present application relates to the medical field, and in particular to a three-dimensional model construction system and method, equipment, and medium. Background Art

[0002] In the field of modern medicine, the demand for precision medicine is growing, especially in the diagnosis and treatment of diseases. Therefore, in order to enable doctors to observe the patient's body structure information more intuitively and to enable doctors to obtain more accurate diagnosis results, various imaging systems have gradually been widely used in the medical field to provide image information of the patient's body structure for doctors to diagnose diseases. In the imaging system, fluorescence imaging and color imaging technology are two important imaging methods. The imaging system built based on fluorescence imaging technology can obtain images of specified areas in the target object with high sensitivity and high specificity through specific fluorescent markers; the images taken by the imaging system built based on color imaging technology can provide rich structural information of the target object.

[0003] In the research work of imaging systems, existing imaging systems are either constructed based on fluorescence imaging technology to obtain a fluorescence image corresponding to a specified area in a target object, or constructed based on color imaging technology to obtain a color image corresponding to the target object, or based on fluorescence imaging and color imaging technology to obtain a fluorescence image and a color image of the target object respectively, and the two are merged to obtain a two-dimensional image of the target object containing the fluorescence area. However, whether the imaging system is constructed based on single-modality (based on fluorescence imaging technology or color imaging technology) or multi-modality (based on fluorescence imaging technology and color imaging technology) imaging technology, what is ultimately provided is a two-dimensional image of the target object, and detailed three-dimensional structural information of the target object cannot be provided. Therefore, when the existing imaging system is used to obtain an image of a patient's target area, only a two-dimensional image of the patient's target area can be obtained, and the two-dimensional image cannot provide more detailed structural information of the patient's target area, making it difficult to assist doctors in obtaining more accurate disease diagnosis results. Summary of the invention

[0004] The present application provides a three-dimensional model construction system and method, device, and medium to solve the problem that the two-dimensional image of the patient's target area obtained by the existing imaging system cannot provide more detailed structural information of the patient's target area, making it difficult to assist doctors in obtaining more accurate disease diagnosis results.

[0005] A first aspect of the present application provides a three-dimensional model construction system, the system is used to construct a three-dimensional model corresponding to a target object, the system comprising:

[0006] An image acquisition module, used for acquiring color images of a target object at different viewing angles and a fluorescent image of a target area in the target object;

[0007] A control module, used to control the image acquisition module so that the image acquisition module can normally complete the acquisition of color images of the target object at different viewing angles and the fluorescence image of the target area in the target object;

[0008] The image processing module is used to obtain a three-dimensional model corresponding to the target object based on the color images of the target object at different viewing angles and the fluorescent image of the target area in the target object.

[0009] In some embodiments of the present application, the image acquisition module includes:

[0010] A fluorescence camera, used for collecting a fluorescence image of a target area in the target object;

[0011] A color camera assembly, used to collect color images of the target object at different viewing angles, wherein the color camera assembly includes a plurality of binocular color cameras, each of which is used to collect color images of the target object at two viewing angles, and each of which is composed of two color cameras placed at a preset distance, each of which is used to collect a color image of the target object at one viewing angle;

[0012] The laser light source is used to irradiate the target area in the target object with laser so that the fluorescence camera can collect the fluorescence image of the target area in the target object.

[0013] In some embodiments of the present application, the image acquisition module further includes:

[0014] A filter, used to enable the fluorescence camera to collect a fluorescence image of a target area in the target object in a specified wavelength band;

[0015] The lighting lamp is used to provide illumination to assist the color camera assembly in collecting color images of the target object at different viewing angles and the fluorescence camera in collecting fluorescence images of the target area in the target object.

[0016] In some embodiments of the present application, the image processing module is configured to obtain the three-dimensional model corresponding to the target object in the following manner:

[0017] Performing denoising and / or pseudo-colorization processing on the fluorescence image of the target area in the target object to obtain a first fluorescence image of the target area in the target object;

[0018] Arbitrarily selecting the color images of the target object from two viewing angles captured by the binocular color camera, and registering the first fluorescent image of the target area in the target object with the selected color images of each viewing angle of the target object, so as to obtain the first fluorescent image of the target area in each viewing angle of the target object;

[0019] Fusing the first fluorescent image of each viewing angle of the target area in the target object with the color image of the target object at the viewing angle to obtain the color image of each viewing angle of the target object containing the fluorescent area;

[0020] Calibrate a selected binocular color camera to obtain an intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera;

[0021] Based on the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, stereo correction is performed on the color image of each viewing angle containing the fluorescent area of ​​the target object, and stereo matching is performed on the color images of the two viewing angles containing the fluorescent area of ​​the target object after stereo correction, so as to obtain a disparity map between the color images of the two viewing angles containing the fluorescent area of ​​the target object;

[0022] Based on the disparity map between the color images of the two perspectives containing the fluorescent area of ​​the target object and the intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of the binocular color camera, a depth map corresponding to the target object is obtained;

[0023] The depth map corresponding to the target object is back-projected to obtain a three-dimensional point cloud corresponding to the target object, and the three-dimensional point cloud corresponding to the target object is three-dimensionally reconstructed to obtain a three-dimensional model corresponding to the target object.

[0024] In some embodiments of the present application, the image processing module is further configured to:

[0025] Arbitrarily select the color images of the target object from two perspectives captured by the binocular color camera, and respectively align the first fluorescent image of the target area in the target object with the selected color images of each perspective of the target object, and segment and / or perform block extraction processing on the registered fluorescent images of each perspective of the target area in the target object to obtain the first fluorescent image of each perspective of the target area in the target object.

[0026] In some embodiments of the present application, the image processing module is further configured to:

[0027] Using a perspective transformation method, the first fluorescent image of the target area in the target object is respectively registered with the selected color image of each viewing angle of the target object to obtain the first fluorescent image of each viewing angle of the target area in the target object;

[0028] Performing stereo matching on the color images of the two perspectives containing the fluorescent area in the target object after stereo correction by using a semi-global block matching method to obtain an initial disparity map between the color images of the two perspectives containing the fluorescent area in the target object, and optimizing the initial disparity map between the color images of the two perspectives containing the fluorescent area in the target object by using a fast marching method to obtain a disparity map between the color images of the two perspectives containing the fluorescent area in the target object;

[0029] The three-dimensional point cloud corresponding to the target object is three-dimensionally reconstructed using a greedy projection triangulation method to obtain a three-dimensional model corresponding to the target object.

[0030] In some embodiments of the present application, the system further comprises:

[0031] The positioning module is used to extract the coordinates of the four vertices of the circumscribed matrix formed by the fluorescent area in each viewing angle of the color image containing the fluorescent area in the target object after stereo correction, and calculate the length and width of the fluorescent area based on the coordinates of the four vertices.

[0032] A second aspect of the present application provides a three-dimensional model construction method, the method comprising:

[0033] Acquire color images of a target object at two viewing angles and a fluorescent image of a target area in the target object, wherein the color images of the target object at two viewing angles are obtained by photographing the target object with a binocular color camera, and the fluorescent image of the target area in the target object is obtained by photographing the target object with a fluorescent camera;

[0034] Performing denoising and / or pseudo-colorization processing on the fluorescence image of the target area in the target object to obtain a first fluorescence image of the target area in the target object;

[0035] Registering the first fluorescent image of the target area in the target object with the selected color image of each viewing angle of the target object respectively, so as to obtain the first fluorescent image of each viewing angle of the target area in the target object;

[0036] Fusing the first fluorescent image of each viewing angle of the target area in the target object with the color image of the target object at the viewing angle to obtain the color image of each viewing angle of the target object containing the fluorescent area;

[0037] Acquire an intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of a binocular color camera, wherein the intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of the binocular color camera are obtained by calibrating the binocular color camera;

[0038] Based on the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, stereo correction is performed on the color image of each viewing angle containing the fluorescent area of ​​the target object, and stereo matching is performed on the color images of the two viewing angles containing the fluorescent area of ​​the target object after stereo correction, so as to obtain a disparity map between the color images of the two viewing angles containing the fluorescent area of ​​the target object;

[0039] Based on the disparity map between the color images of the two perspectives containing the fluorescent area of ​​the target object and the intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of the binocular color camera, a depth map corresponding to the target object is obtained;

[0040] The depth map corresponding to the target object is back-projected to obtain a three-dimensional point cloud corresponding to the target object, and the three-dimensional point cloud corresponding to the target object is three-dimensionally reconstructed to obtain a three-dimensional model corresponding to the target object.

[0041] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the second aspect of the above-mentioned embodiments when executing the computer program.

[0042] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the second aspects of the above embodiments is implemented.

[0043] The present application has the following beneficial effects: the three-dimensional model construction scheme of the present application acquires color images of the target object at different viewing angles and fluorescent images of the target area in the target object, and constructs a three-dimensional model corresponding to the target object based on the color images of the target object at different viewing angles and the fluorescent images of the target area in the target object. Therefore, the three-dimensional model construction scheme of the present application obtains a three-dimensional model of the target object, and when the scheme is used to obtain an image of the target area of ​​the patient, a three-dimensional image of the target area of ​​the patient can be obtained, and the three-dimensional image can provide more detailed structural information of the target area of ​​the patient, thereby better assisting doctors in obtaining more accurate disease diagnosis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0045] Figure 1 It is a schematic diagram of the framework of the first embodiment of the three-dimensional model construction system provided by the present application;

[0046] Figure 2It is a schematic diagram of the framework of the second embodiment of the three-dimensional model construction system provided by the present application;

[0047] Figure 3 It is a schematic diagram of the framework of the third embodiment of the three-dimensional model construction system provided by the present application;

[0048] Figure 4 It is a schematic diagram of the framework of the fourth embodiment of the three-dimensional model construction system provided by the present application;

[0049] Figure 5 It is a schematic diagram of the framework of the fifth embodiment of the three-dimensional model construction system provided by the present application;

[0050] Figure 6 It is a schematic diagram of the framework of the sixth embodiment of the three-dimensional model construction system provided by the present application;

[0051] Figure 7 It is a schematic diagram of an example framework of the arrangement of various components in the image acquisition module of the three-dimensional model construction system provided by the present application;

[0052] Figure 8 It is a flow chart of an embodiment of constructing a three-dimensional model of a target object by an image processing module in a three-dimensional model construction system provided by the present application;

[0053] Fig. 9 This is a schematic diagram of an example framework of a comparison diagram before and after pseudo-colorization of an image provided by the present application;

[0054] Fig.10 This is a schematic diagram of an example of the result of registering the fluorescent image of the target object with the color images of two viewing angles, provided in the present application;

[0055] Fig.11 This is a schematic diagram of an example framework of comparison images before and after block extraction processing of an image provided by the present application;

[0056] Fig.12 is a schematic diagram of an example of the result of stereo correction of images from different viewing angles provided by the present application;

[0057] Fig.13 This is a schematic diagram of an example framework for generating a disparity map based on images from two perspectives provided in this application;

[0058] Fig.14 This is a schematic diagram of an example framework of comparison images before and after optimization of a disparity map provided in this application;

[0059] Fig.15 is a schematic diagram of an example of a matrix of circumscribed fluorescent regions in an image provided by the present application;

[0060] Fig.16It is a schematic diagram of experimental results of three-dimensional models of target objects corresponding to different viewing angles obtained by using the three-dimensional model construction system of the present application;

[0061] Fig.17 It is a schematic diagram of a flow chart of an embodiment of a three-dimensional model construction method provided by the present application;

[0062] Fig.18 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;

[0063] Fig.19 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0064] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0065] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0066] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0067] As described in the background technology, when the existing imaging system is used to obtain an image of the patient's target area, only a two-dimensional image of the patient's target area can be obtained. The two-dimensional image cannot provide more detailed structural information of the patient's target area, making it difficult to assist doctors in obtaining more accurate disease diagnosis results.

[0068] To this end, the present application proposes a new solution, in which a color image of a target object and a fluorescent image of a target area in the target object are obtained based on color imaging and fluorescent imaging technology, respectively, and a three-dimensional reconstruction of the target object is performed based on the two images to obtain a three-dimensional model corresponding to the target object. When the solution is used to construct an image of a patient's target area, a three-dimensional image of the patient's target area can be obtained, and the three-dimensional image can provide more detailed structural information of the patient's target area, thereby better assisting doctors in obtaining more accurate disease diagnosis results.

[0069] The present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0070] According to one embodiment of the present application, Figure 1 As shown, the present application provides a three-dimensional model construction system, which is used to construct a three-dimensional model corresponding to a target object. The system includes: an image acquisition module, which is used to acquire color images of the target object at different viewing angles and fluorescent images of a target area in the target object; a control module, which is used to control the image acquisition module so that the image acquisition module can normally complete the acquisition of color images of the target object at different viewing angles and fluorescent images of the target area in the target object; and an image processing module, which is used to obtain the three-dimensional model corresponding to the target object based on the color images of the target object at different viewing angles and the fluorescent images of the target area in the target object.

[0071] It can be seen from the above embodiments that when the three-dimensional model construction system of the present application is used to obtain an image of a patient's target area, a three-dimensional image of the patient's target area can be obtained, and the three-dimensional image can provide more detailed structural information of the patient's target area, thereby better assisting doctors in obtaining more accurate disease diagnosis results.

[0072] According to one embodiment of the present application, Figure 2 As shown, the system further includes: a display module, which is used to display the three-dimensional model corresponding to the target object. In addition, according to an embodiment of the present application, Figure 3 As shown, the system also includes: a positioning module, which is used to extract the coordinates of the four vertices of the circumscribed matrix formed by the fluorescent area in the color image of each perspective containing the fluorescent area in the target object after stereo correction, and calculate the length and width of the fluorescent area based on the coordinates of the four vertices.

[0073] In order to more clearly understand the solution of the present application, each module in the three-dimensional model construction system of the present application is described in detail below.

[0074] 1. Image acquisition module

[0075] The image acquisition module is used to acquire color images of the target object at different viewing angles and fluorescent images of the target area in the target object.

[0076] According to one embodiment of the present application, Figure 4As shown, the image acquisition module includes: a fluorescence camera for collecting the fluorescence image of the target area in the target object; a color camera assembly for collecting color images of different viewing angles of the target object, wherein the color camera assembly includes a plurality of binocular color cameras, each of which is used to collect color images of two viewing angles of the target object, and each of which is composed of two color cameras placed at a preset distance, and each of which is used to collect a color image of one viewing angle of the target object; a laser light source (such as a near-infrared laser light source) for laser irradiation of the target area in the target object, so that the fluorescence camera collects the fluorescence image of the target area in the target object. According to one embodiment of the present application, the fluorescence camera is a near-infrared CMOS camera, and the color camera is a color CMOS camera. The near-infrared CMOS camera can efficiently capture fluorescence signals and convert them into digital image data. It should be noted that the near-infrared CMOS camera and the color CMOS camera here are only one embodiment of the present application. With the development of image acquisition equipment, the specific fluorescence camera and color camera can be further selected according to application requirements, characteristics of the target object, system integration requirements, image processing requirements and other factors. The device with better performance can be replaced, which is not specifically limited here.

[0077] According to one embodiment of the present application, Figure 5 As shown, the image acquisition module further includes: a filter, which is used to enable the fluorescence camera to acquire a fluorescence image of a target area in the target object in a specified wavelength band; an illumination lamp (such as an LED lamp), which is used to provide illumination to assist the color camera component in acquiring color images of the target object from different viewing angles and the fluorescence camera in acquiring a fluorescence image of the target area in the target object. According to one embodiment of the present application, Figure 6 As shown, the image acquisition module also includes: a laser light homogenizer, which is used to adjust the light beam of the laser light source so that the light beam emitted by the laser light source can be evenly irradiated on the target area of ​​the target object.

[0078] It can be seen from the above embodiments that the function of the filter is to select a light source or fluorescent signal of a specific wavelength to improve the signal-to-noise ratio and contrast of the imaging; the near-infrared laser light source is used to excite the ICG fluorescent probe on the target area of ​​the target object to generate a fluorescent signal; the laser homogenizer is used to adjust the light beam of the laser light source to ensure that the light source can evenly irradiate the target area of ​​the target object, while having an appropriate spot size and shape; the LED lamp is used as auxiliary lighting to provide uniform brightness of the surrounding environment; the laser homogenizer enables the light beam emitted by the laser light source to evenly irradiate the target area of ​​the target object. The above embodiments are conducive to improving the imaging quality. It should be noted that the LED lamp and the near-infrared laser light source are only one of the embodiments of the present application. The lighting lamp and the laser light source in the specific three-dimensional model construction system can be further selected according to the application requirements, the characteristics of the target object, the system integration requirements, and the image processing requirements. The factors such as, can further select better performance equipment to replace, and no specific limitation is made here.

[0079] In order to more intuitively understand how the components in the image acquisition module in the above embodiment work together to complete high-quality image acquisition, the configuration of each component is described below with reference to specific examples.

[0080] This application designs a three-camera hardware structure of a dual color camera and a single fluorescence camera as an example to visually illustrate the setting method of the camera in the image acquisition module, such as Figure 7 As shown, in the design of this application, the horizontal distance between the two color cameras is 70 mm, their distances to the laser light source are both 35 mm, the distance from the laser light source to the near-infrared camera (i.e., the fluorescent camera) is 50 mm, and the distance from the center of the near-infrared camera lens to the line connecting the centers of the two color cameras is 70 mm. Figure 7 The design of such a size parameter structure is mainly to ensure that the common field of view of the three cameras at a 40cm object distance can reach 10cm*10cm. It should be noted that the arrangement of the components in the above-mentioned image acquisition module is only one example. The arrangement of the components in the specific image acquisition module is determined according to comprehensive factors such as field of view overlap, optical property matching, interference reduction, physical space limitations, and application scenario requirements, and is not specifically limited here.

[0081] The camera position settings in the above example fully consider the system's requirements for high precision and high sensitivity, ensuring that the designs of the optical modules and mechanical structures of the dual-color camera and the single fluorescence camera meet the system's requirements and specifications, so that high-quality images can be obtained by performing image acquisition under this structure.

[0082] 2. Control Module

[0083] The control module is used to control the image acquisition module so that the image acquisition module can normally complete the acquisition of color images of the target object at different viewing angles and the fluorescence image of the target area in the target object. For example, the main functions of the control module include: setting and adjusting the exposure time and gain of the camera to ensure image quality and clarity; controlling the camera hardware parameters, such as frame rate, resolution, etc., to meet system requirements; real-time monitoring of the camera status, including connection status, image acquisition status, etc. The control module needs to control three cameras (one fluorescence camera and two color cameras) at the same time to complete the acquisition of fluorescence images and color images.

[0084] It can be seen from the above embodiments that by controlling the device-related parameters of the image acquisition module in the image acquisition module through the control module, the quality of the acquired image can be improved and the flexible operability of the system can be improved.

[0085] 3. Image Processing Module

[0086] The image processing module is used to obtain a three-dimensional model corresponding to the target object based on the color images of the target object at different viewing angles and the fluorescent image of the target area in the target object.

[0087] According to one embodiment of the present application, Figure 8 As shown, the image processing module is configured to execute steps S1-S7 to complete the construction of the three-dimensional model corresponding to the target object. The contents of steps S1-S7 are described in detail below.

[0088] In the step S1, the fluorescence image of the target area in the target object is subjected to denoising and / or pseudo-colorization processing to obtain a first fluorescence image of the target area in the target object.

[0089] Fluorescence images taken with a fluorescence camera are usually grayscale images. However, the human eye has a relatively low resolution for grayscale images and can generally only distinguish more than 20 gray levels. In contrast, the human eye is more sensitive to color changes and can simultaneously distinguish thousands of color images with different hues, brightness, and saturation. Therefore, in order to better display the details and features of the image, the grayscale image is generally converted into a color image. The visual effect of the image is enhanced by mapping the grayscale differences that are difficult for the human eye to distinguish into color differences. This is pseudo-colorization. Fig. 9 As shown, it shows the comparison of the fluorescent image of the target object before and after pseudo-colorization, where: Fig. 9 (a) is the grayscale fluorescence image of the target object captured by the fluorescence camera; Fig. 9 (b) is the fluorescent image of the target object after pseudo-color processing. It can be clearly seen that the visual effect of the fluorescent image after pseudo-color processing is better and more consistent with the observation of the human eye.

[0090] In step S2, one of the color images of the target object from two perspectives captured by the binocular color camera is arbitrarily selected, and the first fluorescent image of the target area in the target object is respectively registered with the selected color image of each perspective of the target object to obtain the first fluorescent image of the target area in each perspective of the target object.

[0091] Since the shooting angles and resolutions of the color camera and the fluorescence camera of the system are different, and there are no feature points that can be extracted from the fluorescence area captured by the fluorescence camera, the images captured by the two cameras will have differences in perspective and size and cannot be aligned using traditional feature extraction methods. In order to achieve basic alignment of the two images with different perspectives and sizes, according to one embodiment of the present application, a perspective transformation method is used to align the first fluorescence image of the target area in the target object with the selected color image of each perspective of the target object, so as to obtain the first fluorescence image of each perspective of the target area in the target object. It should be noted that the perspective transformation method for image alignment is only one embodiment of the present application. The specific alignment method needs to be determined based on multiple factors such as task requirements, image type and characteristics, computing resources, etc., and is not specifically limited here.

[0092] For example, Fig.10 As shown, it shows the result of registering a first fluorescent image of a target area in a target object with color images of two selected viewpoints of the target object, wherein: Fig.10 (a) is the result of registering the first fluorescent image of the target area in the target object with the color image of one viewing angle of the target object, Fig.10 (b) is the result of registering the first fluorescent image of the target area in the target object with the color image of the target object from another viewing angle.

[0093] In the above embodiment, the first fluorescent image of the target area in the target object is registered with the selected color image of each viewing angle of the target object, that is, the position of the first fluorescent image of the target area in the target object is aligned with the position of the target area in the color images of the target object at two viewing angles, so that when the fluorescent image is subsequently fused with the color image, the position of the fluorescent area of ​​the target object is accurate.

[0094] Wherein, according to one embodiment of the present application, the image processing module is further configured to: arbitrarily select the color images of the two perspectives of the target object collected by the binocular color camera, and respectively align the first fluorescent image of the target area in the target object with the selected color images of each perspective of the target object, and segment and / or block extract the fluorescent images of each perspective of the target area in the target object after alignment, so as to obtain the first fluorescent image of each perspective of the target area in the target object. Wherein, according to one embodiment of the present application, the image processing module is further configured to: use the OTSU method to segment the fluorescent images of each perspective of the target area in the target object after alignment. It should be noted that the OTSU method is only an embodiment of the present application for image segmentation processing. The specific segmentation method needs to be determined according to multiple factors such as task requirements, image type and characteristics, computing resources, etc., and is not specifically limited here.

[0095] In order to more intuitively understand the concept of block extraction processing in the above embodiment, Fig.11 Block extraction is described below. Fig.11 As shown, if an image contains different objects, each object in the image can be extracted by performing block extraction on the image. That is, in the image in the figure, there is a pentagon, an ellipse and a polygon connected together. After performing block extraction on the image, different objects in the image can be extracted separately.

[0096] The above embodiment performs segmentation and / or block extraction processing on the fluorescence image of each viewing angle of the target area in the target object to obtain a registration image containing only the fluorescence area. The accuracy of subsequent three-dimensional reconstruction of the fluorescence area in the target object is improved. This solution of reconstructing a fluorescence three-dimensional model of the target area in the target object is applied in the medical field, which can improve the accuracy of obtaining a three-dimensional image of the patient's target area, and the doctor can observe and diagnose the condition of the patient's target area more intuitively and carefully.

[0097] In step S3, the first fluorescent image of each viewing angle of the target area in the target object and the color image of the target object at that viewing angle are fused to obtain the color image of each viewing angle containing the fluorescent area in the target object.

[0098] In the step S4, a calibration process is performed on a selected binocular color camera to obtain an intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera.

[0099] In step S5, based on the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, stereo correction is performed on the color image of each perspective containing the fluorescent area in the target object, and stereo matching is performed on the color images of the two perspectives containing the fluorescent area in the target object after stereo correction to obtain a disparity map between the color images of the two perspectives containing the fluorescent area in the target object.

[0100] According to one embodiment of the present application, the image processing module is further configured to: use a semi-global block matching method to stereo match the color images of the two perspectives of the target object containing the fluorescent area after stereo correction to obtain an initial disparity map between the color images of the two perspectives of the target object containing the fluorescent area, and use a fast marching method to optimize the initial disparity map between the color images of the two perspectives of the target object containing the fluorescent area to obtain a disparity map between the color images of the two perspectives of the target object containing the fluorescent area. It should be noted that the semi-global block matching method for stereo matching of images and the fast marching method for optimizing the disparity map are both embodiments of the present application. The specific matching method and optimization method need to be determined based on multiple factors such as task requirements, image type and characteristics, and computing resources, and are not specifically limited here.

[0101] For example, Fig.12 As shown, it shows the Fig.10 The result of stereo correction of the two images in , where Fig.12 (a) for Fig.10 (a) The effect after stereo correction. Fig.12 (b) for Fig.10 (b) The effect of stereo correction. It can be clearly seen that the corresponding points in the two images after stereo correction are on the same horizontal line. Fig.13 As shown, it shows the result of stereo matching the color images of two perspectives of the target object containing the fluorescent area after stereo correction to obtain the disparity map between the two images, wherein: Fig.13 (a) is a color image of a target object containing a fluorescent area at one viewing angle. Fig.13 (b) is a color image of the target object from another perspective containing the fluorescent area. Fig.13 (c) is the disparity map between the color images of the target object containing the fluorescent area from two perspectives. Fig.14 As shown, it shows the comparison of the disparity map before and after optimization, where: Fig.14 (a) is the disparity map between the color images of the two perspectives containing the fluorescent area of ​​the target object before optimization. Fig.14(b) is the disparity map between the color images of the two perspectives containing the fluorescent area in the optimized target object.

[0102] From the above embodiments, it can be seen that stereo correction can eliminate image distortion caused by different camera installation angles, so that the corresponding points in the two images are located on the same horizontal line, thereby simplifying and improving the accuracy and efficiency of subsequent stereo matching; optimizing the disparity map can improve the smoothness and continuity of the disparity map, improve the accuracy of the disparity map, improve the quality of the depth map, enhance robustness, and improve computational efficiency. These optimization measures help generate more accurate three-dimensional models and improve the application effect of medical imaging.

[0103] In step S6, a depth map corresponding to the target object is obtained based on a disparity map between color images of two viewing angles containing a fluorescent area in the target object and an intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of the binocular color camera.

[0104] In step S7, the depth map corresponding to the target object is back-projected to obtain a three-dimensional point cloud corresponding to the target object, and the three-dimensional point cloud corresponding to the target object is three-dimensionally reconstructed to obtain a three-dimensional model corresponding to the target object.

[0105] According to one embodiment of the present application, the image acquisition module is further configured to: perform three-dimensional reconstruction of the three-dimensional point cloud corresponding to the target object using a greedy projection triangulation method to obtain a three-dimensional model corresponding to the target object. It should be noted that the three-dimensional reconstruction of the target object using the greedy projection triangulation method is only one of the embodiments of the present application. The specific reconstruction method needs to be determined based on multiple factors such as task requirements, image type and characteristics, and computing resources, and is not specifically limited here.

[0106] 4. Display module

[0107] The display module is used to display the three-dimensional model corresponding to the target object.

[0108] It can be seen from the above embodiments that when the three-dimensional model construction system of the present application is used to construct a three-dimensional image of a patient's target area, the constructed three-dimensional image of the patient is ultimately displayed through a display module, and the doctor can observe the structural features of the patient's target area more intuitively, thereby making a more accurate diagnosis result.

[0109] 5. Positioning module

[0110] The positioning module is used to extract the coordinates of four vertices of the circumscribed matrix formed by the fluorescent area in each viewing angle of the color image containing the fluorescent area in the target object after stereo correction, and calculate the length and width of the fluorescent area based on the coordinates of the four vertices.

[0111] According to one embodiment of the present application, the coordinates of the four vertices of the circumscribed matrix formed by the fluorescent area in the color image of each viewing angle of the target object containing the fluorescent area after stereo correction are extracted using the connected component labeling method, and the length and width of the fluorescent area are calculated based on the coordinates of the four vertices. It should be noted that the use of the connected component labeling method to extract the vertex coordinates of the circumscribed matrix of the fluorescent area in the image is only one embodiment of the present application, and the specific extraction method needs to be determined according to multiple factors such as task requirements, image type and characteristics, and computing resources, and is not specifically limited here.

[0112] For example, Fig.15 As shown, it shows the circumscribed matrix ABCD formed by the fluorescent area in the target object image. By extracting the coordinates of A, B, C, and D in the three-dimensional space, and calculating the Euclidean distances of AB and AC respectively, the AB and AC values ​​are averaged after multiple calculations, and the length and width of the fluorescent area are measured.

[0113] As can be seen from the above embodiments, the three-dimensional model construction system of the present application is applied to the medical field. By extracting the vertex coordinates of the circumscribed rectangle of the fluorescent area through the fluorescent area positioning module and calculating its length and width, the position of the patient's target area can be determined more accurately, and the positioning accuracy, quantitative analysis ability, and the accuracy of auxiliary diagnosis and treatment planning can be improved, while improving work efficiency and standardization. These benefits are of great significance to improving the effectiveness of medical diagnosis and treatment.

[0114] In order to verify the effectiveness of the three-dimensional model construction system of the above embodiment of the present application, the inventor conducted a series of experiments, and the specific experimental results are as follows:

[0115] like Fig.16 As shown, it shows the three-dimensional models of the target object from different perspectives obtained by using the three-dimensional model construction system of the present application. Fig.16 (a) is the main view of the 3D model corresponding to the target object; Fig.16 (b) is the left side view of the 3D model corresponding to the target object; Fig.16 (c) is the right side view of the three-dimensional model corresponding to the target object; Fig.16 (d) is the attached view of the three-dimensional model corresponding to the target object; Fig.16 (e) is a bottom view of the three-dimensional model corresponding to the target object. The experimental results show that the three-dimensional model construction system of the present application can obtain three-dimensional models of the target object from different perspectives, and the three-dimensional models from different perspectives can provide more comprehensive and detailed structural information of the patient's target area, thereby better assisting doctors in obtaining more accurate disease diagnosis results.

[0116] The three-dimensional model construction scheme of the present application is described in detail from the method point of view.

[0117] According to one embodiment of the present application, Fig.17 As shown, the present application provides a three-dimensional model construction method, the method comprising: T1, acquiring color images of a target object at two viewing angles and a fluorescent image of a target area in the target object, wherein the color images of the target object at two viewing angles are obtained by photographing the target object with a binocular color camera, and the fluorescent image of the target area in the target object is obtained by photographing the target object with a fluorescent camera; T2, denoising and / or pseudo-colorizing the fluorescent image of the target area in the target object to obtain a first fluorescent image of the target area in the target object; T3, registering the first fluorescent image of the target area in the target object with the selected color image of each viewing angle of the target object, respectively, to obtain a first fluorescent image of the target area in each viewing angle of the target object; T4, fusing the first fluorescent image of each viewing angle of the target area in the target object with the color image of the target object at that viewing angle, to obtain a color image of each viewing angle of the target object containing the fluorescent area; T5, acquiring the internal image of the binocular color camera The method comprises the steps of: generating a stereoscopic image of each viewing angle of the target object containing the fluorescent area based on the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, wherein the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera are obtained by calibrating the binocular color camera; T6, based on the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, stereo correction is performed on the color images of each viewing angle of the target object containing the fluorescent area, and stereo matching is performed on the color images of the two viewing angles of the target object containing the fluorescent area after the stereo correction to obtain a disparity map between the color images of the two viewing angles of the target object containing the fluorescent area; T7, based on the disparity map between the color images of the two viewing angles of the target object containing the fluorescent area and the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, a depth map corresponding to the target object is obtained; T8, back-projecting the depth map corresponding to the target object to obtain a three-dimensional point cloud corresponding to the target object, and three-dimensionally reconstructing the three-dimensional point cloud corresponding to the target object to obtain a three-dimensional model corresponding to the target object.

[0118] Among them, according to one embodiment of the present application, the step T3 includes: respectively aligning the first fluorescence image of the target area in the target object with the selected color image of each viewing angle of the target object, and segmenting and / or block extraction processing the aligned fluorescence image of each viewing angle of the target area in the target object to obtain the first fluorescence image of each viewing angle of the target area in the target object.

[0119] To summarize, compared to the existing imaging systems applied in the medical field that can only obtain two-dimensional images of the patient's target area, the three-dimensional model construction system of the present application can obtain a three-dimensional image of the patient's target area when used to obtain images of the patient's target area in the medical field. The three-dimensional image can provide more detailed structural information of the patient's target area, thereby better assisting doctors in obtaining more accurate disease diagnosis and treatment results.

[0120] In addition, according to an embodiment of the present application, the present application provides a three-dimensional model construction method, the method comprising: using the three-dimensional model construction system or method described in any of the above embodiments to perform three-dimensional reconstruction of the target object.

[0121] It can be seen from the above embodiments that the three-dimensional model construction system of the present application is used to obtain a three-dimensional image of the patient's target area, which can provide more detailed structural information of the patient's target area, thereby better assisting doctors in obtaining more accurate disease diagnosis results.

[0122] Based on the inventive concept of the above embodiments, the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiments when executing the computer program. Fig.18 Provide detailed explanation.

[0123] like Fig.18 As shown, it shows the electronic device of the present application, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0124] The processor 110 is used to control the operation of the electronic device. The processor 110 may also be referred to as a CPU (Central Processing Unit). The processor 110 may be an integrated circuit chip having the ability to process signals. The processor 110 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 110 may also be any conventional processor, etc.

[0125] The memory 120 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0126] The processor 110 is used to execute the computer program stored in the memory 120 to implement the methods described in the various method embodiments of the present application.

[0127] In some embodiments, the electronic device may further include: a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, the memory 120 and the peripheral terminal interface 130 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160 and a power supply 170.

[0128] The peripheral terminal interface 130 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 110 and the memory 120. In some embodiments, the processor 110, the memory 120, and the peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 110, the memory 120, and the peripheral terminal interface 130 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0129] The radio frequency circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 140 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 140 is the communication circuit of the electronic device. The radio frequency circuit 140 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 140 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 140 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 140 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0130] The display screen 150 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 150 is a touch display screen, the display screen 150 also has the ability to collect touch signals on the surface or above the surface of the display screen 150. The touch signal can be input to the processor 110 as a control signal for processing. At this time, the display screen 150 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 150 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 150 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 150 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 150 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 150 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0131] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 110 for processing, or input them into the radio frequency circuit 140 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0132] The power supply 170 is used to power various components in the electronic device. The power supply 170 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0133] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] Based on the inventive concept of the above embodiment, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above projection area calculation method. Fig.19 The execution process of the above-mentioned embodiment in a computer-readable storage medium is described.

[0138] like Fig.19 As shown, it shows the computer-readable storage medium of the present application. If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 200. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage medium.

[0139] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0140] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0141] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

Claims

1. A three-dimensional model construction system, the system is used to construct a three-dimensional model corresponding to a target object, characterized in that: The system comprises: An image acquisition module, used for acquiring color images of a target object at different viewing angles and a fluorescent image of a target area in the target object; A control module, used to control the image acquisition module so that the image acquisition module can normally complete the acquisition of color images of the target object at different viewing angles and the fluorescence image of the target area in the target object; The image processing module is used to obtain a three-dimensional model corresponding to the target object based on the color images of the target object at different viewing angles and the fluorescent image of the target area in the target object.

2. The system according to claim 1, characterized in that The image acquisition module comprises: A fluorescence camera, used for collecting a fluorescence image of a target area in the target object; A color camera assembly, used to collect color images of the target object at different viewing angles, wherein the color camera assembly includes a plurality of binocular color cameras, each of which is used to collect color images of the target object at two viewing angles, and each of which is composed of two color cameras placed at a preset distance, each of which is used to collect a color image of the target object at one viewing angle; The laser light source is used to irradiate the target area in the target object with laser so that the fluorescence camera can collect the fluorescence image of the target area in the target object.

3. The system according to claim 2, characterized in that The image acquisition module also includes: A filter, used to enable the fluorescence camera to collect a fluorescence image of a target area in the target object in a specified wavelength band; The lighting lamp is used to provide illumination to assist the color camera assembly in collecting color images of the target object at different viewing angles and the fluorescence camera in collecting fluorescence images of the target area in the target object.

4. The system according to claim 2, characterized in that The image processing module is configured to obtain the three-dimensional model corresponding to the target object in the following manner: Performing denoising and / or pseudo-colorization processing on the fluorescence image of the target area in the target object to obtain a first fluorescence image of the target area in the target object; Arbitrarily selecting the color images of the target object from two viewing angles captured by the binocular color camera, and registering the first fluorescent image of the target area in the target object with the selected color images of each viewing angle of the target object, so as to obtain the first fluorescent image of the target area in each viewing angle of the target object; Fusing the first fluorescent image of each viewing angle of the target area in the target object with the color image of the target object at the viewing angle to obtain the color image of each viewing angle containing the fluorescent area in the target object; Calibrate a selected binocular color camera to obtain an intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera; Based on the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, stereo correction is performed on the color image of each viewing angle containing the fluorescent area of ​​the target object, and stereo matching is performed on the color images of the two viewing angles containing the fluorescent area of ​​the target object after stereo correction, so as to obtain a disparity map between the color images of the two viewing angles containing the fluorescent area of ​​the target object; Obtaining a depth map corresponding to the target object based on a disparity map between color images of two perspectives containing a fluorescent area of ​​the target object and an intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of the binocular color camera; The depth map corresponding to the target object is back-projected to obtain a three-dimensional point cloud corresponding to the target object, and the three-dimensional point cloud corresponding to the target object is three-dimensionally reconstructed to obtain a three-dimensional model corresponding to the target object.

5. The system according to claim 4, characterized in that The image processing module is further configured to: Arbitrarily select the color images of the target object from two perspectives captured by the binocular color camera, and respectively align the first fluorescent image of the target area in the target object with the selected color images of each perspective of the target object, and segment and / or perform block extraction processing on the registered fluorescent images of each perspective of the target area in the target object to obtain the first fluorescent image of each perspective of the target area in the target object.

6. The system according to claim 4, characterized in that The image processing module is further configured to: Using a perspective transformation method, the first fluorescent image of the target area in the target object is respectively registered with the selected color image of each viewing angle of the target object to obtain the first fluorescent image of each viewing angle of the target area in the target object; Performing stereo matching on the color images of the two perspectives containing the fluorescent area in the target object after stereo correction by using a semi-global block matching method to obtain an initial disparity map between the color images of the two perspectives containing the fluorescent area in the target object, and optimizing the initial disparity map between the color images of the two perspectives containing the fluorescent area in the target object by using a fast marching method to obtain a disparity map between the color images of the two perspectives containing the fluorescent area in the target object; The three-dimensional point cloud corresponding to the target object is three-dimensionally reconstructed using a greedy projection triangulation method to obtain a three-dimensional model corresponding to the target object.

7. The system according to claim 6, characterized in that The system further comprises: The positioning module is used to extract the coordinates of the four vertices of the circumscribed matrix formed by the fluorescent area in each viewing angle of the color image containing the fluorescent area in the target object after stereo correction, and calculate the length and width of the fluorescent area based on the coordinates of the four vertices.

8. A three-dimensional model construction method based on the system according to any one of claims 1 to 7, characterized in that: The method comprises: Acquire color images of a target object at two viewing angles and a fluorescent image of a target area in the target object, wherein the color images of the target object at two viewing angles are obtained by photographing the target object with a binocular color camera, and the fluorescent image of the target area in the target object is obtained by photographing the target object with a fluorescent camera; Performing denoising and / or pseudo-colorization processing on the fluorescence image of the target area in the target object to obtain a first fluorescence image of the target area in the target object; Registering the first fluorescent image of the target area in the target object with the selected color image of each viewing angle of the target object respectively, so as to obtain the first fluorescent image of each viewing angle of the target area in the target object; Fusing the first fluorescent image of each viewing angle of the target area in the target object with the color image of the target object at the viewing angle to obtain the color image of each viewing angle containing the fluorescent area in the target object; Acquire an intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of a binocular color camera, wherein the intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of the binocular color camera are obtained by calibrating the binocular color camera; Based on the intrinsic parameter matrix, relative extrinsic parameters and distortion parameters of the binocular color camera, stereo correction is performed on the color image of each viewing angle containing the fluorescent area of ​​the target object, and stereo matching is performed on the color images of the two viewing angles containing the fluorescent area of ​​the target object after stereo correction, so as to obtain a disparity map between the color images of the two viewing angles containing the fluorescent area of ​​the target object; Obtaining a depth map corresponding to the target object based on a disparity map between color images of two perspectives containing a fluorescent area of ​​the target object and an intrinsic parameter matrix, relative extrinsic parameters, and distortion parameters of the binocular color camera; The depth map corresponding to the target object is back-projected to obtain a three-dimensional point cloud corresponding to the target object, and the three-dimensional point cloud corresponding to the target object is three-dimensionally reconstructed to obtain a three-dimensional model corresponding to the target object.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the method according to claim 8 are implemented when the processor executes the computer program.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method of claim 8 is implemented.

Images