Depth image construction method and device based on endoscope, equipment and medium

Abstract

The invention provides a depth image construction method and device based on an endoscope, equipment and a medium. The method comprises the following steps: acquiring a plurality of sub-aperture images obtained after the same light path passes through a plurality of micro lenses in the same plane; feature points in the different sub-aperture images are extracted, the feature points in the different sub-aperture images are matched to obtain matching point pairs, and the matching point pairs are used for representing a set of feature points of the same type distributed in the different sub-aperture images; and determining depth values of the corresponding matching point pairs according to the focal lengths of the micro-lenses, the distances between the different micro-lenses and the parallax of the matching point pairs in the different sub-aperture images, so as to construct a corresponding depth map based on the depth values. According to the method and the device, the depth image of the endoscope can be constructed, a user side can judge the general condition of the target object based on the depth image, and richer image information is provided compared with an independent two-dimensional image.

Description

Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to an endoscope-based depth image construction method, device, equipment and medium. Background Art

[0002] Traditional medical endoscopes typically use a single optical path and a single sensor. This single optical path's light signals are used to generate only two-dimensional images, providing only a 2D image of the lesion without any depth information. Diagnosis and treatment based solely on 2D images can easily lead to misdiagnosis or missed diagnoses. Summary of the Invention

[0003] In view of the above problems existing in the prior art, the present invention proposes a method, device, equipment and medium for constructing a depth image based on an endoscope, which mainly solves the problem that traditional endoscopes lack depth information, thereby affecting the diagnostic effect.

[0004] In order to achieve the above-mentioned and other purposes, the technical solutions adopted by the present invention are as follows.

[0005] The present application provides an endoscope-based depth image construction method, the method comprising: obtaining multiple sub-aperture images obtained after the same optical path passes through multiple microlenses in the same plane; extracting feature points in different sub-aperture images, and matching the feature points in different sub-aperture images to obtain matching point pairs, wherein the matching point pairs are used to represent a set of feature points of the same type distributed in different sub-aperture images; determining the depth value of the corresponding matching point pair based on the focal length of the microlens, the spacing between different microlenses, and the parallax of the matching point pair in the different sub-aperture images, so as to construct a corresponding depth map based on the depth value.

[0006] In one embodiment of the present application, the step of obtaining multiple sub-aperture images obtained after the same light path passes through multiple microlenses in the same plane includes: splitting the main light path of the endoscope to obtain a split light path, so that the split light path passes through a microlens array composed of multiple microlenses, and imaging through a first image sensor to obtain multiple sub-aperture images, wherein the sub-aperture images correspond one-to-one to the microlenses.

[0007] In an embodiment of the present application, the step of constructing a corresponding depth map based on the depth values includes: mapping the depth values corresponding to different matching point pairs to a two-dimensional matrix, and generating a smooth depth map by interpolation.

[0008] The present application also provides an imaging device applied to the endoscope-based depth image construction method, the imaging device comprising: an optical lens having a main light path arranged therein; a spectroscopic unit arranged at the end of the optical lens, for dividing the main light path into two light paths for transmission, one being an imaging light path and the other being a spectroscopic light path; a microlens array arranged on the spectroscopic light path; an imaging unit for receiving the imaging light path to obtain a two-dimensional image; and further for receiving the light path on the light-emitting side of each microlens in the microlens array to obtain multiple sub-aperture images, and generating a depth map corresponding to the two-dimensional image based on the sub-aperture images.

[0009] In one embodiment of the present application, the imaging unit includes a first image sensor, a second image sensor and a processor, the second image sensor is used to receive the two-dimensional image obtained by the imaging light path; the first image sensor is used to receive the light path on the light-emitting side of the microlens array to obtain multiple sub-aperture images, and the processor solves the sub-aperture images to obtain a depth map corresponding to the two-dimensional image.

[0010] In one embodiment of the present application, the optical lens includes a first lens, a second lens, a third lens, a fourth lens, a fifth lens, a sixth lens and a seventh lens arranged in sequence from the object side to the image side along the main optical path; wherein the first lens is a plane lens; the second lens is a lens with negative optical power, the object side surface of which is concave, and the image side surface is concave or flat; the third lens is a lens with positive optical power, the object side surface of which is convex or flat, and the image side surface is convex; the fourth lens is a lens with positive optical power, the image side surface of which is convex; the fifth lens is a lens with positive optical power, the object side surface of which is convex; the sixth lens is a lens with positive optical power, the object side surface of which is convex, and the image side surface is concave; the seventh lens is a lens with positive optical power, the object side surface of which is convex, and the image side surface is concave.

[0011] In one embodiment of the present application, the sixth lens and the seventh lens form a cemented lens.

[0012] In one embodiment of the present application, the microlenses in the microlens array are located in the same plane and have positive optical power.

[0013] The present application also provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the steps of the endoscope-based depth image construction method.

[0014] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is caused to execute the steps of the endoscope-based depth image construction method.

[0015] As described above, the endoscope-based depth image construction method, device, equipment and medium proposed in this application have the following beneficial effects.

[0016] This application generates multiple sub-aperture images through multiple microlenses, and then uses different sub-aperture images to match feature points. The depth values of the feature points in the sub-aperture images can be calculated by combining the microlens parallax, spacing and focal length to form a depth image, so that the user end can judge the approximate situation of the target object based on the depth image, providing richer image information than a single two-dimensional image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the process of constructing a depth image based on an endoscope in one embodiment of the present application.

[0018] Figure 2 Schematic diagram of the optical structure of an optical lens in one embodiment of the present application.

[0019] Figure 3 FIG. 1 is a schematic structural diagram of a microlens array in an embodiment of the present application.

[0020] Figure 4 FIG. 1 is a schematic diagram of the structure of an imaging device in one embodiment of the present application.

[0021] Figure 5 FIG. 1 is a schematic diagram of the structure of an imaging device in another embodiment of the present application.

[0022] Figure 6 Schematic diagram of the structure of an electronic device in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0024] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0025] The inventors have found that:

[0026] Traditional two-dimensional endoscopes use a single optical path and single sensor design (such as white light CMOS (Complementary Metal-Oxide Semiconductor) or CCD (Charge-Coupled Device)), which can only provide two-dimensional surface images of diseased tissues and cannot obtain depth information. This makes it difficult for doctors to judge key parameters such as the three-dimensional position and infiltration depth of the lesion, which can easily lead to misdiagnosis or missed diagnosis. Traditional two-dimensional endoscopes lack depth information, while existing three-dimensional imaging endoscopes need to sacrifice resolution and real-time performance. Three-dimensional imaging technology is easily affected by ambient light, and the depth of field is insufficient in minimally invasive environments, resulting in large errors in depth calculation.

[0027] Based on the problems existing in the above-mentioned related technologies, the present application proposes a method, device, equipment and medium for constructing a depth image based on an endoscope. The technical solution of the present application is elaborated in detail below in combination with specific embodiments.

[0028] See also Figure 1 , Figure 1 The figure is a flow chart of a method for constructing a depth image based on an endoscope in one embodiment of the present application. The method comprises the following steps:

[0029] Step S100 , obtaining a plurality of sub-aperture images obtained after the same light path passes through a plurality of micro-lenses in the same plane.

[0030] In one embodiment, the step of obtaining multiple sub-aperture images obtained after the same optical path passes through multiple microlenses in the same plane includes: splitting the main optical path of the endoscope to obtain split optical paths, allowing the split optical paths to pass through a microlens array composed of multiple microlenses, and imaging the multiple sub-aperture images using a first image sensor, wherein the sub-aperture images correspond one-to-one to the microlenses. The first image sensor can be a CMOS sensor, for example.

[0031] In one embodiment, the optical structure of the optical lens is as follows: Figure 2As shown, the optical lens includes a first lens 1, a second lens 3, a third lens 4, a fourth lens 5, a fifth lens 6, a sixth lens 7, a seventh lens 8, an eighth lens 9 and a ninth lens 10, which are arranged on an axis in sequence from the object side to the image side of the main light path, wherein the first lens 1 is a flat protective lens, and the first lens 1 cooperates with the optical lens housing 12 to prevent other lenses from contacting the external environment, and the optical lens housing 12 is a straight rod structure or a lens barrel structure; the second lens 3 is a lens with negative optical power, whose object side is concave and the image side is concave or flat; an aperture 2 can also be set between the first lens 1 and the second lens 3, and the aperture 2 can adjust the width of the main light path (that is, the width of the light path from the first lens 1 to the second lens). The third lens 4 is a lens with positive focal power, whose object side is convex or flat, and whose image side is convex; the fourth lens 5 is a lens with positive focal power, whose object side is flat, and whose image side is convex; the fifth lens 6 is a lens with positive focal power, whose object side is convex, and whose image side is flat; the sixth lens 7 is a lens with positive focal power, whose object side is convex, and whose image side is concave; the seventh lens 8 is a lens with negative focal power, whose object side is convex, and whose image side is concave. The sixth lens 7 and the seventh lens 8 can be bonded together to form a cemented lens. By bonding lenses of different materials and refractive indices, aberrations can be eliminated and image quality can be improved. At the same time, the cemented lens can reduce the number of interfaces between air and glass, thereby reducing reflection loss. In addition, the cemented lens can compensate for the difference in curvature radius of the cemented surfaces, thereby reducing the requirements for processing accuracy and simplifying the processing process. The eighth lens element 9 is a beam splitter. Part of the main optical path passes through the beam splitter and enters the ninth lens element 10. Another part of the light is reflected by the eighth lens element 9, forming a split optical path. For example, this split optical path forms a 90-degree angle with the main optical path, but other angles are possible depending on the needs and are not limited here. The ninth lens element 10 is a protective flat glass. A second image sensor 11 (such as a CMOS or CCD) can be located on the image side of the ninth lens element 10 to convert the optical signal from the main optical path into an electrical signal, which is then transmitted to the processor to generate the corresponding two-dimensional image. The reflectivity and transmittance of the eighth lens element 9 can be set and adjusted according to actual application requirements and are not limited here. The second lens element 3 uses negative power to diverge the incident parallel light beam, shifting the main plane backward. This achieves a longer working distance than the actual focal length, even with a shorter physical focal length. The intermediate lenses use positive power to focus the diverged light. The combined negative and positive power distribution significantly reduces the incident angle of off-axis light beams, achieving a wider field of view. The seventh lens 8 uses a negative power lens to further adjust the optical path, adapt to the image sensor, and avoid edge image quality degradation.

[0032] A micro lens array 13 may be provided on the light splitting path. The structure of the micro lens array 13 is as follows: Figure 3As shown. The microlens array 13 is obtained by arranging a plurality of microlenses in the same plane according to a certain pattern. The object side of each microlens faces the spectroscope to receive the light transmitted by the spectroscopic optical path, and the image side is provided with a tenth lens 14. The tenth lens 14 is a protective flat glass. A first image sensor 15 is provided on the side of the tenth lens 14 away from the microlens array 13, which is used to convert the light focused by each microlens into a corresponding electrical signal, thereby generating a corresponding sub-aperture image. Each microlens has a positive optical focal length. The specific number of microlenses in the microlens array 13 and the size of each microlens can be set and adjusted according to actual application requirements, and are not limited here. Of course, the arrangement of the microlenses in the microlens array 13 can be a symmetrical structure or other asymmetric arrangements can be adopted as needed, and are not limited here. By setting a spectroscope to divide the light path into two, this solution can be implemented by only making minor changes to the original endoscope structure, which can effectively save design and manufacturing costs.

[0033] In one embodiment, the beam splitter may not be used, and the microlens array 13 may be directly set on the main light path. This method provides a relatively simple output image, but it can also achieve the technical solution of the embodiment of the present application.

[0034] In one embodiment, a method of extracting corresponding sub-aperture images for different microlens positions can be expressed as follows:

[0035]

[0036] Among them, L[x, y, u, v] is a four-dimensional square function, (x, y) is the coordinate of the main lens plane; (u, v) is the coordinate of the micro lens plane, I s,t (x, y) is the extracted sub-aperture image; Δ is the viewing angle step size, and i and j are the center coordinates of the corresponding microlens in the microlens array.

[0037] Step S110, extract the feature points in different sub-aperture images, and match the feature points in different sub-aperture images to obtain matching point pairs, wherein the matching point pairs are used to characterize the set of feature points of the same type distributed in different sub-aperture images. Specifically, the feature points can be pixel points corresponding to the lesion tissue. The pixel points of the same lesion tissue are regarded as the same type of feature points, and the feature points corresponding to the lesion tissue in different sub-aperture images are matched, and then the feature points belonging to the same lesion on different sub-aperture images are formed into matching point pairs. Of course, the feature points can also be determined according to the specific application scenario of the endoscope. This is only an example and should not be regarded as a limitation to the embodiments of the present application.

[0038] In one embodiment, before extracting image feature points, median filtering and Gaussian denoising preprocessing may be used to eliminate interference such as blood and mucus in the endoscope environment, thereby ensuring the reliability of the subsequently generated depth map.

[0039] In one embodiment, a scale-invariant feature transform (SIFT) or a speeded-up robust feature (SURF) algorithm can be used to extract feature points in each sub-aperture image that are invariant to scale, rotation, and illumination. A brute-force matching algorithm, combined with the microlens focus obtained through a ratio test, can then be used to match the feature points of different sub-aperture images to obtain matching point pairs. The specific feature extraction and brute-force matching processes are well known in the art and will not be detailed here.

[0040] Step S120 , determining depth values of corresponding matching point pairs according to the focal length of the microlenses, the spacing between different microlenses, and the parallax of the matching point pairs in different sub-aperture images, so as to construct a corresponding depth map based on the depth values.

[0041] In one embodiment, the disparity between the matching point pairs may be calculated using a disparity matching cost function, which may be expressed as:

[0042]

[0043] Where d is the disparity, C(x, y, d) is the matching cost, and I 0，0 is the reference sub-aperture image, I s，t is the target sub-aperture image, and W is the matching window.

[0044] After obtaining the parallax, the corresponding depth value can be calculated through the depth solution. The specific depth solution formula can be expressed as:

[0045]

[0046] in, is the virtual baseline length, p is the microlens pitch, and f0 is the microlens focal length.

[0047] After obtaining the depth values of the pixel points corresponding to the feature points in the sub-aperture image based on the above steps, the depth values can be mapped to a two-dimensional matrix to obtain a depth matrix. Furthermore, in order to obtain a smooth depth map, the depth matrix can be interpolated (such as using bilinear interpolation, Lagrange interpolation or Newton interpolation, etc.). The generated depth map is a two-dimensional image, which can reflect the contour and depth information of the target object. After receiving the corresponding depth map, doctors or relevant personnel can make a diagnosis in combination with the visible light two-dimensional image obtained by the main optical path, which greatly avoids the risk of misdiagnosis and missed diagnosis.

[0048] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an imaging device in one embodiment of the present application. The imaging device provided in this embodiment of the present application includes:

[0049] an optical lens 40, in which a main light path is provided;

[0050] A microlens array 13 is provided on the main light path;

[0051] The imaging unit 41 is configured to receive the light path of the light-emitting side of each microlens in the microlens array to obtain a plurality of sub-aperture images, and generate a depth map according to the sub-aperture images.

[0052] The optical lens 40 may include the aforementioned first to seventh lenses and the ninth lens. The microlens array 13 may be disposed at the end of the optical lens 40. The optical lens 40 may be a lens barrel structure. The imaging unit 41 may be disposed in the endoscope camera handle.

[0053] See also Figure 5 As an optional embodiment, the three-dimensional imaging device further includes a light splitting unit 42, which is disposed at the end of the optical lens 40 to split the main light path into two light paths for transmission, one of which is an imaging light path and the other is a light splitting light path;

[0054] A microlens array 13 is provided on the light splitting path;

[0055] The imaging unit 41 is used to receive the imaging light path to obtain a two-dimensional image; it is also used to receive the light path on the light-emitting side of each microlens in the microlens array 13 to obtain multiple sub-aperture images, and generate a depth map corresponding to the two-dimensional image based on the sub-aperture images.

[0056] Specifically, the imaging unit 41 includes a first image sensor 15, a second image sensor 11 and a processor. The second image sensor 11 is used to receive the imaging light path to obtain a two-dimensional image. The first image sensor 15 is used to receive the light path on the light-emitting side of each microlens in the microlens array 13 to obtain multiple sub-aperture images. The processor is used to generate a depth map corresponding to the two-dimensional image based on the sub-aperture image.

[0057] In one embodiment, the light splitting unit 42 is an eighth lens 9 , which can be disposed between the seventh lens 8 and the ninth lens 10 .

[0058] The specific execution process of the imaging unit 41 has been described in detail in the aforementioned method embodiment and will not be repeated here.

[0059] Based on the technical solutions of the above embodiments of the present application, the incident light is divided into two paths through a beam splitter prism, respectively realizing high-resolution two-dimensional imaging (4K resolution, 60fps) and depth calculation function based on a microlens array. Doctors can observe the surface details and spatial position of the lesion at the same time without switching devices or modes; it overcomes the problem of lack of depth information of traditional two-dimensional endoscopes and the equipment redundancy caused by the need for additional light sources or dual cameras in existing three-dimensional endoscopes, realizing "one mirror for multiple uses"; using a microlens array to generate multi-view sub-aperture images, combined with SIFT / SURF algorithms and triangulation principles, the depth calculation error is better than binocular stereo vision and structured light technology; dual 4K image sensors can be used to synchronously capture images, and the processor processes the visible light image of the main light path and the depth image of the split light path in parallel to ensure low latency and meet real-time intraoperative requirements, while maintaining the two-dimensional image resolution of 3840×2160 pixels, avoiding the loss of details caused by the sacrifice of resolution in existing three-dimensional imaging technology. The modular design supports seamless integration with existing endoscope systems, and doctors can use it directly without additional training, shortening the clinical adaptation cycle. Depth map data is compatible with AI-assisted diagnosis systems, providing a three-dimensional data basis for automated lesion identification (such as early tumor markers), thereby improving diagnostic efficiency.

[0060] like Figure 6 As shown, computer system 600 includes a central processing unit 601, which can perform various appropriate actions and processes according to programs stored in read-only memory 602 or programs loaded from storage unit 608 into random access memory 603, such as executing the methods described in the above embodiments. Random access memory 603 also stores various programs and data required for system operation. Central processing unit 601, read-only memory 602, and random access memory 603 are connected to each other via bus 604. Input / output interface 605 is also connected to bus 604.

[0061] The following components are connected to the input / output interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), and a speaker; a storage section 608 including devices such as a hard disk; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed into the storage section 608 as needed.

[0062] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609 and / or installed from a removable medium 611. When the computer program is executed by the central processing unit 601, the various functions defined in the system of the present application are performed.

[0063] The computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage device, magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0064] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0065] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0066] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When executed by a computer processor, the computer program causes the computer to perform the aforementioned endoscope-based depth image construction method. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device.

[0067] Another aspect of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the endoscope-based depth image construction method provided in each of the above embodiments.

[0068] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A method for constructing a depth image based on an endoscope, characterized in that: The method comprises: Acquire multiple sub-aperture images obtained after the same optical path passes through multiple microlenses in the same plane; Extracting feature points from different sub-aperture images, and matching the feature points in different sub-aperture images to obtain matching point pairs, wherein the matching point pairs are used to represent a set of feature points of the same type distributed in different sub-aperture images; Determine the depth value of the corresponding matching point pair according to the focal length of the microlens, the spacing between different microlenses, and the parallax of the matching point pair in different sub-aperture images, and then construct a corresponding depth map based on the depth value.

2. The method for constructing a depth image based on an endoscope according to claim 1, wherein: The step of obtaining multiple sub-aperture images obtained after the same light path passes through multiple microlenses in the same plane comprises: The main light path of the endoscope is split to obtain a split light path, so that the split light path passes through a microlens array composed of multiple microlenses, and is imaged by a first image sensor to obtain multiple sub-aperture images, wherein the sub-aperture images correspond one-to-one to the microlenses.

3. The method for constructing a depth image based on an endoscope according to claim 1, wherein: The step of constructing a corresponding depth map based on the depth values includes mapping the depth values corresponding to different matching point pairs to a two-dimensional matrix, and generating a smooth depth map by interpolation.

4. An imaging device applied to the endoscope-based depth image construction method according to any one of 1 to 3, characterized in that: The imaging device comprises: an optical lens having a main light path disposed therein; A light splitting unit is provided at the end of the optical lens, and is used to split the main light path into two light paths for transmission, one being an imaging light path and the other being a light splitting light path; a microlens array, which is arranged on the light splitting path; An imaging unit is used to receive the imaging light path to obtain a two-dimensional image; it is also used to receive the light path of the light-emitting side of each microlens in the microlens array to obtain multiple sub-aperture images, and generate a depth map corresponding to the two-dimensional image based on the sub-aperture images.

5. The imaging device according to claim 4, wherein The imaging unit includes a first image sensor, a second image sensor and a processor, the second image sensor is used to receive the imaging light path to obtain a two-dimensional image, the first image sensor is used to receive the light path on the light-emitting side of the microlens array to obtain multiple sub-aperture images, and the processor is used to solve the sub-aperture images to obtain a depth map corresponding to the two-dimensional image.

6. The imaging device according to claim 4, wherein The optical lens comprises a first lens, a second lens, a third lens, a fourth lens, a fifth lens, a sixth lens and a seventh lens arranged in sequence from the object side to the image side along the main optical path; wherein the first lens is a plane lens; the second lens is a lens with negative optical power, the object side surface of which is concave, and the image side surface is concave or flat; the third lens is a lens with positive optical power, the object side surface of which is convex or flat, and the image side surface is convex; the fourth lens is a lens with positive optical power, the image side surface of which is convex; the fifth lens is a lens with positive optical power, the object side surface of which is convex; the sixth lens is a lens with positive optical power, the object side surface of which is convex, and the image side surface is concave; the seventh lens is a lens with positive optical power, the object side surface of which is convex, and the image side surface is concave.

7. The imaging device according to claim 6, wherein The sixth lens and the seventh lens form a cemented lens.

8. The imaging device according to claim 4, wherein The microlenses in the microlens array are located in the same plane and have positive optical power.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the steps of the endoscope-based depth image construction method as described in any one of claims 1 to 3.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the steps of the endoscope-based depth image construction method as described in any one of claims 1 to 3.

Images