A surgical microscope and a monocular depth estimation method based on the surgical microscope

Abstract

The present invention provides a surgical microscope and a monocular depth estimation method based on the surgical microscope, belonging to the technical field of medical devices, and specifically comprising: determining a depth estimation part scheme of the surgical microscope according to the depth estimation credibility of different parts and the depth estimation type, performing monocular depth estimation of the surgical microscope using the depth estimation part scheme, dividing the surgical types into different operation processes, and determining that there is an operation process switch according to the similarity between the monitoring image of the surgical microscope and the division feature images of different operation processes, and optimizing the depth estimation part scheme of the surgical microscope based on the operation data of different surgical operation types matched with the operation process and the changes of the image features of different parts, thereby improving the accuracy of the depth estimation process.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical devices, and in particular relates to a surgical microscope and a monocular depth estimation method based on the surgical microscope. Background Art

[0002] Surgical microscopes are key equipment for certain surgical operations. How to use depth estimation information to determine the distance between the surgical microscope and the surgical target is crucial to improving the doctor's operating accuracy and efficiency.

[0003] Specifically, in the invention patent application CN202011372210.5 "A retinal surgical robot imaging method integrating microscope and OCT", the needle tip position is located and tracked in real time through real-time depth information, and the needle insertion depth is observed based on the real-time depth information, which improves the accuracy of the operation. However, it is not difficult to find through analysis that there are the following technical problems:

[0004] In different types of surgeries, due to the differences in the surgical objects, there are great differences in the accuracy requirements for different types of surgical operations, such as the insertion depth of surgical devices and the peeling depth of organs. Therefore, if a differentiated monocular depth estimation strategy cannot be determined according to the differences in surgical types, the accuracy of the surgical operation cannot be guaranteed.

[0005] In order to solve the above technical problems, the present invention provides a surgical microscope and a monocular depth estimation method based on the surgical microscope. Summary of the invention

[0006] To achieve the purpose of the present invention, the present invention adopts the following technical solutions:

[0007] According to one aspect of the present invention, a monocular depth estimation method based on a surgical microscope is provided.

[0008] A monocular depth estimation method based on a surgical microscope, specifically comprising:

[0009] S1 determines the operation requirements of different surgical operation types by using the surgical type, and determines the depth estimation type of the surgical microscope in combination with the surgical operation object corresponding to the surgical type;

[0010] S2: determining image features of different parts of the surgical object according to the analysis result of the image of the surgical microscope, and determining the reliability of depth estimation of the parts based on the image features of the different parts and the changes of the image features of the different parts in the surgical type, and determining the depth estimation part scheme of the surgical microscope according to the reliability of depth estimation of different parts and the depth estimation type;

[0011] S3: using the depth estimation part scheme to perform monocular depth estimation of the surgical microscope, classifying the surgical type into different operation procedures, and when it is determined that there is an operation procedure switch according to the similarity between the monitoring image of the surgical microscope and the division feature images of different operation procedures, proceeding to the next step;

[0012] S4 optimizes the depth estimation site plan of the surgical microscope based on the operation data of different surgical operation types matched with the operation flow and the changes in the image features of different sites.

[0013] A further technical solution is that the type of surgery is determined based on the input data of the operating doctor into the surgical microscope.

[0014] A further technical solution is that the operation requirement includes a limited operation time and a limited operation error.

[0015] A further technical solution is that a method for determining the depth estimation type of the surgical microscope is:

[0016] Determining the object accuracy requirement of the surgical microscope according to the surgical operation object corresponding to the surgical type;

[0017] Determine the limited operation time and limited operation error of different surgical operation types based on the operation requirements of different surgical operation types, and determine the operation accuracy requirement of the surgical microscope according to the limited operation time and limited operation error of different surgical operation types;

[0018] The operation reliability requirement of the surgical microscope is determined by the operation accuracy requirement and the object precision requirement of the surgical microscope, and the depth estimation type of the surgical microscope is determined by using the operation reliability requirement.

[0019] A further technical solution is that the depth estimation type of the surgical microscope includes a first-level accuracy estimation type, a second-level accuracy estimation type and a third-level accuracy estimation type, and the depth estimation type of the surgical microscope is specifically determined according to the reliability requirement interval in which the operational reliability requirements of the surgical microscope are located.

[0020] A further technical solution is to divide the surgical types into different operation procedures, specifically including:

[0021] The operation steps of the surgical type are divided according to a preset division method to obtain different operation processes.

[0022] A further technical solution is to determine whether there is an operation process switch, specifically including:

[0023] Determining the image similarity between the monitoring image and the segmentation feature images based on the similarity between the monitoring image and the segmentation feature images of different operation processes, and determining similar images in the segmentation feature images using the image similarity;

[0024] The operation processes before and after the similar images are close are respectively used as the front-end operation process and the back-end operation process, and the switching credibility coefficient of the monitoring image is determined according to the similarity between the monitoring image and the historical monitoring images of the front-end operation process and the historical monitoring images of the back-end operation process;

[0025] Whether there is an operation flow switch is determined by the switch credibility coefficient.

[0026] A further technical solution is that when the switching credibility coefficient is greater than a preset credibility threshold, it is determined that there is an operation process switch.

[0027] On the other hand, the present invention provides a surgical microscope, which adopts the above-mentioned monocular depth estimation method based on a surgical microscope, specifically comprising:

[0028] Type classification module, estimation scheme screening module, process switching module, and optimization processing module;

[0029] The type classification module is responsible for determining the operation requirements of different surgical operation types by using the surgical type, and determining the depth estimation type of the surgical microscope in combination with the surgical operation object corresponding to the surgical type;

[0030] The estimation scheme screening module is responsible for determining the image features of different parts of the surgical operation object according to the analysis results of the image of the surgical microscope, and determining the reliability of the depth estimation of the parts based on the image features of the different parts and the changes of the image features of the different parts in the surgical type, and determining the depth estimation part scheme of the surgical microscope according to the reliability of the depth estimation of different parts and the depth estimation type;

[0031] The process switching module is responsible for performing monocular depth estimation of the surgical microscope using the depth estimation part scheme, classifying the surgical type into different operation processes, and determining whether there is an operation process switching according to the similarity between the monitoring image of the surgical microscope and the division feature images of different operation processes;

[0032] The optimization processing module is responsible for optimizing the depth estimation part scheme of the surgical microscope based on the operation data of different surgical operation types matched with the operation flow and the changes in the image features of different parts.

[0033] The beneficial effects of the present invention are:

[0034] 1. In the present invention, the depth estimation type of the surgical microscope is determined according to the operation requirements of the surgical operation type and the surgical operation object corresponding to the surgical operation type, thereby achieving accurate evaluation of the depth estimation type of the surgical microscope from multiple angles such as the operation duration and operation accuracy requirements of the surgical operation type, and the surgical operation object, thereby laying a foundation for determining differentiated depth estimation schemes for different surgical microscopes and improving the overall processing efficiency and accuracy of the depth estimation.

[0035] 2. In the present invention, the depth estimation part scheme of the surgical microscope is optimized based on the operation data of different surgical operation types matched with the operation process and the changes in the image features of different parts, thereby avoiding the technical problem that the accuracy of depth estimation does not meet the requirements due to the use of a single depth estimation part scheme, ensuring the accuracy of depth estimation for operation processes with more complex operations and operation processes with large changes in image features, and improving the accuracy and efficiency of surgical operations.

[0036] Other features and advantages will be described in the following description. The objects and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and drawings.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0039] Figure 1 is a flow chart of a monocular depth estimation method based on a surgical microscope;

[0040] Figure 2 is a flow chart of a method for determining a depth estimation type of a surgical microscope;

[0041] Figure 3 is a flow chart of a method for determining the reliability of a depth estimate of a part;

[0042] Figure 4 is a flow chart of a method for determining a depth estimation site plan of a surgical microscope;

[0043] Figure 5 A schematic diagram of a surgical microscope. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0045] To solve the above problems, according to one aspect of the present invention, Figure 1 As shown, a monocular depth estimation method based on a surgical microscope is provided, which specifically includes:

[0046] S1 determines the operation requirements of different surgical operation types by using the surgical type, and determines the depth estimation type of the surgical microscope in combination with the surgical operation object corresponding to the surgical type;

[0047] Furthermore, the type of surgery is determined based on the data input by the surgeon into the surgical microscope.

[0048] It can be understood that the operation requirements include a limited operation time and a limited operation error.

[0049] Specifically, Figure 2 As shown, the method for determining the depth estimation type of the surgical microscope is:

[0050] Determining the object accuracy requirement of the surgical microscope according to the surgical operation object corresponding to the surgical type;

[0051] Determine the limited operation time and limited operation error of different surgical operation types based on the operation requirements of different surgical operation types, and determine the operation accuracy requirement of the surgical microscope according to the limited operation time and limited operation error of different surgical operation types;

[0052] The operation reliability requirement of the surgical microscope is determined by the operation accuracy requirement and the object precision requirement of the surgical microscope, and the depth estimation type of the surgical microscope is determined by using the operation reliability requirement.

[0053] Furthermore, the depth estimation type of the surgical microscope includes a primary accuracy estimation type, a secondary accuracy estimation type and a tertiary accuracy estimation type, and the depth estimation type of the surgical microscope is specifically determined according to the reliability requirement interval of the operational reliability requirement of the surgical microscope.

[0054] It should also be noted that the method for determining the depth estimation type of the surgical microscope is:

[0055] Determining the object accuracy requirement of the surgical microscope according to the surgical operation object corresponding to the surgical type;

[0056] Determine the limited operation time and limited operation error of different surgical operation types based on the operation requirements of different surgical operation types, and determine the comprehensive limited error amount of different surgical operation types according to the limited operation time and limited operation error of different surgical operation types;

[0057] The operation accuracy requirement of the surgical microscope is determined by using the average value of the comprehensive limited error amounts of different surgical operation types, the operation reliability requirement of the surgical microscope is determined by the operation accuracy requirement of the surgical microscope and the object accuracy requirement, and the depth estimation type of the surgical microscope is determined by using the operation reliability requirement.

[0058] It should also be noted that the method for determining the depth estimation type of the surgical microscope is:

[0059] Obtaining a surgical operation object corresponding to the surgical type, determining whether the surgical operation object corresponding to the surgical type is a specific operation object, and if so, determining a depth estimation type of the surgical microscope based on a preset estimation type, and if not, proceeding to the next step;

[0060] Determine the operation limit errors of different surgical operation types based on the operation requirements of different surgical operation types of the surgical type, judge whether the number of surgical operation types whose operation limit errors are less than a preset error threshold is greater than a preset number, if so, determine the depth estimation type of the surgical microscope based on the preset estimation type, if not, proceed to the next step;

[0061] Determine the comprehensive limit error of different surgical operation types according to the operation limit time and operation limit error of different surgical operation types, and judge whether there is a surgical operation type whose comprehensive limit error is less than the preset error threshold. If so, proceed to the next step; if not, determine the depth estimation type of the surgical microscope based on the three-level accuracy estimation type.

[0062] Determine whether the number of surgical operation types whose comprehensive limited error amount is less than a preset error threshold is greater than a preset number, if so, determine the depth estimation type of the surgical microscope based on the preset estimation type, if not, proceed to the next step;

[0063] The object accuracy requirement of the surgical microscope is determined according to the surgical operation object corresponding to the surgical type, the operation accuracy requirement of the surgical microscope is determined using the average value of the comprehensive limited error amounts of different surgical operation types, the operation reliability requirement of the surgical microscope is determined based on the operation accuracy requirement and the object accuracy requirement of the surgical microscope, and the depth estimation type of the surgical microscope is determined using the operation reliability requirement.

[0064] S2: determining image features of different parts of the surgical object according to the analysis result of the image of the surgical microscope, and determining the reliability of depth estimation of the parts based on the image features of the different parts and the changes of the image features of the different parts in the surgical type, and determining the depth estimation part scheme of the surgical microscope according to the reliability of depth estimation of different parts and the depth estimation type;

[0065] It should be noted that the image features include the grayscale value in the grayscale space and the number of image pixels.

[0066] Specifically, the changes in the image features of the different parts in the surgery type are determined based on the analysis results of the historical monitoring images of the surgical microscope of the surgery type.

[0067] It is understandable that if Figure 3 As shown, the method for determining the reliability of the depth estimation of the part is:

[0068] Determine the grayscale values ​​and the number of image pixels of the different parts in the grayscale space according to the image features of the different parts, and determine the feature significance of the different parts based on the grayscale values ​​and the number of image pixels in the grayscale space;

[0069] Determine the change of the grayscale value of the different parts in the grayscale space by the change of the image features of the different parts in the type of surgery, and determine the stability of the image features of the different parts in combination with the change of the number of image pixels of the different parts;

[0070] The confidence level of the depth estimation of the part is determined based on the feature saliency and image feature stability.

[0071] Furthermore, the value range of the reliability of the depth estimation of the part is between 0 and 1, wherein the higher the reliability of the depth estimation of the part is, the greater the reliability of the position for depth estimation is.

[0072] Specifically, Figure 4 As shown, the method for determining the depth estimation site plan of the surgical microscope is:

[0073] Determine credible parts among the parts according to the depth estimation credibility of different parts and the depth estimation type, and freely combine the credible parts to generate multiple available depth estimation alternatives;

[0074] Determine the solution estimation credibility of different available depth estimation alternative solutions by weighting the depth estimation credibility of different credible parts of different available depth estimation alternative solutions;

[0075] Determining the depth estimation difficulty of different available depth estimation alternatives according to the number of credible parts of different available depth estimation alternatives;

[0076] Based on the depth estimation difficulty and the scheme estimation credibility, the usable coefficients of different available depth estimation alternative schemes are determined, and the depth estimation site scheme of the surgical microscope is determined using the usable coefficients.

[0077] Further, using the available coefficient to determine the depth estimation location scheme of the surgical microscope specifically includes:

[0078] The available depth estimation alternative with the largest available coefficient is taken as the depth estimation part solution.

[0079] It can be understood that the method for determining the depth estimation site plan of the surgical microscope is:

[0080] Determining an estimation credibility threshold of the depth estimation part solution according to the depth estimation type;

[0081] Generate multiple groups of alternative plans based on different parts, estimate the weight of the credibility of the different alternative plans according to the depth of the parts, and determine the credibility of the plan estimation of different alternative plans;

[0082] The depth estimation site solution of the surgical microscope is determined based on the estimation credibility of different alternative solutions, with the estimation credibility threshold and the minimum number of sites as constraints.

[0083] It should be further explained that the method for determining the depth estimation location scheme of the surgical microscope is:

[0084] Determine the credible parts among the parts according to the depth estimation credibility of different parts and the depth estimation type, determine whether the sum of the depth estimation credibility of the credible parts meets the requirement based on the depth estimation type, if so, proceed to the next step, if not, generate the depth estimation part scheme of the surgical microscope based on the credible parts;

[0085] The credible parts are freely combined to generate a plurality of available depth estimation alternatives, and it is determined whether the sum of the depth estimation credibility of different credible parts of the available depth estimation alternatives meets the requirements, if so, proceeding to the next step, if not, determining that the available depth estimation alternative does not belong to the depth estimation part solution of the surgical microscope;

[0086] Obtaining the number of credible parts of the available depth estimation alternative solution, determining whether the number of credible parts of the available depth estimation alternative solution is greater than a preset number of parts, if so, determining that the available depth estimation alternative solution does not belong to the depth estimation part solution of the surgical microscope, if not, proceeding to the next step;

[0087] Determine the solution estimation credibility of different available depth estimation alternative solutions by weighting the depth estimation credibility of different credible parts of different available depth estimation alternative solutions;

[0088] Determining the depth estimation difficulty of different available depth estimation alternatives according to the number of credible parts of different available depth estimation alternatives;

[0089] Based on the depth estimation difficulty and the scheme estimation credibility, the usable coefficients of different available depth estimation alternative schemes are determined, and the depth estimation site scheme of the surgical microscope is determined using the usable coefficients.

[0090] S3: using the depth estimation part scheme to perform monocular depth estimation of the surgical microscope, classifying the surgical type into different operation procedures, and when it is determined that there is an operation procedure switch according to the similarity between the monitoring image of the surgical microscope and the division feature images of different operation procedures, proceeding to the next step;

[0091] Furthermore, performing monocular depth estimation of the surgical microscope specifically includes:

[0092] Determine a credible part corresponding to the depth estimation part scheme and use it as the estimated part;

[0093] The surgical microscope is controlled to switch between multiple angles to obtain multiple monitoring images, and the monocular depth estimation of the surgical microscope is performed according to the change of the gray value of the estimated part in different monitoring images.

[0094] It is understandable that the types of surgeries are divided into different operation procedures, specifically including:

[0095] The operation steps of the surgical type are divided according to a preset division method to obtain different operation processes.

[0096] Further, it is determined that there is an operation process switch, specifically including:

[0097] Determining the image similarity between the monitoring image and the segmentation feature images based on the similarity between the monitoring image and the segmentation feature images of different operation processes, and determining similar images in the segmentation feature images using the image similarity;

[0098] The operation processes before and after the similar images are close are respectively used as the front-end operation process and the back-end operation process, and the switching credibility coefficient of the monitoring image is determined according to the similarity between the monitoring image and the historical monitoring images of the front-end operation process and the historical monitoring images of the back-end operation process;

[0099] Whether there is an operation flow switch is determined by the switch credibility coefficient.

[0100] It can be understood that when the switching credibility coefficient is greater than a preset credibility threshold, it is determined that there is an operation process switching.

[0101] Specifically, it should be noted that determining whether there is an operation process switch includes:

[0102] Determine the image similarity between the monitoring image and the dividing feature images based on the similarity between the monitoring image and the dividing feature images of different operation processes, and judge whether there is a dividing feature image whose image similarity is greater than a preset similarity, if so, proceed to the next step, if not, determine that there is no operation process switching;

[0103] Determine similar images in the divided feature images by using the image similarity, use the operation flows before and after the similar images as the front-end operation flow and the back-end operation flow respectively, and determine whether there is a historical monitoring image in the front-end operation flow with a similarity greater than a preset similarity according to the similarity between the monitoring image and the historical monitoring image of the front-end operation flow, if not, proceed to the next step, if yes, determine that there is no operation flow switching;

[0104] Determine whether there is a historical monitoring image in the backend operation process according to the similarity between the monitoring image and the historical monitoring image of the backend operation process, and if not, proceed to the next step; if so, determine that there is no operation process switching;

[0105] The switching credibility coefficient of the monitoring image is determined according to the similarity between the monitoring image and the historical monitoring images of the front-end operation process and the historical monitoring images of the back-end operation process, and whether there is an operation process switch is determined by the switching credibility coefficient.

[0106] S4 optimizes the depth estimation site plan of the surgical microscope based on the operation data of different surgical operation types matched with the operation flow and the changes in the image features of different sites.

[0107] Specifically, the method for optimizing the depth estimation part scheme of the surgical microscope is as follows:

[0108] Determine the operation times of different surgical operation types based on the operation data of different surgical operation types matched with the operation flow, and determine the operation reliability requirement coefficient of the operation flow based on the operation requirements of different surgical operation types;

[0109] Determine the change of the image features of the credible parts in the depth estimation part scheme according to the change of the image features of different parts, and determine the change amount of the depth estimation credibility of the depth estimation part scheme by using the change of the image features of the credible parts;

[0110] Based on the operation reliability requirement coefficient of the operation process, the depth estimation credibility of the depth estimation part scheme and the change amount of the depth estimation credibility, when it is determined that the depth estimation part scheme does not meet the requirements, based on the change of image features of different parts, the credible image features and the image feature credibility of the credible image features are determined;

[0111] The number of feature selections of the credible image features is determined by the operation reliability requirement coefficient, and the optimization processing result of the depth estimation part scheme of the surgical microscope is determined based on the image feature credibility and the number of feature selections.

[0112] Furthermore, based on the credibility of the image features and the number of selected features, the optimization processing result of the depth estimation part scheme of the surgical microscope is determined, specifically including:

[0113] Based on the number of selected features, the credible image features are selected from large to small according to the credibility of the image features to obtain a selection result, and the selection result is used to determine the optimization processing result of the depth estimation part scheme of the surgical microscope.

[0114] Example 2

[0115] On the other hand, Figure 5 As shown, the present invention provides a surgical microscope, which adopts the above-mentioned monocular depth estimation method based on a surgical microscope, specifically comprising:

[0116] Type classification module, estimation scheme screening module, process switching module, and optimization processing module;

[0117] The type classification module is responsible for determining the operation requirements of different surgical operation types by using the surgical type, and determining the depth estimation type of the surgical microscope in combination with the surgical operation object corresponding to the surgical type;

[0118] The estimation scheme screening module is responsible for determining the image features of different parts of the surgical operation object according to the analysis results of the image of the surgical microscope, and determining the reliability of the depth estimation of the parts based on the image features of the different parts and the changes of the image features of the different parts in the surgical type, and determining the depth estimation part scheme of the surgical microscope according to the reliability of the depth estimation of different parts and the depth estimation type;

[0119] The process switching module is responsible for performing monocular depth estimation of the surgical microscope using the depth estimation part scheme, classifying the surgical type into different operation processes, and determining whether there is an operation process switching according to the similarity between the monitoring image of the surgical microscope and the division feature images of different operation processes;

[0120] The optimization processing module is responsible for optimizing the depth estimation part scheme of the surgical microscope based on the operation data of different surgical operation types matched with the operation flow and the changes in the image features of different parts.

[0121] Based on the above embodiments, the present invention achieves the following beneficial effects:

[0122] 1. In the present invention, the depth estimation type of the surgical microscope is determined according to the operation requirements of the surgical operation type and the surgical operation object corresponding to the surgical operation type, thereby achieving accurate evaluation of the depth estimation type of the surgical microscope from multiple angles such as the operation duration and operation accuracy requirements of the surgical operation type, and the surgical operation object, thereby laying a foundation for determining differentiated depth estimation schemes for different surgical microscopes and improving the overall processing efficiency and accuracy of the depth estimation.

[0123] 2. In the present invention, the depth estimation part scheme of the surgical microscope is optimized based on the operation data of different surgical operation types matched with the operation process and the changes in the image features of different parts, thereby avoiding the technical problem that the accuracy of depth estimation does not meet the requirements due to the use of a single depth estimation part scheme, ensuring the accuracy of depth estimation for operation processes with more complex operations and operation processes with large changes in image features, and improving the accuracy and efficiency of surgical operations.

[0124] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0125] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A monocular depth estimation method based on a surgical microscope, characterized in that: Specifically include: Determine the operation requirements of different surgical operation types by using the surgical type, and determine the depth estimation type of the surgical microscope in combination with the surgical operation object corresponding to the surgical type; Determine the image features of different parts of the surgical operation object according to the analysis results of the image of the surgical microscope, and determine the reliability of depth estimation of the parts based on the image features of the different parts and the changes of the image features of the different parts in the surgical type, and determine the depth estimation part scheme of the surgical microscope according to the reliability of depth estimation of different parts and the depth estimation type; Using the depth estimation part scheme to perform monocular depth estimation of the surgical microscope, the surgical type is divided into different operation processes, and when it is determined that there is an operation process switch according to the similarity between the monitoring image of the surgical microscope and the division feature images of different operation processes, the next step is entered; Based on the operation data of different surgical operation types matched with the operation flow and the changes of image features of different parts, the depth estimation part scheme of the surgical microscope is optimized; The method for determining the depth estimation site plan of the surgical microscope is: Determine credible parts among the parts according to the depth estimation credibility of different parts and the depth estimation type, and freely combine the credible parts to generate multiple available depth estimation alternatives; Determine the solution estimation credibility of different available depth estimation alternative solutions by weighting the depth estimation credibility of different credible parts of different available depth estimation alternative solutions; Determining the depth estimation difficulty of different available depth estimation alternatives according to the number of credible parts of different available depth estimation alternatives; Determine the usable coefficients of different available depth estimation alternative solutions based on the depth estimation difficulty and solution estimation credibility, and determine the depth estimation site solution of the surgical microscope using the usable coefficients; The method for optimizing the depth estimation part scheme of the surgical microscope is as follows: Determine the operation times of different surgical operation types based on the operation data of different surgical operation types matched with the operation flow, and determine the operation reliability requirement coefficient of the operation flow based on the operation requirements of different surgical operation types; Determine the change of the image features of the credible parts in the depth estimation part scheme according to the change of the image features of different parts, and determine the change amount of the depth estimation credibility of the depth estimation part scheme by using the change of the image features of the credible parts; Based on the operation reliability requirement coefficient of the operation process, the depth estimation credibility of the depth estimation part scheme and the change amount of the depth estimation credibility, when it is determined that the depth estimation part scheme does not meet the requirements, based on the change of image features of different parts, the credible image features and the image feature credibility of the credible image features are determined; The number of feature selections of the credible image features is determined by the operation reliability requirement coefficient, and the optimization processing result of the depth estimation part scheme of the surgical microscope is determined based on the image feature credibility and the number of feature selections.

2. The monocular depth estimation method based on a surgical microscope according to claim 1, characterized in that: The type of surgery is determined based on the data input by the surgeon into the surgical microscope.

3. The monocular depth estimation method based on a surgical microscope according to claim 1, characterized in that: The operation requirements include an operation limit time and an operation limit error.

4. The monocular depth estimation method based on a surgical microscope according to claim 1, characterized in that: The method for determining the depth estimation type of the surgical microscope is: Determining the object accuracy requirement of the surgical microscope according to the surgical operation object corresponding to the surgical type; Determine the limited operation time and limited operation error of different surgical operation types based on the operation requirements of different surgical operation types, and determine the operation accuracy requirement of the surgical microscope according to the limited operation time and limited operation error of different surgical operation types; The operation reliability requirement of the surgical microscope is determined by the operation accuracy requirement and the object precision requirement of the surgical microscope, and the depth estimation type of the surgical microscope is determined by using the operation reliability requirement.

5. The monocular depth estimation method based on surgical microscope according to claim 1, characterized in that: The image features include grayscale values ​​in grayscale space and the number of image pixels.

6. The monocular depth estimation method based on a surgical microscope according to claim 1, characterized in that: The method for determining the reliability of the depth estimation of the part is: Determine the grayscale values ​​and the number of image pixels of the different parts in the grayscale space according to the image features of the different parts, and determine the feature significance of the different parts based on the grayscale values ​​and the number of image pixels in the grayscale space; Determine the change of the grayscale value of the different parts in the grayscale space by the change of the image features of the different parts in the type of surgery, and determine the stability of the image features of the different parts in combination with the change of the number of image pixels of the different parts; The confidence level of the depth estimation of the part is determined based on the feature saliency and image feature stability.

7. The monocular depth estimation method based on a surgical microscope according to claim 6, characterized in that: The value range of the depth estimation credibility of the part is between 0 and 1, wherein the higher the depth estimation credibility of the part is, the greater the credibility of the part in depth estimation is.

8. The monocular depth estimation method based on a surgical microscope according to claim 1, characterized in that: The types of surgeries are divided into different operation procedures, including: The operation steps of the surgical type are divided according to a preset division method to obtain different operation processes.

9. A surgical microscope, using the monocular depth estimation method based on a surgical microscope according to any one of claims 1 to 8, characterized in that: Specifically include: Type classification module, estimation scheme screening module, process switching module, and optimization processing module; The type classification module is responsible for determining the operation requirements of different surgical operation types by using the surgical type, and determining the depth estimation type of the surgical microscope in combination with the surgical operation object corresponding to the surgical type; The estimation scheme screening module is responsible for determining the image features of different parts of the surgical operation object according to the analysis results of the image of the surgical microscope, and determining the reliability of the depth estimation of the parts based on the image features of the different parts and the changes of the image features of the different parts in the surgical type, and determining the depth estimation part scheme of the surgical microscope according to the reliability of the depth estimation of different parts and the depth estimation type; The process switching module is responsible for performing monocular depth estimation of the surgical microscope using the depth estimation part scheme, classifying the surgical type into different operation processes, and determining whether there is an operation process switching according to the similarity between the monitoring image of the surgical microscope and the division feature images of different operation processes; The optimization processing module is responsible for optimizing the depth estimation part scheme of the surgical microscope based on the operation data of different surgical operation types matched with the operation flow and the changes in the image features of different parts.

Images