Stereoscopic endoscope imaging optimization method and system based on objective lens optics
By reading device information and feature recognition within the image, establishing a response parameter set and reconstructing the light control parameters, the problem of poor parameters and environment combination during stereoscopic endoscopy imaging is solved, and imaging quality and efficiency are improved.
Patent Information
- Application Number
- CN202411030727.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-30
AI Technical Summary
The existing stereo endoscopes cannot accurately combine parameters and environment during imaging, resulting in poor imaging results.
By reading device information, aberration analysis is carried out, a response parameter set is established, and light control parameters are reconstructed based on in-image feature recognition and depth identification, imaging fitting and adaptive enhancement are performed, and optimized response parameters are generated to control the imaging process.
Improve imaging quality, efficiency and stability, and achieve effective imaging quality control and improvement of local details in different areas.
Smart Images

Figure CN119006299A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of imaging optimization, and in particular to a stereo endoscope imaging optimization method and system based on objective lens optics. Background Art
[0002] A stereo endoscope is a medical device that uses the principle of stereo imaging to achieve three-dimensional image observation. It generates a stereo image by installing two cameras at a certain distance on the optical lens, shooting and synthesizing the images captured by the two cameras. In the application process of stereo endoscopes, the imaging effect of stereo endoscopes is affected by many factors, and the stereo endoscopes in the prior art cannot accurately combine parameters and environment. Therefore, there is a technical problem of poor imaging effect in the process of imaging control. Summary of the invention
[0003] The present application provides a stereoscopic endoscope imaging optimization method based on objective lens optics, aiming to solve the technical problem that the stereoscopic endoscope in the prior art cannot accurately combine parameters and environment, resulting in poor imaging effect during the imaging control process.
[0004] In view of the above problems, the present application provides a stereoscopic endoscope imaging optimization method and system based on objective lens optics.
[0005] The first aspect disclosed in the present application provides a stereoscopic endoscope imaging optimization method based on objective lens optics, the method comprising: reading device information of the stereoscopic endoscope, and performing aberration analysis of the objective lens based on the device information to establish a response parameter set, wherein the response parameter set has a sample distance and a light state identifier; performing image pre-capture of a target area with the stereoscopic endoscope, and capturing objective lens imaging to establish a pre-captured image, wherein the pre-captured image has a depth identifier; performing image feature recognition on the pre-captured image, and distributing an attention area with the feature recognition result, reconstructing light control parameters with the depth identifier and the regional features of the attention area, and performing imaging fitting with the light control parameters; performing parameter matching of the response parameter set with the imaging fitting result to generate optimized response parameters, performing imaging control by optimizing the response parameters and the light control parameters, and configuring image adaptive enhancement parameters with the imaging fitting result; performing image fusion after enhancing the imaging result based on the adaptive enhancement parameter, and generating stereoscopic imaging with the fusion result.
[0006] The second aspect disclosed in the present application provides a stereoscopic endoscope imaging optimization system based on objective lens optics, the system is used for the above-mentioned stereoscopic endoscope imaging optimization method based on objective lens optics, the system comprises: an aberration analysis module, the aberration analysis module is used to read the device information of the stereoscopic endoscope, and based on the device information, perform aberration analysis of the objective lens, establish a response parameter set, the response parameter set has a sample distance and a light state identifier; an image pre-capture module, the image pre-capture module is used to perform image pre-capture of a target area with the stereoscopic endoscope, and capture objective lens imaging to establish a pre-captured image, wherein the pre-captured image has a depth identifier; an imaging fitting module The imaging fitting module is used to perform intra-image feature recognition on the pre-collected image, distribute attention areas based on the feature recognition results, reconstruct light control parameters based on the depth mark and the regional features of the attention areas, and perform imaging fitting based on the light control parameters; the imaging control module is used to perform parameter matching of the response parameter set based on the imaging fitting results, generate optimized response parameters, perform imaging control by optimizing the response parameters and light control parameters, and configure image adaptive enhancement parameters based on the imaging fitting results; the image fusion module is used to perform image fusion after enhancing the imaging results based on the adaptive enhancement parameters, and generate stereoscopic imaging based on the fusion results.
[0007] The third aspect disclosed in the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, any step of the first aspect disclosed in the present application is implemented.
[0008] According to a fourth aspect disclosed in the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step of the first aspect disclosed in the present application is implemented.
[0009] One or more technical solutions provided in this application have at least the following technical effects or advantages:
[0010] By reading the device information and performing aberration analysis, a response parameter set can be established to accurately describe the optical characteristics of the objective lens, so as to optimize it in the subsequent imaging process; in the imaging process, since different target areas may have different characteristics and depths, by allocating attention to the area according to the characteristics in the image, and reconstructing the light control parameters according to the depth mark and the characteristics of the attention area, the imaging quality of different areas can be effectively controlled; by matching the parameters of the response parameter set according to the imaging fitting results, the optimized response parameters are generated, and the parameters are adjusted automatically, which improves the efficiency and stability of the imaging process; by enhancing the imaging results based on adaptive enhancement parameters, different areas of the image can be enhanced, thereby improving the clarity and consistency of local details of the imaging results. In summary, the stereoscopic endoscope imaging optimization method based on objective lens optics effectively solves many technical problems existing in the stereoscopic endoscope imaging process by adopting the above-mentioned technology, and improves the imaging quality, efficiency and stability.
[0011] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 A schematic flow chart of a stereoscopic endoscope imaging optimization method based on objective lens optics provided in an embodiment of the present application;
[0013] Figure 2 A schematic diagram of the structure of a stereoscopic endoscope imaging optimization system based on objective lens optics provided in an embodiment of the present application;
[0014] Figure 3 This is a diagram of the internal structure of a computer device provided in an embodiment of the present application.
[0015] Description of reference numerals: aberration analysis module 10 , image pre-collection module 20 , imaging fitting module 30 , imaging control module 40 , image fusion module 50 . DETAILED DESCRIPTION
[0016] The embodiment of the present application solves the technical problem that the stereoscopic endoscope in the prior art cannot accurately combine parameters and environment, resulting in poor imaging effect during imaging control, by providing a stereoscopic endoscope imaging optimization method based on objective lens optics.
[0017] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically introduced below in conjunction with the accompanying drawings of the specification. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0018] like Figure 1 As shown, the embodiment of the present application provides a stereoscopic endoscope imaging optimization method based on objective lens optics, the method comprising:
[0019] Reading device information of the stereo endoscope, and performing aberration analysis of the objective lens based on the device information to establish a response parameter set, wherein the response parameter set has a sample distance and a light state identifier;
[0020] The device information of the stereo endoscope is read, including hardware information, such as camera model, lens parameters, etc., and software information, such as the resolution and frame rate of the acquired image. The aberration analysis of the objective lens is performed based on the read device information. The aberration refers to the difference between the actual imaging and the ideal imaging, which is usually caused by various factors, such as errors in the lens manufacturing process, the propagation of light in the lens, etc. The optical simulation method can be used to simulate and record the aberration conditions under various possible light conditions to obtain the aberration information of the device. A response parameter set is established based on the obtained aberration information, including the sample distance and the light state identifier. The sample distance refers to the aberration conditions at different imaging distances, and the light state identifier refers to the aberration conditions under different light conditions, such as strong light, weak light, etc.
[0021] Performing image pre-capture of a target area with the stereo endoscope, and capturing an objective lens image to establish a pre-captured image, wherein the pre-captured image carries a depth mark;
[0022] The stereo endoscope is navigated to the target area and image pre-capture is performed to obtain image data of the target area. The objective lens performs imaging to obtain optical images of the target area. These images are images captured in real time by the stereo endoscope and are used to observe the situation of the target area in real time. The pre-captured image data is saved to form a data set of pre-captured images.
[0023] When saving the pre-captured image, the depth information of the object in the image is obtained through a depth sensor or other depth estimation method, and the depth information is associated with the image data to form a pre-captured image with a depth mark.
[0024] Performing image feature recognition on the pre-collected image, distributing attention areas based on the feature recognition results, reconstructing light control parameters based on the depth mark and regional features of the attention areas, and performing imaging fitting based on the light control parameters;
[0025] Perform image processing and feature extraction on the pre-collected images to identify various features in the images, including edges, textures, shapes, etc. These features are used to understand the image content and structure. Based on the feature recognition results, determine the attention areas in the image, that is, the areas that need special attention. These areas are target areas or important structures in the image for subsequent imaging analysis.
[0026] Combined with the pre-acquired depth mark and the regional features of the attention area, the light control parameters are reconstructed. These parameters include the color, intensity, direction, etc. of the light. By adjusting these parameters, the imaging effect can be optimized, so that the attention area can obtain better imaging results. The image is fitted with the reconstructed light control parameters, such as using optical simulation or imaging algorithm, and the light control parameters are adjusted to make the image more consistent with the actual scene, so as to achieve the best imaging effect.
[0027] Perform parameter matching of the response parameter set based on the imaging fitting results to generate optimized response parameters, perform imaging control by optimizing the response parameters and light control parameters, and configure image adaptive enhancement parameters based on the imaging fitting results;
[0028] The imaging fitting results are analyzed to obtain the imaging differences and optimization space, including comparing the differences between the fitting results and the actual scenes, and evaluating the impact of various parameter adjustments on the imaging effects.
[0029] Based on the imaging fitting results, the parameters in the response parameter set are adjusted, including adjusting the sample distance, light state identification and other parameters in the parameter set according to the characteristics of the fitting results, so that the parameter set can better reflect the actual imaging situation. Based on the results of parameter matching, the optimized response parameters are generated, which are adjusted and optimized to meet the current imaging requirements and scene characteristics.
[0030] By optimizing the response parameters and light control parameters, imaging control is performed, including adjusting the light source and lens parameters, to make the imaging process more accurate. Finally, the imaging fitting results are used to configure the image adaptive enhancement parameters, including image sharpening, contrast enhancement, noise removal, etc., to further improve the image quality and clarity.
[0031] After enhancing the imaging results based on the adaptive enhancement parameters, image fusion is performed to generate stereoscopic imaging with the fusion results.
[0032] The obtained adaptive enhancement parameters are used to enhance the imaging results, including contrast adjustment, sharpening, denoising, etc. of the image to improve the quality and clarity of the image. The enhanced imaging results are fused. Image fusion can be a method of superimposing multiple images, weighted averaging, etc. to obtain richer and more comprehensive information. In the case of stereoscopic imaging, the imaging results of the left and right eyes can be fused. Stereoscopic imaging is generated based on the results of image fusion, that is, the fused images are applied to the left and right eyes respectively, so that the human eye can perceive the stereoscopic effect.
[0033] Furthermore, the performing image feature recognition on the pre-collected image and distributing attention areas according to the feature recognition results includes:
[0034] Establish a feature library of features to be identified;
[0035] Performing image traversal of the pre-collected image using the feature library, and generating a matching degree mark under two-dimensional image coordinates according to the image traversal result;
[0036] The distribution of attention areas is completed through matching degree identification results.
[0037] Determine the features that need to be identified in the image. These features are related to the target area and can also be used to identify other important structures in the image. Organize the identified feature data to form a feature library, which contains descriptive information of various features and their related identifiers.
[0038] By sliding a window on the pre-collected image, the image is traversed, and for each region, corresponding features are extracted. These features are the features to be identified that have been previously defined in the feature library. Each extracted feature is matched with the features in the feature library, for example, by calculating the similarity between the features, such as using Euclidean distance, cosine similarity, etc., to perform matching. Based on the matching results, a matching degree identifier is generated for each region in the image. The matching degree identifier is a numerical value that indicates the degree of matching between the region and the features in the feature library. The generated matching degree identifier is mapped to the two-dimensional image coordinates of the pre-collected image to obtain a matching degree identifier corresponding to the pixels of the pre-collected image.
[0039] A threshold is set based on the result of the matching degree identification. The threshold can be determined according to the specific situation and is used to determine whether an area belongs to the attention area. According to the set threshold, the areas exceeding the threshold in the matching degree identification result are determined as attention areas. These areas are considered to have a high degree of matching with the features in the feature library and may be important areas in the pre-collected image. The determined attention areas are marked in the pre-collected image, which can be achieved by drawing a bounding box on the image and marking with colors.
[0040] Furthermore, reconstructing light control parameters based on the depth mark and the regional features of the attention area, and performing imaging fitting based on the light control parameters, includes:
[0041] Acquiring the accuracy requirement of imaging, performing adaptive aggregation of regions based on the accuracy requirement, the depth mark and the attention region, and generating a region adaptive clustering result, wherein the region adaptive clustering result has a mapping focus mark;
[0042] The depth mark, the regional features of the attention area and the focus mark are input into the light response model to generate light control parameters, wherein the light control parameters include light color, light intensity and lighting direction, and imaging fitting is performed using the light control parameters.
[0043] The precision requirement of imaging is obtained, that is, how high the imaging quality and accuracy need to be, including the requirements for imaging resolution, clarity, contrast, etc. In combination with the imaging precision requirement, depth identification and attention area, each area in the pre-collected image is adaptively aggregated, that is, according to the imaging precision requirement, pixels of different depths and attention areas are aggregated to improve the quality and accuracy of imaging, and the results of adaptive aggregation are used to generate regional adaptive clustering results, including grouping image pixels into different clusters, each cluster represents a specific area with similar imaging characteristics. For each regional adaptive clustering result, the mapping focus is identified, which are the areas that need special attention during the imaging process.
[0044] First, a light response model is established. This model can generate light control parameters suitable for binocular imaging based on the depth mark, regional features of the attention area and the focus mark. These parameters include the color, intensity and lighting direction of the light. The depth mark, regional features of the attention area and the focus mark are used as input data and input into the light response model. In the light response model, light control parameters are generated based on the input data. These parameters are used to control the color, intensity and lighting direction of the light during the imaging process to meet the needs of binocular imaging.
[0045] The generated light control parameters are used to perform imaging fitting on the pre-collected image, including using optical simulation or imaging algorithm, and adjusting the light control parameters to make the image more consistent with the actual scene to achieve the best imaging effect.
[0046] Since this imaging is binocular, binocular fusion can establish stereo imaging. Therefore, for each camera, the above steps need to be performed to obtain light control parameters suitable for the camera. The two sets of solutions will be different because binocular imaging usually needs to take into account the differences between the left and right eyes and the viewing angle.
[0047] Furthermore, the step of inputting the depth mark, the regional features of the attention area and the focus mark into the light response model to generate the light control parameters includes:
[0048] Obtain the objective lens deviation angle of binocular imaging;
[0049] After the light control parameters of any objective lens are set, the lighting direction compensation of the second objective lens is performed according to the objective lens deviation angle, and the light control parameters of the second objective lens are established based on the lighting direction compensation result.
[0050] In binocular imaging, the objective lens deviation angle refers to the angular deviation between the left and right cameras, that is, the difference in viewing angle between the two cameras. This angular deviation can affect the effect of stereo imaging. A common method to obtain the objective lens deviation angle of binocular imaging is through camera calibration. Camera calibration refers to determining the internal parameters of the camera, such as focal length, optical center, and external parameters, such as rotation matrix, translation vector. In binocular camera calibration, images are taken at different angles through a specific calibration plate, and the relative position between the cameras is calculated using image processing and computer vision methods to obtain the objective lens deviation angle.
[0051] By obtaining the objective lens deviation angle of binocular imaging, this angle deviation can be taken into account in the subsequent image processing and imaging fitting process to ensure that the images of the left and right cameras can be correctly matched, thereby obtaining a more accurate stereoscopic imaging effect.
[0052] In binocular imaging, when the light control parameters of any objective lens are set, the lighting direction of the second objective lens is compensated according to the objective lens deviation angle to ensure that the imaging between the left and right cameras can match each other and maintain the accuracy and stability of stereoscopic imaging.
[0053] Specifically, a light control parameter of one objective lens, such as the left camera, is selected and set to a desired value. The light control parameter of another objective lens, such as the right camera, is compensated for the lighting direction according to the deviation angle of the objective lens, that is, the original light control parameter is adjusted so that the imaging direction of the right camera matches that of the left camera. According to the compensation result of the lighting direction, new light control parameters of the second objective lens, such as the right camera, are established. These parameters are used in the imaging process of the right camera to ensure consistency with the imaging of the left camera.
[0054] Furthermore, after enhancing the imaging result based on the adaptive enhancement parameter, image fusion is performed, including:
[0055] Sampling the attention area and non-attention area of the imaging results, and verifying the consistency of the imaging fitting results based on the sampling results;
[0056] According to the consistency verification result, adaptive enhancement parameter incremental compensation is performed, where the incremental compensation control is as follows:
[0057] S1: Establish an adaptive enhancement model based on big data, extract image features of imaging fitting results, input them as input features into the adaptive enhancement model, and output adaptive enhancement parameters;
[0058] S2: Configure an incremental fine-tuning interval, use the incremental fine-tuning interval as a hidden layer constraint, input the consistency verification result into an incremental training model with hidden layer constraints, and complete the incremental compensation of adaptive enhancement parameters, wherein the incremental training model is obtained by incremental training based on the adaptive enhancement model.
[0059] A certain number of sample points are selected from the imaging results, which are from the attention area and the non-attention area respectively. The attention area is determined according to the previous analysis, and the non-attention area is other areas except the attention area. For each sample point, the corresponding data is obtained from the imaging fitting results, including image pixel value, color information, depth information, etc.
[0060] The sampled data of the attention area and the non-attention area are compared to verify their consistency, which can be achieved by calculating indicators such as the difference in pixel values, color consistency, and consistency of depth information. According to the results of the consistency verification, it is analyzed whether the imaging results are consistent in the attention area and the non-attention area. If the consistency is good, it means that the imaging fitting results have good accuracy and stability in the entire image range.
[0061] According to the consistency verification results, adaptive enhancement parameter incremental compensation is performed to improve the quality and accuracy of imaging and meet the imaging requirements in different scenarios. The incremental compensation control is as follows:
[0062] First, collect a large amount of image data, which covers a variety of scenes, lighting conditions, shooting angles, etc. These data can be images from real scenes or images generated through simulation or synthesis. Based on the large amount of image data collected, train an adaptive enhancement model, which can be a variety of machine learning models, such as deep neural networks, support vector machines, random forests, etc. The goal of the model is to learn the relationship between image features and the optimal enhancement parameters.
[0063] Feature extraction is performed on the image of the imaging fitting result. These features include color histogram, gradient information, texture features, etc. The purpose of feature extraction is to convert the image information into a numerical form that can be processed by the machine learning model.
[0064] The extracted image features are taken as input and input into the established adaptive enhancement model. The model outputs adaptive enhancement parameters corresponding to the input features based on the mapping relationship between the learned image features and the optimal enhancement parameters. These parameters are dynamically adjusted according to the characteristics of the input image to adapt to different imaging scenarios and requirements.
[0065] Determine the incremental fine-tuning interval range, which can be determined based on factors such as consistency verification indicators and model performance. Use the configured incremental fine-tuning interval as a hidden layer constraint, which limits the range of parameter changes of the incremental training model during the fine-tuning process to ensure that the performance of the fine-tuned model is still within an acceptable range.
[0066] The incremental training model with hidden layer constraints is applied to the adaptive enhancement model. This incremental training model is obtained by incremental training based on the adaptive enhancement model. It can fine-tune the adaptive enhancement parameters according to the consistency verification results while maintaining the overall performance of the model.
[0067] The consistency verification result is used as input to the incremental training model with hidden layer constraints. This result includes various indicators about image consistency, such as pixel difference, color consistency, contrast, etc. Under the action of the incremental training model, the adaptive enhancement parameters are incrementally compensated according to the consistency verification result, so that the parameters can be dynamically adjusted according to the verification result to improve the consistency and accuracy of imaging, thereby optimizing the quality and effect of imaging.
[0068] Furthermore, the method further comprises:
[0069] Performing same-position alignment on the enhanced images, and performing image feature splicing based on the same-position alignment results;
[0070] Image fusion is completed by stitching the results of image features to establish stereoscopic imaging.
[0071] The enhanced images are aligned at the same position, which can be achieved through methods such as image registration or feature point matching. The goal is to make the same position of different images have the same pixel coordinates for subsequent feature stitching.
[0072] Extract features from the aligned images. These features include local features, global features, or depth features. The purpose of feature extraction is to capture the key information of the image. Align the extracted image features, for example, transform the coordinates of the feature points so that they are in the same coordinate system. Splice the aligned image features. The spliced features contain information from different images and have a more comprehensive and rich representation.
[0073] The features obtained by feature concatenation are fused, for example, weighted averaging of features, fusion of depth information, fusion of color information, etc. The goal is to fuse information from different perspectives into an overall feature representation. Stereoscopic imaging is generated using the fused features. The goal of stereoscopic imaging is to allow the observer to perceive the three-dimensional effect of the image, that is, the objects seen in the image have a sense of depth.
[0074] Furthermore, the method further comprises:
[0075] Performing feature matching on the stereoscopic images, and generating additional identification based on the matching similarity and the matching subject, wherein the additional identification includes a warning level identification, and different warning levels have different warning features;
[0076] The additional identification is used to manage the annotation of the stereoscopic imaging.
[0077] Features are extracted from stereoscopic imaging. These features may include local features, global features, depth features, etc. of the image. The extracted features are matched, including comparison of feature descriptors and matching of feature points. The goal is to find similar feature points or feature areas in the image.
[0078] According to the result of feature matching, the matching similarity is calculated, including comparing the number of matching points, the similarity of feature descriptors and other indicators. According to the matching similarity and the matching subject, additional identification is generated, including a warning level identification. The warning level identification can be determined according to the size of the matching similarity, that is, the higher the similarity, the higher the warning level. In addition, the warning features may also vary according to the matching subject. For example, different warning measures may be required for different types of targets or different matching situations.
[0079] The additional identifiers generated by the analysis include warning level identifiers. According to the additional identifiers after analysis, the corresponding annotation information is extracted, including matching similarity, matching subject, warning level and other information. A labeling management system is established to record and manage the labeling information of stereoscopic imaging. The extracted labeling information is recorded in the labeling management system, including associating matching similarity, matching subject, warning level and other information with the corresponding stereoscopic imaging. A query function is provided so that users can retrieve the annotation information as needed, such as querying related stereoscopic imaging according to a specific warning level, or analyzing according to the matching subject. Such a labeling management system can provide users with convenient and fast labeling query functions, which helps to better understand and utilize the information of stereoscopic imaging.
[0080] In summary, the stereoscopic endoscope imaging optimization method based on objective lens optics provided in the embodiments of the present application has the following technical effects:
[0081] 1. By reading the device information and performing aberration analysis, a set of response parameters can be established to accurately describe the optical properties of the objective lens, thereby optimizing it in the subsequent imaging process;
[0082] 2. During the imaging process, since different target areas may have different features and depths, the imaging quality of different areas can be effectively controlled by allocating attention to the area according to the features in the image and reconstructing the light control parameters according to the depth mark and the features of the attention area;
[0083] 3. By matching the parameters of the response parameter set according to the imaging fitting results, the optimized response parameters are generated, and the parameters are adjusted automatically, thus improving the efficiency and stability of the imaging process;
[0084] 4. By enhancing the imaging results based on adaptive enhancement parameters, enhancement processing can be performed on different areas of the image, thereby improving the clarity and consistency of local details of the imaging results.
[0085] In summary, this stereoscopic endoscope imaging optimization method based on objective lens optics effectively solves many technical problems existing in the stereoscopic endoscope imaging process by adopting the above-mentioned technology, and improves the imaging quality, efficiency and stability.
[0086] Based on the same inventive concept as the stereoscopic endoscope imaging optimization method based on objective lens optics in the aforementioned embodiment, Figure 2 As shown, the present application provides a stereoscopic endoscope imaging optimization system based on objective lens optics, the system comprising:
[0087] An aberration analysis module 10, which is used to read device information of the stereo endoscope, and perform aberration analysis of the objective lens based on the device information to establish a response parameter set, wherein the response parameter set has a sample distance and a light state identifier;
[0088] An image pre-collection module 20, the image pre-collection module 20 is used to perform image pre-collection of a target area with the stereo endoscope, collect objective lens imaging, and establish a pre-collected image, wherein the pre-collected image has a depth mark;
[0089] An imaging fitting module 30, the imaging fitting module 30 is used to perform image feature recognition on the pre-collected image, distribute attention areas based on the feature recognition results, reconstruct light control parameters based on the depth mark and the regional features of the attention areas, and perform imaging fitting based on the light control parameters;
[0090] An imaging control module 40, the imaging control module 40 is used to perform parameter matching of a response parameter set based on an imaging fitting result, generate optimized response parameters, perform imaging control by optimizing the response parameters and light control parameters, and configure image adaptive enhancement parameters based on the imaging fitting result;
[0091] The image fusion module 50 is used to enhance the imaging result based on the adaptive enhancement parameters and then perform image fusion to generate stereoscopic imaging with the fusion result.
[0092] Furthermore, the system further includes an attention area distribution module to perform the following operation steps:
[0093] Establish a feature library of features to be identified;
[0094] Performing image traversal of the pre-collected image using the feature library, and generating a matching degree mark under two-dimensional image coordinates according to the image traversal result;
[0095] The distribution of attention areas is completed through matching degree identification results.
[0096] Furthermore, the system further includes an imaging fitting module to perform the following operation steps:
[0097] Acquiring the accuracy requirement of imaging, performing adaptive aggregation of regions based on the accuracy requirement, the depth mark and the attention region, and generating a region adaptive clustering result, wherein the region adaptive clustering result has a mapping focus mark;
[0098] The depth mark, the regional features of the attention area and the focus mark are input into the light response model to generate light control parameters, wherein the light control parameters include light color, light intensity and lighting direction, and imaging fitting is performed using the light control parameters.
[0099] Furthermore, the system further includes a lighting direction compensation module to perform the following operation steps:
[0100] Obtain the objective lens deviation angle of binocular imaging;
[0101] After the light control parameters of any objective lens are set, the lighting direction compensation of the second objective lens is performed according to the objective lens deviation angle, and the light control parameters of the second objective lens are established based on the lighting direction compensation result.
[0102] Furthermore, the system further includes an incremental compensation module to perform the following operation steps:
[0103] Sampling the attention area and non-attention area of the imaging results, and verifying the consistency of the imaging fitting results based on the sampling results;
[0104] According to the consistency verification result, adaptive enhancement parameter incremental compensation is performed, where the incremental compensation control is as follows:
[0105] S1: Establish an adaptive enhancement model based on big data, extract image features of imaging fitting results, input them as input features into the adaptive enhancement model, and output adaptive enhancement parameters;
[0106] S2: Configure an incremental fine-tuning interval, use the incremental fine-tuning interval as a hidden layer constraint, input the consistency verification result into an incremental training model with hidden layer constraints, and complete the incremental compensation of adaptive enhancement parameters, wherein the incremental training model is obtained by incremental training based on the adaptive enhancement model.
[0107] Furthermore, the system further includes a stereoscopic imaging establishment module to perform the following operation steps:
[0108] Performing same-position alignment on the enhanced images, and performing image feature splicing based on the same-position alignment results;
[0109] Image fusion is completed by stitching the results of image features to establish stereoscopic imaging.
[0110] Furthermore, the system further includes a labeling management module to perform the following operation steps:
[0111] Performing feature matching on the stereoscopic images, and generating additional identification based on the matching similarity and the matching subject, wherein the additional identification includes a warning level identification, and different warning levels have different warning features;
[0112] The additional identification is used to manage the annotation of the stereoscopic imaging.
[0113] Through the aforementioned detailed description of the stereoscopic endoscope imaging optimization method based on objective lens optics in this specification, those skilled in the art can clearly understand the stereoscopic endoscope imaging optimization system based on objective lens optics in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.
[0114] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 3As shown. The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the processor of the computer device is used to provide computing and control capabilities; the memory of the computer device includes a non-volatile storage medium and an internal memory, the non-volatile storage medium stores an operating system, a computer program and a database, and the internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium; the network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a stereoscopic endoscope imaging optimization method based on objective lens optics.
[0115] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0116] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0117] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A stereoscopic endoscope imaging optimization method based on objective lens optics, characterized in that: The method comprises: Reading device information of the stereo endoscope, and performing aberration analysis of the objective lens based on the device information to establish a response parameter set, wherein the response parameter set has a sample distance and a light state identifier; Performing image pre-capture of a target area with the stereo endoscope, and capturing an objective lens image to establish a pre-captured image, wherein the pre-captured image carries a depth mark; Performing image feature recognition on the pre-collected image, distributing attention areas based on the feature recognition results, reconstructing light control parameters based on the depth mark and the regional features of the attention areas, and performing imaging fitting based on the light control parameters; Perform parameter matching of the response parameter set based on the imaging fitting results to generate optimized response parameters, perform imaging control by optimizing the response parameters and light control parameters, and configure image adaptive enhancement parameters based on the imaging fitting results; After enhancing the imaging results based on the adaptive enhancement parameters, image fusion is performed to generate stereoscopic imaging with the fusion results.
2. The method according to claim 1, characterized in that The performing image feature recognition on the pre-collected image and distributing attention areas according to the feature recognition results includes: Establish a feature library of features to be identified; Performing image traversal of the pre-collected image using the feature library, and generating a matching degree mark under two-dimensional image coordinates according to the image traversal result; The distribution of attention areas is completed through matching degree identification results.
3. The method according to claim 2, characterized in that The reconstructing the light control parameters based on the depth mark and the regional features of the attention area, and performing imaging fitting based on the light control parameters, comprises: Acquiring the accuracy requirement of imaging, performing adaptive aggregation of regions based on the accuracy requirement, the depth mark and the attention region, and generating a region adaptive clustering result, wherein the region adaptive clustering result has a mapping focus mark; The depth mark, the regional features of the attention area and the focus mark are input into the light response model to generate light control parameters, wherein the light control parameters include light color, light intensity and lighting direction, and imaging fitting is performed using the light control parameters.
4. The method according to claim 3, characterized in that The step of inputting the depth mark, the regional features of the attention area and the focus mark into the light response model to generate the light control parameters includes: Obtain the objective lens deviation angle of binocular imaging; After the light control parameters of any objective lens are set, the lighting direction compensation of the second objective lens is performed according to the objective lens deviation angle, and the light control parameters of the second objective lens are established based on the lighting direction compensation result.
5. The method according to claim 1, characterized in that After the imaging result is enhanced based on the adaptive enhancement parameter, image fusion is performed, including: Sampling the attention area and non-attention area of the imaging results, and verifying the consistency of the imaging fitting results based on the sampling results; According to the consistency verification result, adaptive enhancement parameter incremental compensation is performed, where the incremental compensation control is as follows: S1: Establish an adaptive enhancement model based on big data, extract image features of imaging fitting results, input them as input features into the adaptive enhancement model, and output adaptive enhancement parameters; S2: Configure an incremental fine-tuning interval, use the incremental fine-tuning interval as a hidden layer constraint, input the consistency verification result into an incremental training model with hidden layer constraints, and complete the incremental compensation of adaptive enhancement parameters, wherein the incremental training model is obtained by incremental training based on the adaptive enhancement model.
6. The method according to claim 5, characterized in that The method further comprises: Performing same-position alignment on the enhanced images, and performing image feature splicing based on the same-position alignment results; Image fusion is completed by stitching the results of image features to establish stereoscopic imaging.
7. The method according to claim 1, characterized in that The method further comprises: Performing feature matching on the stereoscopic images, and generating additional identification based on the matching similarity and the matching subject, wherein the additional identification includes a warning level identification, and different warning levels have different warning features; The additional identification is used to manage the annotation of the stereoscopic imaging.
8. A stereoscopic endoscope imaging optimization system based on objective lens optics, characterized in that: The system is used to implement the stereoscopic endoscope imaging optimization method based on objective lens optics according to any one of claims 1 to 7, the system comprising: An aberration analysis module, the aberration analysis module is used to read the device information of the stereo endoscope, and perform aberration analysis of the objective lens based on the device information to establish a response parameter set, wherein the response parameter set has a sample distance and a light state identifier; An image pre-capture module, the image pre-capture module is used to perform image pre-capture of a target area with the stereo endoscope, and to capture an objective lens image to establish a pre-captured image, wherein the pre-captured image carries a depth mark; An imaging fitting module, the imaging fitting module is used to perform image feature recognition on the pre-collected image, distribute attention areas based on the feature recognition results, reconstruct light control parameters based on the depth mark and the regional features of the attention areas, and perform imaging fitting based on the light control parameters; An imaging control module, the imaging control module is used to perform parameter matching of a response parameter set based on an imaging fitting result, generate optimized response parameters, perform imaging control by optimizing the response parameters and light control parameters, and configure image adaptive enhancement parameters based on the imaging fitting result; The image fusion module is used to enhance the imaging result based on the adaptive enhancement parameters and then perform image fusion to generate stereoscopic imaging with the fusion result.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the objective lens optics-based stereoscopic endoscopic imaging optimization method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the objective lens optics-based stereoscopic endoscopic imaging optimization method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Double-channel endoscope
CN104434006A
Stereovision endoscope
JP1996122666A
Stereoscopic endoscope
JP2002159440A
Three-dimensional shape acquisition device from stereoscopic endoscopic image
JP2013240590A
Convergence optics for stereoscopic imaging systems
US20050154256A1
