Endoscope storage method and device with accurate selection, storage medium and equipment

Abstract

The invention relates to an endoscope storage method and device with accurate selection, a storage medium and equipment. The method comprises the steps that a planned propelling path for propelling an endoscope is obtained; performing path segmentation on the planned propulsion path to obtain multiple sections of planned propulsion sub-paths; determining a positioning coordinate of each planned propulsion sub-path; constructing a positioning coordinate set corresponding to the endoscope image according to the positioning coordinates corresponding to each planning propulsion sub-path; in the advancing process of the endoscope, if the advancing coordinate of the endoscope coincides with any positioning coordinate in the positioning coordinate set, establishing an index relationship between the advancing coordinate and the corresponding endoscope image; and storing the endoscope image collected by the endoscope at the coinciding position of the advancing coordinate and the positioning coordinate and the index relation into an image database. The planning propulsion path is divided into multiple sections of planning propulsion sub-paths, and then image acquisition and storage are performed at the positioning coordinates of the planning propulsion sub-paths, so that the data volume of image storage is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image preservation and display, and in particular, to an endoscope preservation method, device, storage medium and equipment for precision selection. Background Art

[0002] An endoscope can be used to capture images inside the human body, which can not only help doctors determine the lesion area and extent of the patient's body tissue, but also help doctors determine the exact position and posture of surgical instruments in the human body during surgery. However, all images taken by the endoscope are saved, which not only results in a large amount of image data, but also makes it impossible to focus on displaying many images during the presentation stage. In addition, during the advancement of the endoscope, the images captured may have deviations in brightness and color due to uneven exposure time, resulting in certain deviations in the image presentation. Summary of the invention

[0003] In view of the above technical problems, the purpose of the present disclosure is to provide an endoscope preservation method, device, storage medium and equipment for precision selection.

[0004] In order to achieve the above-mentioned object, a first aspect of the embodiment of the present disclosure provides an endoscope preservation method for precision selection, comprising:

[0005] Acquiring a planned advancement path for advancing the endoscope;

[0006] Dividing the planned advancement path into multiple planned advancement sub-paths, wherein each of the planned advancement sub-paths is approximately a straight line path;

[0007] For each of the planned advancement sub-paths, determining the positioning coordinates of the planned advancement sub-path;

[0008] Constructing a set of positioning coordinates corresponding to the endoscopic image according to the positioning coordinates corresponding to each of the planned advancement subpaths;

[0009] During the advancement of the endoscope, if the travel coordinates of the endoscope coincide with any positioning coordinate in the positioning coordinate set, an index relationship between the travel coordinates and the corresponding endoscope image is established;

[0010] The endoscopic image collected by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship are saved in an image database.

[0011] Optionally, the step of saving the endoscopic image acquired by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship into an image database comprises:

[0012] In the case where the travel coordinates coincide with any positioning coordinate in the positioning coordinate set, determining whether the endoscope has captured a plurality of endoscopic images at the position where the coordinates coincide;

[0013] If the endoscope captures a plurality of endoscopic images at the position where the coordinates overlap, determining the endoscopic image to be saved from the plurality of endoscopic images;

[0014] The endoscopic images to be saved in the endoscopic images and the corresponding index relationships are saved in the image database.

[0015] Optionally, if the endoscope captures a plurality of endoscopic images at a position where coordinates overlap, the step of determining an endoscopic image to be saved from the plurality of endoscopic images comprises:

[0016] If the endoscope captures multiple endoscopic images at the coordinate coincident position, each of the endoscopic images is divided into grids to obtain an endoscopic image corresponding to each grid;

[0017] determining a contrast index for each of the grids based on an edge detection operator of the endoscopic image of each grid;

[0018] Determine a quality index corresponding to each grid according to the brightness of the endoscopic image of each grid;

[0019] Determining the comprehensive quality of each of the endoscopic images according to the contrast index and the quality index corresponding to each grid of each of the endoscopic images;

[0020] An endoscopic image to be saved is determined from the plurality of endoscopic images according to the comprehensive quality.

[0021] Optionally, the step of determining the quality index corresponding to each grid according to the brightness of the endoscopic image of each grid comprises:

[0022] According to the brightness of the endoscope image corresponding to each grid, a brightness histogram corresponding to each grid is constructed;

[0023] According to the preset selection parameters, a target bin is determined from the bins of each brightness histogram;

[0024] Determine the number of pixels contained in each bin in the target bin and the number of pixels contained in each bin in the brightness histogram;

[0025] The quality index corresponding to each grid is determined according to the weight value corresponding to each bin in the brightness histogram, the number of pixels contained in each bin in the brightness histogram, and the number of pixels contained in each bin corresponding to each target bin.

[0026] Optionally, the step of determining the contrast index of each grid according to the edge detection operator of the endoscopic image of each grid comprises:

[0027] Performing edge detection on the endoscopic image corresponding to each grid, and calculating the absolute value of the edge detection operator of each pixel coordinate in each of the endoscopic images;

[0028] Sum the absolute values ​​of the edge detection operators of the pixel coordinates corresponding to each of the grids to obtain the edge detection sum value of each of the grids;

[0029] The contrast index corresponding to each grid is determined according to the grid weight corresponding to each grid and the edge detection sum value.

[0030] Optionally, the step of determining the comprehensive quality of each of the endoscopic images according to the contrast index and the quality index corresponding to each grid of each of the endoscopic images comprises:

[0031] Taking the maximum contrast index corresponding to each of the endoscopic images as a target contrast index, and taking the maximum quality index corresponding to each of the endoscopic images as a target quality index;

[0032] The product of the index weight of the contrast index and the contrast index of each grid of the endoscopic image is divided by the target contrast index, and the sum of the quotients is obtained to obtain a contrast index sum value;

[0033] summing the product of the index weight of the quality index and the quality index of each grid of the endoscopic image and dividing the product by the target quality index to obtain a quality index sum value;

[0034] The sum of the contrast index and value and the quality index and value is determined as the comprehensive quality of the endoscopic image.

[0035] Optionally, the step of determining the positioning coordinates of each planned advancement sub-path includes:

[0036] For each of the planned advancement sub-paths, determining the path coordinates of the planned advancement sub-path;

[0037] Determine the path coordinates in the planned advancement sub-path that coincide with the coordinates of the preset positioning point as the positioning coordinates of the planned advancement sub-path;

[0038] The preset positioning point includes a midpoint, any one of the two end points, and any one of the turning points during the propulsion process.

[0039] A second aspect of the disclosed embodiment provides an endoscope storage device for precision selection, comprising:

[0040] a path acquisition module, configured to acquire a planned advancement path for advancing the endoscope;

[0041] A segmentation module is configured to segment the planned advancement path into multiple segments of planned advancement sub-paths, wherein each of the planned advancement sub-paths is approximately a straight line path;

[0042] A determination module is configured to determine the positioning coordinates of each planned advancement sub-path;

[0043] A construction module, configured to construct a set of positioning coordinates corresponding to the endoscopic image according to the positioning coordinates corresponding to each of the planned advancement sub-paths;

[0044] a coordinate acquisition module configured to establish an index relationship between the travel coordinates and the corresponding endoscope image if the travel coordinates of the endoscope coincide with any positioning coordinates in the positioning coordinate set during the advancement of the endoscope;

[0045] The saving module is configured to save the endoscopic image collected by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship into an image database.

[0046] Optionally, the storage module is configured to:

[0047] In the case where the travel coordinates coincide with any positioning coordinate in the positioning coordinate set, determining whether the endoscope has captured a plurality of endoscopic images at the position where the coordinates coincide;

[0048] If the endoscope captures a plurality of endoscopic images at the position where the coordinates overlap, determining the endoscopic image to be saved from the plurality of endoscopic images;

[0049] The endoscopic images to be saved in the endoscopic images and the corresponding index relationships are saved in the image database.

[0050] Optionally, the storage module is configured to:

[0051] If the endoscope captures multiple endoscopic images at the coordinate coincident position, each of the endoscopic images is divided into grids to obtain an endoscopic image corresponding to each grid;

[0052] determining a contrast index for each of the grids based on an edge detection operator of the endoscopic image of each grid;

[0053] Determine a quality index corresponding to each grid according to the brightness of the endoscopic image of each grid;

[0054] Determining the comprehensive quality of each of the endoscopic images according to the contrast index and the quality index corresponding to each grid of each of the endoscopic images;

[0055] An endoscopic image to be saved is determined from the plurality of endoscopic images according to the comprehensive quality.

[0056] Optionally, the storage module is configured to:

[0057] According to the brightness of the endoscope image corresponding to each grid, a brightness histogram corresponding to each grid is constructed;

[0058] According to the preset selection parameters, a target bin is determined from the bins of each brightness histogram;

[0059] Determine the number of pixels contained in each bin in the target bin and the number of pixels contained in each bin in the brightness histogram;

[0060] The quality index corresponding to each grid is determined according to the weight value corresponding to each bin in the brightness histogram, the number of pixels contained in each bin in the brightness histogram, and the number of pixels contained in each bin corresponding to each target bin.

[0061] Optionally, the storage module is configured to:

[0062] Performing edge detection on the endoscopic image corresponding to each grid, and calculating the absolute value of the edge detection operator of each pixel coordinate in each of the endoscopic images;

[0063] Sum the absolute values ​​of the edge detection operators of the pixel coordinates corresponding to each of the grids to obtain the edge detection sum value of each of the grids;

[0064] The contrast index corresponding to each grid is determined according to the grid weight corresponding to each grid and the edge detection sum value.

[0065] Optionally, the storage module is configured to:

[0066] Taking the maximum contrast index corresponding to each of the endoscopic images as a target contrast index, and taking the maximum quality index corresponding to each of the endoscopic images as a target quality index;

[0067] The product of the index weight of the contrast index and the contrast index of each grid of the endoscopic image is divided by the target contrast index, and the sum of the quotients is obtained to obtain a contrast index sum value;

[0068] summing the product of the index weight of the quality index and the quality index of each grid of the endoscopic image and dividing the product by the target quality index to obtain a quality index sum value;

[0069] The sum of the contrast index and value and the quality index and value is determined as the comprehensive quality of the endoscopic image.

[0070] Optionally, the segmentation module is configured as follows:

[0071] For each of the planned advancement sub-paths, determining the path coordinates of the planned advancement sub-path;

[0072] Determine the path coordinates in the planned advancement sub-path that coincide with the coordinates of the preset positioning point as the positioning coordinates of the planned advancement sub-path;

[0073] The preset positioning point includes a midpoint, any one of the two end points, and any one of the turning points during the propulsion process.

[0074] According to a third aspect of the embodiments of the present disclosure, there is provided a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps of the endoscope preservation method for accuracy selection described in any one of the first aspects.

[0075] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, including:

[0076] a first memory having a computer program stored thereon;

[0077] A second processor is used to execute the computer program in the first memory to implement the steps of the endoscope preservation method with precision selection described in any one of the first aspects.

[0078] Through the above technical solution, at least the following beneficial effects can be achieved:

[0079] The planned advancement path of the endoscope is obtained; the planned advancement path is segmented to obtain multiple planned advancement sub-paths, each of which is approximately a straight line path; the positioning coordinates of each planned advancement sub-path are determined; according to the positioning coordinates corresponding to each planned advancement sub-path, a positioning coordinate set corresponding to the endoscope image is constructed; during the advancement of the endoscope, if the travel coordinates of the endoscope coincide with any positioning coordinate in the positioning coordinate set, an index relationship between the travel coordinates and the corresponding endoscope image is established; the endoscope image collected by the endoscope at the travel coordinates and the index relationship are saved in the image database. The planned advancement path is divided into multiple planned advancement sub-paths, and then the image is collected and saved at the positioning coordinates of the planned advancement sub-path, thereby reducing the amount of image data saved. By establishing an index relationship between the travel coordinates and the corresponding endoscope image, the corresponding endoscope image can be quickly found according to the input coordinates in the image presentation stage.

[0080] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0082] Figure 1 The present invention is a flowchart of an endoscope preservation method for precision selection according to an embodiment of the present invention.

[0083] Figure 2 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flowchart of step S16 in FIG.

[0084] Figure 3 This is an implementation shown according to an embodiment of the present disclosure. Figure 2 Flowchart of step S162 in FIG.

[0085] Figure 4 This is an implementation shown according to an embodiment of the present disclosure. Figure 1 Flowchart of step S12 in FIG.

[0086] Figure 5 It is a block diagram of an endoscope storage device showing accuracy selection according to an embodiment of the present disclosure.

[0087] Figure 6 It is a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0088] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0089] Before introducing the precision-selected endoscope preservation method, apparatus, storage medium and device provided by the present invention, the application scenarios of the present invention are first introduced. If all images collected by the endoscope are saved, not only the amount of collection during the advancement process is large, but the amount of data of the saved endoscopic images is also large.

[0090] To this end, the present disclosure provides an endoscope storage method with precision selection, aiming to reduce the amount of image acquisition and the amount of data stored by the endoscope, see Figure 1 As shown, the method includes the following steps.

[0091] In step S11 , a planned advancement path for advancing the endoscope is obtained.

[0092] In the disclosed embodiment, the planned advancement path may be a preset advancement path for the current image acquisition before the endoscope is advanced. For example, the approximate location of the lesion may be inferred based on X-rays or B-ultrasound, and then the planned advancement path for advancing the endoscope may be obtained based on the starting point of the endoscope and the approximate location of the lesion.

[0093] In step S12, the planned advancement path is segmented to obtain multiple planned advancement sub-paths, wherein each planned advancement sub-path is approximately a straight line path.

[0094] In one embodiment of the present disclosure, a curve mathematical modeling can be performed on the planned advancement path. For example, a polynomial function is established for approximation processing to obtain a planned advancement sub-path that is approximately a straight line path. In specific implementation, the planned path is segmented by the stored mathematical model parameters. The space required for storing the parameters is very small, and only all the coordinates on the planned advancement path need to be fitted and calculated.

[0095] In another embodiment of the present disclosure, the planned advancement path is discretized into a curve, and a set of control points is used to replace the planned advancement path, so that the coordinate values ​​of the control points can be directly read to complete the segmentation without complex calculations.

[0096] In another embodiment of the present disclosure, a sampling interval parameter can be selected from the control parameter set to divide the planned propulsion path into straight line segments of equal length. For example, after experiments, the sampling interval parameter can be set to 10 mm, that is, the planned propulsion path is divided into multiple planned propulsion sub-paths of 10 mm in length. The sampling interval parameter of 10 mm can well adapt to the natural curvature of various human tissues.

[0097] It should be noted that there is no strict restriction on the length requirement of each planned advancement sub-path. The above 10mm is only a preferred method, and it is not necessary to divide the lengths equally. The actual length of some planned advancement sub-paths is allowed to be larger or smaller than the average value. If the system default sampling interval parameter is not suitable for some scenarios, the user is allowed to change it to other values.

[0098] In step S13, for each planned advancement sub-path, the positioning coordinates of the planned advancement sub-path are determined.

[0099] In the disclosed embodiment, the positioning coordinates of each planned advancement sub-path can be specified, for example, the midpoint of each planned advancement sub-path can be specified as the positioning coordinates. Of course, other coordinates with obvious features on the planned advancement sub-path can also be used as positioning coordinates, and the effect is exactly the same. For example, the position where the space becomes narrow on the planned advancement sub-path.

[0100] In step S14, a set of positioning coordinates corresponding to the endoscopic image is constructed according to the positioning coordinates corresponding to each planned advancement sub-path.

[0101] It can be understood that the positioning coordinate set includes all positioning coordinate points of the planned advancement path for this endoscopic image acquisition.

[0102] In step S15, during the advancement of the endoscope, if the travel coordinates of the endoscope coincide with any positioning coordinates in the positioning coordinate set, an index relationship between the travel coordinates and the corresponding endoscope image is established.

[0103] Here, an index relationship between the traveling coordinates and the corresponding endoscopic image is established, so that the corresponding endoscopic image can be found through the traveling coordinates.

[0104] In step S16, the endoscopic image collected by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship are saved in the image database.

[0105] In the disclosed embodiment, the index relationship between the traveling coordinates and the endoscopic image, as well as the endoscopic image captured at the position where the traveling coordinates and the positioning coordinates coincide, are saved in the image database. In this way, the endoscopic image can be queried based on the index relationship by inputting any coordinate.

[0106] In one embodiment, if the input coordinates are not coordinates in the positioning coordinate set, the advancement distance between the input coordinates and each positioning coordinate in the positioning coordinate set corresponding to the endoscopic image can be calculated; wherein the advancement distance is the distance from the image coordinate to the positioning coordinate, or the advancement distance is the distance from the positioning coordinate to the image coordinate; the positioning coordinate with the smallest advancement distance in the positioning coordinate set is determined as the target image coordinate, and then according to the index relationship, the endoscopic image corresponding to the target image coordinate is displayed as the target endoscopic image.

[0107] Specifically, in response to the received specified image coordinates of the endoscopic image designated to display a specific location, the advancement distance between the specified image coordinates and each positioning coordinate in the positioning coordinate set corresponding to the endoscopic image can be calculated according to the advancement path of the endoscope when acquiring the endoscopic image.

[0108] The advancing distance is the distance the endoscope is advanced from the designated image coordinates to the positioning coordinates, or the advancing distance is the distance the endoscope is advanced from the positioning coordinates to the designated image coordinates.

[0109] The positioning coordinate with the smallest advancement distance in the positioning coordinate set is determined as the target image coordinate of the specified image coordinate.

[0110] According to the index relationship between the propulsion coordinates of the endoscope and the endoscopic images to be saved, multiple endoscopic images to be saved corresponding to the target image coordinates are queried from the image database, and the image database stores multiple endoscopic images to be saved corresponding to each propulsion coordinate.

[0111] In the present disclosure, when a doctor issues a control instruction through the human-computer interaction interface and requests to replay a historical image of a specific location, the closest positioning coordinates are found from the positioning coordinate set based on the specified image coordinates input by the doctor, and then multiple endoscopic images to be saved that have been saved for the positioning coordinates are found from the image database.

[0112] Each target endoscopic image is subjected to image equalization and restoration processing, and then fused to generate an endoscopic display image.

[0113] Optionally, the steps of performing image equalization and restoration processing on each target endoscopic image and fusing them to generate an endoscopic display image include:

[0114] The target endoscopic image is subjected to brightness and chromaticity equalization processing to obtain a first backup endoscopic image.

[0115] In the disclosed embodiment, brightness and chromaticity balancing processing can reduce the interference to the image processing algorithm caused by the time non-uniformity of the endoscope exposure and the time non-uniformity of the endoscope illumination light source.

[0116] The first backup endoscopic image is subjected to glare repair to obtain a second backup endoscopic image.

[0117] In the disclosed embodiment, glare repair can reduce the interference of local highlights of the image caused by reflections from human tissue on the image processing algorithm.

[0118] The second backup endoscopic image is transformed from the Cartesian coordinate system to the polar coordinate system to obtain a polar coordinate endoscopic image.

[0119] Image feature point detection is performed on the polar coordinate endoscopic image to obtain endoscopic image feature points.

[0120] In the disclosed embodiment, feature points in an image may be detected using algorithms such as SIFT and SURF.

[0121] Based on the RANSAC algorithm, normal feature points and abnormal feature points in the endoscopic image feature points are determined.

[0122] In this way, the feature points can be screened, high-quality feature points can be retained, and unreliable feature points can be eliminated.

[0123] Abnormal feature points are removed from the feature points of the endoscopic image to obtain an endoscopic display image that only retains normal feature points.

[0124] Optionally, the step of removing abnormal feature points from the feature points of the endoscope image to obtain an endoscope display image retaining only normal feature points includes:

[0125] The area where the normal feature points in each endoscopic display image are located is used as the endoscopic image area to be fused of the corresponding endoscopic display image.

[0126] Based on the Gaussian pyramid algorithm, convolution and down-sampling are performed on each endoscopic image region to be fused, and Gaussian pyramid endoscopic images with multiple different resolution scales corresponding to each endoscopic image region to be fused are obtained.

[0127] Based on the Laplacian pyramid algorithm, the underlying image of each Gaussian pyramid endoscopic image is upsampled to generate the corresponding Laplacian endoscopic image.

[0128] The average value of the pixels corresponding to each layer in each Laplacian endoscopic image is calculated.

[0129] An image layer having the same pixel average value is used as a common endoscopic image layer, and each Laplacian endoscopic image is fused to obtain an endoscopic display image.

[0130] The endoscope display image is displayed in the viewport of the human-computer interaction interface.

[0131] Performing image equalization and restoration processing on multiple endoscopic images to be saved, and fusing them to generate an endoscopic display image, which is then displayed in the viewport of the human-computer interaction interface, can improve the accuracy of image display.

[0132] The above technical solution reduces the amount of image data saved by dividing the planned advancement path into multiple planned advancement sub-paths, and then collecting and saving images at the positioning coordinates of the planned advancement sub-paths. By establishing an index relationship between the travel coordinates and the corresponding endoscopic image, the corresponding endoscopic image can be quickly found according to the input coordinates during the image presentation stage.

[0133] In one embodiment, see Figure 2 As shown, in step S16, the step of saving the endoscopic image collected by the endoscope at the moving coordinates and the index relationship into the image database includes the following steps.

[0134] In step S161, when the traveling coordinates coincide with any positioning coordinate in the positioning coordinate set, it is determined whether the endoscope has captured a plurality of endoscopic images at the position where the coordinates coincide.

[0135] In one embodiment of the present disclosure, when the traveling coordinates coincide with any positioning coordinate in the positioning coordinate set, the length of time the endoscope stays at the position where the coordinates coincide is determined. When the length of stay exceeds a preset length threshold, it is determined whether the endoscope captures multiple endoscopic images at the position where the coordinates coincide, wherein the preset length threshold may be determined according to the image capturing frequency of the endoscope.

[0136] In another embodiment of the present disclosure, when the traveling coordinates coincide with any positioning coordinate in the positioning coordinate set, the number of coincidences between the endoscope and the positioning coordinate is determined. When the number of coincidences is greater than a preset threshold of the number of coincidences, it is determined whether the endoscope has captured multiple endoscopic images at the position where the coordinates coincide.

[0137] In step S162, if the endoscope captures a plurality of endoscopic images at the position where the coordinates overlap, an endoscopic image to be saved is determined from the plurality of endoscopic images.

[0138] In the embodiment of the present disclosure, if the endoscope captures multiple endoscopic images at positions where coordinates overlap, the multiple endoscopic images whose acquisition time is closest to the current time are determined as endoscopic images to be saved; alternatively, the shooting angle of the endoscope when each endoscopic image is taken is determined, and the endoscopic images for which there is no overlap in shooting angle are determined as endoscopic images to be saved.

[0139] In one embodiment, since the endoscope captures images at a certain shooting frequency, the endoscope may not be at the shooting moment when the traveling coordinates and the positioning coordinates coincide. An endoscopic image can be captured when the spatial distance between the traveling coordinates and the positioning coordinates is minimum, as the endoscopic image to be saved at the coordinate coincidence position.

[0140] In step S163, the endoscopic images to be saved and the corresponding index relationships in the endoscopic images are saved in the image database.

[0141] In the embodiments of the present disclosure, see Figure 3 As shown, in step S162, if the endoscope captures multiple endoscopic images at the coordinate overlapping position, the step of determining the endoscopic image to be saved from the multiple endoscopic images includes the following steps.

[0142] In step S1621, if the endoscope captures multiple endoscopic images at positions where coordinates overlap, each endoscopic image is divided into grids to obtain an endoscopic image corresponding to each grid.

[0143] For example, each endoscopic image is divided into M×N grids.

[0144] In step S1622, the contrast index of each grid is determined based on the edge detection operator of the endoscopic image of each grid.

[0145] In the embodiments of the present disclosure, the edge detection operator may be obtained by a Sobel detection algorithm, or may be an edge detection operator of other algorithms, and the present disclosure does not impose any particular limitation on this.

[0146] Optionally, in step S1622, the step of determining the contrast index of each grid according to the edge detection operator of the endoscopic image of each grid includes:

[0147] Edge detection is performed on the endoscopic image corresponding to each grid, and the absolute value of the edge detection operator of each pixel coordinate in each endoscopic image is calculated.

[0148] The absolute values ​​of the edge detection operators of the pixel coordinates corresponding to each grid are summed to obtain the edge detection sum value of each grid.

[0149] According to the grid weight corresponding to each grid and the edge detection and value, the contrast index corresponding to each grid is determined.

[0150] In the embodiment of the present disclosure, the contrast index C(i,j) corresponding to each grid is determined by the following calculation formula:

[0151]

[0152] Where (i, j) is the pixel coordinate of the grid in the endoscopic image, w(i, j) is the grid weight corresponding to the (i, j)th grid, where the weights of each grid can be exactly the same, or the network weight of the central area grid can be greater than the grid weights of the edge and corner grids. |S(i, j)| is the absolute value of the edge detection operator of the (i, j)th grid.

[0153] In step S1623, the quality index corresponding to each grid is determined according to the brightness of the endoscopic image of each grid.

[0154] Optionally, in step S1623, the step of determining the quality index corresponding to each grid according to the brightness of the endoscope image of each grid includes:

[0155] According to the brightness of the endoscope image corresponding to each grid, a brightness histogram corresponding to each grid is constructed.

[0156] In one embodiment, according to the brightness of the endoscopic image corresponding to each grid, a 5-bin histogram is used to construct a brightness histogram corresponding to each grid.

[0157] According to the preset selection parameters, the target bin is determined from the bins of each brightness histogram.

[0158] The above embodiment is used for explanation. For example, the number of bins of the brightness histogram is 5, and the preset selection parameter is to use the 3rd, 4th, and 5th bins among the 5 bins as target bins. That is, the preset selection parameter may be that the number of target bins is 3, and the starting bin value of the target bin is 3. Then, the 3rd, 4th, and 5th bins may be selected as target bins from the brightness histogram of the 5 bins corresponding to each grid.

[0159] Determine the number of pixels contained in each bin in the target bin and the number of pixels contained in each bin in the brightness histogram.

[0160] In the disclosed embodiment, the number of pixels contained in each bin is determined by the maximum prime number of the endoscope camera, and the number of pixels contained in each bin is the product of the number of pixels in the horizontal direction and the number of pixels in the vertical direction of the camera in the bin.

[0161] The quality index corresponding to each grid is determined according to the weight value corresponding to each bin in the brightness histogram, the number of pixels contained in each bin in the brightness histogram, and the number of pixels contained in each bin corresponding to each target bin.

[0162] In the embodiment of the present disclosure, the quality index Q corresponding to each grid can be determined by the following calculation formula:

[0163]

[0164] Where m and n are preset selection parameters, Hist(i) is the number of pixels contained in the i-th target bin, Hist(j) is the number of pixels contained in the j-th bin in the brightness histogram, N is the total number of bins in the brightness histogram, and ω(j) is the weight function of the j-th bin in the brightness histogram.

[0165] In step S1624, the comprehensive quality of each endoscopic image is determined based on the contrast index and the quality index corresponding to each grid of each endoscopic image.

[0166] Optionally, in step S1624, the step of determining the comprehensive quality of each endoscopic image according to the contrast index and the quality index corresponding to each grid of each endoscopic image includes:

[0167] The maximum contrast index corresponding to each endoscopic image is used as the target contrast index, and the maximum quality index corresponding to each endoscopic image is used as the target quality index.

[0168] The product of the index weight of the contrast index and the contrast index of each grid of the endoscopic image is divided by the target contrast index and the sum of the quotients is obtained to obtain a contrast index sum value.

[0169] The product of the index weight of the quality index and the quality index of each grid of the endoscopic image is divided by the target quality index and the sum of the quotients is obtained to obtain the quality index sum value.

[0170] The sum of the contrast index and the quality index is determined as the comprehensive quality of the endoscopic image.

[0171] In the embodiment of the present disclosure, the total quality Z of the endoscopic image can be determined by the following calculation formula:

[0172]

[0173] Where i is the i-th grid in the endoscopic image, i = 1, 2, ..., M × N; C max is the target quality index of the endoscopic image, i.e., the maximum contrast index; Q max The target quality index of the endoscopic image is the quality index with the maximum value. s is the adjustment coefficient used to adjust the weights of the quality index and the contrast index.

[0174] In step S1625, an endoscopic image to be saved is determined from the plurality of endoscopic images based on the comprehensive quality.

[0175] In the disclosed embodiment, the number of endoscopic images to be saved at each positioning coordinate can be specified in the control parameter set. For example, the number of endoscopic images to be saved at each positioning coordinate is specified to be 5, and the endoscopic image to be saved at the positioning coordinate will be determined from the 5 endoscopic images with the largest comprehensive quality values.

[0176] Alternatively, see Figure 4 As shown, in step S12, for each planned advancement sub-path, the step of determining the positioning coordinates of the planned advancement sub-path includes the following steps.

[0177] In step S121 , for each planned advancement sub-path, the path coordinates of the planned advancement sub-path are determined.

[0178] The path coordinates referred to in this step are the coordinates of all points on the planned advancement subpath, that is, the coordinates of all points on the planned advancement subpath during the advancement of the endoscope.

[0179] In step S122, the path coordinates in the planned advancement sub-path that coincide with the coordinates of the preset positioning point are determined as the positioning coordinates of the planned advancement sub-path.

[0180] The preset positioning point includes a midpoint, any one of the two end points, and any one of the turning points during the advancement process.

[0181] In the disclosed embodiment, a planned advancement sub-path has only one positioning coordinate, so the preset positioning point includes the midpoint, any one of the two endpoints, and any one of the turning points during advancement. For example, when the preset positioning point is designated as the midpoint, the coordinates in the path coordinates of the planned advancement sub-path that coincide with the midpoint are used as the positioning coordinates. For another example, when the preset positioning point is designated as the endpoint to be reached first by advancement, the coordinates in the path coordinates of the planned advancement sub-path that coincide with the endpoint are used as the positioning coordinates.

[0182] Based on the same concept, the present disclosure also provides an endoscope storage device with precision selection, see Figure 5 As shown, the endoscope storage device 500 for accuracy selection includes: a path acquisition module 510, a segmentation module 520, a determination module 530, a construction module 540, a coordinate acquisition module 550 and a storage module 560.

[0183] Wherein, the path acquisition module 510 is configured to acquire a planned advancement path for advancing the endoscope;

[0184] The segmentation module 520 is configured to segment the planned advancement path into multiple planned advancement sub-paths, wherein each of the planned advancement sub-paths is approximately a straight line path;

[0185] The determination module 530 is configured to determine the positioning coordinates of each of the planned advancement sub-paths;

[0186] A construction module 540 is configured to construct a set of positioning coordinates corresponding to the endoscopic image according to the positioning coordinates corresponding to each of the planned advancement sub-paths;

[0187] The coordinate acquisition module 550 is configured to establish an index relationship between the travel coordinate and the corresponding endoscope image if the travel coordinate of the endoscope coincides with any positioning coordinate in the positioning coordinate set during the advancement of the endoscope;

[0188] The saving module 560 is configured to save the endoscopic image collected by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship into an image database.

[0189] Optionally, the saving module 560 is configured to:

[0190] In the case where the travel coordinates coincide with any positioning coordinate in the positioning coordinate set, determining whether the endoscope has captured a plurality of endoscopic images at the position where the coordinates coincide;

[0191] If the endoscope captures a plurality of endoscopic images at the position where the coordinates overlap, determining the endoscopic image to be saved from the plurality of endoscopic images;

[0192] The endoscopic images to be saved in the endoscopic images and the corresponding index relationships are saved in the image database.

[0193] Optionally, the saving module 560 is configured to:

[0194] If the endoscope captures multiple endoscopic images at the coordinate coincident position, each of the endoscopic images is divided into grids to obtain an endoscopic image corresponding to each grid;

[0195] determining a contrast index for each of the grids based on an edge detection operator of the endoscopic image of each grid;

[0196] Determine a quality index corresponding to each grid according to the brightness of the endoscopic image of each grid;

[0197] Determining the comprehensive quality of each of the endoscopic images according to the contrast index and the quality index corresponding to each grid of each of the endoscopic images;

[0198] An endoscopic image to be saved is determined from the plurality of endoscopic images according to the comprehensive quality.

[0199] Optionally, the saving module 560 is configured to:

[0200] According to the brightness of the endoscope image corresponding to each grid, a brightness histogram corresponding to each grid is constructed;

[0201] According to the preset selection parameters, a target bin is determined from the bins of each brightness histogram;

[0202] Determine the number of pixels contained in each bin in the target bin and the number of pixels contained in each bin in the brightness histogram;

[0203] The quality index corresponding to each grid is determined according to the weight value corresponding to each bin in the brightness histogram, the number of pixels contained in each bin in the brightness histogram, and the number of pixels contained in each bin corresponding to each target bin.

[0204] Optionally, the saving module 560 is configured to:

[0205] Performing edge detection on the endoscopic image corresponding to each grid, and calculating the absolute value of the edge detection operator of each pixel coordinate in each of the endoscopic images;

[0206] Sum the absolute values ​​of the edge detection operators of the pixel coordinates corresponding to each of the grids to obtain the edge detection sum value of each of the grids;

[0207] The contrast index corresponding to each grid is determined according to the grid weight corresponding to each grid and the edge detection sum value.

[0208] Optionally, the saving module 560 is configured to:

[0209] Taking the maximum contrast index corresponding to each of the endoscopic images as a target contrast index, and taking the maximum quality index corresponding to each of the endoscopic images as a target quality index;

[0210] The product of the index weight of the contrast index and the contrast index of each grid of the endoscopic image is divided by the target contrast index, and the sum of the quotients is obtained to obtain a contrast index sum value;

[0211] summing the product of the index weight of the quality index and the quality index of each grid of the endoscopic image and dividing the product by the target quality index to obtain a quality index sum value;

[0212] The sum of the contrast index and value and the quality index and value is determined as the comprehensive quality of the endoscopic image.

[0213] Optionally, the segmentation module 520 is configured to:

[0214] For each of the planned advancement sub-paths, determining the path coordinates of the planned advancement sub-path;

[0215] Determine the path coordinates in the planned advancement sub-path that coincide with the coordinates of the preset positioning point as the positioning coordinates of the planned advancement sub-path;

[0216] The preset positioning point includes a midpoint, any one of the two end points, and any one of the turning points during the propulsion process.

[0217] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0218] Those skilled in the art should understand that the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple modules can be combined or integrated into one module. In addition, the modules described as separate components may or may not be physically separated. Moreover, each module may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. When implemented using hardware, it may be implemented in whole or in part in the form of an integrated circuit or chip.

[0219] The embodiment of the present disclosure also provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a first processor, implements the steps of the endoscope preservation method for accuracy selection described in any one of the first aspects.

[0220] The present disclosure also provides an electronic device, including:

[0221] a first memory having a computer program stored thereon;

[0222] A second processor is used to execute the computer program in the first memory to implement the steps of the endoscope preservation method with precision selection described in any one of the first aspects.

[0223] Figure 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. The electronic device 600 can be configured as an endoscope storage device for precision selection, such as Figure 6 As shown, the electronic device 600 may include: a third processor 601 , a second memory 602 . The electronic device 600 may also include one or more of a multimedia component 603 , an input / output (I / O) interface 604 , and a communication component 605 .

[0224] The third processor 601 is used to control the overall operation of the electronic device 600 to complete all or part of the steps in the above-mentioned precision selected endoscope preservation method. The second memory 602 is used to store various types of data to support the operation of the electronic device 600, and these data may include, for example, instructions for any application or method used to operate on the electronic device 600, and application-related data, such as pictures, etc. The second memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the second memory 602 or sent through the communication component 605. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 604 provides an interface between the third processor 601 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons for operating endoscopic image preservation. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 605 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0225] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned precision selection endoscope preservation method.

[0226] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the endoscope preservation method of the above-mentioned accuracy selection are implemented. For example, the computer-readable storage medium can be the second memory 602 including the program instructions, and the program instructions can be executed by the third processor 601 of the electronic device 600 to complete the endoscope preservation method of the above-mentioned accuracy selection.

[0227] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0228] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0229] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for storing an endoscope with precision selection, characterized in that: include: Acquiring a planned advancement path for advancing the endoscope; Dividing the planned advancement path into multiple planned advancement sub-paths, wherein each of the planned advancement sub-paths is approximately a straight line path; For each of the planned advancement sub-paths, determining the positioning coordinates of the planned advancement sub-path; Constructing a set of positioning coordinates corresponding to the endoscopic image according to the positioning coordinates corresponding to each of the planned advancement subpaths; During the advancement of the endoscope, if the travel coordinates of the endoscope coincide with any positioning coordinate in the positioning coordinate set, an index relationship between the travel coordinates and the corresponding endoscope image is established; The endoscopic image collected by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship are saved in an image database.

2. The method for storing endoscopes by precision selection according to claim 1, characterized in that: The step of saving the endoscopic image collected by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship into an image database comprises: In the case where the travel coordinates coincide with any positioning coordinate in the positioning coordinate set, determining whether the endoscope has captured a plurality of endoscopic images at the position where the coordinates coincide; If the endoscope captures a plurality of endoscopic images at the position where the coordinates overlap, determining the endoscopic image to be saved from the plurality of endoscopic images; The endoscopic images to be saved in the endoscopic images and the corresponding index relationships are saved in the image database.

3. The method for storing endoscopes by precision selection according to claim 2, characterized in that: If the endoscope captures a plurality of endoscopic images at the position where the coordinates overlap, the step of determining the endoscopic image to be saved from the plurality of endoscopic images comprises: If the endoscope captures multiple endoscopic images at the coordinate coincident position, each of the endoscopic images is divided into grids to obtain an endoscopic image corresponding to each grid; determining a contrast index for each of the grids based on an edge detection operator of the endoscopic image of each grid; Determine a quality index corresponding to each grid according to the brightness of the endoscopic image of each grid; Determining the comprehensive quality of each of the endoscopic images according to the contrast index and the quality index corresponding to each grid of each of the endoscopic images; An endoscopic image to be saved is determined from the plurality of endoscopic images according to the comprehensive quality.

4. The method for storing endoscopes by precision selection according to claim 3, characterized in that: The step of determining the quality index corresponding to each grid according to the brightness of the endoscopic image of each grid comprises: According to the brightness of the endoscope image corresponding to each grid, a brightness histogram corresponding to each grid is constructed; According to the preset selection parameters, a target bin is determined from the bins of each brightness histogram; Determine the number of pixels contained in each bin in the target bin and the number of pixels contained in each bin in the brightness histogram; The quality index corresponding to each grid is determined according to the weight value corresponding to each bin in the brightness histogram, the number of pixels contained in each bin in the brightness histogram, and the number of pixels contained in each bin corresponding to each target bin.

5. The method for storing endoscopes by precision selection according to claim 3, characterized in that: The step of determining the contrast index of each grid according to the edge detection operator of the endoscopic image of each grid comprises: Performing edge detection on the endoscopic image corresponding to each grid, and calculating the absolute value of the edge detection operator of each pixel coordinate in each of the endoscopic images; Sum the absolute values ​​of the edge detection operators of the pixel coordinates corresponding to each of the grids to obtain the edge detection sum value of each of the grids; The contrast index corresponding to each grid is determined according to the grid weight corresponding to each grid and the edge detection sum value.

6. The method for storing endoscopes by precision selection according to claim 3, characterized in that: The step of determining the comprehensive quality of each of the endoscopic images according to the contrast index and the quality index corresponding to each grid of each of the endoscopic images comprises: Taking the maximum contrast index corresponding to each of the endoscopic images as a target contrast index, and taking the maximum quality index corresponding to each of the endoscopic images as a target quality index; The product of the index weight of the contrast index and the contrast index of each grid of the endoscopic image is divided by the target contrast index, and the sum of the quotients is obtained to obtain a contrast index sum value; summing the product of the index weight of the quality index and the quality index of each grid of the endoscopic image and dividing the product by the target quality index to obtain a quality index sum value; The sum of the contrast index and value and the quality index and value is determined as the comprehensive quality of the endoscopic image.

7. The method for storing an endoscope according to any one of claims 1 to 6, characterized in that: The step of determining the positioning coordinates of each planned advancement sub-path includes: For each of the planned advancement sub-paths, determining the path coordinates of the planned advancement sub-path; Determine the path coordinates in the planned advancement sub-path that coincide with the coordinates of the preset positioning point as the positioning coordinates of the planned advancement sub-path; The preset positioning point includes a midpoint, any one of the two end points, and any one of the turning points during the propulsion process.

8. An endoscope storage device for precision selection, characterized in that: include: a path acquisition module, configured to acquire a planned advancement path for advancing the endoscope; A segmentation module is configured to segment the planned advancement path into multiple segments of planned advancement sub-paths, wherein each of the planned advancement sub-paths is approximately a straight line path; A determination module is configured to determine the positioning coordinates of each planned advancement sub-path; A construction module, configured to construct a set of positioning coordinates corresponding to the endoscopic image according to the positioning coordinates corresponding to each of the planned advancement sub-paths; a coordinate acquisition module configured to establish an index relationship between the travel coordinates and the corresponding endoscope image if the travel coordinates of the endoscope coincide with any positioning coordinates in the positioning coordinate set during the advancement of the endoscope; The saving module is configured to save the endoscopic image collected by the endoscope at the position where the traveling coordinates and the positioning coordinates coincide with each other and the index relationship into an image database.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the first processor, the steps of the endoscope preservation method for accuracy selection described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a first memory having a computer program stored thereon; A second processor is used to execute the computer program in the first memory to implement the steps of the precision selected endoscope preservation method according to any one of claims 1-7.

Images