Method, device and equipment for adjusting view angle of endoscope and storage medium

Abstract

The invention relates to an endoscope view angle adjusting method and device, equipment and a storage medium. The method comprises the steps that a first image shot by a camera is acquired, and a target angle is acquired; identifying a pattern feature of a preset marker in the first image and a shielding position of the preset marker in the first image; generating an angle recognition pattern according to at least part of pattern features of the preset marker; calculating an inclination angle of the angle identification pattern; calculating the current angle of the preset marker in the visual field of the endoscope according to the inclination angle and the shielding position; judging whether the current angle is consistent with the target angle; and if not, rotating the first image until the current angle of the preset marker in the first image is rotated to the target angle, forming a second image, and displaying the second image on the display screen. The visual field angle, displayed on the display screen, of the endoscope conforms to the operation inertia of a doctor, the doctor has better experience feeling and convenience in the endoscope using process, and the accuracy and safety of an operation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopes, and in particular to an endoscope field of view angle adjustment method, device, equipment and storage medium. Background Art

[0002] Endoscopes are currently widely used in clinical practice. Taking the oral endoscope as an example, the oral endoscope is used as a visual aid in oral medical scenarios, which can display areas in the oral cavity that are invisible to the human eye. Based on the real-time images sent back by the endoscope, the doctor can clearly see the position of the surgical instruments in the oral cavity, improving the accuracy and safety of diagnosis and treatment. The oral endoscope makes diagnosis and treatment simpler, but the angle at which the endoscope is placed in the user's mouth is random, resulting in random imaging angles, which causes the endoscopic imaging seen by the doctor on the display screen to be inconsistent with the doctor's visual intuition or proprioception. For example, when the doctor moves the endoscope upward, the field of view in the image / video sent back on the display screen is not moving upward. This prevents the doctor from quickly obtaining images of the target area, and it takes a certain amount of time to adjust and try to determine the current direction of the endoscope's field of view, which makes it inconvenient for the doctor to operate.

[0003] Therefore, there is an urgent need to propose a solution for adjusting the angle of the endoscope field of view. Summary of the invention

[0004] The main purpose of the present invention is to provide an endoscope field of view angle adjustment method, device, equipment and storage medium to solve the problem of inconvenience in operation when adjusting the field of view angle of the image displayed on the display screen to match the actual feeling in the prior art.

[0005] To achieve the above object, a first aspect of the present invention provides a method for adjusting the field of view angle of an endoscope, wherein the endoscope comprises a camera and a display screen, and the method comprises:

[0006] Acquire a first image captured by the camera, wherein the first image includes a preset marker image and other object images, and acquire a target angle;

[0007] Identifying pattern features of a preset marker in the first image and an occluded position of the preset marker in the first image;

[0008] generating an angle recognition pattern according to at least a portion of the pattern features of the preset marker;

[0009] Calculating the tilt angle of the angle recognition pattern;

[0010] Calculating a current angle of the preset marker in the endoscope field of view according to the tilt angle and the blocking position;

[0011] Determining whether the current angle is consistent with the target angle;

[0012] If they are inconsistent, the first image is rotated until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and the second image is presented on the display screen.

[0013] Furthermore, the step of identifying the pattern features of the preset marker in the first image and the occlusion position of the preset marker in the first image includes:

[0014] Preprocessing the first image, wherein the preprocessing includes at least one of grayscale conversion, denoising, and contrast enhancement, to obtain a preprocessed image;

[0015] Extracting a region of the preset marker from the preprocessed image according to predefined features of the preset marker;

[0016] Using a feature extraction algorithm, extracting the pattern features of the preset marker from the area of ​​the preset marker;

[0017] A predefined positioning area is selected from the area of ​​the preset marker, coordinate information of the positioning area in the first image is determined, and the coordinate information is used as the occlusion position of the preset marker in the first image.

[0018] Furthermore, the preset marker is a fan-shaped structure, the fan-shaped structure includes an outer circular arc, the outer circular arc includes a first circular arc vertex and a second circular arc vertex, and the step of generating an angle recognition pattern according to at least part of the pattern features of the preset marker includes:

[0019] Generate an enclosing rectangular frame outside the preset marker, the midpoint of the outer circular arc is tangent to one side of the enclosing rectangular frame, and use the enclosing rectangular frame as the angle recognition pattern; or,

[0020] The first arc vertex and the second arc vertex are connected to obtain a vertex line, and the vertex line is used as the angle recognition pattern.

[0021] Furthermore, the step of calculating the current angle of the preset marker in the endoscope field of view according to the tilt angle and the shielding position includes:

[0022] Comparing the occlusion position with the center position of the first image to determine a relative positional relationship between the occlusion position and the center position;

[0023] A compensation angle is calculated according to the relative position relationship, and the compensation angle is added to the tilt angle to obtain the current angle.

[0024] Further, the step of calculating a compensation angle according to the relative position relationship and adding the compensation angle to the tilt angle to obtain the current angle includes:

[0025]

[0026] Among them, angle is the tilt angle, r_angle is the current angle, (boxx, boxy) is the occlusion position coordinate, and (centerx, centery) is the center position coordinate.

[0027] Furthermore, before the step of acquiring the first image captured by the camera in real time, the step further includes:

[0028] An endoscope field of view angle adjustment instruction is received, wherein the endoscope field of view angle adjustment instruction is generated instantly or with a delay based on the action of inserting the camera into the endoscope baffle, and the preset marker image is an image formed by the front end of the endoscope baffle in the camera field of view.

[0029] Furthermore, after the step of rotating the first image until the current angle of the preset marker in the first image is rotated to a target angle to form a second image and presenting the second image on the display screen, the step further includes:

[0030] Acquire the video captured by the camera in real time;

[0031] According to the rotation direction of the first image to be rotated into the second image, the image in the video is rotated frame by frame in real time by the difference angle between the target angle and the current angle, and the rotated image is displayed on the display screen in real time.

[0032] A second aspect of the present invention provides an endoscope field of view angle adjustment device, the endoscope comprising a camera and a display screen, the device comprising:

[0033] A first image acquisition module, used to acquire a first image taken by the camera, wherein the first image includes a preset marker image and other object images, and acquire a target angle;

[0034] A marker recognition module, used to recognize pattern features of a preset marker in the first image and an occluded position of the preset marker in the first image;

[0035] An angle recognition pattern generation module, used to generate an angle recognition pattern according to at least part of the pattern features of the preset marker;

[0036] An inclination angle calculation module, used to calculate the inclination angle of the angle recognition pattern;

[0037] A field of view angle calculation module, used to calculate the current angle of the preset marker in the field of view of the endoscope according to the tilt angle and the blocking position;

[0038] A judging module, used to judge whether the current angle is consistent with the target angle;

[0039] The image rotation module is used to rotate the first image if there is no inconsistency, until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and present the second image on the display screen.

[0040] A third aspect of the present invention provides a device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned endoscope field angle adjustment method when executing the computer program.

[0041] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-mentioned endoscope field of view angle adjustment method.

[0042] The method, device, equipment and storage medium for adjusting the field of view of an endoscope provided by the present invention obtain a first image taken by the camera, identify the pattern features of a preset marker in the first image and the occlusion position of the preset marker in the first image; generate an angle recognition pattern according to at least part of the pattern features of the preset marker; calculate the inclination angle of the angle recognition pattern; calculate the current angle of the preset marker in the field of view of the endoscope according to the inclination angle and the occlusion position; determine whether the current angle is consistent with the target angle; if not, rotate the first image until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and present the second image on the display screen; so as to achieve the effect of displaying the rotated second image in real time on the display screen, so that the field of view angle of the endoscope displayed on the display screen conforms to the doctor's operating inertia, so that the doctor has a better experience and convenience in the process of using the endoscope, and significantly improves the accuracy and safety of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of the exploded structure of the baffle and the camera before assembly in one embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of a first image using a baffle as a preset marker in an embodiment of the present invention;

[0045] Figure 3is a schematic diagram of a first image using a scale inside a baffle as a preset marker in an embodiment of the present invention;

[0046] Figure 4 is a schematic flow chart of a method for adjusting the angle of view of an endoscope in one embodiment of the present invention;

[0047] Figure 5 is a schematic diagram of imaging of an angle recognition pattern in one embodiment of the present invention;

[0048] Figure 6 is a schematic diagram of imaging of an angle recognition pattern in another embodiment of the present invention;

[0049] Figure 7 is a schematic diagram of imaging of an angle recognition pattern in another embodiment of the present invention;

[0050] Figure 8 is a structural block diagram of an endoscope field angle adjustment device in one embodiment of the present invention;

[0051] Fig. 9 It is a schematic block diagram of the structure of an endoscope device according to an embodiment of the present invention.

[0052] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] The endoscope of the embodiment of the present invention is a medical device used to examine the internal organs and tissues of the human body. It inserts a tiny endoscopic camera into the body cavity such as the mouth, throat, stomach, intestine, etc., so that the operator (such as a doctor) can observe and photograph the internal conditions. Endoscopic equipment includes, for example, oral scopes, gastroscopes, colonoscopes, laparoscopes, cystoscopes, etc.

[0055] In particular, the embodiments of the present invention are applicable to oral endoscopes. When using an oral endoscope, the camera of the endoscope needs to be placed in a handheld endoscope baffle, which scrapes the gums to allow the camera to capture images of the subgingival area.

[0056] like Figure 1The figure shows a schematic diagram of the decomposed structure of the front end portion 1 including the baffle 100 and the endoscope assembly 2 including the camera 200 before assembly in the dental endoscope. The baffle 100 is a hollow tube with a cutting free tip at one end. When the front end 1 and the endoscope assembly 2 are assembled together, the camera 200 is inserted into the baffle 100, and the front end of the camera 200 is retracted at the top of the cutting free tip, so that at least part of the cutting free tip is within the field of view of the camera and does not occupy the entire field of view of the camera. The camera 200 can be assembled with different baffles 100, but in the process of calling the endoscope field angle adjustment method of the present invention, the position of the camera 200 and the baffle 100 remains relatively fixed. The position fixation of the camera 200 and the baffle 100 can be completed by using any feasible structural components in the prior art. Since the innovation of the present invention is not in this structural part, the present invention will not repeat this specific embodiment here. The cutting free tip is formed by beveling one end of the hollow tube, and the cutting free tip includes a top and a bottom, the top being the tip, and the bottom being the connection with the non-cutting part of the hollow tube. Common cutting free tip shapes can refer to, for example, the tip of a syringe needle. In the use of a dental endoscope device, the camera 200 needs to be inserted into the periodontal pocket, so it is necessary to insert the cutting free tip of the baffle 100 into the periodontal pocket, so that the camera 200 installed inside the baffle 100 is brought to the proximal end of the oral tissue of the operator. In some embodiments, the front end of the camera 200 is located between the top and the bottom of the cutting free tip, and in some embodiments, the front end of the camera 200 is flush with the bottom of the cutting free tip; the embodiment of the present invention further creatively finds that since the camera 200 is set inside the baffle 100 of the above structure, there will be a partial image of the baffle 100 in the image captured by the camera 200. In a specific embodiment, the image of the partial baffle 100 is the above-mentioned preset marker image. As Figure 2 The diagram is a schematic diagram of a first image in an embodiment, wherein the shaded portion is a schematic diagram of a preset marker image, and the blank portion is a schematic diagram of other object images. In another specific embodiment, a pattern is further provided on the baffle, such as a baffle scale 300, which is formed by etching or stamping the outer side of the baffle 100 to form the baffle outer scale 300. Correspondingly, a raised or darkened baffle inner scale 400 is formed at a position corresponding to the baffle outer scale 300 on the inner side of the baffle 100, which is usually a circular sector pattern, such as Figure 3 As shown, the image of the scale 400 in the baffle can be used as the above-mentioned preset marker image, and the other object images are the remaining baffle images and the images shown in the blank part. It can be understood that the pattern set on the baffle can also be other preset images, such as letters, texts, numbers, arbitrary graphics, etc. The present invention does not enumerate them one by one here, but they should all be included in the protection scope of the present invention.

[0057] Reference Figure 4 , a method for adjusting the field of view angle of an endoscope provided in an embodiment of the present invention, the endoscope includes a camera and a display screen, and the method includes:

[0058] S1. Acquire a first image captured by the camera, wherein the first image includes a preset marker image and other object images, and acquire a target angle;

[0059] S2. Identify pattern features of a preset marker in the first image and an occluded position of the preset marker in the first image;

[0060] S3, generating an angle recognition pattern according to at least part of the pattern features of the preset marker;

[0061] S4, calculating the tilt angle of the angle recognition pattern;

[0062] S5. Calculating a current angle of the preset marker in the endoscope field of view according to the tilt angle and the blocking position;

[0063] S6, determining whether the current angle is consistent with the target angle;

[0064] S7. If they are inconsistent, rotate the first image until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and present the second image on the display screen.

[0065] In this embodiment, the camera is used to collect images in real time, and the display screen is used to synchronously display the images collected by the endoscope camera. This embodiment aims to display the second image after rotation on the display screen in real time through the above steps S1-S7, so that the field of view of the endoscope displayed on the display screen conforms to the doctor's operating inertia, so that the doctor has a better experience and convenience in the process of using the endoscope, and significantly improves the accuracy and safety of the operation.

[0066] In the above step S1, in endoscopic imaging, the above first image is usually set to be circular. Since the endoscope is placed in the user's mouth at a random angle, the first image at this time is often inconsistent with the doctor's visual intuition or proprioception. Figure 2 and Figure 3 As shown, the above-mentioned preset marker image is the image of the aforementioned part of the baffle 100, the image of the scale 400 inside the baffle, or other preset images on the baffle.

[0067] The target angle can be given by the doctor when he / she determines which tooth to treat, or can be automatically calculated by the doctor after he / she takes his / her seat by automatically identifying the doctor's current operating position, operating approach and the tooth position to be treated, according to a preset angle setting rule. Since the setting of the target angle is not the core innovation of the present invention, the present invention does not elaborate on the specific implementation method of the target angle setting.

[0068] In the above step S2, for the preset marker, which is an image of a portion of the baffle 100 or an image of the scale 400 inside the baffle, its outline is a fan-shaped structure. Specifically, the fan-shaped structure includes a fan-shaped, an annular fan-shaped (e.g. Figure 3 In some embodiments, a computer image recognition method may be used to recognize the first image according to pattern features (such as shape, gloss, color, etc.) corresponding to a preset marker.

[0069] Since the preset marker forms a partial occlusion on the image collected by the endoscope camera, the above-mentioned occlusion position is the spatial representation of the occlusion area of ​​the preset marker in the first image. For example, for the circular first image, it can be the coordinates of the center of the preset marker image in the coordinate axis with the upper left corner of the rectangle containing the first image as the origin and the horizontal line as the x-axis; it can also be the coordinates of the center point of the arc of the outer circumference of the preset marker in the aforementioned coordinate axis; etc.; other methods that can be used to identify the occlusion position not mentioned in the embodiments of the present invention should be included in the overall concept of the present invention. Exemplarily, the rectangle containing the first image is a rectangular shape corresponding to the camera imaging sensor. When the dental endoscope is used, a circular first image is formed in the rectangular shape. For the convenience of calculation, the upper left corner of the rectangle can be directly used as the origin. It can be understood that other rectangles containing the first image can also be used.

[0070] In a specific embodiment, the step S2 of identifying the pattern features of the preset marker in the first image and the occluded position of the preset marker in the first image includes:

[0071] S201, preprocessing the first image, wherein the preprocessing includes at least one of grayscale conversion, denoising, and contrast enhancement, to obtain a preprocessed image;

[0072] S202, extracting a region of the preset marker from the preprocessed image according to predefined features of the preset marker;

[0073] S203, using a feature extraction algorithm to extract pattern features of the preset marker from the area of ​​the preset marker;

[0074] S204: Select a predefined positioning area from the area of ​​the preset marker, determine coordinate information of the positioning area in the first image, and use the coordinate information as the occlusion position of the preset marker in the first image.

[0075] Specifically, in the above step S201, the grayscale conversion is to convert the first image from a color image to a grayscale image to speed up the image recognition speed. The denoising process is to use a denoising algorithm (such as Gaussian filtering, median filtering, etc.) on the first image to reduce the noise in the image to improve the accuracy of subsequent processing. Contrast enhancement is to improve the image quality through techniques such as histogram equalization or adaptive contrast enhancement to make the characteristics of the preset marker more obvious. In the above step S202, according to the characteristics of the preset marker (such as shape, size, color, etc.), the image is screened to remove areas that are not related to the preset marker. In the above step S203, a feature extraction algorithm is used to extract the pattern features of the preset marker from the screened area. In the above step S204, the above positioning area can be, for example, its center of mass (or center point), so as to determine the position of the preset marker in the first image.

[0076] In the above step S3, the above angle recognition pattern is a graphic with a tilt direction meaning, which may be, for example, a rectangle, a line segment, etc. The angle recognition pattern may be generated by a pre-trained recognition model based on deep learning.

[0077] In a specific embodiment, Figure 2 and Figure 3 As shown, the preset marker is a fan-shaped structure, and the fan-shaped structure includes an outer circular arc. The step S3 of generating an angle recognition pattern according to at least part of the pattern features of the preset marker includes:

[0078] S301, generating an enclosing rectangular frame outside the preset marker, wherein the midpoint of the outer circular arc is tangent to one side of the enclosing rectangular frame, and using the enclosing rectangular frame as the angle recognition pattern.

[0079] Specifically, the formation of the enclosing rectangular box can be based on a deep learning recognition model. By inputting a large amount of endoscopic historical image data and manual marking results into the initial OBB (Oriented Bounding Boxes) model, multiple iterative training is used to obtain an angle recognition pattern generation model suitable for the present invention. The trained model is converted to a c++ version, and the inference frame rate reaches more than 20 frames in a linux+cpu environment to meet the real-time performance of endoscopic image acquisition.

[0080] In a specific embodiment in which the image of a portion of the baffle 100 is used as a preset marker, a surrounding rectangular frame is generated outside the fan-shaped structure corresponding to the image of the baffle 100 by the model (see Figure 5 In the specific embodiment in which the image of the scale 400 in the baffle is used as the preset marker, the fan-shaped structure corresponding to the image of the scale 400 in the baffle is obtained by the model (see Figure 6 Generate an enclosing rectangular frame outside the black bold rectangular frame in the figure.

[0081] In another specific embodiment, Figure 7 As shown, the preset marker is a fan-shaped structure, the fan-shaped structure includes an outer circular arc, the outer circular arc includes a first circular arc vertex and a second circular arc vertex, and the step S3 of generating an angle recognition pattern according to at least part of the pattern features of the preset marker includes:

[0082] S302: Connect the first arc vertex and the second arc vertex to obtain a vertex line, and use the vertex line as the angle recognition pattern.

[0083] Specifically, in a specific embodiment in which the image of a portion of the baffle 100 is used as a preset marker, two vertices of the outer circumference arc of the fan-shaped structure corresponding to the image of the baffle 100 are connected to generate a vertex connection line (see Figure 7 In the specific embodiment in which the image of the scale 400 inside the baffle is used as the preset marker, two vertices of the outer circumference arc of the fan-shaped structure corresponding to the image of the scale 400 inside the baffle are connected to generate a vertex connection line.

[0084] In another specific embodiment, the preset marker is a fan-shaped structure, the fan-shaped structure includes an inner arc, the inner arc is an arc in the fan-shaped structure closer to the center of the first image, the inner arc includes a third arc vertex and a fourth arc vertex, and the step S3 of generating an angle recognition pattern according to at least part of the pattern features of the preset marker includes:

[0085] S303, connecting the third arc vertex and the fourth arc vertex to obtain a vertex line, and using the vertex line as the angle recognition pattern.

[0086] It is understandable that specific methods of generating an angle recognition pattern based on at least part of the pattern features of a preset marker that are not listed in the present invention should be included in the inventive concept of the present invention.

[0087] In the above step S4, the above tilt angle is the angle between the angle recognition pattern and the horizontal line in the first image, and the range of the above tilt angle is 0° to 90°. Specifically, after the deep learning-based recognition model in step S3 generates the angle recognition pattern, it can further calculate its tilt angle and output it.

[0088] For the implementation method in which the angle recognition pattern is an enclosing rectangular frame, the angle between the direction of the side tangent to the midpoint of the outer circular arc and the horizontal line can be calculated as the above-mentioned inclination angle.

[0089] For the implementation method in which the angle recognition pattern is a vertex line, the angle between the vertex line and the horizontal line can be calculated as the above-mentioned inclination angle.

[0090] In the above steps S5 and S6, the current angle of the preset marker in the endoscope field of view in the first image is calculated based on the above tilt angle and the occlusion position. Thus, the current angle can be compared with the target angle. If they are inconsistent, the endoscope field of view angle is adjusted. If they are consistent, there is no need to adjust the field of view angle.

[0091] In a specific embodiment, the step S5 of calculating the current angle of the preset marker in the endoscope field of view according to the tilt angle and the occlusion position includes:

[0092] S501, comparing the occlusion position with the center position of the first image to determine a relative position relationship between the occlusion position and the center position;

[0093] S502: Calculate a compensation angle according to the quadrant position of the relative position relationship, and add the compensation angle to the tilt angle to obtain the current angle.

[0094] In this embodiment, the above-mentioned occlusion position can be, in a custom coordinate axis, the center position of a preset marker image, the center point position of a preset marker outer arc, or the center point position of a preset marker inner arc, etc. For example, for the center position of the preset marker image, the center coordinates (boxx, boxy) of the enclosing rectangular box can be obtained and compared with the center position coordinates (centerx, centery) of the first image to determine their relative position relationship.

[0095] Specifically, in a coordinate axis with the upper left corner of a rectangle including the first image as the origin and the horizontal line as the x-axis as an example, the compensation angle can be calculated based on the comparison of the following positional relationships. Specifically, the step S502 of calculating the compensation angle based on the relative positional relationship and adding the compensation angle to the tilt angle to obtain the current angle adopts the following formula:

[0096]

[0097] Among them, angle is the tilt angle, r_angle is the current angle, (boxx, boxy) is the occlusion position coordinate, and (centerx, centery) is the center position coordinate.

[0098] In the above step S7, the angle of the field of view of the rotated second image is consistent with the doctor's visual intuition or proprioception. The doctor can operate directly according to his or her own visual intuition without having to try to adjust the surgical position many times, which greatly improves the convenience of operation.

[0099] In a specific embodiment, before the step S1 of acquiring the first image captured by the camera in real time, the method further includes:

[0100] S01. Receive an endoscope field of view angle adjustment instruction, wherein the endoscope field of view angle adjustment instruction is generated instantly or with a delay based on the action of inserting the camera into the endoscope baffle, and the preset marker image is an image formed by the front end of the endoscope baffle in the camera field of view.

[0101] In this embodiment, the angle of the camera is random when it is inserted into the endoscope baffle. After the insertion, the relative angle between the camera and the endoscope baffle is fixed. Until the camera is taken out and inserted into the endoscope baffle again, its relative angle will change again. Therefore, when the action of inserting the camera into the endoscope baffle is detected, it is necessary to re-identify and adjust the endoscope field of view angle to trigger the execution of the endoscope field of view angle adjustment method. The above instructions can be generated automatically or manually. The above instructions can be generated instantly; they can also be generated with a delay. For example, after the camera is inserted into the endoscope baffle, when it is detected that the endoscope baffle is placed in the patient's periodontal pocket, specifically, it can be determined whether it has been placed in the patient's periodontal pocket by identifying the image captured by the endoscope camera, because it is convenient to synchronously obtain the target angle at this time, thereby rotating the first image to a suitable second image.

[0102] In a specific embodiment, the step of rotating the first image until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and presenting the second image on the display screen after step S6, further includes:

[0103] S7, acquiring the video captured by the camera in real time;

[0104] S8. According to the rotation direction of the first image to the second image, the image in the video is rotated frame by frame in real time by the difference angle between the target angle and the current angle, and the rotated image is displayed on the display screen in real time.

[0105] In this embodiment, during the doctor's operation, for the continuously captured images, it is only necessary to rotate the images frame by frame at the corresponding angle. For each image, the rotation direction is consistent with the rotation direction of the first angle to the second image, and the rotation angle is consistent with the difference between the target angle and the current angle. The frame processing speed of image rotation can achieve real-time performance, and the rotated image is displayed in real time on the display screen, so that the endoscope field of view angle conforms to the doctor's operating inertia, allowing the doctor to have a better experience and convenience in the process of using the endoscope, thereby significantly improving the accuracy and safety of the operation.

[0106] Reference Figure 8 , is an endoscope field of view angle adjustment device provided by an embodiment of the present invention, the endoscope includes a camera and a display screen, and the device includes:

[0107] A first image acquisition module 10 is used to acquire a first image captured by the camera, wherein the first image includes a preset marker image and other object images, and acquire a target angle;

[0108] A marker recognition module 20, configured to recognize pattern features of a preset marker in the first image and an occluded position of the preset marker in the first image;

[0109] An angle recognition pattern generating module 30, used to generate an angle recognition pattern according to at least part of the pattern features of the preset marker;

[0110] An inclination angle calculation module 40 is used to calculate the inclination angle of the angle recognition pattern;

[0111] A field of view angle calculation module 50, used to calculate the current angle of the preset marker in the field of view of the endoscope according to the tilt angle and the blocking position;

[0112] A judging module 60, configured to judge whether the current angle is consistent with the target angle;

[0113] The image rotation module 70 is used to rotate the first image if there is no inconsistency, until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and present the second image on the display screen.

[0114] In a specific embodiment, the marker recognition module 20 includes:

[0115] A preprocessing unit, configured to preprocess the first image, wherein the preprocessing includes at least one of grayscale conversion, denoising, and contrast enhancement, to obtain a preprocessed image;

[0116] an extraction unit, configured to extract a region of the preset marker from the preprocessed image according to predefined features of the preset marker;

[0117] A feature extraction unit, configured to extract pattern features of the preset marker from the region of the preset marker using a feature extraction algorithm;

[0118] A position determination unit is used to select a predefined positioning area from the area of ​​the preset marker, determine the coordinate information of the positioning area in the first image, and use the coordinate information as the occlusion position of the preset marker in the first image.

[0119] In a specific embodiment, the preset marker is a fan-shaped structure, the fan-shaped structure includes an outer circular arc, the outer circular arc includes a first circular arc vertex and a second circular arc vertex, and the angle recognition pattern generation module 30 includes:

[0120] A first recognition unit is used to generate an enclosing rectangular frame outside the preset marker, wherein the midpoint of the outer circular arc is tangent to one side of the enclosing rectangular frame, and the enclosing rectangular frame is used as the angle recognition pattern; or

[0121] The second recognition unit is used to connect the first arc vertex and the second arc vertex to obtain a vertex line, and use the vertex line as the angle recognition pattern.

[0122] In a specific embodiment, the viewing angle calculation module 50 includes:

[0123] A comparing unit, configured to compare the occlusion position with the center position of the first image to determine a relative positional relationship between the occlusion position and the center position;

[0124] A calculation unit is used to calculate a compensation angle according to the relative position relationship, and add the compensation angle to the tilt angle to obtain the current angle.

[0125] In a specific embodiment, the viewing angle calculation module 50 further includes:

[0126]

[0127] Among them, angle is the tilt angle, r_angle is the current angle, (boxx, boxy) is the occlusion position coordinate, and (centerx, centery) is the center position coordinate.

[0128] In a specific embodiment, the device further comprises:

[0129] A trigger module is used to receive an endoscope field of view angle adjustment instruction, wherein the endoscope field of view angle adjustment instruction is generated instantly or with a delay based on the action of the camera inserting into the endoscope baffle, and the preset marker image is the image formed by the front end of the endoscope baffle in the camera field of view.

[0130] In a specific embodiment, the device further comprises:

[0131] A video acquisition module, used to acquire the video captured by the camera in real time;

[0132] The frame-by-frame rotation module is used to rotate the image in the video frame by the difference angle between the target angle and the current angle in real time according to the rotation direction of the first image to the second image, and display the rotated image on the display screen in real time.

[0133] In the above device embodiment, the specific implementation and technical effects of each module and unit can be found in the above method embodiment, which will not be described in detail here.

[0134] Reference Fig. 9 The present invention also provides an endoscope device in an embodiment, and its internal structure can be as follows Figure 3 As shown. The endoscope device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed for the endoscope device is used to provide computing and control capabilities. The memory of the endoscope device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the endoscope device is used to store the corresponding data in this embodiment. The network interface of the endoscope device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned endoscope field angle adjustment method is implemented.

[0135] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0136] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0137] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0138] The embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0139] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0140] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0141] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for adjusting the angle of view of an endoscope, characterized in that: The endoscope comprises a camera and a display screen, and the method comprises: Acquire a first image captured by the camera, wherein the first image includes a preset marker image and other object images, and acquire a target angle; Identifying pattern features of a preset marker in the first image and an occluded position of the preset marker in the first image; generating an angle recognition pattern according to at least a portion of the pattern features of the preset marker; Calculating the tilt angle of the angle recognition pattern; Calculating a current angle of the preset marker in the endoscope field of view according to the tilt angle and the blocking position; Determining whether the current angle is consistent with the target angle; If they are inconsistent, the first image is rotated until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and the second image is presented on the display screen.

2. The method for adjusting the endoscope field of view angle according to claim 1, characterized in that: The step of identifying the pattern features of the preset marker in the first image and the occluded position of the preset marker in the first image includes: Preprocessing the first image, wherein the preprocessing includes at least one of grayscale conversion, denoising, and contrast enhancement, to obtain a preprocessed image; Extracting a region of the preset marker from the preprocessed image according to predefined features of the preset marker; Using a feature extraction algorithm, extracting the pattern features of the preset marker from the area of ​​the preset marker; A predefined positioning area is selected from the area of ​​the preset marker, coordinate information of the positioning area in the first image is determined, and the coordinate information is used as the occlusion position of the preset marker in the first image.

3. The method for adjusting the endoscope field of view angle according to claim 1, characterized in that: The preset marker is a fan-shaped structure, the fan-shaped structure includes an outer circular arc, the outer circular arc includes a first circular arc vertex and a second circular arc vertex, and the step of generating an angle recognition pattern according to at least part of the pattern features of the preset marker includes: Generate an enclosing rectangular frame outside the preset marker, the midpoint of the outer circular arc is tangent to one side of the enclosing rectangular frame, and use the enclosing rectangular frame as the angle recognition pattern; or, The first arc vertex and the second arc vertex are connected to obtain a vertex line, and the vertex line is used as the angle recognition pattern.

4. The method for adjusting the endoscope field of view angle according to claim 1, characterized in that: The step of calculating the current angle of the preset marker in the endoscope field of view according to the tilt angle and the shielding position comprises: Comparing the occlusion position with the center position of the first image to determine a relative positional relationship between the occlusion position and the center position; A compensation angle is calculated according to the relative position relationship, and the compensation angle is added to the tilt angle to obtain the current angle.

5. The method for adjusting the endoscope field of view angle according to claim 4, characterized in that: The step of calculating the compensation angle according to the relative position relationship and adding the compensation angle to the tilt angle to obtain the current angle comprises: Among them, angle is the tilt angle, r_angle is the current angle, (boxx, boxy) is the occlusion position coordinate, and (centerx, centery) is the center position coordinate.

6. The method for adjusting the endoscope field of view angle according to claim 1, characterized in that: Before the step of acquiring the first image captured by the camera in real time, the method further includes: An endoscope field of view angle adjustment instruction is received, wherein the endoscope field of view angle adjustment instruction is generated instantly or with a delay based on the action of inserting the camera into the endoscope baffle, and the preset marker image is an image formed by the front end of the endoscope baffle in the camera field of view.

7. The method for adjusting the endoscope field of view angle according to claim 1, characterized in that: After the step of rotating the first image until the current angle of the preset marker in the first image is rotated to a target angle to form a second image and presenting the second image on the display screen, the method further includes: Acquire the video captured by the camera in real time; According to the rotation direction of the first image to be rotated into the second image, the image in the video is rotated frame by frame in real time by the difference angle between the target angle and the current angle, and the rotated image is displayed on the display screen in real time.

8. An endoscope field of view angle adjustment device, characterized in that: The endoscope comprises a camera and a display screen, and the device comprises: A first image acquisition module, used to acquire a first image taken by the camera, wherein the first image includes a preset marker image and other object images, and acquire a target angle; A marker recognition module, used to recognize pattern features of a preset marker in the first image and an occluded position of the preset marker in the first image; An angle recognition pattern generation module, used to generate an angle recognition pattern according to at least part of the pattern features of the preset marker; An inclination angle calculation module, used to calculate the inclination angle of the angle recognition pattern; A field of view angle calculation module, used to calculate the current angle of the preset marker in the field of view of the endoscope according to the tilt angle and the blocking position; A judging module, used to judge whether the current angle is consistent with the target angle; The image rotation module is used to rotate the first image if there is no inconsistency, until the current angle of the preset marker in the first image is rotated to the target angle to form a second image, and present the second image on the display screen.

9. A device comprising a memory and a processor, wherein a computer program is stored in the memory, characterized in that: When the processor executes the computer program, the steps of the endoscope field angle adjustment method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the endoscope field angle adjustment method according to any one of claims 1 to 7 are implemented.

Images