Image processing system, image processing device and image processing method

Abstract

An endoscope system includes circuitry configured to set a plurality of evaluation regions in an endoscopic image captured by an image sensor via a scope, wherein adjacent regions of the plurality of evaluation regions are spatially separated from each other, to calculate an evaluation value for each of the plurality of evaluation regions, to compare the evaluation values ​​of the plurality of evaluation regions, and to adjust image processing of the endoscopic image based on the comparison results.

Description

Technical Field

[0001] This disclosure relates to image processing systems, image processing apparatuses, and image processing methods, and more specifically, to image processing systems, image processing apparatuses, and image processing methods capable of determining the type of a lens with minimal computational effort.

[0002] <Cross-reference to related applications>

[0003] This application claims the benefit of Japanese priority patent application JP2019-065381, filed on March 29, 2019, the entire contents of which are incorporated herein by reference. Background Technology

[0004] Typically, surgical endoscopes use a camera head attached to the endoscope body, which is then inserted into the patient's body to observe the surgical area.

[0005] The scope body is detachable, and a variety of scope body types can be selected for use. Mechanical vignetting and other imaging characteristics vary depending on the scope body type. Since subsequent image processing needs to be adjusted according to the scope body type, a method for determining the scope body type is required.

[0006] For example, Patent Document 1 discloses a method for determining the type of mirror body.

[0007] [List of Citations]

[0008] [Patent Literature]

[0009] Patent Document 1: JP 2004-33487A Summary of the Invention

[0010] [Technical Issues]

[0011] However, as disclosed in Patent Document 1, determining the lens type requires detecting all straight edges in the image, which uses a large amount of computation. Therefore, there is a need for a method to determine the lens type with less computation.

[0012] This disclosure was made in view of this situation and enables the determination of the lens type with a small amount of computation.

[0013] [Solution to the problem]

[0014] One aspect of this disclosure is an image processing system comprising a control unit that sets up a plurality of evaluation frames arranged at predetermined intervals for calculating evaluation values ​​with respect to the respective plurality of evaluation frames using endoscopic images captured by the endoscope, and estimating the type of the endoscope based on the relationships between the calculated evaluation values.

[0015] One aspect of the image processing apparatus disclosed herein includes a control unit that sets up a plurality of evaluation frames arranged at predetermined intervals for calculating evaluation values ​​with respect to the respective plurality of evaluation frames using endoscopic images captured by an endoscope, and performing signal processing corresponding to the type of endoscope based on the relationships between the calculated evaluation values.

[0016] One aspect of this disclosure is an image processing method comprising: setting up a plurality of evaluation frames arranged at predetermined intervals using an image processing apparatus, calculating evaluation values ​​for the respective plurality of evaluation frames using endoscopic images captured by an endoscope, and performing signal processing corresponding to the type of the endoscope based on the relationship between the calculated evaluation values.

[0017] In one aspect of the image processing system, image processing apparatus, and image processing method of this disclosure, a plurality of evaluation frames arranged at predetermined intervals are set up for calculating evaluation values ​​with respect to the respective plurality of evaluation frames using endoscopic images captured by the endoscope, and estimating the type of the endoscope based on the relationship between the calculated evaluation values.

[0018] Note that the image processing apparatus of one aspect of this disclosure may be a standalone device or an internal block constituting a device. Attached Figure Description

[0019] [ Figure 1 ] Figure 1 This is a diagram illustrating an example of a schematic configuration of an image processing system that applies the technology according to this disclosure.

[0020] [ Figure 2 ] Figure 2 This is a diagram illustrating an example of an endoscope configuration.

[0021] [ Figure 3 ] Figure 3 This is a block diagram illustrating an example of the functional configuration of the camera head and CCU.

[0022] [ Figure 4 ] Figure 4 This is a diagram showing a first example of an endoscopic image.

[0023] [ Figure 5 ] Figure 5 This is a diagram showing a second example of an endoscopic image.

[0024] [ Figure 6 ] Figure 6 This is a flowchart illustrating the process of the first step.

[0025] [ Figure 7 ] Figure 7This is a diagram illustrating an example of multiple evaluation frames set up in the first process.

[0026] [ Figure 8 ] Figure 8 This is a flowchart illustrating the process of the second processing step.

[0027] [ Figure 9 ] Figure 9 This is a diagram illustrating an example of multiple evaluation frames set up in the second process.

[0028] [ Figure 10 ] Figure 10 This is a flowchart illustrating the process of the third processing step.

[0029] [ Figure 11 ] Figure 11 This is a diagram illustrating an example of multiple evaluation frames set up in the third process.

[0030] [ Figure 12 ] Figure 12 This is a flowchart illustrating the process of the fourth step.

[0031] [ Figure 13 ] Figure 13 This is a flowchart illustrating the process of the fourth step.

[0032] [ Figure 14 ] Figure 14 This is a diagram illustrating an example of multiple evaluation frames set up in the fourth process.

[0033] [ Figure 15 ] Figure 15 This is a flowchart illustrating the process of the fifth step.

[0034] [ Figure 16 ] Figure 16 This is a flowchart illustrating the process of the fifth step.

[0035] [ Figure 17 ] Figure 17 This is a diagram showing an example of multiple evaluation frames in the fifth process.

[0036] [ Figure 18 ] Figure 18 It is a flowchart illustrating the process of determining the procedure.

[0037] [ Figure 19 ] Figure 19 This is a diagram illustrating an example of the coordinate system for the evaluation frame.

[0038] [ Figure 20 ] Figure 20 This is a diagram illustrating an example of a computer configuration. Detailed Implementation

[0039] Hereinafter, embodiments of the technology according to this disclosure (the technology) will be described with reference to the accompanying drawings. Please note that the description will proceed in the following order.

[0040] 1. Embodiments of this technology

[0041] 2. Modification

[0042] 3. Computer configuration

[0043] <1. Embodiments of this technology>

[0044] (System Overview)

[0045] First, an overview of systems to which the technology according to this disclosure can be applied will be described. Figure 1 An example of a schematic configuration of an image processing system applying the technology according to this disclosure is shown.

[0046] Figure 1 This illustrates the state of an operator (surgeon) 3 performing surgery on a patient 4 on a hospital bed 2 using an endoscopic surgical system 1. Figure 1 In this endoscopic surgical system 1, there are: an endoscope 10; other surgical instruments 20, such as a pneumoperitoneum tube 21, an energy processing instrument 22 and forceps 23; a support arm device 30 for supporting the endoscope 10; and a trolley 50 equipped with various devices for endoscopic surgery.

[0047] Endoscope 10 includes a body 101 inserted into the body cavity of patient 4 from a predetermined distance distal to the end, and a camera head 102 connected to the proximal end of the body 101. Note that... Figure 1 An endoscope 10 is shown that is configured to include a so-called rigid endoscope body 101, but the endoscope 10 can be configured to include a so-called flexible endoscope body 101.

[0048] An opening is provided at the distal end of the endoscope body 101, into which the objective lens is fitted. A light source device 53 is connected to the endoscope 10, and the light (illumination light) generated by the light source device 53 is guided to the distal end of the endoscope tube through a light guide extending inside the endoscope body 101, and the light is emitted through the objective lens toward the target of observation in the body cavity of the patient 4. Note that the endoscope 10 may be a forward-looking endoscope, a slant-looking endoscope, or a lateral-looking endoscope.

[0049] An optical system and imaging element are housed inside the camera head 102. Reflected light from the observed target (observation light) is focused onto the imaging element by the optical system. The imaging element performs photoelectric conversion on the observation light to generate an image signal corresponding to the subject image. The image signal is sent as RAW data (RAW image) to the camera control unit (CCU) 51.

[0050] The CCU 51 includes processors, such as a central processing unit (CPU) and a graphics processing unit (GPU), and provides overall control over the operation of the endoscope 10 and the display device 52. Furthermore, the CCU 51 receives image signals from the camera head 102 and performs various types of image processing on the image signals, such as image processing (de-mosaic processing), to display the observed image (display image) based on the image signals.

[0051] The display device 52 displays an image based on an image signal that has undergone image processing by the CCU 51, according to the control from the CCU 51.

[0052] The light source device 53 includes a light source, such as a light-emitting diode (LED), and provides illumination light to the endoscope 10 for imaging surgical parts, etc.

[0053] Input device 54 is the input interface of endoscopic surgery system 1. Users can input various types of information and commands into endoscopic surgery system 1 via input device 54. For example, users can input commands to change the imaging conditions of endoscope 10 (type of illumination light, magnification, focal length, etc.).

[0054] The processing tool control device 55 controls the drive of the energy processing tool 22 for tissue cauterization, incision, and vascular closure. The pneumoperitoneum device 56 injects gas into the body cavity via the pneumoperitoneum tube 21 to inflate the patient's body cavity, ensuring the field of vision of the endoscope 10 and the operator's working space.

[0055] Recorder 57 is a device capable of recording various types of information about the surgery. Printer 58 is a device capable of printing various types of information about the surgery in various formats such as text, images, and graphics.

[0056] (Detailed configuration of the endoscope)

[0057] Figure 2 It shows Figure 1 An example of the detailed configuration of the endoscope 10.

[0058] exist Figure 2 In this endoscope 10, the endoscope body 101 and the camera head 102 are connected. Furthermore, in the endoscope 10, the endoscope body 101 is connected to the light source device 53 via a light guide 121, and the camera head 102 is connected to the CCU 51 via a transmission cable 122. Additionally, the CCU 51 is connected to the display device 52 via a transmission cable 123, and to the light source device 53 via a transmission cable 124.

[0059] The endoscope 101 is configured as a rigid endoscope. In other words, the endoscope 101 is rigid or at least partially flexible and has an elongated insertion portion (endoscope tube) that is inserted into the body cavity of the patient 4. The endoscope 101 provides an optical system for configuring and focusing an image of the subject using one or more lenses.

[0060] The light source device 53 is connected to one end of the light guide 121 and provides illumination light to one end of the light guide 121 to illuminate the interior of the body cavity according to the control of the CCU 51. One end of the light guide 121 is detachably connected to the light source device 53, and the other end is detachably connected to the mirror body 101.

[0061] Then, the light guide 121 transmits the illumination light provided by the light source device 53 from one end to the other end and provides the light to the mirror body 101. The illumination light provided to the mirror body 101 is emitted from the distal end of the mirror body 101 and emitted into the body cavity. The observation light (the image of the subject) emitted into the body cavity and reflected in the body cavity is focused by the optical system in the mirror body 101.

[0062] Camera head 102 is detachably connected to the proximal end (eyepiece 111) of lens body 101. Camera head 102 then captures the observation light (subject image) focused by lens body 101 under the control of CCU 51 and outputs the resulting image signal (RAW data). The image signal is, for example, an image signal corresponding to 4K resolution (e.g., 3840 × 2160 pixels). Note that reference will be made later. Figure 3 Describe the detailed configuration of camera head 102.

[0063] One end of the transmission cable 122 is detachably connected to the camera head 102 via connector 131, and the other end is detachably connected to the CCU 51 via connector 132. Then, the transmission cable 122 sends the image signal, etc., output from the camera head 102 to the CCU 51, and sends each of the control signal, synchronization signal, power, etc. output from the CCU 51 to the camera head 102.

[0064] Note that when transmitting image signals, etc., from camera head 102 to CCU 51 via transmission cable 122, the image signals, etc., can be transmitted as optical signals or as electrical signals. This also applies to control signals, synchronization signals, and clocks transmitted from CCU 51 to camera head 102 via transmission cable 122. Furthermore, communication between camera head 102 and CCU 51 is not limited to wired communication using transmission cable 122, and wireless communication can be performed according to a predetermined communication scheme.

[0065] The display device 52 displays images based on image signals from the CCU 51 under control from the CCU 51, and outputs sound based on control signals from the CCU 51.

[0066] One end of the transmission cable 123 is detachably connected to the display device 52, and the other end is detachably connected to the CCU 51. The transmission cable 123 then sends the image signal processed by the CCU 51 and the control signal output from the CCU 51 to the display device 52.

[0067] The CCU 51 includes a CPU and other components, and comprehensively controls the operation of the light source device 53, the camera head 102, and the display device 52. Note that detailed configuration of the CCU 51 will be provided later. Figure 3 describe.

[0068] One end of the transmission cable 124 is detachably connected to the light source device 53, and the other end is detachably connected to the CCU 51. Then, the transmission cable 124 transmits control signals from the CCU 51 to the light source device 53.

[0069] (Detailed configuration of camera head and CCU)

[0070] Figure 3 It is shown Figure 1 and Figure 2 A block diagram illustrating an example of the functional configuration of the camera head 102 and CCU 51.

[0071] The camera head 102 includes a lens unit 151, an imaging unit 152, a drive unit 153, a communication unit 154, and a camera head control unit 155. The camera head 102 and the CCU 51 are communicatively connected to each other via a transmission cable 122.

[0072] Lens unit 151 is an optical system disposed at the connection portion with lens body 101. Observation light collected from the distal end of lens body 101 is guided to camera head 102 and incident on lens unit 151. Lens unit 151 includes multiple lenses, such as zoom lenses, focusing lenses, etc.

[0073] Imaging unit 152 includes imaging elements, such as complementary metal-oxide-semiconductor (CMOS) image sensors, charge-coupled device (CCD) image sensors, etc. The imaging elements constituting imaging unit 152 can be a single element (so-called single-plate type) or multiple elements (so-called multi-plate type). For example, in the case where imaging unit 152 includes a multi-plate type, image signals corresponding to R, G, and B can be generated by the respective imaging elements, and a color image can be obtained by synthesizing the image signals.

[0074] Alternatively, imaging unit 152 may include a pair of imaging elements for acquiring right-eye and left-eye image signals respectively corresponding to a three-dimensional (3D) display. Performing a 3D display allows operator 3 to more accurately determine the depth of living tissue in the surgical area. Note that in the case where imaging unit 152 is of a multi-plate type, multiple systems of lens units 151 may be provided corresponding to the respective imaging elements.

[0075] Furthermore, the imaging unit 152 does not necessarily have to be located in the camera head 102. For example, the imaging unit 152 can be located immediately after the objective lens, inside the lens body 101.

[0076] The drive unit 153 includes actuators and the like, and, under control from the camera head control unit 155, moves one or more of the plurality of lenses included in the lens unit 151 a predetermined distance along the optical axis. As a result, the magnification and focus of the image captured by the imaging unit 152 can be appropriately adjusted.

[0077] The communication unit 154 includes communication means for sending / receiving various types of information to / from the CCU 51. The communication unit 154 transmits the image signal obtained from the imaging unit 152 as RAW data to the CCU 51 via the transmission cable 122.

[0078] Furthermore, the communication unit 154 receives control signals from the CCU 51 for controlling the drive of the camera head 102 and provides these control signals to the camera head control unit 155. The control signals include, for example, information about imaging conditions, such as information specifying the frame rate of the image, information specifying the exposure value during imaging, or information specifying the magnification and focus of the image.

[0079] Note that imaging conditions such as frame rate, exposure value, magnification, and focus can be appropriately specified by the user or automatically set by the control unit 161 of CCU 51 based on the acquired image signal. That is, in the latter case, so-called automatic exposure (AE), automatic focus (AF), and automatic white balance (AWB) functions are installed in the endoscope 10.

[0080] The camera head control unit 155 controls the driving of the camera head 102 based on the control signals received from the CCU 51 via the communication unit 154.

[0081] CCU 51 is an image processing device that includes a control unit 161, a communication unit 162, and an image processing unit 163.

[0082] The control unit 161 performs various types of control regarding the imaging of surgical parts, etc., by the endoscope 10 and the display of endoscopic images (medical images) obtained by imaging surgical parts, etc. For example, the control unit 161 generates control signals for controlling the drive of the camera head 102.

[0083] Furthermore, based on the image signal processed by the image processing unit 163, the control unit 161 causes the display device 52 to display a display image (endoscopic image) of the surgical procedure, etc. At this time, the control unit 161 can identify various objects in the image by using various image recognition technologies.

[0084] For example, the control unit 161 detects the color, edge shape, etc. of objects included in the image, thereby enabling it to identify surgical instruments such as forceps, specific body parts, bleeding, and mist when using the energy processing tool 22. When the display device 52 displays an image, the control unit 161 can overlay and display various types of surgical assistance information on the image of the surgical section using the recognition results. The surgical assistance information is overlaid and displayed and presented to the operator 3, thereby reducing the burden on the operator 3 and allowing the operator 3 to reliably perform the surgery.

[0085] The communication unit 162 includes communication means for sending / receiving various types of information to / from the camera head 102. The communication unit 162 receives image signals transmitted from the camera head 102 via the transmission cable 122.

[0086] In addition, the communication unit 162 sends control signals for controlling the camera head 102 to the camera head 102. Image signals and control signals can be transmitted via electrical communication, optical communication, etc.

[0087] The image processing unit 163 performs various types of image processing on image signals including RAW data sent from the camera head 102.

[0088] In addition, the control unit 161 includes an evaluation frame setting unit 171, an evaluation value calculation unit 172, an evaluation value determination unit 173, an evaluation result setting unit 174, and a type estimation unit 175.

[0089] The evaluation frame setting unit 171 sets multiple evaluation frames or evaluation regions arranged at predetermined intervals for the endoscopic image (RAW image) corresponding to the image signal (RAW data). Here, each evaluation frame is an area arbitrarily set for the region (region of the RAW image) corresponding to the imaging surface of the imaging element having a predetermined array pattern (e.g., Bayer array), and each evaluation frame can be, for example, a frame used to obtain evaluation values ​​such as information about brightness (brightness information).

[0090] The evaluation value calculation unit 172 calculates evaluation values ​​for the corresponding plurality of evaluation frames set by the evaluation frame setting unit 171. As an evaluation value, for example, luminance information (luminance value) in each evaluation frame is calculated. Note that the evaluation value is not limited to luminance value, and can be, for example, a quantitative value representing edges, black areas, etc. included in the evaluation frame (e.g., feature values ​​such as edge amount or black area amount).

[0091] The evaluation value determination unit 173 determines the relationship (correlation) between the evaluation values ​​of the corresponding plurality of evaluation frames calculated by the evaluation value calculation unit 172. Here, for example, the relationship between the evaluation values ​​corresponding to evaluation frames arranged adjacently or arranged at constant intervals is determined.

[0092] The evaluation result setting unit 174 performs settings for the evaluation results based on the relationship between the evaluation values ​​determined by the evaluation value determination unit 173.

[0093] The type estimation unit 175 estimates the endoscope body 101 type of the endoscope 10 based on the relationship between the evaluation values ​​determined by the evaluation value determination unit 173. For example, based on the relationship between the evaluation values, the endoscope body 101 type can be estimated by detecting the mask edge, which is the boundary between a black area (hereinafter also referred to as a mask area) where mechanical vignetting occurs due to the endoscope body 101 and an area of ​​the subject image (where effective areas where mechanical vignetting does not occur).

[0094] The type of mirror 101 is defined, for example, the diameter or shape of mirror 101. Since the size of the area of ​​the subject image is determined by the type of mirror 101, the type of mirror 101 can be obtained from the size of the subject image if the correlation between the size of the area of ​​the subject image and the type of mirror 101 is recorded in advance.

[0095] Note that, for example, depending on the type of mirror body 101, there may be cases where the area of ​​the subject image is not circular, but rectangular or octagonal. In the case where the area is not circular, for example, the length of the straight line that maximizes the vertical or horizontal length of the area of ​​the subject image can be used as the diameter of the mirror body 101. Furthermore, the central position of the mirror body 101 can be estimated along with its diameter.

[0096] In addition, as described above, in the endoscope 10, the endoscope body 101 connected to the camera head 102 is inserted into the body cavity of the patient 4, thereby allowing the operator 3 to observe the surgical area.

[0097] For example, Figure 4An example of an endoscope image 200 corresponding to an image signal obtained by imaging a subject image focused by the endoscope body 101 through a camera head 102 is shown. In the endoscope image 200, the left black area and the right black area each represent a mask area in which mechanical vignetting occurs, and the boundary between the area of ​​the subject image and the mask area (black area) is the mask edge 220.

[0098] In other words, in the endoscope 10, the elongated mirror body 101 is attached, and the shape of the image of the subject focused by the mirror body 101 and the shape of the imaging surface of the imaging element of the camera head 102 (imaging unit 152) do not match each other, thus causing mechanical vignetting due to the mirror body 101.

[0099] Note that the endoscopic image 200 is displayed as a display image by the display device 52 after undergoing various types of image processing. For example, such as Figure 5 As shown, the operator 3 can perform procedures such as removing the affected area by using surgical tools 20, such as energy processing tools 22, while viewing the displayed images in real time.

[0100] Here, in the endoscope 10, the endoscope body 101 is detachable, and the endoscope body 101 to be used is selected from a variety of endoscope bodies. In this case, the mechanical vignetting and optical imaging characteristics differ depending on the type of endoscope body 101 used. Since subsequent image processing depends on the type of endoscope body 101 to be adjusted, a method is needed to determine the type of endoscope body 101 used.

[0101] For example, signal processing related to AF, AE, etc. is performed on a region of the subject image, and various problems occur when focusing or exposing a region including a mask, so it is necessary to determine the type of lens body 101 used.

[0102] As stated above, Patent Document 1 requires detecting all straight edges in the image to determine the mirror type, which increases the computational load. Therefore, a method is needed to determine the type of mirror 101 used with less computation.

[0103] Therefore, in this technology, a method is designed to solve such problems and determine the type of mirror body 101 to be used with a small amount of computation. The details of this technology will now be described with reference to the accompanying drawings.

[0104] Note that in the endoscopic surgical system 1 (CCU 51), since the first to fifth processes and the determination process are performed as processes for determining the type of endoscope 101 used, these types of processes will be described sequentially. Furthermore, in the following description, the image of the imaging surface corresponding to the imaging element of the camera head 102 will also be referred to as an endoscopic image.

[0105] (First processing flow)

[0106] First, refer to Figure 6 The flowchart is used to describe the flow of the first process performed by CCU 51.

[0107] In step S10, the evaluation frame setting unit 171 sets multiple evaluation frames for the image (RAW image) corresponding to the image signal from the camera head 102. In setting the evaluation frames, such as... Figure 7 As shown, evaluation frames 210 are set at the central part and four peripheral corners of the endoscopic image 200.

[0108] Specifically, in the endoscopic image 200, a rectangular evaluation frame 210-0 (hereinafter also referred to as frame 0) is set in the central part, and rectangular evaluation frames 210-1, 210-2, 210-3 and 210-4 (hereinafter also referred to as frames 1, 2, 3 and 4) are set at the four corners of the upper left, lower left, upper right and lower right.

[0109] However, the size of the rectangles in evaluation frames 210-1 to 210-4 is smaller than the size of the rectangle in evaluation frame 210-0.

[0110] In addition, Figure 7 In the image 200, four circles of different diameters, centered approximately at the centroid, are indicated to be superimposed on the endoscopic image 200. These circles correspond to the mask edges 220, each being a boundary between the subject image region and the mask region.

[0111] In other words, the diameter of the endoscope body 101 used in the endoscope 10 corresponds to the mask edge 220 in the endoscope image 200, and since the location of the mask edge 220 is assumed to be detected in the endoscope image 200 is known in advance by design, evaluation frames 210-0 to 210-4 are set here to determine whether the mask area is included.

[0112] Note that in the following description, four types are assumed as the types of mirror body 101 used, and it is assumed that as the mask type of each mask edge 220, "TYPE1" is assigned to mask edge 220-1, "TYPE2" is assigned to mask edge 220-2, "TYPE3" is assigned to mask edge 220-3, and "TYPE4" is assigned to mask edge 220-4.

[0113] Return to Figure 6 As described above, in step S11, the evaluation value calculation unit 172 calculates and... Figure 7The evaluation values ​​corresponding to frame 0 at the center and frames 1 to 4 at the four peripheral corners are shown. As evaluation values, for example, feature values ​​obtained from the endoscopic image 200, such as brightness values, can be used.

[0114] Note that in step S11, for ease of explanation, the evaluation values ​​for the corresponding multiple evaluation frames 210 are calculated at once. However, for each of the determination processing steps (S12, S14, S15, S16) described later, the evaluation values ​​for the target evaluation frame 210 can be calculated sequentially. The method for calculating the evaluation values ​​in the second processing is similar to that in the fifth processing described later.

[0115] In step S12, the evaluation value determination unit 173 determines, based on the calculation results of the evaluation value of each frame, whether the evaluation value of at least one of the four peripheral corner frames 1 to 4 is greater than the evaluation value of the central frame 0.

[0116] If, in the determination process of step S12, it is determined that the evaluation values ​​of frames 1 to 4 at the four peripheral corners are greater than the evaluation value of frame 0 at the center, in other words, the central part of the endoscope image 200 is dark while the peripheral part is bright, the process proceeds to step S13. Then, in step S13, the evaluation result setting unit 174 sets the mask type to "rigid mirror body".

[0117] Furthermore, if in the determination process of step S12 it is determined that the evaluation values ​​of frames 1 to 4 at the four peripheral corners are less than the evaluation value of frame 0 at the center, the process proceeds to step S14. In step S14, the evaluation value determination unit 173 determines, based on the calculation result of the evaluation value of the target frame, whether at least one evaluation value of frames 1 to 4 at the four peripheral corners exceeds a predetermined threshold (first threshold).

[0118] In the determination process of step S14, if the evaluation values ​​of frames 1 to 4 at the four peripheral corners exceed a predetermined threshold, in other words, the entire image is bright, the process proceeds to step S13, and "no rigid mirror body" is set as the mask type.

[0119] Furthermore, if in the determination process of step S14 it is determined that the evaluation values ​​of frames 1 to 4 at the four peripheral corners are less than or equal to a predetermined threshold, the process proceeds to step S15. In step S15, the evaluation value determination unit 173 determines, based on the calculation result of the evaluation value of the target frame, whether the evaluation value of frame 0 at the center is less than or equal to a predetermined threshold (second threshold).

[0120] In the determination process of step S15, if the evaluation value of frame 0 at the center is determined to be less than or equal to the threshold, in other words, the endoscope image 200 generally reflects a black image, the process proceeds to step S13, and "no rigid endoscope" is set to the mask type.

[0121] Furthermore, if the evaluation value of frame 0 at the center is determined to exceed a predetermined threshold in the determination process of step S15, the process proceeds to step S16. In step S16, the evaluation value determination unit 173 determines, based on the calculation result of the evaluation value of the target frame, whether the brightness difference between frame 0 at the center and frames 1 to 4 at the four peripheral corners is less than or equal to a predetermined threshold (third threshold).

[0122] If the brightness difference is determined to be less than or equal to a predetermined threshold in the determination process of step S16, the process proceeds to step S13, and "non-rigid mirror body" is set as the mask type.

[0123] Note that if the determination process in steps S12, S14, S15, and S16 determines the result as "yes", then "no rigid mirror body" is set to the mask type (S13), and the process proceeds to step S17. Then, in step S17, the evaluation result setting unit 174 sets the recognition result to "paused". Afterward, the process returns to step S10, and the first process described above is repeated.

[0124] Furthermore, if the brightness difference exceeds a predetermined threshold during the determination process in step S16, the process proceeds to step S18. In step S18, the evaluation result setting unit 174 sets the recognition result to "recognition successful". Thereafter, in CCU 51, a second process is performed after the first process. Note that details of the second process will be referred to later. Figure 8 Descriptions such as...

[0125] The flow of the first process has been described above. In the first process, such as... Figure 7 As shown, frame 0 at the center and frames 1 to 4 at the four peripheral corners are set as evaluation frames 210 of the endoscope image 200, and it is determined whether the endoscope image 200 includes a mask region based on the relationship (correlation) between the evaluation values ​​of the corresponding evaluation frames 210. Then, the first process is repeated if the mask region is not included, and the second process is performed if the mask region is included.

[0126] (Second processing flow)

[0127] Next, we will refer to Figure 8 The flowchart is used to describe the flow of the second processing performed by CCU 51.

[0128] In step S30, the evaluation frame setting unit 171 sets multiple evaluation frames for the image corresponding to the image signal from the camera head 102. In setting the evaluation frames, such as... Figure 9 As shown, the evaluation frame 210 is set at the central part and the horizontal (X direction) part of the endoscope image 200, respectively.

[0129] Specifically, in the endoscopic image 200, a rectangular evaluation frame 210-4 (hereinafter referred to as evaluation frame 4) is set in the central part including the approximate centroid.

[0130] Furthermore, rectangular evaluation frames 210-0 to 210-3 (hereinafter also referred to as evaluation frames 0, 1, 2 and 3) are set at predetermined intervals on the left side in the horizontal direction, and rectangular evaluation frames 210-5 to 210-8 (hereinafter also referred to as evaluation frames 5, 6, 7 and 8) are set at predetermined intervals on the right side in the horizontal direction, so as to be substantially symmetrical about the central part of the endoscope image 200 (including the approximate centroid in the central part of the endoscope image 200).

[0131] However, the size of the rectangles of evaluation frames 210-0 to 210-3 and evaluation frames 210-5 to 210-8, which are discretely arranged at predetermined intervals on the left and right sides of the central portion, is smaller than the size of the rectangle of evaluation frame 210-4. Note that in this disclosure, "discretely" means that the multiple evaluation frames 210 are not arranged consecutively.

[0132] exist Figure 9 In order to determine which (mask type) mask edge 220 corresponds to the diameter of the endoscope body 101 used, evaluation frames 0 to 8 are discretely set at predetermined intervals in the horizontal direction in the endoscope image 200 among the mask edges 220-1 to 220-4.

[0133] Specifically, for example, since the position of the detected mask edge 220-1 is known in the design, evaluation frames 0 and 1, as well as evaluation frames 7 and 8, are set such that the detection position of the mask edge 220-1 is between evaluation frames 210.

[0134] Similarly, the evaluation frames are configured such that the detection positions of mask edge 220-2 are between evaluation frames 1 and 2 and between evaluation frames 6 and 7, the detection positions of mask edge 220-3 are between evaluation frames 2 and 3 and between evaluation frames 5 and 6, and the detection positions of mask edge 220-4 are between evaluation frames 3 and 4 and between evaluation frames 4 and 5.

[0135] Return to Figure 8 As described above, in step S31, the evaluation value calculation unit 172 calculates and... Figure 9 The evaluation values ​​corresponding to evaluation frames 0 to 8 are shown. As evaluation values, for example, feature values ​​obtained from endoscopic image 200, such as brightness values, can be used.

[0136] In step S32, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 0 and evaluation frame 1, and whether there is an edge between evaluation frame 7 and evaluation frame 8, based on the calculation results of the evaluation values ​​of evaluation frame 0 and evaluation frame 1 and evaluation frame 7 and evaluation frame 8.

[0137] Here, for example, the difference between the brightness value obtained from evaluation frame 0 and the brightness value obtained from evaluation frame 1, and the difference between the brightness value obtained from evaluation frame 7 and the brightness value obtained from evaluation frame 8 are compared with a predetermined threshold (fourth threshold), and it is determined whether these brightness differences exceed the predetermined threshold, thereby determining whether there is an edge (mask edge 220-1) between these evaluation frames 210.

[0138] If the determination process in step S32 is affirmative (“Yes”), the process proceeds to step S33. In step S33, the evaluation result setting unit 174 sets the mask type to “TYPE1”.

[0139] Furthermore, if the determination in step S32 is negative ("No"), the process proceeds to step S34. In step S34, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 1 and evaluation frame 2, and whether there is an edge between evaluation frame 6 and evaluation frame 7, based on the calculation results of the evaluation values ​​of evaluation frame 1 and evaluation frame 2, and evaluation frame 6 and evaluation frame 7.

[0140] If the determination process in step S34 is affirmative, the process proceeds to step S35. In step S35, the evaluation result setting unit 174 sets the mask type to "TYPE2".

[0141] Furthermore, if the determination in step S34 is negative, the process proceeds to step S36. In step S36, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 2 and evaluation frame 3, and whether there is an edge between evaluation frame 5 and evaluation frame 6, based on the calculation results of the evaluation values ​​of evaluation frame 2 and evaluation frame 3, and evaluation frame 5 and evaluation frame 6.

[0142] If the determination process in step S36 is affirmative, the process proceeds to step S37. In step S37, the evaluation result setting unit 174 sets the mask type to "TYPE3".

[0143] Furthermore, if the determination in step S36 is negative, the process proceeds to step S38. In step S38, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 3 and evaluation frame 4, and whether there is an edge between evaluation frame 4 and evaluation frame 5, based on the calculation results of the evaluation values ​​of evaluation frame 3 and evaluation frame 4 and evaluation frame 5.

[0144] If the determination process in step S38 is affirmative, the process proceeds to step S39. In step S39, the evaluation result setting unit 174 sets the mask type to "TYPE4".

[0145] Note that if the determination process in steps S32, S34, S36 and S38 is affirmative, “TYPE1”, “TYPE2”, “TYPE3” and “TYPE4” are set as mask types respectively (S33, S35, S37 and S39), and the process proceeds to step S40.

[0146] Then, in step S40, the evaluation result setting unit 174 sets the recognition result to "recognition successful". Afterwards, in CCU 51, a third process is performed after the second process. Note that details of the third process will be referred to later. Figure 10 Descriptions such as...

[0147] Furthermore, if the determination in step S38 is negative, the process proceeds to step S41. Then, the evaluation result setting unit 174 sets the mask type to "non-rigid mirror" (S41) and the recognition result to "paused" (S42). Afterward, the process returns to... Figure 6 Step S10 is performed, and the first process described above is executed.

[0148] The flow of the second process has been described above. In the second process, such as... Figure 9 As shown, evaluation frames 210-0 to 210-8 are discretely arranged at predetermined intervals in the horizontal direction of the endoscopic image 200, and a mask type depending on the edge location is set based on the relationship (correlation) between the evaluation values ​​of the corresponding evaluation frames 210. Then, the first process is repeated if no edge corresponding to the evaluation frame 210 is detected, and the third process is performed if an edge corresponding to the evaluation frame 210 is detected.

[0149] (Third processing flow)

[0150] Next, we will refer to Figure 10 The flowchart is used to describe the process of the third processing performed by CCU 51.

[0151] In step S50, the evaluation result setting unit 174 determines whether the mask type is set to "TYPE3" or "TYPE4" in the second process. In other words, in this example, since the detection positions in the vertical direction of the mask edges 220-1 and 220-2 of "TYPE1" and "TYPE2" are located outside the endoscope image 200, the processing of the mask edges 220-1 and 220-2 is excluded.

[0152] If the determination process in step S50 is affirmative, the process proceeds to step S51. In step S51, the evaluation frame setting unit 171 sets multiple evaluation frames for the image corresponding to the image signal from the camera head 102.

[0153] In the settings of the evaluation frame, such as Figure 11 As shown, evaluation frames 210 are provided in the central portion and the portion in the vertical direction (Y direction) of the endoscope image 200. Specifically, in the endoscope image 200, a rectangular evaluation frame 210-4 (hereinafter referred to as evaluation frame 4) is provided in the central portion including the approximate centroid.

[0154] Furthermore, rectangular evaluation frames 210-0 and 210-1 (hereinafter also referred to as evaluation frame 0 and evaluation frame 1) are set at predetermined intervals on the upper side in the vertical direction, and rectangular evaluation frames 210-2 and 210-3 (hereinafter also referred to as evaluation frame 2 and evaluation frame 3) are set at predetermined intervals on the lower side in the vertical direction, so as to be substantially symmetrical about the central part of the endoscope image 200 (including the approximate centroid therein).

[0155] However, the size of the rectangles of evaluation frames 210-0 and 210-1, as well as evaluation frames 210-2 and 210-3, which are discretely arranged at predetermined intervals on the upper and lower sides of the central portion, is smaller than the size of the rectangle of evaluation frame 210-4.

[0156] exist Figure 11 In order to determine which (mask type) mask edge 220 corresponds to the diameter of the endoscope body 101 used, evaluation frames 210-0 to 210-3 are discretely set in the vertical direction at predetermined intervals in the endoscope image 200 in the mask edges 220-3 and 220-4.

[0157] Specifically, the evaluation frames are set such that the detection position of mask edge 220-3 is between evaluation frame 0 and evaluation frame 1 and between evaluation frame 2 and evaluation frame 3, and the detection position of mask edge 220-4 is between evaluation frame 1 and evaluation frame 4 and between evaluation frame 2 and evaluation frame 4.

[0158] Return to Figure 10 As described in step S52, the evaluation value calculation unit 172 calculates and... Figure 11 The evaluation values ​​corresponding to evaluation frames 0 to 4 are shown. As evaluation values, for example, feature values ​​obtained from endoscopic image 200, such as brightness values, can be used.

[0159] In step S53, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 0 and evaluation frame 1, and whether there is an edge between evaluation frame 2 and evaluation frame 3, based on the calculation results of the evaluation values ​​of evaluation frame 0 and evaluation frame 1 and evaluation frame 2 and evaluation frame 3.

[0160] Here, for example, the difference between the brightness value obtained from evaluation frame 0 and the brightness value obtained from evaluation frame 1, and the difference between the brightness value obtained from evaluation frame 2 and the brightness value obtained from evaluation frame 3 are compared with a predetermined threshold (fifth threshold), and it is determined whether these brightness differences exceed the predetermined threshold, thereby determining whether there are edges (mask edges 220-3) between these evaluation frames 210.

[0161] If the determination process in step S53 is affirmative, the process proceeds to step S54. In step S54, the evaluation result setting unit 174 sets the mask type determined in the second process to "TYPE3".

[0162] Furthermore, if the determination in step S53 is negative, the process proceeds to step S55. In step S55, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 1 and evaluation frame 4, and whether there is an edge between evaluation frame 2 and evaluation frame 4, based on the calculation results of the evaluation values ​​of evaluation frame 1 and evaluation frame 4, and evaluation frame 2 and evaluation frame 4.

[0163] If the determination process in step S55 is affirmative, the process proceeds to step S56. In step S56, the evaluation result setting unit 174 sets the mask type determined in the second process to "TYPE4".

[0164] When the processing in step S54 or S56 is completed, the processing proceeds to step S57. Then, in step S57, the evaluation result setting unit 174 sets the recognition result to "recognition successful".

[0165] In step S58, the evaluation result setting unit 174 determines whether to set the mask type determined in the second process to "TYPE4" and whether to set the mask type determined in the third process to "TYPE3".

[0166] If the determination process in step S58 is affirmative, the process proceeds to step S59. In step S59, the evaluation result setting unit 174 sets the mask type to "TYPE4". In other words, in this case, assuming, for example, that a larger mask diameter is detected in the vertical direction due to light leakage, the mask diameter detected in the horizontal direction in the second process is adopted. As described above, selecting and determining a narrower mask diameter can more reliably prevent the inclusion of black areas, for example, when performing subsequent processing.

[0167] Furthermore, if the determination in step S58 is negative, the process proceeds to step S60. In step S60, the evaluation value determination unit 173 determines whether to set the mask type determined in the second process to "TYPE3" and to set the mask type determined in the third process to "TYPE4".

[0168] If the determination process in step S60 is affirmative, the process proceeds to step S61. In step S61, the evaluation result setting unit 174 sets the mask type to "TYPE4". In other words, in this case, assuming that a large mask diameter is detected in the horizontal direction due to light leakage, for example, the mask diameter detected in the vertical direction in the third process is used, thereby more reliably preventing the inclusion of black areas.

[0169] If the process ends in step S59 or S61, or if the determination in step S60 is negative, the process proceeds to step S62. Furthermore, if the determination in step S50 is negative, the process proceeds to step S62.

[0170] In step S62, it is determined whether the operation mode is set to high-precision calculation mode, which is a mode used to calculate the mask diameter size more accurately than the normal mode.

[0171] If the determination process in step S62 is affirmative, then in CCU 51, the fourth process is executed after the third process. Note that this will be referred to later. Figure 12 Details of the fourth process are described below.

[0172] Furthermore, if the determination in step S62 is negative, the process returns to... Figure 6 Step S10 is performed, and the first process described above is executed. Note that if the determination is negative in the determination process of step S55, the first process described above is executed similarly.

[0173] The flow of the third processing has been described above. In the third processing, such as... Figure 11 As shown, evaluation frames 210-0 to 210-4 are discretely set at predetermined intervals in the vertical direction of the endoscopic image 200, and a mask type depending on the edge position is set (reset) based on the relationship (correlation) between the evaluation values ​​of the corresponding evaluation frames 210. Then, when the processing returns to the first processing mode when the normal mode is set to the operation mode, the fourth processing is performed when the high-precision calculation mode is set.

[0174] Here, when operating in normal mode, through the second and third processes, one of "TYPE1" to "TYPE4" is set based on the correlation of the evaluation values ​​corresponding to the respective plurality of evaluation frames 210, and the type estimation unit 175 can estimate the diameter of the mirror body 101 by obtaining the mask diameter size based on the set mask type. For example, when the mask type is set to "TYPE4" through the second and third processes, the mask edges 220-4 are detected, thereby obtaining the mask diameter.

[0175] Furthermore, the center position of the mask can also be obtained in both the horizontal and vertical directions. For example, in the case of detecting mask edge 220-4, such as... Figure 11 As shown, the coordinates (x, y) of the center position of the mask can be calculated by using the vertex coordinates of the rectangles in evaluation frames 210-1 and 210-2.

[0176] Specifically, for example, in Figure 11 In the endoscope image 200, the position of the upper left vertex is taken as the origin (0, 0). When the coordinates of the lower right vertex of the rectangle of the evaluation frame 210-1 are (x0, y0) and the coordinates of the upper left vertex of the rectangle of the evaluation frame 210-2 are (x1, y1), the center position (x, y) of the mask in the horizontal and vertical directions is obtained by the following formulas (1) and (2).

[0177] x=(x0-x1) / 2+x1···(1)

[0178] y=(y1-y0) / 2+y0···(2)

[0179] As described above, when operating in normal mode, although the accuracy is lower than when operating in high-precision calculation mode, the diameter and central position of the mask can be calculated with a smaller amount of computation, the diameter and central position of the mirror body 101 can be estimated, and the type of the mirror body 101 can be determined.

[0180] Furthermore, in the second and third processes, since multiple evaluation frames 210 are discretely set at predetermined intervals for the endoscope image 200 and the evaluation frames 210 are separated from each other at predetermined intervals, errors that occur during the installation of the endoscope body 101 can be reduced, for example.

[0181] Note that the above description describes the case where evaluation is performed by setting evaluation frames 210 in both the horizontal and vertical directions through the second and third processes, such that multiple evaluation frames 210 are symmetrical with respect to the approximate centroid of the endoscopic image 200. However, evaluation can also be performed by setting evaluation frames 210 in only one of the horizontal and vertical directions through the second or third processes. However, as mentioned above, when evaluation frames 210 are set in both the horizontal and vertical directions through the second and third processes, the mask type can be set assuming, for example, the presence of light leakage, thereby allowing for more accurate determination of the mask diameter and center position.

[0182] Furthermore, in the second and third processes, the case where multiple evaluation frames 210 are discretely arranged at predetermined intervals is described, i.e., there is a gap between adjacent frames; however, a portion of the evaluation frames 210 can be arranged continuously. Moreover, the number of discretely arranged evaluation frames 210 is arbitrary, and for example, a larger number of evaluation frames 210 can be set for the detection location of the mask edge 220. Furthermore, the shape of each discretely arranged evaluation frame 210 is not limited to a rectangle; it can be other shapes, and not all evaluation frames 210 need to have the same shape. The interval between arranging the multiple evaluation frames 210 does not need to be a constant interval.

[0183] Furthermore, in the second and third processes, examples are described of detecting edges (mask edges 220) by using brightness values ​​as evaluation values ​​and comparing the brightness difference with a predetermined threshold; however, edges can be detected by using quantitative values ​​(e.g., feature values ​​such as edge amount or black area amount) representing edges, black areas, etc. included in frame 210 as evaluation values.

[0184] (Fourth processing step)

[0185] Next, we will refer to Figure 12 and Figure 13 The flowchart will be used to describe the flow of the fourth process performed by CCU 51. However, in particular, the fourth process will describe the case where "TYPE3" is set to the mask type in the second and third processes described above.

[0186] In step S70, the evaluation frame setting unit 171 sets multiple evaluation frames for the image corresponding to the image signal from the camera head 102. In setting the evaluation frames, since "TYPE3" is set as a mask type, such as... Figure 14 As shown, multiple evaluation frames 210 are set according to the detection positions of mask edges 220-3. (And...) Figure 7 , Figure 9 and Figure 11 Compared to spatially separated and discrete multiple evaluation frames, Figure 14The multiple evaluation frames in the model are spatially separated and continuous, meaning that adjacent frames are adjacent to each other without gaps, thus allowing for high-precision detection of edges along the horizontal direction.

[0187] Specifically, evaluation frames 210-0 to 210-4 (hereinafter also referred to as evaluation frames 0, 1, 2, 3 and 4) corresponding to the detection positions of the mask edge 220-3 are continuously set on the left side of the horizontal direction, and evaluation frames 210-5 to 210-9 (hereinafter also referred to as evaluation frames 5, 6, 7, 8 and 9) corresponding to the detection positions of the mask edge 220-3 are continuously set on the right side of the horizontal direction, so as to be basically symmetrical about the approximate centroid of the endoscope image 200 (left-right symmetry with the Y-axis as the axis of symmetry).

[0188] However, the rectangles of evaluation frames 210-0 to 210-4, which are arranged symmetrically and continuously from left to right, and the rectangles of evaluation frames 210-5 to 210-9 have essentially the same shape and essentially the same size.

[0189] Furthermore, in each evaluation frame 210, a start position and an end position are set in the horizontal direction (X direction). The start position indicates the position of the left end in the X direction of each evaluation frame 210, and the end position indicates the position of the right end in the X direction of each evaluation frame 210.

[0190] Return to Figure 12 As described above, in step S71, the evaluation value calculation unit 172 calculates and... Figure 14 The evaluation values ​​corresponding to evaluation frames 0 to 9 are shown. As evaluation values, for example, feature values ​​obtained from endoscopic image 200, such as brightness values, can be used.

[0191] In step S72, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 0 and evaluation frame 2 based on the calculation results of the evaluation values ​​of evaluation frame 0 and evaluation frame 2.

[0192] Here, for example, the difference between the luminance value obtained from evaluation frame 0 and the luminance value obtained from evaluation frame 2 is compared with a predetermined threshold (sixth threshold) to determine whether the luminance difference exceeds the predetermined threshold, thereby determining whether there is an edge between evaluation frame 0 and evaluation frame 2 (mask edge 220-3 in this example).

[0193] If the determination process in step S72 is affirmative, the process proceeds to step S73. In step S73, the evaluation result setting unit 174 sets the X-direction end position of evaluation frame 0 to the left end position of the mask diameter edge.

[0194] Furthermore, if the determination in step S72 is negative, the process proceeds to step S74. In step S74, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 1 and evaluation frame 3 based on the calculation results of the evaluation values ​​of evaluation frame 1 and evaluation frame 3.

[0195] If the determination process in step S74 is affirmative, the process proceeds to step S75. In step S75, the evaluation result setting unit 174 sets the X-direction end position of evaluation frame 1 to the left end position of the mask diameter edge.

[0196] Furthermore, if the determination in step S74 is negative, the process proceeds to step S76. In step S76, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 2 and evaluation frame 4 based on the calculation results of the evaluation values ​​of evaluation frame 2 and evaluation frame 4.

[0197] If the determination process in step S76 is affirmative, the process proceeds to step S77. In step S77, the evaluation result setting unit 174 sets the X-direction end position of evaluation frame 2 to the left end position of the mask diameter edge.

[0198] Furthermore, if the determination in step S76 is negative, the process proceeds to step S78. In step S78, the evaluation result setting unit 174 sets the X-direction end position of evaluation frame 4 to the left end position of the mask diameter edge.

[0199] When the left end position of the mask diameter edge is set in step S73, S75, S77 or S78, the process proceeds to... Figure 13 Step S79.

[0200] In step S79, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 5 and evaluation frame 7 based on the calculation results of the evaluation values ​​of evaluation frame 5 and evaluation frame 7.

[0201] If the determination process in step S79 is affirmative, the process proceeds to step S80. In step S80, the evaluation result setting unit 174 sets the X-direction start position of the evaluation frame 5 to the right end position of the mask diameter edge.

[0202] Furthermore, if the determination in step S79 is negative, the process proceeds to step S81. In step S81, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 6 and evaluation frame 8 based on the calculation results of the evaluation values ​​of evaluation frame 6 and evaluation frame 8.

[0203] If the determination process in step S81 is affirmative, the process proceeds to step S82. In step S82, the evaluation result setting unit 174 sets the starting position of the evaluation frame 6 in the X direction to the right end position of the mask diameter edge.

[0204] Furthermore, if the determination in step S81 is negative, the process proceeds to step S83. In step S83, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 7 and evaluation frame 9 based on the calculation results of the evaluation values ​​of evaluation frame 7 and evaluation frame 9.

[0205] If the determination process in step S83 is affirmative, the process proceeds to step S84. In step S84, the evaluation result setting unit 174 sets the starting position of the evaluation frame 7 in the X direction to the right end position of the mask diameter edge.

[0206] Furthermore, if the determination in step S83 is negative, the process proceeds to step S85. In step S85, the evaluation result setting unit 174 sets the X-direction start position of the evaluation frame 8 to the right end position of the mask diameter edge.

[0207] When the right end position of the mask diameter edge is set in the process of steps S80, S82, S84, or S85, the fifth process is performed in CCU 51 after the fourth process. Note that the details of the fifth process will be referred to later. Figure 15 To describe it.

[0208] The flow of the fourth process has been described above. In the fourth process, based on the evaluation results from the second and third processes described above, a process for calculating the detailed mask edges in the horizontal direction (X direction) is performed, such as... Figure 14 As shown, evaluation frames 210-0 to 210-4 and evaluation frames 210-5 to 210-9 are arranged symmetrically and continuously in the horizontal direction of the endoscope image 200, and the left and right edge positions in the mask diameter are set based on the relationship (correlation) between the evaluation values ​​of the corresponding evaluation frames 210.

[0209] (Fifth processing flow)

[0210] Next, we will refer to Figure 15 and Figure 16 The flowchart will be used to describe the flow of the fifth process performed by CCU 51. However, in the fifth process, similar to the fourth process described above, the case where "TYPE3" is set to the mask type in the second and third processes described above will be described.

[0211] In step S90, the evaluation result setting unit 174 determines whether the mask type is set to "TYPE3" or "TYPE4" in the second process.

[0212] If the determination process in step S90 is affirmative, the process proceeds to step S91. In step S91, the evaluation frame setting unit 171 sets multiple evaluation frames for the image corresponding to the image signal from the camera head 102.

[0213] In the evaluation frame settings, since "TYPE3" is set as the mask type, such as Figure 17 As shown, multiple evaluation frames 210 are set according to the detection positions of mask edges 220-3. Figure 17 Multiple evaluation frames are spatially separated and continuous, allowing for high-precision detection of edges along the vertical direction.

[0214] Specifically, evaluation frames 210-0 to 210-4 (hereinafter also referred to as evaluation frames 0, 1, 2, 3 and 4) corresponding to the detection positions of the mask edge 220-3 are continuously set on the upper side of the vertical direction, and evaluation frames 210-5 to 210-9 (hereinafter also referred to as evaluation frames 5, 6, 7, 8 and 9) corresponding to the detection positions of the mask edge 220-3 are continuously set on the lower side of the vertical direction, so as to be basically symmetrical about the approximate centroid of the endoscope image 200 (vertical symmetry with the X-axis as the axis of symmetry).

[0215] However, the rectangles of evaluation frames 210-0 to 210-4, which are arranged in a vertically symmetrical and continuous manner, and the rectangles of evaluation frames 210-5 to 210-9 have essentially the same shape and essentially the same size.

[0216] Furthermore, in each evaluation frame 210, a start position and an end position are set in the vertical direction (Y direction). The start position indicates the upper position in the Y direction of each evaluation frame 210, and the end position indicates the lower position in the Y direction of each evaluation frame 210.

[0217] Return to Figure 15 As described in step S92, the evaluation value calculation unit 172 calculates and... Figure 17 The evaluation values ​​corresponding to evaluation frames 0 to 9 are shown. As evaluation values, for example, feature values ​​obtained from endoscopic image 200, such as brightness values, can be used.

[0218] In step S93, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 0 and evaluation frame 2 based on the calculation results of the evaluation values ​​of evaluation frame 0 and evaluation frame 2.

[0219] Here, for example, the difference between the luminance value obtained from evaluation frame 0 and the luminance value obtained from evaluation frame 2 is compared with a predetermined threshold (seventh threshold) to determine whether the luminance difference exceeds the predetermined threshold, thereby determining whether there is an edge between evaluation frame 0 and evaluation frame 2 (mask edge 220-3 in this example).

[0220] If the determination process in step S93 is affirmative, the process proceeds to step S94. In step S94, the evaluation result setting unit 174 sets the end position of the Y direction of evaluation frame 0 to the upper edge of the mask diameter.

[0221] Furthermore, if the determination in step S93 is negative, the process proceeds to step S95. In step S95, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 1 and evaluation frame 3 based on the calculation results of the evaluation values ​​of evaluation frame 1 and evaluation frame 3.

[0222] If the determination process in step S95 is affirmative, the process proceeds to step S96. In step S96, the evaluation result setting unit 174 sets the end position of the Y direction of evaluation frame 1 to the upper edge of the mask diameter.

[0223] Furthermore, if the determination in step S95 is negative, the process proceeds to step S97. In step S97, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 2 and evaluation frame 4 based on the calculation results of the evaluation values ​​of evaluation frame 2 and evaluation frame 4.

[0224] If the determination process in step S97 is affirmative, the process proceeds to step S98. In step S98, the evaluation result setting unit 174 sets the end position of the Y direction of the evaluation frame 2 to the upper edge of the mask diameter.

[0225] Furthermore, if the determination in step S97 is negative, the process proceeds to step S99. In step S99, the evaluation result setting unit 174 sets the end position of the Y direction of evaluation frame 4 to the upper edge of the mask diameter.

[0226] When the upper position of the mask diameter edge is set in step S94, S96, S98 or S99, the process proceeds to... Figure 16 Step S100.

[0227] In step S100, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 5 and evaluation frame 7 based on the calculation results of the evaluation values ​​of evaluation frame 5 and evaluation frame 7.

[0228] If the determination process in step S100 is affirmative, the process proceeds to step S101. In step S101, the evaluation result setting unit 174 sets the starting position of the Y direction of the evaluation frame 5 to the lower end position of the mask diameter edge.

[0229] Furthermore, if the determination in step S100 is negative, the process proceeds to step S102. In step S102, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 6 and evaluation frame 8 based on the calculation results of the evaluation values ​​of evaluation frame 6 and evaluation frame 8.

[0230] If the determination process in step S102 is affirmative, the process proceeds to step S103. In step S103, the evaluation result setting unit 174 sets the starting position of the Y direction of the evaluation frame 6 to the lower end position of the mask diameter edge.

[0231] Furthermore, if the determination in step S102 is negative, the process proceeds to step S104. In step S104, the evaluation value determination unit 173 determines whether there is an edge between evaluation frame 7 and evaluation frame 9 based on the calculation results of the evaluation values ​​of evaluation frame 7 and evaluation frame 9.

[0232] If the determination process in step S104 is affirmative, the process proceeds to step S105. In step S105, the evaluation result setting unit 174 sets the starting position of the Y direction of the evaluation frame 7 to the lower end position of the mask diameter edge.

[0233] Furthermore, if the determination in step S104 is negative, the process proceeds to step S106. In step S106, the evaluation result setting unit 174 sets the starting position of the Y direction of the evaluation frame 9 to the lower end position of the mask diameter edge.

[0234] When the lower end position of the mask diameter edge is set in the process of steps S101, S103, S105, or S106, a determination process is performed in CCU51 after the fifth process. Note that this will be referred to later. Figure 18 The details of the process are described and determined.

[0235] The flow of the fifth process has been described above. In the fifth process, based on the evaluation results from the second and third processes described above, a process for calculating the detailed mask edges in the vertical direction (Y direction) is performed, such as... Figure 17 As shown, evaluation frames 210-0 to 210-4 and evaluation frames 210-5 to 210-9 are arranged vertically symmetrically and continuously in the vertical direction of the endoscope image 200, and the upper edge position and lower edge position in the mask diameter are set based on the relationship (correlation) between the evaluation values ​​of the corresponding evaluation frames 210.

[0236] As described above, when operating in high-precision calculation mode, the mask type of the mask region included in the endoscopic image 200 is set based on the correlation of the evaluation values ​​corresponding to the respective plurality of evaluation frames 210 through the first to fifth processes, and each of the left edge position, right edge position, upper edge position, and lower edge position is set in the mask diameter corresponding to the mask type. For example, in cases where the allowable error range is narrow, it is necessary to obtain the mask diameter and center position more accurately, in which case the high-precision calculation mode is set to the operating mode. Therefore, the position of the vignetting region in the endoscopic image, which depends on how the endoscope is attached, and the estimated mask diameter can be estimated.

[0237] Note that in the fourth and fifth processes described above, a plurality of evaluation frames 210 were arranged sequentially in a manner symmetrical with respect to an approximate centroid of the endoscopic image 200; however, similar to the second and third processes, the plurality of evaluation frames 210 can be arranged discretely at predetermined intervals. Furthermore, the number of sequentially arranged evaluation frames 210 is not limited to five, and can be four or fewer, or six or more. Moreover, the shape of each of the sequentially arranged evaluation frames 210 is not limited to a rectangle, but can be other shapes, and not all evaluation frames 210 need to have the same shape.

[0238] Furthermore, in the fourth and fifth processes, examples are described of detecting edges (mask edges 220) by using a brightness value as an evaluation value and comparing the brightness difference with a predetermined threshold; however, edges can be detected by using quantitative values ​​(e.g., feature values ​​such as edge amount or black area amount) representing edges, black areas, etc. included in frame 210 as evaluation values.

[0239] (Determine the processing flow)

[0240] Next, we will refer to Figure 18 The flowchart is used to describe the process of deterministic processing performed by CCU 51.

[0241] In step S111, the type estimation unit 175 calculates the mask diameter size based on the processing results of the first to fifth processes described above.

[0242] For example, when the operation is performed in a high-precision calculation mode, since the left edge position, right edge position, upper edge position and lower edge position of the mask diameter are set respectively, the mask diameter and the center position of the mask can be obtained by using these edge positions.

[0243] Specifically, the diameters in the horizontal and vertical directions of the mask are obtained, for example, by the following formulas (3) and (4). However, the coordinates of the evaluation frame 210 are represented by an orthogonal coordinate system in which the X-axis in the horizontal direction and the Y-axis in the vertical direction are orthogonal to each other, and as shown in the figure. Figure 19 As shown, the position of the upper left vertex of the endoscopic image 200 is the origin (0, 0).

[0244] Mask diameter (horizontal direction) = Right edge position - Left edge position ... (3)

[0245] Mask diameter (vertical direction) = lower edge position - upper edge position ... (4)

[0246] Furthermore, the center positions (x, y) of the mask in the horizontal and vertical directions are obtained, for example, by the following formulas (5) and (6). However, here, the coordinates of the evaluation frame 210 are also obtained by... Figure 19 The orthogonal coordinate system shown is used.

[0247] x = (Right edge position - Left edge position) / 2 + Left edge position ... (5)

[0248] y = (Lower edge position - Upper edge position) / 2 + Upper edge position ... (6)

[0249] As described above, in the type estimation unit 175, the mask diameter included in the endoscope image 200 is calculated, and the mask diameter is highly correlated with the type of the endoscope body 101. Therefore, it can also be said that the type estimation unit 175 estimates the diameter of the endoscope body 101 by calculating the mask diameter. Then, by estimating the diameter of the endoscope body 101, the type of the endoscope body 101 used in the endoscope 10 can be determined. In other words, it can also be said that the type of the endoscope body 101 is determined by the diameter of the endoscope body 101.

[0250] When the mask diameter is calculated in step S111, the process proceeds to step S112. Then, in CCU51, subsequent processing is informed of the identification results obtained in the first to fifth processes described above and the mask diameter information obtained in step S111. However, in addition to the mask diameter, the mask diameter information may include information about the center position of the mask.

[0251] Here, subsequent processing includes, for example, signal processing such as autofocus (AF), auto exposure (AE), auto white balance (AWB), and extended depth of field (EDOF).

[0252] As described above, for example, the endoscopic image 200 includes a mask region (black area) that depends on the diameter of the endoscope body 101, in addition to the area of ​​the subject image. Various problems arise when focusing or exposing the mask region during signal processing such as AF and AE. On the other hand, in this technology, since the type of endoscope body 101 used can be reliably determined with a small computational load, signal processing can be performed on the area of ​​the subject image, and AF, AE, etc., with high accuracy can be achieved.

[0253] Furthermore, for example, in signal processing related to EDOF (e.g., processing for extended depth of field), the mask center position is an important parameter, and the operation is performed in a high-precision calculation mode, thereby calculating a mask center position with high accuracy and improving the accuracy of signal processing related to EDOF.

[0254] The determination process has been described above. In the determination process, when the operation is performed in a high-precision calculation mode, the type of mirror 101 can be determined by calculating the diameter and center position of the mask, and estimating the diameter and center position of the mirror 101 with a small amount of computation and high accuracy based on the left edge position, right edge position, upper edge position and lower edge position of the mask diameter corresponding to the mask type.

[0255] As described above, when the operation is performed in normal mode, the second and third processes are performed, and the mask type is set based on the correlation of the evaluation values ​​corresponding to the corresponding multiple evaluation frames 210, and the mask diameter size corresponding to the mask type is obtained, thereby estimating the type (diameter) of the mirror body 101.

[0256] Furthermore, when the operation is performed in the high-precision calculation mode, in addition to the second and third processes, the fourth and fifth processes are performed, and based on the correlation of the evaluation values ​​corresponding to the corresponding multiple evaluation frames 210, the left edge position, right edge position, upper edge position and lower edge position in the mask diameter corresponding to the mask type are calculated, and the mask diameter size is obtained based on the position information (coordinates) of these types, thereby estimating the type (diameter) of the mirror body 101.

[0257] As described above, in normal mode and high-precision calculation mode, since the mask diameter size is obtained based on the correlation (correlation) of the evaluation values ​​of the corresponding multiple evaluation frames 210 set in the image, it is not necessary to perform the processing for detecting all straight edges in the image as disclosed in the above-mentioned Patent Document 1. Therefore, the type of mirror body 101 can be determined with a small amount of computation.

[0258] <2. Modification>

[0259] Note that in the above description, it is assumed that the first to fifth processes and the determination process are performed by the control unit 161 of the CCU 51; however, these types of processes can be performed by other processing units besides the CCU 51 in the endoscopic surgery system 1. In this case, the evaluation frame setting unit 171 to the type estimation unit 175 are located in another processing unit. Furthermore, in the evaluation frame setting unit 171 to the type estimation unit 175, some blocks can be set in the control unit 161, and other blocks can be set in other processing units.

[0260] Furthermore, in the above description, the image signal corresponding to 4K resolution is described as an image signal output from camera head 102; however, this is not a limitation, and the image signal can be an image signal corresponding to another resolution, such as 8K resolution (e.g., 7680×4320 pixels), 2K resolution (e.g., 1280×720 pixels), etc.

[0261] In the above description, mask diameter information is generated to determine the type of mirror 101, and then subsequent signal processing is performed. However, the type of mirror 101 can be determined by reading parameters corresponding to its type. For example, a table linking mask diameter information to parameters required for signal processing can be pre-stored, and the parameters for signal processing corresponding to the type of mirror 101 can be read based on the mask diameter information.

[0262] Furthermore, the above description has already described the arrangement of multiple evaluation frames 210 in both the vertical and horizontal directions; however, the arrangement is not limited to vertical and horizontal directions, and the multiple evaluation frames 210 can be arranged in any position as long as the mask diameter can be detected. Moreover, regarding the size of the evaluation frames 210, not all evaluation frames 210 need to have the same size; for example, the size of the evaluation frames 210 can vary depending on their arrangement position. Furthermore, the interval between the multiple evaluation frames 210 arranged at predetermined intervals is not limited to the same interval and can be different intervals.

[0263] Note that in the above description, for ease of description, the image corresponding to the imaging surface of the imaging element of the camera head 102 is referred to as an endoscopic image; however, while the image corresponding to the subject image focused by the endoscope body 101 can be called an endoscopic image, the image corresponding to the imaging surface of the imaging element of the camera head 102 can be called an observation image and distinguished. In this case, the observation image includes the endoscopic image, multiple evaluation frames are set for the observation image at predetermined intervals, evaluation values ​​are calculated with respect to the corresponding multiple evaluation frames, and the mask diameter size corresponding to the endoscopic image is calculated based on the relationship between the calculated evaluation values.

[0264] <3. Computer Configuration>

[0265] The series of processing steps described above (e.g., the first through fifth processes and the determination process described above) can be executed by hardware or by software. When the series of processing steps are executed by software, a program constituting the software is installed in the computer of each device. As used herein, 'computer' refers to circuitry that can be configured via the execution of computer-readable instructions, and such circuitry may include one or more local processors (e.g., CPUs), and / or one or more remote processors, such as cloud computing resources, or any combination thereof. For example, this technology can be configured in the form of cloud computing, where a function for processing is collaboratively shared across multiple devices via a network. Furthermore, this technology can be configured as a server or IP converter in a hospital, where a function for processing is collaboratively shared across multiple devices via a network. Figure 20 This is a block diagram illustrating an example of the hardware configuration of a computer that performs the above series of processing steps through a program.

[0266] In the computer, the Central Processing Unit (CPU) 1001, Read-Only Memory (ROM) 1002, and Random Access Memory (RAM) 1003 are interconnected via a bus 1004. Furthermore, an Input / Output Interface 1005 is connected to the bus 1004. The Input / Output Interface 1005 is connected to an Input Unit 1006, an Output Unit 1007, a Storage Unit 1008, a Communication Unit 1009, and a Driver 1010.

[0267] Input unit 1006 includes a microphone, keyboard, mouse, etc. Output unit 1007 includes a speaker, display, etc. Storage unit 1008 includes a hard disk, non-volatile memory, etc. Communication unit 1009 includes a network interface, etc. Driver 1010 drives removable recording media 1011 such as disk, optical disk, magneto-optical disk, or semiconductor memory.

[0268] In the computer configured as described above, the CPU 1001 loads and executes the program recorded in the ROM 1002 or storage unit 1008 into the RAM 1003 via the input / output interface 1005 and the bus 1004, thereby performing the series of processing steps described above.

[0269] For example, a program executed by a computer (CPU 1001) can be provided by being recorded on a removable recording medium 1011, such as a packaging medium. Furthermore, the program can be provided via wired or wireless transmission media, such as a local area network, the Internet, or digital satellite broadcasting.

[0270] In a computer, a program can be installed into the storage unit 1008 via the input / output interface 1005 by installing the removable recording medium 1011 into the drive 1010. Alternatively, a program can be installed into the storage unit 1008 by receiving data via a wired or wireless transmission medium from the communication unit 1009. Additionally, the program can be pre-installed into the ROM 1002 or the storage unit 1008.

[0271] In this specification, the processes executed by a computer according to a program do not necessarily have to be executed sequentially in the order described in the flowchart. In other words, the processes executed by a computer according to a program also include parallel or individual processes (e.g., parallel processing or processing performed by an object). Furthermore, the program may be processed by a single computer (processor), or it may be distributed and processed by multiple computers.

[0272] Note that the embodiments of this technology are not limited to the above embodiments, and various modifications can be made without departing from the scope of this technology.

[0273] In addition, this technology can have the following configurations. (1)

[0275] An image processing system, comprising

[0276] Control Unit

[0277] Multiple evaluation frames are set up at predetermined intervals for endoscopic images captured using the endoscope.

[0278] Calculate the evaluation values ​​for the corresponding multiple evaluation frames with respect to the settings, and

[0279] The type of lens is estimated based on the relationship between calculated evaluation values. (2)

[0281] According to the image processing system described in (1), wherein

[0282] The control unit arranges multiple evaluation frames discretely at predetermined intervals. (3)

[0284] According to the image processing system described in (1) or (2), wherein

[0285] The control unit sets a predetermined interval so that multiple evaluation frames are symmetrical with respect to the approximate centroid of the endoscopic image. (4)

[0287] The image processing system according to any one of (1) to (3), wherein

[0288] The control unit does not keep the predetermined interval constant. (5)

[0290] The image processing system according to any one of (1) to (4), wherein

[0291] The type of mirror body is determined by its diameter. (6)

[0293] According to the image processing system described in (2), wherein

[0294] Control Unit

[0295] In the case of estimating the lens type using the first mode, multiple evaluation frames are discretely arranged, and

[0296] Multiple evaluation frames are arranged consecutively while estimating the lens type using a second mode with higher accuracy than the first mode. (7)

[0298] According to the image processing system described in (6), wherein

[0299] The control unit locates multiple evaluation frames arranged continuously based on the evaluation results corresponding to the evaluation values ​​of the discretely arranged multiple evaluation frames. (8)

[0301] The image processing system according to any one of (1) to (7), wherein

[0302] The control unit estimates the type of the lens based on the difference between the evaluation values ​​corresponding to the respective evaluation frames arranged adjacent to each other or at constant intervals in a plurality of evaluation frames. (9)

[0304] The image processing system according to any one of (1) to (8), wherein

[0305] The control unit estimates the type of the lens and its central position. (10)

[0307] The image processing system according to any one of (6) to (9), wherein

[0308] The control unit makes the size of the corresponding multiple evaluation frames arranged sequentially smaller than the size of the corresponding multiple evaluation frames arranged discretely. (11)

[0310] The image processing system according to any one of (1) to (10), wherein

[0311] Control Unit

[0312] Multiple evaluation frames are arranged in each of the horizontal and vertical directions of the endoscopic image, and

[0313] For each of the horizontal and vertical directions, an evaluation value is calculated based on the feature values ​​detected from the corresponding regions of the corresponding multiple evaluation frames corresponding to the endoscopic image. (12)

[0315] According to the image processing system described in (11), wherein

[0316] The characteristic values ​​include brightness values. (13)

[0318] According to the image processing system described in (11), wherein

[0319] The feature values ​​include edge quantity or black area quantity. (14)

[0321] The image processing system according to any one of (11) to (13), wherein

[0322] The control unit arranges multiple evaluation frames approximately symmetrically about the centroid of the endoscopic image. (15)

[0324] The image processing system according to any one of (1) to (14), wherein

[0325] Control Unit

[0326] Multiple evaluation frames are arranged near the approximate centroid and vertices of the endoscopic images.

[0327] The evaluation value is calculated based on the feature values ​​detected from the regions corresponding to the multiple evaluation frames of the endoscopic image, and

[0328] The calculated evaluation value determines whether the endoscopic image includes a mask region. (16)

[0330] The image processing system according to any one of (1) to (15), wherein

[0331] Based on information corresponding to the estimated lens type, at least one of the signal processing steps is performed in the signal processing concerning autofocus (AF), auto exposure (AE), auto white balance (AWB), and extended depth of field (EDOF). (17)

[0333] The image processing system according to any one of (1) to (16), wherein

[0334] The endoscope body is configured as part of the endoscope. (18)

[0336] An image processing apparatus, comprising

[0337] Control Unit

[0338] Multiple evaluation frames are set up at predetermined intervals for endoscopic images captured using the endoscope.

[0339] Calculate the evaluation values ​​for the corresponding multiple evaluation frames with respect to the settings, and

[0340] Based on the relationship between the calculated evaluation values, signal processing corresponding to the type of endoscope is performed. (19)

[0342] According to the image processing apparatus described in (18), wherein

[0343] Control Unit

[0344] In the case of estimating the lens type using the first mode, multiple evaluation frames are discretely arranged, and

[0345] When estimating the type of the lens body using a second mode with higher accuracy than the first mode, multiple evaluation frames are continuously arranged based on the evaluation results corresponding to the evaluation values ​​of the discretely arranged multiple evaluation frames. (20)

[0347] An image processing method, comprising,

[0348] Through image processing device,

[0349] Multiple evaluation frames are set up at predetermined intervals for endoscopic images captured using the endoscope.

[0350] Calculate the evaluation values ​​for the corresponding multiple evaluation frames with respect to the settings, and

[0351] Based on the relationship between the calculated evaluation values, signal processing corresponding to the type of endoscope is performed. (twenty one)

[0353] An endoscope system comprising:

[0354] The circuit is configured as follows:

[0355] Multiple evaluation regions are set in the endoscopic image captured by the image sensor via the endoscope, and adjacent evaluation regions are spatially separated from each other.

[0356] Calculate an evaluation value for each of the multiple evaluation regions;

[0357] Compare the assessment values ​​of multiple assessment areas; and

[0358] Adjust the image processing of the endoscopic images based on the comparison results. (twenty two)

[0360] According to the endoscope system described in (21), wherein

[0361] Multiple evaluation areas are set discretely at predetermined intervals. (twenty three)

[0363] The endoscope system according to any one of (21) and (22), wherein

[0364] Multiple evaluation regions are set up such that the multiple evaluation regions are symmetrical with respect to the approximate centroid of the endoscopic image. (twenty four)

[0366] According to the endoscope system described in (22), wherein

[0367] The predetermined intervals between multiple assessment areas are not constant. (25)

[0369] The endoscope system according to any one of (21) to (24), wherein

[0370] The circuit is configured to estimate the type of endoscope based on comparison and adjust the image processing of the endoscope image based on the type of endoscope. (26)

[0372] According to the endoscope system described in (25), wherein

[0373] The circuit is configured to recognize the presence of a scope and, when a scope is present, to estimate the type of scope based on the size of a region in the endoscopic image. (27)

[0375] The endoscope system according to any one of (21) to (26), wherein

[0376] The circuit is configured as

[0377] Multiple evaluation regions are set up. In the first mode, adjacent evaluation regions in the multiple evaluation regions are spatially separated and discrete from each other.

[0378] Multiple evaluation areas are set up. In the second mode, adjacent evaluation areas in the multiple evaluation areas are spatially separated and continuous. (28)

[0380] According to the endoscope system described in (27), wherein

[0381] The circuit is configured to estimate the type of endoscope based on a comparison in a first mode, and to estimate the location of the vignetting region in the endoscopic image based on a comparison in a second mode. (29)

[0383] According to the endoscope system described in (27), wherein

[0384] Based on the comparison of the first mode, multiple evaluation areas in the second mode are set at predetermined locations. (30)

[0386] According to the endoscope system described in (27), wherein

[0387] The size of multiple evaluation regions in the second mode is smaller than the size of multiple evaluation regions in the first mode. (31)

[0389] The endoscope system according to any one of (21) to (30), wherein

[0390] The comparison is obtained from the difference between the evaluation values ​​of adjacent areas in multiple evaluation areas. (32)

[0392] The endoscope system according to any one of (21) to (31), wherein

[0393] The circuit is configured as

[0394] Multiple evaluation frames are set in each direction of the endoscopic image, both horizontally and vertically.

[0395] Compare the evaluation values ​​of multiple evaluation areas in the horizontal direction;

[0396] Compare the evaluation values ​​of multiple evaluation regions in the vertical direction; and

[0397] Based on the results of horizontal and vertical comparisons, adjust the image processing on the endoscopic images. (33)

[0399] The endoscope system according to any one of (21) to (32), wherein

[0400] The evaluation value is calculated based on the brightness value of each of the multiple evaluation areas. (34)

[0402] The endoscope system according to any one of (21) to (33), wherein

[0403] Multiple evaluation areas were set as the central region of the endoscopic image and a region substantially symmetrical about that central region. (35)

[0405] According to the endoscope system described in (34), wherein

[0406] The circuit is configured to estimate whether the lens is attached based on the evaluation value of the central region, and to estimate the type of lens based on the comparison of evaluation values ​​with respect to the substantially symmetrical region of the central region. (36)

[0408] According to any one of (21) to (35) the endoscope system, wherein

[0409] Image processing includes at least one of autofocus processing, auto exposure processing, auto white balance processing, or extended depth of field processing. (37)

[0411] The endoscope system according to any one of (21) to (36), wherein

[0412] The circuit is configured to estimate the type of lens by determining the location of the edge of the vignetting caused by the lens based on a comparison. (38)

[0414] The endoscope system according to any one of (21) to (37), wherein

[0415] The circuit is configured to read parameters from a table stored in memory based on comparisons and use the parameters to adjust image processing. (39)

[0417] A non-transitory computer-readable medium having a program stored thereon, which, when executed by a computer, causes the computer to perform processing, the processing comprising:

[0418] Multiple evaluation regions are set in the endoscopic image captured by the image sensor via the endoscope, and adjacent evaluation regions are spatially separated from each other.

[0419] Calculate an evaluation value for each of the multiple evaluation regions;

[0420] Compare the assessment values ​​of multiple assessment areas; and

[0421] Adjust the image processing of the endoscopic images based on the comparison results. (40)

[0423] One method includes:

[0424] Multiple evaluation regions are set in the endoscopic image captured by the image sensor via the endoscope, and adjacent evaluation regions are spatially separated from each other.

[0425] Calculate an evaluation value for each of the multiple evaluation regions;

[0426] Compare the assessment values ​​of multiple assessment areas; and

[0427] Adjust the image processing of the endoscopic images based on the comparison results.

[0428] Those skilled in the art will understand that, within the scope of the appended claims or their equivalents, various modifications, combinations, sub-combinations and alterations may be made based on design requirements and other factors.

[0429] [List of Reference Symbols]

[0430] 1. Endoscopic surgical system

[0431] 10. Endoscope

[0432] 20 Surgical Instruments

[0433] 30 Support arm device

[0434] 51 CCU

[0435] 52 Display devices

[0436] 53 Light source device

[0437] 54 Input Devices

[0438] 55. Handling tool control device

[0439] 56. Pneumoperitoneum device

[0440] 57 Recorder

[0441] 58 Printers

[0442] 101. Lens body

[0443] 102 camera lenses

[0444] 151 Lens Units

[0445] 152 imaging units

[0446] 153 drive units

[0447] 154 communication units

[0448] 155 camera head control unit

[0449] 161 Control Unit

[0450] 162 Communication Units

[0451] 163 Image Processing Units

[0452] 171 Evaluation Frame Setting Unit

[0453] 172 Evaluation Value Calculation Unit

[0454] 173 Evaluation Value Determination Unit

[0455] 174 Evaluation Result Setting Unit

[0456] 175 Type Estimation Units

[0457] 1001 CPU

Claims

1. An endoscope system, comprising: The circuit is configured as follows: Multiple evaluation regions are set in the endoscopic image captured by the image sensor via the endoscope, and adjacent evaluation regions among the multiple evaluation regions are spatially separated from each other; Calculate an evaluation value for each of the plurality of evaluation regions; Compare the evaluation values ​​of multiple evaluation regions; and The image processing of the endoscope image is adjusted according to the estimated type of endoscope, wherein The circuit is configured to estimate whether the lens is attached based on an evaluation value of the central region, and to estimate the type of the lens based on a comparison of evaluation values ​​of regions symmetrical about the central region.

2. The endoscope system according to claim 1, wherein Multiple evaluation regions are set discretely at predetermined intervals.

3. The endoscope system according to claim 2, wherein... The predetermined intervals between the multiple evaluation regions are not constant.

4. The endoscope system according to claim 1, wherein The plurality of evaluation regions are configured such that the plurality of evaluation regions are symmetrical with respect to the approximate centroid of the endoscopic image.

5. The endoscope system according to claim 1, wherein... The circuit is configured to estimate the type of endoscope based on the comparison and adjust the image processing of the endoscope image based on the type of endoscope.

6. The endoscope system according to claim 5, wherein The circuit is configured to recognize the presence of a scope and, when the scope is present, to estimate the type of the scope based on the size of a region in the endoscopic image.

7. The endoscope system according to claim 1, wherein... The circuit is configured as follows: Multiple evaluation regions are set up, and in the first mode, adjacent evaluation regions within the multiple evaluation regions are spatially separated and discrete from each other; and Multiple evaluation regions are set up, and in the second mode, adjacent evaluation regions in the multiple evaluation regions are spatially separated and continuous from each other.

8. The endoscope system according to claim 7, wherein The circuit is configured to estimate the type of the endoscope based on a comparison in the first mode, and to estimate the location of the vignetting region in the endoscopic image based on a comparison in the second mode.

9. The endoscope system according to claim 7, wherein Based on the comparison of the first mode, multiple evaluation areas in the second mode are set at predetermined locations.

10. The endoscope system according to claim 7, wherein The size of the plurality of evaluation regions in the second mode is smaller than the size of the plurality of evaluation regions in the first mode.

11. The endoscope system according to claim 1, wherein The comparison is obtained from the difference between the evaluation values ​​of adjacent regions of the multiple evaluation regions.

12. The endoscope system according to claim 1, wherein The circuit is configured as follows: Multiple evaluation frames are set in each of the horizontal and vertical directions of the endoscopic image; Compare the evaluation values ​​of multiple evaluation regions in the horizontal direction; Compare the evaluation values ​​of multiple evaluation regions in the vertical direction; and Based on the results of the horizontal and vertical comparisons, the image processing on the endoscopic image is adjusted.

13. The endoscope system according to claim 1, wherein... The evaluation value is calculated based on the brightness value of each of the plurality of evaluation regions.

14. The endoscope system according to claim 1, wherein The evaluation regions are set as a central region of the endoscopic image and a region symmetrical about the central region.

15. The endoscope system according to claim 1, wherein... Image processing includes at least one of autofocus processing, auto exposure processing, auto white balance processing, and extended depth of field processing.

16. The endoscope system of claim 1, wherein... The circuit is configured to estimate the type of the lens by determining the position of the edge of the vignetting caused by the lens based on the comparison.

17. The endoscope system of claim 1, wherein... The circuit is configured to read parameters from a table stored in memory based on the comparison, and to adjust the image processing using the parameters.

18. A non-transitory computer-readable medium having a program stored thereon, which, when executed by a computer, causes the computer to perform processing, the processing comprising: Multiple evaluation regions are set in the endoscopic image captured by the image sensor via the endoscope, and adjacent evaluation regions among the multiple evaluation regions are spatially separated from each other; Calculate an evaluation value for each of the plurality of evaluation regions; Compare the evaluation values ​​of multiple evaluation regions; as well as The image processing of the endoscope image is adjusted according to the estimated type of endoscope, wherein The type of the lens is estimated based on the evaluation value of the central region and the lens is estimated based on the comparison of the evaluation values ​​of the regions symmetrical about the central region.

19. A method for image processing, comprising: Multiple evaluation regions are set in the endoscopic image captured by the image sensor via the endoscope, and adjacent evaluation regions among the multiple evaluation regions are spatially separated from each other; Calculate an evaluation value for each of the plurality of evaluation regions; Compare the evaluation values ​​of multiple evaluation regions; as well as The image processing of the endoscope image is adjusted according to the estimated type of endoscope, wherein The type of the lens is estimated based on the evaluation value of the central region and the lens is estimated based on the comparison of the evaluation values ​​of the regions symmetrical about the central region.

Images