Image processing device and camera system

Abstract

To provide a camera system capable of reducing a load on a network and an image processor without causing a trouble in surgery.SOLUTION: An image processing device for processing image data from an image sensor 12 installed on an intraluminal surgery camera 1 to be mounted on a human body wall 3, includes: an area setting part 201 for setting a second area R2 smaller than a first area R1 in the first area R1 photographed by the image sensor 12; and a display control part 191 for displaying a first image G1 corresponding to the first area R1 and a second image G2 corresponding to the second area R2 in different display areas, where a frame rate of the second image G2 is larger than a frame rate of the first image G1.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a camera for intracorporeal surgery, such as laparoscopic surgery, in which a surgical operation is performed under a laparoscope. [Background technology]

[0002] In recent years, laparoscopic surgery has come to be performed in many facilities, and the proportion of laparoscopic surgery at each facility is increasing. Unlike open surgery, laparoscopic surgery is a minimally invasive surgery that performs examinations and treatments without opening the patient's abdomen. As is well known, in laparoscopic surgery, small holes of about 5 to 15 mm are made in the abdominal wall near the patient's navel and the surrounding area (perforation), a trocar (also called a trocar) is inserted (puncture), carbon dioxide gas is injected into the abdominal cavity through the pneumoperitoneum gas supply hole to inflate it (pneumoperitoneum), and treatment tools such as an endoscope (surgical camera), surgical forceps, an electric scalpel, and a suturing stapler are inserted through the trocar. An airtight valve may be installed inside the trocar attached to the abdominal wall to maintain the abdominal cavity in a pneumoperitoneum state. The surgeon performs treatment operations such as incising and suturing organs using these treatment tools while observing the affected area through the endoscope.

[0003] When starting surgery, when the surgeon and the assistant present at the surgery insert forceps or the like into the abdominal cavity, another assistant usually guides the forceps to the affected area while viewing the area near the tip of the forceps inserted into the abdominal cavity using an endoscope. Also, when the surgeon incises or sutures an organ, the endoscope is brought close to the affected organ to take an image, so the field of view becomes very narrow, and the surgeon, assistants, and others cannot grasp the state of the abdominal cavity outside the narrow working area displayed by the endoscope (such as the movement of treatment tools outside the working area). Therefore, there is a risk of serious medical accidents due to unexpected bleeding outside the field of view of the endoscope that occurs when inserting forceps into the abdominal cavity or during surgery, or forgetting to collect residual items such as gauze.

[0004] As a countermeasure to such a problem, Patent Documents 1 to 4 report a technique of photographing the entire abdominal cavity with a separate camera in addition to the endoscope that photographs the above-mentioned affected area. Furthermore, Patent Document 5 reports an imaging device that uses a camera installed on the abdominal wall to photograph the abdominal cavity, and cuts out and displays an image of the affected area from data photographed of the entire abdominal cavity. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 5131661 (issued January 30, 2013) [Patent Document 2] Patent No. 5669666 (issued February 12, 2015) [Patent Document 3] Patent No. 6703684 (issued June 3, 2020) [Patent Document 4] Patent No. 6764491 (issued September 30, 2020) [Patent Document 5] JP 2019-130005 A (Published on August 8, 2019) Summary of the Invention [Problem to be solved by the invention]

[0006] However, with the technology of Patent Document 1, when attaching an internal monitoring camera that captures images of the entire abdominal cavity separately from the endoscope, the connector part at the tip of the needle that is inserted from outside the body and has a built-in video signal line from the camera must be attached to the connector part of the camera inside the abdominal cavity using forceps or the like, and further, when retrieving the camera after a procedure such as surgery is completed, this requires an accordingly delicate and skilled operation.

[0007] Furthermore, the techniques of Patent Documents 2 and 3 simplify the camera attachment and camera retrieval in Patent Document 1 by using a dedicated introduction and retrieval tool and support tube. However, even with these techniques, when attaching an intraperitoneal camera, the work requires the use of an endoscope and lighting equipment such as an LED inserted separately from the camera in a narrow space such as the abdominal cavity that cannot be seen with the direct eye, which is a problem that requires time and effort.

[0008] On the other hand, the technology of Patent Document 4 uses an imaging device (camera) with at least four legs to capture images in almost all directions in a limited space such as the abdominal cavity. However, when the camera is dropped and installed in the abdominal cavity, it is not always possible to install it in the desired location and direction in one go because the shape of the abdominal cavity is complex. In that case, it is necessary to check and adjust (reinstall) the installation location using an endoscope or lighting equipment such as an LED inserted separately from the camera. On the other hand, when retrieving the camera, the same work using an endoscope and lighting equipment is also required. Furthermore, during a treatment such as surgery, the installed camera may get in the way of the work, or the camera may move from the desired position due to vibrations caused by the work, which creates a problem that it is necessary to reinstall it each time.

[0009] Furthermore, the following phenomenon occurs as a problem common to the technologies of Patent Documents 1 to 4. When the affected area is illuminated with a lighting device such as an LED light and the entire abdominal cavity is photographed with a separate camera in addition to the endoscope that photographs an enlarged image of the affected area, the image of the entire abdominal cavity shows a blurred white image (whiteout) due to the strong light irradiated on the affected area, making it difficult to recognize the image of the affected area and its surroundings (operative field).

[0010] On the other hand, the above-mentioned Patent Document 5 describes a technique for installing and fixing an imaging device for imaging the abdominal cavity and an image display system, in which an image of the entire abdominal cavity is displayed on a display device (first display unit 51), while an image of a part of the image (affected area) is extracted from the image and displayed on another display device (second display unit 52) ​​at the same time. However, the frame rate of the first display unit 51 and the second display unit 52 is not mentioned, and if the frame rates of both are the same, the following problems will occur. If the frame rate is reduced (for example, 10 fps), the image of the affected area will not be displayed smoothly, which will interfere with the surgery. If the frame rate is set to the same level as that of a normal video (for example, 30 fps), the amount of image data transmitted to the first display unit 51 and the second display unit 52 will be large, which will increase the load on the network and image processor.

[0011] The present invention has been made in consideration of the above problems, and has an object to provide a camera system that does not interfere with surgery and can reduce the load on the network and image processing processor. [Means for solving the problem]

[0012] In order to solve the above problems, the present invention includes the following aspects. Section 1. 1. An image processing device for processing image data from an image sensor mounted on an intracorporeal surgical camera for attachment to a human body wall, comprising: a region setting unit that sets a second region smaller than the first region within the first region captured by the image sensor; a display control unit that displays a first image corresponding to the first region and a second image corresponding to the second region in different display regions; Equipped with 13. An image processing device, comprising: a frame rate of the second image that is higher than a frame rate of the first image. Section 2. 2. The image processing device according to item 1, wherein the display control unit displays the second image larger than a portion of the first image corresponding to the second region. Section 3. 3. The image processing device according to item 1 or 2, wherein the frame rate of the first image is 1 to 24 fps. Section 4. 3. The image processing device according to claim 1 or 2, wherein the frame rate of the second image is 12 to 60 fps. Section 5. 5. The image processing device according to any one of items 1 to 4, wherein the image sensor has a memory function in each pixel. Section 6. a read control unit that controls a timing for reading out pixel signals from the pixels of the image sensor; 6. The image processing device according to item 5, wherein the read control unit makes a period for reading out pixel signals from pixels in the first region longer than a period for reading out pixel signals from pixels in the second region. Section 7. 7. The image processing device according to claim 6, wherein the read control unit differentiates a timing for reading out pixel signals from pixels in the first region from a timing for reading out pixel signals from pixels in the second region. Section 8. An image processing device according to any one of items 1 to 7, an intracorporeal surgical camera for attachment to a body wall of a human being, connected to said image processing device; A camera system comprising: Section 9. The camera system described in item 8, wherein the intracavity surgical camera is used while attached to a fixed unit, and a lens covering at least the image sensor protrudes into the body cavity so that incident light to the lens is not blocked by the fixed unit and the body wall. Section 10. Item 10. The camera system according to item 8 or 9, wherein the intracorporeal surgery camera is equipped with a wide-angle lens. Effect of the Invention

[0013] According to the present invention, it is possible to provide a camera system that can reduce the load on the network and image processor without interfering with surgery. [Brief description of the drawings]

[0014] [Figure 1] 1 is a schematic cross-sectional view of an intracorporeal surgical camera according to an embodiment of the present invention attached to an abdominal wall via a fixing unit processed into a desired shape so as to be able to photograph the inside of the abdominal cavity. [Diagram 2] 1 is a schematic front view of an intracavity surgical camera according to an embodiment of the present invention, as viewed from the tip side of a wide-angle lens. [Diagram 3] 1 is a schematic diagram of a camera system according to an embodiment of the present invention; [Figure 4] FIG. 2 is a block diagram showing the schematic configuration of a camera, an image processor, a personal computer for controlling imaging conditions, and a monitor. [Diagram 5] FIG. 2 is a schematic diagram showing images of a first region (the entire abdominal cavity) and a second region (operative field) when the inside of the abdominal cavity is photographed with an intracorporeal surgical camera according to an embodiment of the present invention. [Figure 6] 2 is a diagram for explaining a first region, a second region, and a region remaining after removing the second region from the first region. FIG. [Figure 7] 4 is a diagram showing a schematic time series of readout timings of pixel signals in a first region and a second region, and accumulation times of pixel signals in the remaining region and the second region. FIG. [Figure 8] 8 is a schematic diagram showing an image (conceptual diagram) read out when pixel signals in the first and second regions are read out, fitted in accordance with the read timing (t1, t2, ... and u1, u2, ...) of FIG. [Figure 9](a) is an example of an image captured using a normal readout method of a first region when capturing a still object behind a blackout curtain using an intracavity surgical camera according to an embodiment of the present invention, (b) is an example of an image captured of the first region at 10 fps, and (c) is an example of an image captured of a second region set within the first region, output at 30 fps and enlarged approximately three times using the electronic zoom function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Note that in the following description, each drawing is a schematic diagram, and the shape, material, size, etc. of the described configuration, the relationship between the thickness and width of each part, the thickness ratio of each part, etc. do not reflect the actual ones, but have been appropriately changed for the sake of clarity and simplification of the drawings.

[0016] 1 is a schematic cross-sectional view of an intracorporeal surgery camera (hereinafter simply referred to as camera) 1 according to this embodiment attached to an abdominal wall 3 via a fixing unit processed into a desired shape so as to be able to photograph the inside of an abdominal cavity 2. The camera 1 mainly comprises a main body 8 and a camera support tube 9, and a wide-angle lens 6, an LED lighting device 10, and an image sensor 12 are mounted on the abdominal cavity 2 side of the main body 8.

[0017] FIG. 1 shows a case where a trocar 4 and a wound edge protection device 5 are used as a fixed unit. The camera 1 is installed on the abdominal wall 3 by penetrating the trocar 4 with the main body 8 attached to it through the wound edge protection device 5 that is attached by drilling the abdominal wall 3. The tip of the trocar 4 is provided with a stopper 7 so that the camera 1 remains at the desired position, that is, so that the angle of view of the wide-angle lens 6 on the abdominal cavity 2 side of the main body 8 is not obstructed and the distance between the tip surface of the wide-angle lens 6 and the inner wall of the abdominal cavity 2 is minimized. If the main body 8 of the camera 1 and the camera support tube 9 are made of stainless steel, they can be reused by washing, sterilization, etc. By covering the tip of the main body 8 including the wide-angle lens 6 that is inserted into the abdominal cavity 2 with a transparent film, etc., management after use can be simplified. Furthermore, an LED lighting device 10 for illuminating the inside of the abdominal cavity 2 is attached to the tip of the main body 8.

[0018] The fixed unit may be an integrated trocar 4 and wound edge protection device 5. The trocar 4 may be provided with a check valve 11 to minimize leakage of carbon dioxide gas from the abdominal cavity 2, or a stabilizer (not shown) to prevent the image captured by the camera 1 from being blurred due to the movement of the abdominal cavity 2, as necessary. Furthermore, a pneumoperitoneum gas insufflation hole (not shown) for injecting carbon dioxide gas into the abdominal cavity 2 may be attached to the fixed unit. A cable connector is provided at the end of the camera support tube 9 opposite the main body 8, so that cables 13 such as signal lines and power lines can be electrically connected.

[0019] 2 is a schematic front view of the camera 1 as seen from the tip of the wide-angle lens 6. In the figure, four LED lighting devices 10 are attached at equal intervals around the image sensor 12. The number may be changed as necessary, and the shape may be other than circular, for example, a ring or approximately ring shape that surrounds the image sensor 12. As shown in FIG. 1, when the camera 1 is attached to the trocar 4, the stopper 7 provided at the tip of the trocar 4 fits between the LED lighting devices 10 attached at equal intervals, so that the camera 1 can be fixed in a desired position.

[0020] FIG. 3 is a schematic diagram of the camera system 100 according to the present embodiment, and shows a state in which the camera 1 is fixed and held by a fixing unit (trocar 4 and wound edge protection device 5) on the abdominal wall 3 of a patient 14 undergoing laparoscopic surgery, and the entire inside of the abdominal cavity 2 is illuminated and photographed by an LED lighting device 10 attached to the camera 1. The camera system 100 mainly includes the camera 1, an image processing processor 19, a computer 20 for controlling imaging conditions, and monitors 21 and 22. In FIG. 3, two forceps 17 and 18 are inserted into the abdominal cavity 2 through a perforation made at another position in the abdominal wall 3 via trocars 15 and 16. Image data from the image sensor 12 of the camera 1 is transmitted to the image processing processor 19 via a cable 13. Furthermore, the image processing processor 19 is electrically connected to the computer 20 for controlling imaging conditions and the monitors 21 and 22 for viewing the camera images.

[0021] 4 is a block diagram showing the schematic configuration of the camera 1, the image processor 19, the imaging condition control computer 20, and the monitors 21 and 22. The image processor 19 has, as its functional blocks, a display control section 191 and a readout control section 192. The imaging condition control computer 20 has, as its functional blocks, a region setting section 201 and a frame rate setting section 202. The image processor 19 and the imaging condition control computer 20 correspond to the image processing device recited in the claims, and may be configured as a single computer. At least some of the functions of the image processor 19 and the imaging condition control computer 20 may be incorporated in the camera 1.

[0022] Image data of a first region R1 captured by the image sensor 12 mounted on the camera 1 is transmitted to the image processor 19 from the image sensor 12. The first region R1 is a wide region within the abdominal cavity 2 captured using almost all pixels of the image sensor 12. The region setting unit 201 sets a second region R2 smaller than the first region R1 within the first region R1 in response to a user's operation. The second region R2 is a narrow region within the abdominal cavity 2 captured using some of the pixels of the image sensor 12, and in this embodiment, is a surgical field including an affected area 23 where forceps, an electric scalpel, etc. are concentrated.

[0023] The display control unit 191 displays the first image corresponding to the first region R1 and the second image corresponding to the second region R2 in different display regions. In this embodiment, the display control unit 191 displays the first image on the monitor 21 and the second image on the monitor 22. The monitor 21 is mainly for viewing by the surgical assistant, and the monitor 22 is for viewing by the surgeon and the surgical assistant. The first image and the second image may be displayed in different display regions of the same monitor.

[0024] 3 also shows organs 24 with affected area 23 being operated on in abdominal cavity 2, bleeding area 25 that occurs when forceps, an electric scalpel, or the like is inserted into abdominal cavity 2, and gauze 26 as residue after surgery. It should be noted here that monitor 22 viewed by the surgeon shows a second image of second region R2 including affected area 23, but does not show images of bleeding area 25 and gauze 26 that occur outside the surgical field including affected area 23. However, the first image corresponding to first region R1, which is a wide area in the abdominal cavity, also shows these residues (bleeding area 25 and gauze 26), and a surgical assistant or the like viewing monitor 21 on which the first image is displayed can recognize the presence of the residues.

[0025] 5 is a schematic diagram of a first image G1 and a second image G2 corresponding to a first region R1 and a second region R2 when the inside of the abdominal cavity 2 is photographed by the camera 1 of this embodiment. FIG. 5(a) is an image displayed on a monitor (not shown) of the imaging condition control computer 20, and the monitor displays the first region R1. The first region R1 includes the affected area 23, as well as a bleeding area 25 of an organ 24 and gauze 26. The region setting unit 201 of the imaging condition control computer 20 sets a second region R2, indicated by a dashed frame, within the first region R1. The second region R2 is a surgical field that mainly includes the affected area 23.

[0026] Fig. 5(b) shows the first image G1 displayed on the monitor 21, and Fig. 5(c) shows the second image G2 displayed on the monitor 22. In order to allow the surgeon to more easily view the affected area 23, the display control unit 191 of the image processing processor 19 uses electronic zoom to display the second image G2 larger (approximately three times larger) than the portion of the first image G1 corresponding to the second region R2.

[0027] If the viewing angle of the wide-angle lens 6 mounted on the camera 1 is 100° and the distance from the wide-angle lens 6 to the organs 24 in the abdominal cavity 2 is approximately 150 mm, the first region R1 will be approximately 300 mm x 170 mm when the aspect ratio is 16:9, and almost the entire area inside the abdominal cavity 2 can be photographed.

[0028] The surgeon performs procedures such as surgery while mainly looking at the screen of the monitor 22 on which the second image G2 (FIG. 5(c)) is displayed. Other than the surgeon, such as surgical assistants, provide appropriate support to ensure that the surgeon's processing work proceeds smoothly while periodically looking at the first image G1 and the second image G2 displayed on the monitors 21 and 22. In addition, when the surgeon issues instructions to change the position and size of the second region R2 and to enlarge (zoom) the image, the region setting unit 201 of the imaging condition control computer 20 can change the position and size or enlarge the image according to the surgeon's instructions.

[0029] If an abnormality is found while the surgeon is performing a treatment, for example, if a bleeding area 25 as shown in FIG. 5(a) is found in a place (not shown on the monitor 22 on which the second image G2 is displayed) away from the affected area 23 and the vicinity of the affected area (operating field) during surgery, a surgical assistant other than the surgeon, who is mainly watching the monitor 21 displaying the first image G1, can immediately notice the abnormality and take appropriate measures such as stopping the bleeding. In addition, after the treatment is completed, if the surgeon checks the entire inside of the abdominal cavity 2 on the monitor 21 and finds, for example, gauze 26 as shown in FIG. 5(a), he or she can immediately take appropriate measures such as removing it using forceps. If images during the treatment are recorded, they can be used to confirm the contents of the treatment or to investigate the cause of the abnormality.

[0030] (Method of image signal processing) As shown in FIG. 6, the first region R1 is a wide region (region ABCD) captured using almost all pixels of the image sensor 12 mounted on the camera 1, the second region R2 is a narrow region (region EFGH) captured using some of the pixels of the image sensor 12, and region R1' is the remaining region obtained by removing the second region R2 from the first region R1 (shown as a solid region in FIG. 6).

[0031] In this embodiment, the image sensor 12 is a global shutter compatible CMOS image sensor having a memory function in each pixel, and the read control unit 192 of the image processor 19 controls the timing of reading image data from the pixels of the image sensor 12. Specifically, the read control unit 192 makes the period of reading image data from the pixels corresponding to the first region R1 longer than the period of reading image data from the pixels corresponding to the second region R2. In addition, the read control unit 192 makes the timing of reading image data from the pixels corresponding to the first region R1 different from the timing of reading image data from the pixels corresponding to the second region R2. An example of such control will be described with reference to Figs. 7 and 8.

[0032] 7 is a diagram showing a time series of pixel signal readout timings for the first region R1 and the second region R2, and the accumulation times for the remaining regions R1' and the second region R2. As an example, it is assumed that the readout speeds (frame rates) for the pixel signals for the first region R1 and the second region R2 are 10 fps and 30 fps, respectively, and that while image data for the first region R1 is read out once, the second region R2 is read out three times. In addition, the circles in the diagram represent pixel signals (charges) read out from pixels in the region R1', and the black circles represent pixel signals read out from pixels in the second region R2.

[0033] At time t1, pixel signals for one frame are read out from the pixels (photodiodes) in the first region R1, and the read pixel signals are output as pixel data until t2, which is 1 / 10th of a second later. Until this output is completed at time t2, the pixels in the first region R1 start accumulating pixel signals for the next frame. Next, at time t2, the pixel signals of the pixels in the first region R1 that have been accumulated between times t1 and t2 are read out, and output as pixel data until t3, which is 1 / 10th of a second later, and until this output is completed at time t3, pixel signals are accumulated in the pixels in the first region R1. This is repeated. The pixel signals accumulated in 1 / 10th of a second are equivalent to the pixel capacitance (represented by 30 circles in the figure) in the region R1', but as will be described later, in the second region R2, they are 1 / 6 of the capacitance that is originally accumulated in the pixel (represented by 5 circles in the figure). That is, the pixel signals read out from the pixels in the first region R1 at times t1, t2, . . . are represented by 30 circles and 5 black circles in the drawing.

[0034] On the other hand, as for the timing of reading out pixel signals from the second region R2 and outputting pixel data, as shown in FIG. 7, pixel signals for one frame are read out from the pixels of the second region R2 at time u1, 1 / 60 seconds after time t1, and the pixel signals read out are output as pixel data until time u2, 1 / 30 seconds after time t1. Accumulation of pixel signals for the next frame begins in the pixels of the second region R2 until time u2, when this output is completed. Next, at time u2, the charges of the pixels of the second region R2 accumulated between times u1 and u2 are read out and output as pixel data until time u3, 1 / 30 seconds after time t1, and charges are accumulated in the pixels of the second region R2 until time u3, when this output is completed. This is repeated at times u3, u4, ....

[0035] The amount of pixel signals read out from the second region R2 when the image signal of the first region R1 is read out (times t1, t2, ...) is 1 / 6 of the pixel signals accumulated during 1 / 10 seconds because the accumulation time is, for example, 1 / 60 seconds from time u3 to t2. Therefore, the amount of pixel signals read out from the second region R2 when the image signal of the first region R1 is read out is half the amount of pixel signals read out when the image signal of only the second region R2 is read out (times u1, u2, ...) (accumulation time 1 / 30 seconds), and the luminance of the second region R2 is reduced to 1 / 6 of the surroundings (region R1'). This is because, for example, the charge amount accumulated in the pixels of the region R1' at time t2 is photoelectrically converted during 1 / 10 seconds between time t1 and time t2, but the charge has already been read out three times from the pixels of the second region R2 at times u1, u2, and u3 every 1 / 30 seconds between time t1 and time t2, so the charge amount accumulated in the pixels of the second region R2 at time t2 is photoelectrically converted during 1 / 60 seconds after time u3. However, in reality, since the surgeon performing laparoscopic surgery enlarges the image (operative field) of the second region R2 and views it, it is not a practical problem even if the image of the second region R2 in the image of the first region R1 (the entire abdominal cavity) is a little dark. However, if the reduction in brightness of the image of the second region R2 is a concern, an operation to increase the gain may be performed when outputting image data of the first region R1.

[0036] The amount of image signal read out at the time of pixel signal readout of the second region R2 immediately after time t2 (time u4) is also half of the pixel signal read out at time u3, because the charge accumulated in the pixels of the second region R2 at time u4 is photoelectrically converted during 1 / 60 second between time t2 and time u4 (the charge accumulated in the pixels of the second region R2 photoelectrically converted during 1 / 60 second between time u3 and time t2 has already been read out as the signal of the first region R1 at time t2). However, the number of frames in the second region R2 where the luminance is halved is 1 / 3 of the total frames, so there is no effect on how the surgeon sees it.

[0037] In FIG. 7, the image signal readout and data output operations of the second region R2 are followed in chronological order as follows. (1) Pixel signal (7) readout: Starts at time t1 (2) Pixel signal (3) readout: Starts at time u1 (1 / 60 seconds after time t1) (3) Output of pixel data read out in (2): Completed at time u2 (4) Pixel signal (5) readout: Starts at time u2 (1 / 30 seconds after time u1) (5) Output of pixel data read out in (4): Completed at time u3 (6) Pixel signal (8) readout: Starts at time u3 (1 / 30 seconds after time u2) (7) Output of pixel data read out in (1): Completed at time t2 (8) Output of pixel data read out in (6): Completed at time u4

[0038] If the image sensor 12 mounted on the camera 1 is an image sensor having a memory function within the pixel (for example, a CMOS image sensor compatible with a global shutter), the pixel signals read from the pixels of the first region R1 at time t1 are temporarily transferred to the memory, and the pixel data is output within 1 / 10 seconds from time t1. Meanwhile, the pixel signals read from the pixels of the second region R2 at times u1, u2, and u3 every 1 / 30 seconds ((2), (4), and (6) above) are also similarly transferred to the memory, and image data is output each in 1 / 30 seconds ((3), (5), and (8) above).

[0039] FIG. 8 is a schematic diagram showing images (conceptual diagrams) read out when pixel signals of the first region R1 and the second region R2 are read out, in accordance with the read timings (t1, t2, ..., and u1, u2, ...) of FIG. 7. In FIG. 7 and FIG. 8, the frame rates of the first region R1 and the second region R2 are set to 10 fps and 30 fps, respectively, but are not necessarily limited thereto. It is sufficient that the frame rate of the image of the second region R2 is higher than the frame rate of the image of the first region R1, and that the image data of the second region R2 is read out and output at least once during the period in which the image data of the first region R1 is read out and output. That is, the readout control unit 192 may set the period for reading out pixel signals from the pixels of the first region R1 to be longer than the period for reading out pixel signals from the pixels of the second region R2. As a result, since the frame rate of the image of the first region R1 is low, the load on the network and image processor 19 can be reduced compared to the case where the frame rate of each image of the first region R1 and the second region R2 are both 30 fps. Also, since the frame rate of the image of the second region R2 is high, the surgery is not hindered. Therefore, the load on the network and image processor can be reduced without hindering the surgery.

[0040] In the first region R1, the frame rate of the image of the first region R1 is preferably 1 to 24 fps since it is only necessary to check the presence or absence of bleeding or residue in the region other than the second region R2. On the other hand, since the second region R2 includes an affected area, the frame rate of the image of the second region R2 is preferably 12 to 60 fps. If possible, it is preferable that the readout control unit 192 reads out pixel signals from the pixels of the first region R1 at different timings from the timing of reading out pixel signals from the pixels of the second region R2. In other words, it is preferable that the readout control unit 192 is set so that the image data of the first region R1 and the second region R2 are not output simultaneously.

[0041] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention. EXAMPLES

[0042] Hereinafter, an embodiment of the present invention will be described, but since the purpose here is to provide an example of imaging when image data is read out from the image sensor of the present technology and output, it should be noted that the subject is not an example of imaging in an actual laparoscopic surgery, but a still life of fruit arranged in a space surrounded by a blackout curtain. For convenience of explanation, the same reference numerals are used for components having the same functions as those described in the above embodiment, and their explanations are omitted.

[0043] The camera used in this embodiment has the same configuration as the camera 1 shown in FIG. 1. The camera module incorporated in the main body 8 of the camera 1 is equipped with a 2 / 3-type 5M CMOS sensor compatible with a global shutter, and an ultra-compact LED lighting device 10 and an image sensor 12 are attached to the abdominal cavity 2 side of the main body 8. The distance from the image sensor 12 to the still object, which is the subject, is about 180 mm. The camera module also incorporates an optical image stabilizer (OIS), and is equipped with a wide-angle lens 6 with a viewing angle of 86°. The signal extracted from the camera module is transmitted to a computer 20 for controlling imaging conditions and two monitors 21 and 22 via an image processor 19 shown in FIG. 3, so that the image of the subject can be confirmed.

[0044] The images captured in this embodiment are shown in Fig. 9. Fig. 9(a) is an example of an image captured by a normal readout method of the wide first region R1 so that the image will be approximately 280mm x 160mm when displayed on a monitor with an aspect ratio of 16:9. Fig. 9(b) is an example of an image captured by capturing the wide first region R1 at 10fps with an aspect ratio of 16:9 and 1,920 horizontal pixels, and Fig. 9(c) is an example of an image captured by capturing the narrow second region R2 set within the wide first region R1 at 30fps with an aspect ratio of 16:9 and 640 horizontal pixels, and then enlarged approximately 3 times using the electronic zoom function.

[0045] In the imaging example of the first region R1 in Figure 9(b), the image of the region surrounded by the white dashed line (second region R2) is darker than the surrounding area (region R1') because, as mentioned above, the amount of pixel signals read out from the pixels in the second region R2 is only 1 / 6 of that in region R1'. However, since the image of the second region R2 can be viewed on the other monitor, this is not a problem in practice, and if necessary, the gain can be increased when outputting image data for the first region R1.

[0046] In laparoscopic surgery, as shown in FIG. 3, the abdominal wall 3 may vibrate due to the movement of treatment tools such as forceps 17, 18, an electric scalpel, and a suture stapler inserted into the abdominal cavity 2 via trocars 15, 16, and this may cause blurring of the image captured by the camera 1. The effect of blurring is particularly large in enlarged images. In this embodiment, an optical image stabilizer (OIS) is incorporated in the camera module, so that the effect of the vibration can be suppressed. A stabilizer (not shown) for suppressing the blurring may be installed, and further, a device may be provided to suspend the fixing unit that fixes the camera 1 from above, so that the camera 1 can be stably installed.

[0047] [Additional Notes] In the embodiment described above, the imaging condition control computer 20 is electrically connected, but instead, a tablet or smartphone that can be operated with a touch panel can be connected wirelessly to remotely change the position and size of the second area R2 and perform electronic zoom operation. [Industrial Applicability]

[0048] In the above embodiment, the conventional laparoscopic surgery performed by injecting carbon dioxide gas into the abdominal cavity is taken as an example, but the present invention is not limited to the above embodiment, and can be applied to cameras for thoracoscope surgery other than laparoscopic surgery, and by changing the fixing unit, can also be applied to medical cameras such as colposcopes used in cervical cancer examinations and head-mounted surgical cameras, and even professional wearable cameras used at construction sites, etc. For example, if the fixing unit used to attach the camera is changed so that it can be used as a professional wearable camera, not only can it be installed on a body wall, but it can also be used to view not only a wide area of ​​the site, but also a narrow area of ​​interest, and can be enlarged by electronic zoom remotely, allowing the work to be carried out smoothly while grasping the surrounding situation that is often overlooked. [Explanation of symbols]

[0049] 1: Intrabody surgery camera 2: Abdominal cavity 3: Abdominal wall 4, 15, 16: Trocca 5: Wound edge protection device 6: Wide-angle lens 7: Stopper 8: Main body 9: Camera support tube 10:LED lighting device 11: Check valve 12: Image sensor 13: Cable 14:Patient 17, 18: Forceps 19: Image processor 191: Display control unit 192: Read control unit 20: Computer for controlling imaging conditions 201: Area setting section 202: Frame rate setting section 21, 22: Monitor 23: Affected area 24: Organs 25: Bleeding area 26: Gauze 100: Camera system

Claims

1. An image processing device that processes image data from an image sensor mounted on a camera, a region setting unit that sets a second region smaller than the first region within a first region captured by the image sensor; a display control unit that displays a first image corresponding to the first region and a second image corresponding to the second region in different display regions; Equipped with a frame rate of the second image is greater than a frame rate of the first image; The image sensor has a memory function in each pixel, The image processing device includes: a read control unit that controls a timing for reading out pixel signals from the pixels of the image sensor; The image processing device, characterized in that the read control unit makes the period for reading pixel signals from the pixels of the first region longer than the period for reading pixel signals from the pixels of the second region, and makes the timing for reading pixel signals from the pixels of the first region different from the timing for reading pixel signals from the pixels of the second region.

2. The image processing device according to claim 1 , wherein the display control unit displays the second image larger than a portion of the first image that corresponds to the second region.

3. The image processing device according to claim 1 , wherein the frame rate of the first image is 1 to 24 fps.

4. The image processing device according to claim 1 , wherein the frame rate of the second image is 12 to 60 fps.

5. The image processing device according to claim 1 , wherein the camera is an intracorporeal surgical camera for attachment to a body wall of a human.

6. An image processing device according to any one of claims 1 to 5, a camera connected to the image processing device.

Images