Medical control system, image processing server, image conversion device, and control method

Abstract

The present disclosure pertains to a medical control system, an image processing server, an image conversion device, and a control method wherein the time period in which the display of medical information, etc., disappears in association with switching of an image transmission path can be reduced. An image conversion device on a transmission side transmits an image for transmission to a network. The image processing server subjects the image for transmission to image processing, and transmits the image generated through image processing to the network. An image conversion device on a reception side outputs, to a display device, the image for transmission and a display image converted from the image generated through image processing. Furthermore, a control delay time based on a difference between a delay time of a first transmission path that does not pass through the image processing server and a delay time of a second transmission path that passes through the image processing server is determined on the basis of characteristics of the display device, and the timing for outputting images to the display device is controlled. The present technology can be applied to, for example, a medical image transmission system.

Description

Technical Field

[0001] The present disclosure relates to a medical control system, an image processing server, an image conversion device, and a control method, and more particularly to a medical control system, an image processing server, an image conversion device, and a control method that can reduce the period during which medical information and the like are not displayed when switching the image transmission path.

Background Art

[0002] In recent years, various medical devices have been connected network-wise, and a system has been proposed that performs a more efficient operation by integrating information from multiple medical devices and displaying it on a monitor. For example, it is assumed that various medical devices and a monitor are connected via an IP network using an IP (Internet Protocol) converter or the like.

[0003] At this time, devices used during diagnosis and surgery, such as endoscopes and microscopes, need to display images in real time so as not to deviate from the operator's sense, and it is required to realize the process from imaging to display with low latency. It is expected that this will enable a more efficient operation.

[0004] For example, in the medical video device disclosed in Patent Document 1, a configuration is proposed in which an IP converter that performs IP transmission of surgical video in an endoscope system or the like has an image processing function.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] By the way, conventionally, when switching the transmission path of an image via an IP network, frame rate resynchronization may be executed on the monitor side, and as a result, there may be a long period during which medical information and the like are not displayed on the monitor.

[0007] The present disclosure has been made in view of such a situation, and is intended to reduce the period during which medical information and the like are not displayed when switching the transmission path of an image.

Means for Solving the Problem

[0008] A medical control system according to an aspect of the present disclosure includes a transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, an image processing server that performs image processing on the image transmitted via the network from the transmission-side image conversion device and transmits the image generated by the image processing to the network, and a reception-side image conversion device that receives the transmission image and the image generated by the image processing via the network, converts the transmission image and the image generated by the image processing into a display image based on the transmission image and the image generated by the image processing, and outputs the display image to a display device. A control delay time based on a difference between a delay time of a first transmission path not passing through the image processing server and a delay time of a second transmission path passing through the image processing server which is an integral multiple of the reciprocal of the frame rate of the display device is obtained based on the characteristics of the display device, and includes a delay time control unit that controls the timing of outputting an image to the display device.

[0009] An image processing server according to an aspect of the present disclosure is an image processing server that constitutes a medical control system together with a transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, and a reception-side image conversion device that receives the transmission image and an image generated by image processing via the network, converts the transmission image and the image generated by image processing into a display image based on the transmission image and the image generated by image processing, and outputs the display image to a display device. The image processing server performs image processing on an image transmitted from the transmission-side image conversion device via the network, transmits the image generated by the image processing to the network, and a control delay time based on a difference between a delay time of a first transmission path not passing through the image processing server and a delay time of a second transmission path passing through the image processing server which is an integral multiple of the reciprocal of the frame rate of the display device obtains the control delay time based on characteristics of the display device, and has a delay time control unit that controls a timing of outputting an image to the display device.

[0010] A reception-side image conversion device according to an aspect of the present disclosure is a reception-side image conversion device that constitutes a medical control system together with a transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, and an image processing server that performs image processing on an image transmitted from the transmission-side image conversion device via the network and transmits the image generated by the image processing to the network. The reception-side image conversion device receives the transmission image and the image generated by image processing via the network, converts the transmission image and the image generated by image processing into a display image based on the transmission image and the image generated by image processing, and outputs the display image to a display device. The reception-side image conversion device obtains a control delay time based on a difference between a delay time of a first transmission path not passing through the image processing server and a delay time of a second transmission path passing through the image processing server which is an integral multiple of the reciprocal of the frame rate of the display device obtains the control delay time based on characteristics of the display device, and has a delay time control unit that controls a timing of outputting an image to the display device.

[0011] The control method according to one aspect of the present disclosure converts an image captured by a medical device into a transmission image for transmission via a network, and transmits the image to the network. The control method includes a transmission-side image conversion device, an image processing server, and a reception-side image conversion device. The transmission-side image conversion device converts an image captured by a medical device into a transmission image for transmission via a network, and transmits the image to the network. The image processing server performs image processing on the image transmitted via the network from the transmission-side image conversion device, and transmits the image generated by the image processing to the network. The reception-side image conversion device receives the transmission image and the image generated by the image processing via the network, converts the received images into a display image based on the transmission image and the image generated by the image processing, and outputs the display image to a display device. A control delay time based on a difference between a delay time of a first transmission path not passing through the image processing server and a delay time of a second transmission path passing through the image processing server is obtained based on characteristics of the display device, and timing for outputting an image to the display device is controlled. which is an integral multiple of the reciprocal of the frame rate of the display device is obtained based on the characteristics of the display device, and includes controlling the timing for outputting an image to the display device.

[0012] In one aspect of the present disclosure, a medical control system includes a transmission-side image conversion device, an image processing server, and a reception-side image conversion device. The transmission-side image conversion device converts an image captured by a medical device into a transmission image for transmission via a network, and transmits the image to the network. The image processing server performs image processing on the image transmitted via the network from the transmission-side image conversion device, and transmits the image generated by the image processing to the network. The reception-side image conversion device receives the transmission image and the image generated by the image processing via the network, converts the received images into a display image based on the transmission image and the image generated by the image processing, and outputs the display image to a display device. A control delay time based on a difference between a delay time of a first transmission path not passing through the image processing server and a delay time of a second transmission path passing through the image processing server which is an integral multiple of the reciprocal of the frame rate of the display device is obtained based on the characteristics of the display device, and the timing for outputting an image to the display device is controlled.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0014] Hereinafter, specific embodiments to which this technology is applied will be described in detail with reference to the drawings.

[0015] <Configuration Example of Medical Image Transmission System> FIG. 1 is a block diagram showing a configuration example of an embodiment of a medical image transmission system to which this technology is applied.

[0016] As shown in FIG. 1, the medical image transmission system 11 includes M medical imaging devices 21-1 to 21-M, M transmission devices 22-1 to 22-M, an IP switch 23, an image processing server 24, N reception devices 25-1 to 25-N, and N image reception devices 26-1 to 26-N.

[0017] The medical imaging devices 21-1 to 21-M are devices that capture images used for medical purposes, such as endoscopes, surgical microscopes, X-ray imaging devices, and operating room cameras, and supply the images to the corresponding transmission devices 22-1 to 22-M. Hereinafter, when it is not necessary to distinguish the medical imaging devices 21-1 to 21-M, they are simply referred to as medical imaging devices 21.

[0018] The transmission devices 22-1 to 22-M perform IP conversion on the images supplied from the corresponding medical imaging devices 21-1 to 21-M and transmit them via the IP switch 23. Hereinafter, when it is not necessary to distinguish the transmission devices 22-1 to 22-M, they are simply referred to as transmission devices 22. Further, as shown in the figure, the transmission device 22 includes an IP converter 31 for performing IP conversion on the image supplied from the medical imaging device 21 into a transmission image for transmission via the IP network.

[0019] The IP switch 23 switches the image transmission path between the transmission devices 22-1 to 22-M and the reception devices 25-1 to 25-N, and the image transmission path via the image processing server 24. For example, the IP switch 23 can switch between a transmission path (first transmission path) for transmitting an image from the transmission device 22 to the reception device 25 without passing through the image processing server 24 and a transmission path (second transmission path) for transmitting an image from the transmission device 22 to the reception device 25 via the image processing server 24.

[0020] The image processing server 24 is connected to the IP switch 23 via the IP network, performs various necessary image processes on the images captured by the medical imaging device 21 supplied via the IP switch 23, and supplies them to the IP switch 23.

[0021] The receiving devices 25-1 to 25-N receive the IP-converted images supplied via the IP switch 23, reconstruct them into the original images (display images), and supply them to the corresponding image receiving devices 26-1 to 26-N, respectively. Hereinafter, when it is not necessary to distinguish the receiving devices 25-1 to 25-N, they are simply referred to as the receiving device 25. Further, as shown in the figure, the receiving device 25 includes an IP converter 41 for converting the image IP-converted as the transmission image into the original image.

[0022] The image receiving devices 26-1 to 26-N receive the images supplied from the corresponding receiving devices 25-1 to 25-N, respectively. For example, the image receiving devices 26-1 to 26-N are displays that display images at a predetermined frame rate (e.g., 60 Hz), and can receive and display the images captured by the medical imaging device 21. Hereinafter, when it is not necessary to distinguish the image receiving devices 26-1 to 26-N, they are simply referred to as the image receiving device 26.

[0023] Here, between the receiving device 25 and the image receiving device 26, they are connected according to a standard such as HDMI (registered trademark) (High-Definition Multimedia Interface) or Displayport, and can transmit and receive EDID (Extended Display Identification Data), which is device-specific identification data. Thereby, the image receiving device 26 can notify the receiving device 25 of the frame rate, for example.

[0024] In the medical image transmission system 11 configured as described above, the time from when the image captured by the medical imaging device 21 is input to the transmission device 22 until it is output from the receiving device 25 to the image receiving device 26 is the delay time. And in the medical image transmission system 11, depending on the time required for image processing in the image processing server 24, the delay time in the transmission path without passing through the image processing server 24 and the delay time in the transmission path passing through the image processing server 24 are different.

[0025] For example, in the medical image transmission system 11, while performing dynamic (real-time) image processing in the image processing server 24, IP converter 31, IP converter 41, etc., the image is transmitted to the image receiving device 26 with low latency. At this time, at the request of the surgeon or the like, information of other external devices may be superimposed and displayed, or switched to another video, and it is preferable to switch quickly so as not to interfere with the procedure. However, when the timing of the frame rate of the image receiving device 26 and the timing of the image signal received by the image receiving device 26 are misaligned, the image receiving device 26 needs to execute a resynchronization process to resynchronize the frame rate according to the image signal. For example, since the resynchronization process may take about several seconds to several tens of seconds, there is a concern that a non-display period will occur during which medical information cannot be displayed on the image receiving device 26 while the resynchronization process is being executed.

[0026] Therefore, the medical image transmission system 11 can execute a delay time control process that controls the delay time so that the timing at which the image is output from the receiving device 25 matches the timing of the frame rate of the image receiving device 26 even when the transmission path of the image is switched. As a result, the medical image transmission system 11 can reduce (such as avoiding or shortening the non-display period) the occurrence of the non-display period associated with the resynchronization process because the image receiving device 26 does not need to execute the resynchronization process.

[0027] Here, with reference to FIGS. 2 to 6, in the medical image transmission system 11, the delay time generated in the image transmitted via the IP network and the processing when switching the transmission path will be described.

[0028] FIG. 2 is a diagram for explaining the delay time in the transmission path in which the transmission device 22 and the reception device 25 are connected via the IP network without passing through the image processing server 24.

[0029] In addition to the IP converter 31, the transmission device 22 includes an interface 32 and an encoder 33. The interface 32 acquires, for example, an image transmitted from the medical imaging device 21 according to the SDI (Serial Digital Interface) standard and supplies it to the encoder 33, and the encoder 33 encodes the image and supplies it to the IP converter 31.

[0030] In addition to the IP converter 41, the reception device 25 includes a decoder 42 and an interface 43. The decoder 42 decodes the image supplied from the IP converter 41 and supplies it to the interface 43, and the SDI 43 transmits the image to the image reception device 26 according to the SDI standard, for example.

[0031] Then, as shown in FIG. 2, in the transmission path in which the transmission device 22 and the reception device 25 are connected via the IP network without passing through the image processing server 24, the delay time until the image input to the transmission device 22 is output from the reception device 25 is defined as A [msec].

[0032] FIG. 3 is a diagram for explaining the delay time in the transmission path via the image processing server 24 when the delay time control process is not performed. Note that the transmission device 22 and the reception device 25 are the same as the configuration example described with reference to FIG. 2.

[0033] The image processing server 24 includes IP converters 51-1 and 51-2, a decoder 52, a frame buffer 53, an image processing unit 54, a frame buffer 55, and an encoder 56. In the image processing server 24, the IP converters 51-1 and 51-2, the decoder 52, the frame buffer 53, the frame buffer 55, and the encoder 56 surrounded by the dashed line are network interfaces.

[0034] The IP converter 51-1 receives the IP-converted image transmitted from the transmission device 22 via the IP network, reconstructs it into the original image, and supplies it to the decoder 52. The decoder 52 decodes the image supplied from the IP converter 51-1 and supplies it to the frame buffer 53, and the frame buffer 53 temporarily stores the image.

[0035] The image processing unit 54 is constituted by, for example, a GPU (Graphics Processing Unit), reads out the image stored in the frame buffer 53, performs necessary image processing on the image, and supplies the image after the image processing to the frame buffer 55.

[0036] The frame buffer 55 temporarily stores the image on which the image processing has been performed in the image processing unit 54, and the encoder 56 encodes the image read from the frame buffer 55 and supplies it to the IP converter 51-2. The IP converter 51-2 IP-converts the image and transmits it to the receiving device 25 via the IP network.

[0037] And, as shown in FIG. 3, in the transmission path via the image processing server 24, the delay time from when the image input to the transmission device 22 is output from the receiving device 25 is the delay time A [msec] described with reference to FIG. 2 plus the image processing delay time C [msec] until the image input to the image processing server 24 is output, resulting in a total delay time of A + C [msec].

[0038] Here, when the image processing delay time C [msec] is not an integer multiple of the reciprocal of the frame rate of the image receiving device 26, the image receiving device 26 executes a resynchronization process to resynchronize the frame rate in accordance with the timing at which an image is output from the receiving device 25.

[0039] For example, referring to FIG. 4, when the frame rate of the image receiving device 26 is 60 Hz, the process for switching the transmission path when the delay time control process is not performed will be described.

[0040] When the frame rate of the image receiving device 26 is 60 Hz, the image receiving device 26 displays images at intervals of 16.6 msec, which is the reciprocal thereof. In the following description, the reciprocal of the frame rate is referred to as the image display interval.

[0041] First, when an image transmitted through a transmission path with a delay time A [msec] without passing through the image processing server 24 is output from the receiving device 25, the image receiving device 26 synchronizes the frame rate in accordance with the output timing of the delay time A [msec]. As a result, the image receiving device 26 can display the image output from the receiving device 25 at an image display interval of 16.6 msec.

[0042] Thereafter, when the transmission path is switched to the one passing through the image processing server 24, as a result, the total delay time becomes A + C [msec], and thus the output timing at which an image is output from the receiving device 25 will deviate from the frame rate of the image receiving device 26.

[0043] Therefore, the image receiving device 26 can display the image output from the receiving device 25 at an image display interval of 16.6 msec by performing a resynchronization process to resynchronize the frame rate in accordance with the output timing of the total delay time A + C [msec].

[0044] FIG. 5 is a diagram for explaining the delay time in the transmission path via the image processing server 24 when the delay time control process is performed. Note that the transmission device 22, the image processing server 24, and the reception device 25 are the same as the configuration examples described with reference to FIG. 3.

[0045] As shown in FIG. 5, in the medical image transmission system 11, the delay time control process is performed such that the delay time until the image input to the image processing server 24 is output is an integer multiple of the image display interval B, which is the reciprocal of the frame rate of the image reception device 26. As a result, the image input to the image processing server 24 is output at the timing when the control delay time B×n [msec] (n = integer) has elapsed.

[0046] For example, in the image processing server 24, the control delay time can be set by adjusting the time for temporarily storing the image in the frame buffer 53 or 55. Note that the time required for image processing in the image processing server 24 does not change compared to the case where the delay time control process is not performed. Therefore, the control delay time B×n [msec] is equal to or greater than the image processing delay time C [msec] (i.e., B×n = C + α).

[0047] Therefore, in the transmission path via the image processing server 24, the delay time from when the image input to the transmission device 22 is output from the reception device 25 is the control total delay time A+(B×n) [msec], which is obtained by adding the control delay time B×n [msec] until the image input to the image processing server 24 is output to the delay time A [msec] described with reference to FIG. 2. Note that the control delay time B×n [msec] is the difference between the delay time A [msec] of the transmission path not passing through the image processing server 24 and the control total delay time A+(B×n) [msec] of the transmission path passing through the image processing server 24.

[0048] Here, with reference to FIG. 6, when the frame rate of the image reception device 26 is 60 Hz, the process for switching the transmission path when the delay time control process is performed will be described.

[0049] Similar to the explanation with reference to FIG. 4 above, the image receiving device 26 synchronizes the frame rate in accordance with the output timing at which the image transmitted through the transmission path via the image processing server 24 is output from the receiving device 25.

[0050] And even if the switching is performed to the transmission path via the image processing server 24, as a result of the total control delay time being A+(B×n) [msec], the output timing at which the image is output from the receiving device 25 matches the frame rate of the image receiving device 26. Thereby, the image receiving device 26 can immediately display the image output from the receiving device 25 at the image display interval of 16.6 msec without the need to perform the resynchronization process as described above.

[0051] Here, the delay time A [msec] is a design value, and in the delay time control process, it is not necessary to recognize the delay time A [msec] which is the delay amount of the IP network. Note that since the communication speed of the IP network is several microseconds, it is a negligible time. Also, the image processing delay time C [msec] is uniquely determined by the calculation amount of the application executed in the image processing server 24. For example, when the image processing server 24 transmits a packet to the receiving device 25, it also sends time information at the same time, and the IP converter 41 can synchronize based on this time information.

[0052] <Delay Time Control Device and Delay Time Control Process> FIG. 7 is a block diagram showing a configuration example of a delay time control device that executes a delay time control process in the medical image transmission system 11.

[0053] As shown in FIG. 7, the delay time control device 61 includes a frame rate acquisition unit 71, a delay time estimation unit 72, a control delay time calculation unit 73, and a control delay time setting unit 74, and controls the delay time in the delay execution unit 62.

[0054] The frame rate acquisition unit 71 communicates with the image receiving device 26 to acquire the frame rate notified from the image receiving device 26, and supplies the image display interval B, which is the reciprocal of the frame rate, to the control delay time calculation unit 73.

[0055] The delay time estimation unit 72 estimates the image processing delay time C [msec] corresponding to one or more image processes set to be executed in the image processing server 24, and supplies it to the control delay time calculation unit 73.

[0056] Here, when a plurality of image processes are performed in the image processing server 24, the delay time estimation unit 72 estimates the image processing delay time according to the sum of the times required for each image process. For example, as shown in FIG. 8, when three image processes A, image process B, and image process C are performed, the delay time estimation unit 72 estimates the image processing delay time according to the sum of the times required for the three image processes.

[0057] The control delay time calculation unit 73 determines an integer n so that the image processing delay time C [msec] is equal to or greater than the image processing delay time C [msec] based on the image display interval B notified from the frame rate acquisition unit 71 and the image processing delay time C [msec] notified from the delay time estimation unit 72, and calculates the control delay time B × n [msec].

[0058] The control delay time setting unit 74 sets the control delay time B × n [msec] calculated by the control delay time calculation unit 73 for the delay execution unit 62.

[0059] When the delay execution unit 62 adjusts the delay time in the frame buffer 53 or 55, for example, as shown in FIG. 5 described above, it executes a delay according to the control delay time B×n [msec] in the image processing server 24. Alternatively, the receiving device 25 may be configured to include a frame buffer (not shown) that serves as the delay execution unit 62. In this configuration, a delay according to the control delay time B×n [msec] is executed in the receiving device 25. That is, the delay execution unit 62 may be provided anywhere as long as it is on the image transmission path via the image processing server 24 in the medical image transmission system 11.

[0060] For example, when the IP converter 41 includes the delay execution unit 62, when displaying images on a plurality of image receiving devices 26, the output timing of the images in the IP converter 41 of the corresponding receiving device 25 can be controlled according to the control delay time based on the frame rate of each image receiving device 26. Therefore, in this case, it is possible to easily perform the process of displaying images with the control delay time for each image receiving device 26, and for example, it is possible to reduce the complexity of the process compared to a configuration in which the process of controlling the output timing of images in the image processing server 24 is performed.

[0061] Also, for example, when the image processing server 24 includes the delay execution unit 62, when displaying images on a plurality of image receiving devices 26, the output timing of the images can be controlled according to the same control delay time for those image receiving devices 26. At this time, as the control delay time, for example, the longest control delay time among the control delay times based on the frame rates of the individual image receiving devices 26 can be used. By controlling the output timing of the images according to the same control delay time for the plurality of image receiving devices 26 in this way, it is possible to reduce the occurrence of deviation in the images displayed on those image receiving devices 26, and for example, it is possible to suppress the medical staff from feeling discomfort.

[0062] Similarly, the delay time control device 61 can also be configured to be included in any of the devices constituting the medical image transmission system 11. For example, the medical image transmission system 11 can adopt a configuration in which the image processing server 24 includes a delay execution unit 62, or a configuration in which the receiving device 25 includes a delay execution unit 62. In the case where the image processing server 24 includes the delay time control device 61, it is necessary to configure such that the receiving device 25 acquires the frame rate from the image receiving device 26 and the frame rate is notified from the receiving device 25 to the image processing server 24.

[0063] As described above, the delay time control device 61 controls the delay time in the delay execution unit 62 to be the control delay time B×n [msec], so that even when the transmission path without passing through the image processing server 24 and the transmission path passing through the image processing server 24 are switched, resynchronization processing in the image receiving device 26 can be avoided from being executed. Thereby, in the image receiving device 26, a non-display period due to the switching of the transmission path does not occur, and for example, a state that hinders the operation of a surgeon or the like can be reduced.

[0064] Also, in the medical image transmission system 11, as shown in FIG. 8, the time required for processing for each of a plurality of image processes may be presented to the user in a bar graph GUI (Graphical User Interface) as shown in the figure. Thereby, the user can select, via the GUI, the image process to be executed by the image processing server 24 with reference to the time required for each image process. In this way, by visualizing which image process causes the delay for each delay time, medical staff can use it as a basis for determining which image process should not be performed when they want to reduce the delay.

[0065] FIG. 9 is a flowchart for explaining the delay time control process executed in the delay time control device 61.

[0066] For example, the process starts at the timing when the medical image transmission system 11 is activated. In step S11, the frame rate acquisition unit 71 communicates with the image receiving device 26 to acquire the frame rate notified from the image receiving device 26. Then, the frame rate acquisition unit 71 supplies the image display interval B, which is the reciprocal of the acquired frame rate, to the control delay time calculation unit 73.

[0067] In step S12, the delay time estimation unit 72 estimates the image processing delay time C [msec] corresponding to the image processing set to be executed in the image processing server 24, and supplies it to the control delay time calculation unit 73.

[0068] In step S13, the control delay time calculation unit 73 determines an integer n based on the image display interval B notified from the frame rate acquisition unit 71 in step S11 and the image processing delay time C [msec] notified from the delay time estimation unit 72 in step S12, so that it is equal to or greater than the image processing delay time C [msec], and calculates the control delay time B×n [msec].

[0069] In step S14, the control delay time setting unit 74 sets the control delay time B×n [msec] calculated by the control delay time calculation unit 73 in step S13 for the delay execution unit 62, and then the process ends.

[0070] As described above, the delay time control device 61 can set the control delay time B×n [msec] for the delay execution unit 62 so that it is an integer multiple of the reciprocal of the frame rate. That is, the delay time control device 61 determines the control delay time B×n [msec] based on the difference between the delay time of the transmission path not passing through the image processing server 24 and the delay time of the transmission path passing through the image processing server 24, and based on the frame rate which is a characteristic of the image receiving device 26, and can control the timing at which the image receiving device 26 outputs an image from the receiving device 25. Thereby, when switching is performed by the IP switch 23 between the transmission path not passing through the image processing server 24 and the transmission path passing through the image processing server 24, it is possible to reduce the situation where medical information and the like are not displayed.

[0071] Note that the delay time control device 61 may acquire information of the medical imaging device 21 connected to the transmission device 22 and set the control delay time based on the characteristics of the medical imaging device 21. For example, since real-time performance is important for endoscopes and microscopes, the delay time control device 61 determines the priority for each medical imaging device 21, and for those with high priority, sets the control delay time so that the latency is minimized, or sets the control delay time so as not to cause a delay. Conversely, for medical imaging devices 21 such as an operating room camera where a certain amount of delay is acceptable, the delay time control device 61 reduces the priority and sets the control delay time.

[0072] Also, the delay time control device 61 can adjust the control delay time in accordance with the processing of the application executed by the image processing server 24 or the IP converter 41. This is because it is assumed that the calculation amount of the application executed by the image processing server 24 or the IP converter 41 fluctuates. For example, when the calculation amount of the application is large, it can be adjusted to reduce the control delay time, and if it is still not in time even after reducing the control delay time, conversely, it can be adjusted to increase the control delay time.

[0073] In addition, the delay time control device 61 may set the control delay time so that a delay of two frames always occurs, and adjust it to reduce the control delay time according to the worst case. At this time, when the control delay time increases too much, it is preferable to notify the medical staff of this and prompt them to stop the process that caused the increase in the control delay time. Specifically, in a situation where a large amount of smoke is generated when using an electric scalpel, if the computational amount of the smoke removal process increases, the processing time may increase in the image processing server 24. At this time, an adjustment is made to reduce the conventional control delay time according to the increased processing time.

[0074] Furthermore, when the frame rate resynchronization process is executed in the image receiving device 26, the delay time control device 61 can reset the control delay time according to the frame rate after resynchronization. For example, when the image receiving device 26 restarts, information on the restart is notified from the image receiving device 26 to the receiving device 25, and the delay time control device 61 can reset the control delay time according to the notification. Also, when the power supply to the image receiving device 26 is cut off or the connection between the image receiving device 26 and the receiving device 25 is disconnected, the delay time control device 61 may stop the delay time control process until the frame rate is notified from the image receiving device 26.

[0075] Also, in the medical image transmission system 11, it is assumed that various medical devices such as endoscopes, microscopes, ultrasonic devices, and MRI (Magnetic Resonance Imaging) are used as the medical imaging device 21. In this case, when the frame rates output from the respective medical imaging devices 21 are different, the delay time control device 61 preferably executes the delay time control process in accordance with the image receiving device 26. That is, the delay time control device 61 can set the control delay time in accordance with the characteristics of the medical imaging device 21 and the characteristics of the image receiving device 26.

[0076] For example, when the image processing unit 54 cannot process signals at 60 frames per second (fps), it processes at 29.97 and outputs images two at a time. Also, when the image processing unit 54 has the ability to process at 60 fps and an image with a high processing load is to be processed, instead of outputting to the image receiving device 26 at 30 fps, the image processing unit 54 processes at 30 fps, adds one copy, and outputs to the image receiving device 26 at 60 fps. That is, when the ultrasonic device, which is the medical imaging device 21, outputs at 30 fps and the frame rate of the image receiving device 26 is 60 fps, the image processing unit 54 can process at 30 fps, add a copy, and output at 60 fps.

[0077] The frame rate acquisition unit 71 acquires the frame rate from the image receiving device 26. Additionally, for example, it may also acquire the characteristics of the image receiving device 26. For example, the frame rate acquisition unit 71 can acquire the type of the image receiving device 26 by holding a table in which the types of the image receiving device 26 and the frame rates are associated, and acquire the frame rate by referring to that table. Or, the delay time control device 61 may be configured such that a preset frame rate is input as the characteristic of the image receiving device 26. For example, a nurse may input the frame rate found by examining the image receiving device 26. Thus, various methods can be used to acquire the frame rate.

[0078] <Application Example> The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure may be applied to an operating room system.

[0079] FIG. 10 is a diagram schematically showing the overall configuration of an operating room system 5100 to which the technology according to the present disclosure can be applied. Referring to FIG. 10, the operating room system 5100 is configured by connecting a group of devices installed in the operating room so as to be able to cooperate with each other via an IP switch 5107 and an audio-visual controller (AV Controller) 5109. This operating room system is composed of an IP network capable of transmitting and receiving 4K / 8K video, and input / output video and control information for each device are transmitted and received via the IP network.

[0080] Various devices can be installed in the operating room. In FIG. 10, as an example, a group of various devices 5101 for endoscopic surgery, a ceiling camera 5187 provided on the ceiling of the operating room for imaging the operator's hands, an operating room camera 5189 provided on the ceiling of the operating room for imaging the entire state of the operating room, a plurality of display devices 5103A to 5103D, a recorder 5105, a patient bed 5183, and lighting 5191 are illustrated.

[0081] The IP converter 31 converts the video from individual medical imaging devices (endoscope, surgical microscope, X-ray imaging device, operating room camera, etc.) into IP and transmits it onto the network. The IP converter 41 on the monitor side converts the video transmitted via the network into a monitor-specific format and outputs it. This IP converter 41 may have various image processing functions, and may include resolution conversion processing according to the output destination, rotation and shake correction of endoscopic video, object recognition processing, etc. Further, it may include partial processing such as feature information extraction for analysis by the image processing server 24. These image processing functions may be specific to the connected medical imaging device or may be upgradable from the outside. In the IP converter 41 on the display side, it is also possible to perform processing such as synthesis of a plurality of videos (PinP processing, etc.) and superimposition of annotation information.

[0082] Here, among these devices, the device group 5101 belongs to the endoscopic surgery system 5113 and includes an endoscope and a display device for displaying an image captured by the endoscope. Each device belonging to the endoscopic surgery system 5113 is also referred to as a medical device. On the other hand, the display devices 5103A to 5103D, the recorder 5105, the patient bed 5183, and the illumination 5191 are devices installed in, for example, the operating room separately from the endoscopic surgery system 5113. Each of these devices not belonging to the endoscopic surgery system 5113 is also referred to as a non-medical device. The IP switch 5107 and / or the audiovisual controller 5109 controls the operations of these medical devices and non-medical devices in cooperation with each other.

[0083] Similarly, as shown in the figure, when the device group 5101 includes a surgical robot (master-slave for surgery) system 5114, a medical image acquisition device such as an X-ray imaging device 5115, etc. in the operating room, those devices can also be connected.

[0084] The audiovisual controller 5109 comprehensively controls the processing related to image display in medical devices and non-medical devices. Specifically, among the devices included in the operating room system 5100, the device group 5101, the ceiling camera 5187, and the field camera 5189 can be devices having a function of transmitting information to be displayed during surgery (hereinafter also referred to as display information) (hereinafter also referred to as the transmitting source device). Also, the display devices 5103A to 5103D can be devices to which the display information is output (hereinafter also referred to as the output destination device). Further, the recorder 5105 can be a device corresponding to both the transmitting source device and the output destination device. The audiovisual controller 5109 has a function of controlling the operations of the transmitting source device and the output destination device, acquiring the display information from the transmitting source device, and transmitting the display information to the output destination device for display or recording. Note that the display information includes various images captured during surgery and various information related to the surgery (for example, the patient's physical information, past examination results, information about the surgical procedure, etc.).

[0085] Specifically, information about the image of the surgical site in the patient's body cavity captured by the endoscope can be transmitted as display information from the device group 5101 to the audiovisual controller 5109. Also, information about the image of the operator's hands captured by the sealing camera 5187 can be transmitted as display information from the sealing camera 5187. Further, information about the image showing the entire operating room captured by the operating room camera 5189 can be transmitted as display information from the operating room camera 5189. If there are other devices with imaging functions in the operating room system 5100, the audiovisual controller 5109 may also acquire information about the images captured by such other devices as display information from the other devices.

[0086] Alternatively, for example, information about these images captured in the past is recorded by the audiovisual controller 5109 in the recorder 5105. The audiovisual controller 5109 can acquire information about the images captured in the past from the recorder 5105 as display information. Note that various types of information related to the surgery may also be pre-recorded in the recorder 5105.

[0087] The audiovisual controller 5109 causes at least one of the output destination devices, i.e., the display devices 5103A to 5103D, to display the acquired display information (i.e., the images captured during the surgery and various types of information related to the surgery). In the illustrated example, the display device 5103A is a display device suspended from the ceiling of the operating room, the display device 5103B is a display device installed on the wall surface of the operating room, the display device 5103C is a display device installed on the table in the operating room, and the display device 5103D is a mobile device (e.g., a tablet PC (Personal Computer)) having a display function.

[0088] In addition, the operating room system 5100 may include devices outside the operating room. The devices outside the operating room can be, for example, servers connected to a network constructed inside and outside the hospital, PCs used by medical staff, projectors installed in the hospital's conference rooms, and the like. When such external devices are outside the hospital, the visual controller 5109 can also display the display information on the display devices of other hospitals via a video conferencing system or the like for telemedicine.

[0089] In addition, servers and clouds outside the operating room may be used for image analysis and data analysis. They may transmit the video information in the operating room to an external server, generate additional information through big data analysis on the server side and recognition / analysis processing using AI (machine learning), and feedback it to the display device in the operating room. In this case, the IP converter connected to the video equipment in the operating room transmits data to the server and analyzes the video. The data to be transmitted may be the surgical video itself of an endoscope or the like, metadata extracted from the video, data indicating the operating status of the connected devices, and the like.

[0090] The operating room control device (not shown) comprehensively controls processes other than those related to image display in non-medical devices. For example, the operating room control device controls the driving of the patient bed 5183, the ceiling camera 5187, the operating field camera 5189, and the lighting 5191.

[0091] The operating room system 5100 is provided with a centralized operation panel 5111. The user can give instructions regarding image display such as switching of input / output devices to the visual controller 5109 or give instructions regarding the operation of non-medical devices to the operating room control device via the centralized operation panel 5111. The centralized operation panel 5111 is configured with a touch panel provided on the display surface of the display device.

[0092] The technology according to the present disclosure can be suitably applied to the image processing server 24 or the IP converter 41 among the configurations described above. Thereby, when the transmission path is switched by the IP switch 5107 to pass through the image processing server 24, as described above, resynchronization in the display devices 5103A to 5103D is avoided, and image switching can be performed instantaneously. As a result, the time during which medical information cannot be displayed on the display devices 5103A to 5103D can be reduced, and for example, a state that hinders the procedures of an operator or the like can be eliminated.

[0093] <Configuration example of computer> Next, the above-described series of processes (control method) can be performed by hardware or by software. When the series of processes are performed by software, the program constituting the software is installed in a general-purpose computer or the like.

[0094] FIG. 11 is a block diagram showing a configuration example of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0095] The program can be pre-recorded in a hard disk 105 or a ROM 103 as a recording medium built in the computer.

[0096] Alternatively, the program can be stored (recorded) in a removable recording medium 111 driven by a drive 109. Such a removable recording medium 111 can be provided as so-called package software. Here, examples of the removable recording medium 111 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, a semiconductor memory, and the like.

[0097] In addition to being installed on the computer from the removable recording medium 111 as described above, the program can be downloaded to the computer via a communication network or a broadcast network and installed on the built-in hard disk 105. That is, the program can be wirelessly transferred to the computer from, for example, a download site via an artificial satellite for digital satellite broadcasting, or can be wiredly transferred to the computer via a network such as a LAN (Local Area Network) or the Internet.

[0098] The computer incorporates a CPU (Central Processing Unit) 102, and an input / output interface 110 is connected to the CPU 102 via a bus 101.

[0099] When a command is input by the user operating the input unit 107 via the input / output interface 110, the CPU 102 executes the program stored in the ROM (Read Only Memory) 103 accordingly. Alternatively, the CPU 102 loads the program stored in the hard disk 105 into the RAM (Random Access Memory) 104 and executes it.

[0100] Thereby, the CPU 102 performs the processing according to the above-described flowchart or the processing performed according to the configuration of the above-described block diagram. Then, the CPU 102 outputs the processing result from the output unit 106 via the input / output interface 110 as needed, or transmits it from the communication unit 108, or records it on the hard disk 105, etc.

[0101] The input unit 107 is composed of a keyboard, a mouse, a microphone, etc. The output unit 106 is composed of an LCD (Liquid Crystal Display), a speaker, etc.

[0102] Here, in this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order along the order described as a flowchart. That is, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0103] Also, the program may be processed by one computer (processor), or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer for execution.

[0104] Furthermore, in this specification, a system means a collection of a plurality of components (devices, modules (parts), etc.), regardless of whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device in which a plurality of modules are housed in one housing are both systems.

[0105] Also, for example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configurations described as a plurality of devices (or processing units) above may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or another processing unit).

[0106] Also, for example, the present technology can adopt a cloud computing configuration in which one function is shared and jointly processed by a plurality of devices via a network.

[0107] Also, for example, the above-described program can be executed on any device. In that case, it suffices if the device has the necessary functions (such as functional blocks) and can obtain the necessary information.

[0108] Also, for example, each step described in the above flowchart can be executed not only on one device but also in parallel on a plurality of devices. Further, when a plurality of processes are included in one step, the plurality of processes included in that one step can be executed not only on one device but also in parallel on a plurality of devices. In other words, the plurality of processes included in one step can also be executed as the processes of a plurality of steps. Conversely, the processes described as a plurality of steps can also be executed together as one step.

[0109] Note that the program executed by the computer may be such that the processing of the steps of writing the program is executed in time series along the order described in this specification, or may be executed individually in parallel or at a necessary timing such as when a call is made. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the order described above. Further, the processing of the steps of writing this program may be executed in parallel with the processing of other programs, or may be executed in combination with the processing of other programs.

[0110] Note that the technologies described multiple times in this specification can be implemented independently and individually as long as there is no contradiction. Of course, any plurality of these technologies can also be implemented in combination. For example, a part or all of the technology described in any one embodiment can be implemented in combination with a part or all of the technology described in another embodiment. Also, a part or all of any of the above-described technologies can be implemented in combination with other technologies not described above.

[0111] <Example of configuration combination> Note that the present technology can also have the following configurations. (1) An image conversion device on the transmission side that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, An image processing server that performs image processing on the image transmitted via the network from the image conversion device on the transmission side and transmits the image generated by the image processing to the network, A receiving-side image conversion device that receives the transmission image and the image generated by image processing via the network, converts the images into a display image based on the transmission image and the image generated by image processing, and outputs the display image to a display device and includes, A delay time control unit that obtains a control delay time based on a difference between a delay time of a first transmission path that does not pass through the image processing server and a delay time of a second transmission path that passes through the image processing server, and controls the timing of outputting an image to the display device based on the characteristics of the display device A medical control system. (2) The delay time control unit obtains the frame rate of the display device as the characteristic, obtains the control delay time, and sets the control delay time for a delay execution unit provided on the second transmission path. The medical control system according to (1) above. (3) The delay time control unit obtains the control delay time that is an integer multiple of the reciprocal of the frame rate of the display device. The medical control system according to (2) above. (4) The delay time control unit determines an integer such that the control delay time is equal to or greater than an image processing delay time estimated from one or more image processes set to be executed in the image processing server, and calculates the control delay time. The medical control system according to (3) above. (5) The delay execution unit is a frame buffer included in the image processing server or the receiving-side image conversion device. The medical control system according to any one of (2) to (4) above. (6) When images are to be displayed on the plurality of the display devices, a plurality of the receiving-side image conversion devices for outputting images to the respective display devices are provided. The delay time control unit controls the timing of outputting an image in each corresponding receiving-side image conversion device according to the control delay time based on the frame rate of each individual display device. The medical control system according to any one of (2) to (5) above. (7) When the delay time control unit causes images to be displayed on the plurality of display devices, it controls the timing of outputting images to all the display devices according to the longest control delay time among the control delay times based on the frame rates of the individual display devices. The medical control system according to any one of (2) to (6) above. (8) The delay time control unit acquires information on the medical device connected to the transmitting-side image conversion device, and sets the control delay time based on the characteristics of the medical device. The medical control system according to any one of (1) to (7) above. (9) For one or more image processes performed in the image processing server, the time required for each process is presented to the user via a GUI (Graphical User Interface). The medical control system according to any one of (1) to (8) above. (10) The delay time control unit adjusts the control delay time in accordance with the processing of the application executed in the image processing server or the receiving-side image conversion device. The medical control system according to any one of (1) to (9) above. (11) When a frame rate resynchronization process is executed in the display device, the delay time control unit re-sets the control delay time according to the frame rate after resynchronization. The medical control system according to any one of (1) to (10) above. (12) A transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, A reception-side image conversion device that receives the transmission image and an image generated by image processing via the network, converts the images into a display image based on the transmission image and the image generated by image processing, and outputs the display image to a display device, and an image processing server that constitutes a medical control system together with the above, which performs image processing on an image transmitted via the network from the transmission-side image conversion device, transmits the image generated by the image processing to the network, and has a delay time control unit that obtains a control delay time based on a difference between a delay time of a first transmission path that does not pass through the image processing server and a delay time of a second transmission path that passes through the image processing server, and controls a timing for outputting an image to the display device based on characteristics of the display device. Image processing server. (13) A transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, An image processing server that performs image processing on an image transmitted via the network from the transmission-side image conversion device and transmits the image generated by the image processing to the network, and a reception-side image conversion device that constitutes a medical control system together with the above, which receives the transmission image and an image generated by image processing via the network, converts the images into a display image based on the transmission image and the image generated by image processing, and outputs the display image to a display device. A delay time control unit that obtains a control delay time based on a difference between a delay time of a first transmission path not passing through the image processing server and a delay time of a second transmission path passing through the image processing server, based on the characteristics of the display device, and controls the timing of outputting an image to the display device. Image conversion device. (14) A transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits it to the network, An image processing server that performs image processing on an image transmitted via the network from the transmission-side image conversion device and transmits the image generated by the image processing to the network, A reception-side image conversion device that receives the transmission image and the image generated by image processing via the network, converts them into a display image based on the transmission image and the image generated by image processing, and outputs it to a display device A medical control system comprising: Obtaining a control delay time based on a difference between a delay time of a first transmission path not passing through the image processing server and a delay time of a second transmission path passing through the image processing server, based on the characteristics of the display device, and controlling the timing of outputting an image to the display device A control method including:

[0112] Note that the present embodiment is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present disclosure. Also, the effects described in this specification are merely examples and are not limiting, and there may be other effects.

Description of Reference Numerals

[0113] 11 Medical image transmission system, 21 Medical imaging device, 22 Transmitting device, 23 IP switch, 24 Image processing server, 25 Receiving device, 26 Image receiving device, 31 IP converter, 32 Interface, 33 Encoder, 41 IP converter, 42 Decoder, 43 Interface, 51 IP converter, 52 Decoder, 53 Frame buffer, 54 Image processing unit, 55 Frame buffer, 56 Encoder, 61 Delay time control device, 62 Delay execution unit, 71 Frame rate acquisition unit, 72 Delay time estimation unit, 73 Control delay time calculation unit, 74 Control delay time setting unit

Claims

1. A transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, An image processing server that performs image processing on an image transmitted via the network from the transmission-side image conversion device and transmits the image generated by the image processing to the network, A reception-side image conversion device that receives the transmission image and the image generated by the image processing via the network, converts the images into a display image based on the transmission image and the image generated by the image processing, and outputs the display image to a display device and includes A control delay time based on a difference between a delay time of a first transmission path that does not pass through the image processing server and a delay time of a second transmission path that passes through the image processing server, and the control delay time that is an integer multiple of the reciprocal of the frame rate of the display device is obtained based on the characteristics of the display device, and a delay time control unit that controls the timing of outputting an image to the display device A medical control system.

2. The delay time control unit obtains the frame rate of the display device as the characteristic, obtains the control delay time, and sets the control delay time for a delay execution unit provided on the second transmission path. The medical control system according to claim 1.

3. The delay time control unit determines an integer such that the control delay time is equal to or greater than an image processing delay time estimated from one or more image processes set to be executed in the image processing server, and calculates the control delay time. The medical control system according to claim 1.

4. The delay execution unit is a frame buffer provided in the image processing server or the reception-side image conversion device. The medical control system according to claim 2.

5. When images are to be displayed on the plurality of the display devices, a plurality of the receiving-side image conversion devices for outputting images to the respective display devices are provided. The delay time control unit controls the timing of outputting an image in the corresponding receiving-side image conversion device according to the control delay time based on the frame rate of each of the display devices. The medical control system according to claim 2.

6. When images are to be displayed on the plurality of the display devices, the delay time control unit controls the timing of outputting an image to all of the display devices according to the longest control delay time among the control delay times based on the frame rates of the individual display devices. The medical control system according to claim 2.

7. The delay time control unit acquires information on the medical device connected to the transmitting-side image conversion device, and sets the control delay time based on the characteristics of the medical device. The medical control system according to claim 1.

8. Regarding one or more image processes performed in the image processing server, the time required for each process is presented to the user via a GUI (Graphical User Interface). The medical control system according to claim 1.

9. The delay time control unit adjusts the control delay time according to the process of the application executed in the image processing server or the receiving-side image conversion device. The medical control system according to claim 1.

10. When resynchronization processing of the frame rate is executed in the display device, the delay time control unit re-sets the control delay time according to the frame rate after resynchronization. The medical control system according to claim 1.

11. An image conversion device on the transmission side that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, A receiving-side image conversion device that receives the transmission image and an image generated by image processing via the network, converts the transmission image and the image generated by image processing into a display image based on the images, and outputs the display image to a display device An image processing server that constitutes a medical control system together with the above, Performs image processing on the image transmitted from the transmission-side image conversion device via the network, transmits the image generated by the image processing to the network, A control delay time based on the difference between the delay time of the first transmission path not passing through the image processing server and the delay time of the second transmission path passing through the image processing server, and the control delay time that is an integer multiple of the reciprocal of the frame rate of the display device is obtained based on the characteristics of the display device, and a delay time control unit that controls the timing of outputting an image to the display device is provided Image processing server.

12. An image conversion device on the transmission side that converts an image captured by a medical device into a transmission image for transmission via a network and transmits the image to the network, An image processing server that performs image processing on the image transmitted from the transmission-side image conversion device via the network and transmits the image generated by the image processing to the network A receiving-side image conversion device that constitutes a medical control system together with the above, Receives the transmission image and the image generated by image processing via the network, converts the transmission image and the image generated by image processing into a display image based on the images, and outputs the display image to a display device, A control delay time based on the difference between the delay time of a first transmission path not passing through the image processing server and the delay time of a second transmission path passing through the image processing server, the control delay time being an integer multiple of the reciprocal of the frame rate of the display device, obtaining the control delay time based on the characteristics of the display device, and having a delay time control unit that controls the timing of outputting an image to the display device Image conversion device.

13. A transmission-side image conversion device that converts an image captured by a medical device into a transmission image for transmission via a network and transmits it to the network, An image processing server that performs image processing on an image transmitted via the network from the transmission-side image conversion device and transmits the image generated by the image processing to the network, A reception-side image conversion device that receives the transmission image and the image generated by the image processing via the network, converts them into a display image based on the transmission image and the image generated by the image processing, and outputs the display image to a display device A medical control system comprising: A control delay time based on the difference between the delay time of a first transmission path not passing through the image processing server and the delay time of a second transmission path passing through the image processing server, obtaining the control delay time that is an integer multiple of the reciprocal of the frame rate of the display device based on the characteristics of the display device, and controlling the timing of outputting an image to the display device A control method including the above.

Images