Image processing device, display device, image processing system, control method and program for image processing device

Abstract

The objective is to provide an image processing device, a display device, an image processing system, a control method for the image processing device, and a program that can complete image processing in accordance with the frame rate during distribution. [Solution] The image processing device 110 is connected to the HMD 120 which displays the left eye image and the right eye image in a communicative manner. The image processing device 110 includes an acquisition means (control unit 205) capable of acquiring a fisheye image, which is a video captured using a fisheye lens, and an image processing means (control unit 205) that performs image processing to reduce distortion on the fisheye image. If the fisheye image includes a left eye image and a right eye image, the image processing means performs image processing on one of the left eye image and the right eye image, and if the image processing on one image is performed within a predetermined time, it performs image processing on the other image.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a display apparatus, an image processing system, a control method of an image processing apparatus, and a program.

Background Art

[0002] With the spread of the Internet, live distribution that distributes videos in real time is carried out. Also, a camera equipped with two fisheye lenses having a 180-degree viewing angle on the left and right is known. This camera can capture VR180 videos. "VR180" is a standard for viewing a VR video with a 180-degree viewing angle in stereoscopic vision. When performing VR live distribution of a VR180 video in real time using this camera, it is necessary to convert the fisheye images obtained by each fisheye lens into orthographic cylindrical projection format images in real time according to the frame rate at the time of distribution. Further, the VR180 video is distributed to a head-mounted display and displayed in a stereoscopically viewable manner. When a VR180 video is viewed on a head-mounted display, the parallax of the left and right images is corrected. This correction is, for example, performed at the distribution source of the VR180 video because meta information of a camera or a lens is used. Patent Document 1 discloses a system including a device that divides one screen and transmits each video for the left eye and the right eye when performing 3D broadcast, and a device that alternately displays each video temporally.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the system described in Patent Document 1, when VR180 video is streamed live in VR in real time, there is a risk that processing for the left eye image and the right eye image may not be completed in accordance with the frame rate at the time of streaming.

[0005] The present invention has been made in view of the above-mentioned problems. The object of the present invention is to provide an image processing apparatus, a display device, an image processing system, a control method for the image processing apparatus, and a program that can complete image processing in accordance with the frame rate at the time of distribution. [Means for solving the problem]

[0006] To achieve the above objective, the present invention provides an image processing apparatus that is communicatively connected to a display device that displays a left-eye image visible to the left eye and a right-eye image visible to the right eye, and comprises: acquisition means capable of acquiring a fisheye image which is a video captured using a fisheye lens; and image processing means that perform image processing to reduce distortion on the fisheye image, wherein the image processing means performs the image processing on one of the left-eye image and the right-eye image if the fisheye image includes the left-eye image and the right-eye image, and if the image processing on one of the images is performed within a predetermined time, the image processing is performed on the other image. [Effects of the Invention]

[0007] According to the present invention, image processing can be completed in accordance with the frame rate during distribution. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing an example of the overall configuration of an image processing system. [Figure 2] This is a block diagram showing an example of the hardware configuration of an image processing device. [Figure 3] This is a flowchart showing the processes performed by the image processing device. [Figure 4A]Figure 3 is a diagram illustrating the processes in steps S305 and S309 of the flowchart shown. [Figure 4B] This diagram illustrates the processes in steps S308 and S310 of the flowchart shown in Figure 3. [Figure 5] This is a flowchart showing the processes performed by the HMD. [Figure 6A] This diagram illustrates the process at step S503 of the flowchart shown in Figure 5. [Figure 6B] This diagram illustrates the process at step S505 of the flowchart shown in Figure 5. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings. However, the configurations described in the following embodiments are merely illustrative, and the scope of the present invention is not limited to the configurations described in the embodiments. For example, each part constituting the present invention can be replaced with any configuration that can perform a similar function. In addition, any configuration may be added.

[0010] Figure 1 is a schematic diagram showing an example of the overall configuration of an image processing system. As shown in Figure 1, the image processing system (information processing system) 1000 includes a digital camera 100, an image processing device 110, and an HMD (head-mounted display) 120. The digital camera 100 is equipped with a detachable lens 105. The lens 105 is a twin-lens (VR180 lens) with fisheye lens functionality. This lens 105 allows for the capture of still images and videos as twin-lens fisheye images (circular fisheye images) with parallax. In this embodiment, videos are the target of processing by the image processing device 110. Furthermore, a "twin-lens fisheye image" refers to a left-eye image (circular fisheye image for the left eye) visible to the left eye of a user 130 wearing the HMD 120, and a right-eye image (circular fisheye image for the right eye) visible to the right eye. The image processing device 110 is connected to the digital camera 100 via an HDMI® cable 115 for communication. As a result, the image processing device 110 can acquire a two-lens fisheye image captured by the digital camera 100 (acquisition step). The image processing device 110 can then perform image processing on the two-lens fisheye image. The image processing device 110 is not particularly limited and can be, for example, a desktop or notebook personal computer, a tablet terminal, or a smartphone. In the configuration shown in Figure 1, the digital camera 100 and the image processing device 110 are configured as separate units, but this is not limited to this configuration. For example, they may be configured as an integrated unit, meaning that the functions of the image processing device 110 may be built into the digital camera 100. The HMD 120 is connected to the image processing device 110 in a communicative manner. As a result, the user 130 can view the two-lens fisheye image processed by the image processing device 110 while wearing the HMD 120 on their head. In this case, the image processing device 110 converts the two-lens fisheye image into a streamable format and transmits the two-lens fisheye image directly to the HMD 120, or distributes it to multiple users 130's HMDs 120 via the cloud 125. The method of communication between the HMD120 and the image processing device 110 is not particularly limited and may be, for example, wireless communication or wired communication.

[0011] Figure 2 is a block diagram showing an example of the hardware configuration of an image processing device. As shown in Figure 2, the image processing device 110 includes a control unit 205, ROM 210, RAM 215, external storage device 220, operation unit 225, display unit 230, communication unit 235, and external I / F 240, which are connected to each other via a system bus 245 so as to be able to communicate with each other. The control unit 205 is a computer that controls the entire image processing device 110, for example, and has a CPU. The ROM 210 stores, for example, programs and parameters. These programs include, for example, programs that cause the control unit 205 to execute each process (control method of the image processing device) which will be described later. The RAM 215 temporarily stores, for example, programs and data supplied from external devices. The external storage device 220 stores various programs and data. The external storage device 220 is not particularly limited and may include, for example, a hard disk or flash memory fixed to the image processing device 110, or an optical disc such as an FD or CD that is detachably installed to the image processing device 110. Other external storage devices 220 include magnetic cards, optical cards, IC cards, and memory cards. The operation unit 225 accepts user input, such as buttons or a touch panel for inputting data. The display unit 230 displays data stored in the image processing device 110, for example. The display unit 230 is not particularly limited and can include, for example, a liquid crystal display. The communication unit 235 is connected to the internet, etc. The external I / F 240 can receive twin-lens fisheye images from the digital camera 100. The external I / F 240 can also transmit data stored in the image processing device 110 to an external device.

[0012] Figure 3 is a flowchart of the processes performed by the image processing device. Figure 4A is a diagram illustrating the processes in steps S305 and S309 of the flowchart shown in Figure 3. Figure 4B is a diagram illustrating the processes in steps S308 and S310 of the flowchart shown in Figure 3. As shown in Figure 3, in step S300, the control unit 205 of the image processing device 110 determines whether or not it has received an HDMI signal output from the digital camera 100 via the external I / F 240. If the control unit 205 determines, as a result of the determination in step S300, that it has received an HDMI signal, the process proceeds to step S301. On the other hand, if the control unit 205 determines, as a result of the determination in step S300, that it has not received an HDMI signal, the process remains in a waiting state at step S300.

[0013] In step S301, the control unit 205 determines whether or not the output of the HDMI signal from the digital camera 100 has stopped. If the control unit 205 determines, based on the result of the determination in step S301, that the output of the HDMI signal has stopped, the process ends. On the other hand, if the control unit 205 determines, based on the result of the determination in step S301, that the output of the HDMI signal has not stopped, the process proceeds to step S302.

[0014] In step S302, the control unit 205 reads the image data contained in the HDMI signal received in step S300. If this image data includes data for a circular fisheye image, the control unit 205 can acquire the circular fisheye image. Thus, in this embodiment, the control unit 205 functions as an acquisition means for acquiring a circular fisheye image. Note that in the image processing device 110, the part that functions as an acquisition means may be provided separately from the control unit 205.

[0015] In step S303, the control unit 205 determines whether the image data read in step S302 includes the left eye image and the right eye image, which are the circular fisheye images described above. Thus, in this embodiment, the control unit 205 functions as a determination means for determining whether the left eye image and the right eye image are included. In the image processing apparatus 110, the part that functions as a determination means may be provided separately from the control unit 205. The determination in step S303 is made based on a known method, for example, using the brightness of the image. If, as a result of the determination in step S303, the control unit 205 determines that the left eye image and the right eye image are included, the process proceeds to step S304. On the other hand, if, as a result of the determination in step S303, the control unit 205 determines that the left eye image and the right eye image are not included, the process returns to step S301 and the subsequent steps are executed in order. In this case, the image processing described later is omitted.

[0016] In step S304, the control unit 205 determines whether the image data read in step S302 is an even-numbered frame. If the control unit 205 determines in step S304 that it is an even-numbered frame, the process proceeds to step S305. On the other hand, if the control unit 205 determines in step S304 that it is not an even-numbered frame, i.e., an odd-numbered frame, the process proceeds to step S306.

[0017] In step S305, the control unit 205 performs image processing for reducing distortion on the left-eye image (one of the images) among these images (circumferential fisheye images) determined to include the left-eye image and the right-eye image in step S303 (image processing step). Thus, in this embodiment, the control unit 205 functions as image processing means for executing image processing for reducing distortion on the circumferential fisheye image. Note that in the image processing apparatus 110, the part that functions as image processing means may be provided separately from the control unit 205. The circumferential fisheye image 400 in FIG. 4A is an image captured by the digital camera 100 equipped with the lens 105. This circumferential fisheye image 400 includes a left-eye image 401L formed by the left lens of the lens 105 and a right-eye image 401R formed by the right lens of the lens 105. Then, in step S305, as image processing for the left-eye image 401L, a process of converting the image based on the orthographic cylindrical projection method is executed. As a result, the left-eye image 401L becomes a left-eye image 401L' with reduced distortion. Note that in step S305, preferably, a parallax correction process for correcting the parallax between the left-eye image and the right-eye image is executed for the left-eye image and the right-eye image. Thereby, in the HMD 120, the image can be viewed more stereoscopically. The parallax correction process is executed in consideration of, for example, individual differences such as the assembly error of the lens 105, the posture of the digital camera 100 at the time of imaging, and the influence of the deviation of the optical axes of the left lens and the right lens of the lens 105 based on meta-information such as the temperature environment at the time of imaging. Note that in the parallax correction process, only when one of the left-eye image and the right-eye image is selected, the other image may be fixed and the parallax correction of one image may be performed.

[0018] In step S306, the control unit 205 performs image processing for reducing distortion on the right-eye image (one of the images) among these images determined to include the left-eye image and the right-eye image in step S303, in the same manner as in step S305 (image processing step).

[0019] In step S307, the control unit 205 determines whether the image processing in step S305 or step S306 has been executed and completed within a predetermined time. In this embodiment, as an example, it is assumed that 60 frames of images are received per second in step S300. In this case, it is necessary to execute the image processing in step S305 or step S306 in about 16 msec. And in order to perform image processing on the left-eye image and the right-eye image in 16 msec under this condition, it is necessary for the image processing for each image to end within 8 msec. Therefore, when the image processing for one of the left-eye image and the right-eye image is completed within 8 msec, it is determined that the image processing for the other image can be completed within the remaining 8 msec (= 16 msec - 8 msec). And as a result of the determination in step S307, when the control unit 205 determines that the image processing has been completed within the predetermined time (8 msec), the process proceeds to step S308. On the other hand, as a result of the determination in step S307, when the control unit 205 determines that the image processing has not been completed within the predetermined time (8 msec), it means that the image processing for the other image cannot be completed in time, and the process proceeds to step S309. Note that in this embodiment, the determination in step S307 is configured to determine whether the processing has ended within the predetermined time when the processing of the monocular image ends, but this is not limiting. For example, the determination in step S307 may be configured to determine whether the processing for both-eye images has ended when a predetermined time has elapsed since the start of the processing.

[0020] In step S308, the control unit 205 performs image processing to reduce distortion on the other image (image processing step). That is, if step S305 has been completed, the control unit 205 performs image processing on the right eye image, and if step S306 has been completed, it performs image processing on the left eye image. The image processing for each image is the same as the image processing in step S305. The circular fisheye image 400 in Figure 4B is an image taken with a digital camera 100 equipped with a lens 105, similar to Figure 4A. This circular fisheye image 400 includes the left eye image 401L and the right eye image 401R. In step S305, image processing has already been performed on the left eye image 401L. As a result, the left eye image 401L becomes the left eye image 401L' with reduced distortion. In step S308, as image processing for the right eye image 401R, a process is performed to transform the image based on equirectangular projection. As a result, the right eye image 401R becomes the right eye image 401R' with reduced distortion.

[0021] In step S309, the control unit 205 outputs the left eye image (one image) processed in step S305, or the right eye image (one image) processed in step S306, to the HMD 120 via the communication unit 235 (output means). Note that the state shown in Figure 4A is the state in which the left eye image 401L' has been output.

[0022] In step S310, the control unit 205 outputs the image processed in step S305 or step S306 (one image) and the image processed in step S308 (the other image) to the HMD120 via the communication unit 235. The state shown in Figure 4B is when the left eye image 401L' and the right eye image 401R' have been output.

[0023] Figure 5 is a flowchart showing the processes performed by the HMD. Figure 6A is a diagram illustrating the process at step S503 of the flowchart shown in Figure 5. Figure 6B is a diagram illustrating the process at step S505 of the flowchart shown in Figure 5. As shown in Figure 5, in step S500, the control unit of the HMD 120 (hereinafter referred to as the "HMD control unit") determines whether or not it has received an image signal output from the image processing device 110 (communication unit 235). This image signal is either an image signal containing the image output in step S309 (see Figure 3) or an image signal containing the image output in step S310 (see Figure 3). If, as a result of the determination in step S500, the HMD control unit determines that it has received an image signal, the process proceeds to step S501. On the other hand, if, as a result of the determination in step S500, the HMD control unit determines that it has not received an image signal, the process remains in a waiting state at step S500.

[0024] In step S501, the HMD control unit determines whether or not the output of the image signal from the image processing device 110 has stopped. If the HMD control unit determines, as a result of the determination in step S501, that the output of the image signal has stopped, the process ends. On the other hand, if the HMD control unit determines, as a result of the determination in step S501, that the output of the image signal has not stopped, the process proceeds to step S502.

[0025] In step S502, the HMD control unit determines whether the circular fisheye image included in the image signal received in step S500 is a left-eye image or a right-eye image, that is, whether it is a monocular image (an image of one eye). Thus, in this embodiment, the HMD control unit functions as a determination means for determining whether the circular fisheye image acquired by the HMD control unit (acquisition means) is a monocular image. Note that in the HMD 120, the part that functions as a determination means may be provided separately from the HMD control unit. If, as a result of the determination in step S502, the HMD control unit determines that it is a monocular image, the process proceeds to step S503. On the other hand, if, as a result of the determination in step S502, the HMD control unit determines that it is not a monocular image, the process proceeds to step S504.

[0026] In step S503, the HMD control unit updates the monocular image displayed on the HMD 120 with the monocular image included in the image signal received in step S500 and for which image processing has been performed. This update is performed only on the region corresponding to the monocular image included in the image signal received in step S500. Regions not corresponding to the monocular image continue to be displayed as they were in the previous frame. Thus, in this embodiment, the HMD control unit functions as an update means for updating the image. In addition, the part of the HMD 120 that functions as an update means may be provided separately from the HMD control unit. The circular fisheye image 600 in Figure 6A is the image (video) displayed on the HMD 120 in the "Nth frame". For example, if only the image-processed left-eye image is transmitted from the image processing device 110 in the "N+1th frame", the HMD 120 receives only this left-eye image and updates the display to that left-eye image. On the other hand, the right-eye image from the Nth frame continues to be displayed as the right-eye image. Similarly, if only the processed right eye image is transmitted from the image processing device 110 in the "N+2 frame," the HMD 120 receives only this right eye image and updates the display to that right eye image. On the other hand, the left eye image from the "N+1 frame" continues to be displayed as the left eye image.

[0027] In step S504, the HMD control unit determines whether the circular fisheye image included in the image signal received in step S500 is a binocular image, i.e., a left-eye image and a right-eye image. If the HMD control unit determines in step S504 that it is a binocular image, the process proceeds to step S505. On the other hand, if the HMD control unit determines in step S504 that it is not a binocular image, the process returns to step S501 and the subsequent steps are executed in order.

[0028] In step S505, the HMD control unit updates the binocular image displayed on the HMD 120 with the binocular image included in the image signal received in step S500 and after image processing has been performed. The circular fisheye image 600 in Figure 6B is the image displayed on the HMD 120 in the Nth frame, similar to Figure 6A. For example, if the image-processed binocular image is transmitted from the image processing device 110 in the N+1th frame, the HMD 120 receives the binocular image and updates the display with that image. Note that the update processing in steps S503 and S505 is performed by the HMD control unit, but is not limited to this. For example, the update processing in steps S503 and S505 may be performed by the control unit 205 (update control means) of the image processing device 110. In this case, the processing burden on the HMD 120 can be reduced.

[0029] As described above, the image processing system 1000 can live stream fisheye images (videos) captured by the digital camera 100 to the HMD 120 in real time via the image processing device 110. At that time, the image processing device 110 can complete image processing for the left eye image and the right eye image in accordance with the frame rate at the time of streaming. In addition, the HMD 120 can update the currently displayed image with the image processed by the image processing device 110, i.e., the image that has been processed. As a result, the HMD 120 can observe live-streamed video smoothly.

[0030] While preferred embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of its gist. The present invention provides a program that implements one or more of the functions of the above embodiments to a system or device via a network or storage medium. It can also be implemented by one or more general-purpose processors (ASICs) of the computer of the system or device reading and executing the program. Furthermore, the present invention can also be implemented by a dedicated processor (e.g., an ASIC or FPGA) that implements one or more functions. Moreover, the present invention can also be implemented by a combination of a general-purpose processor and a dedicated processor. Here, "processor" refers to a processor in a broad sense and includes both general-purpose processors and dedicated processors. Furthermore, the process of implementing the present invention may be executed by only one processor, or by the cooperation of multiple processors located in physically separate locations.

[0031] This embodiment includes the following configurations, methods, and programs. (Configuration 1) An image processing device that is communicatively connected to a display device that displays a left-eye image visible to the left eye and a right-eye image visible to the right eye, A means for acquiring fisheye images, which are videos captured using a fisheye lens, The system includes an image processing means that performs image processing to reduce distortion on the aforementioned fisheye image, The image processing means is characterized in that, if the fisheye image includes the left eye image and the right eye image, it performs the image processing on one of the left eye image and the right eye image, and if the image processing on one of the images is performed within a predetermined time, it performs the image processing on the other image. (Configuration 2) The configuration includes a determination means for determining whether the fisheye image acquired by the acquisition means includes the left eye image and the right eye image, The image processing apparatus according to Configuration 1, characterized in that, as a result of the determination by the determination means, if it is determined that the fisheye image includes the left eye image and the right eye image, the image processing is performed on one of the images; if the image processing on one of the images is performed within a predetermined time, the image processing is performed on the other image; and if it is determined that the fisheye image does not include the left eye image and the right eye image, the image processing is omitted. (Configuration 3) The image processing apparatus according to Configuration 2, characterized in that, if the determination by the determination means determines that the fisheye image includes the left eye image and the right eye image, the image processing means is capable of performing a process to correct the parallax between the left eye image and the right eye image for the left eye image and the right eye image. (Configuration 4) An image processing apparatus according to any one of Configurations 1 to 3, characterized in that when the image processing is performed on the other image, it is provided with output means for outputting the one image on which the image processing has been performed and the other image on which the image processing has been performed to the display device. (Configuration 5) The image processing apparatus according to Configuration 4, characterized in that if the image processing on one of the images is not performed within a predetermined time, the output means outputs the one of the images on which the image processing has been performed to the display device. (Configuration 6) The image processing apparatus according to any one of Configurations 1 to 5, characterized in that when the image processing is performed on one of the images, the image processing is performed on the left eye image if the fisheye image is in an even-numbered frame, and on the right eye image if the fisheye image is in an odd-numbered frame. (Configuration 7) The image processing apparatus according to any one of Configurations 1 to 6, characterized in that the image processing means is capable of performing a process of converting the fisheye image based on equirectangular projection as the image processing. (Configuration 8) The image processing apparatus according to any one of Configurations 1 to 7, characterized in that the acquisition means acquires a circular fisheye image as the fisheye image. (Configuration 9) The display device is provided with update control means for performing control to update the fisheye image displayed on the display device, The image processing apparatus according to any one of configurations 1 to 8, characterized in that, if the fisheye image includes one of the images on which the image processing has been performed, the update control means updates the one of the images displayed on the display device to the one of the images on which the image processing has been performed. (Configuration 10) A display device that is communicatively connected to an image processing device described in any one of Configurations 1 to 9, An acquisition means for acquiring the fisheye image from the image processing device, A display means for displaying the fisheye image acquired by the acquisition means, The system includes an update means for updating the fisheye image displayed on the display means, The display device is characterized in that, if the fisheye image includes one of the images on which the image processing has been performed, the updating means updates the one of the images currently displayed on the display means to the one of the images on which the image processing has been performed. (Configuration 11) The fisheye image acquired by the acquisition means includes a determination means for determining whether or not the one image on which the image processing has been performed is included, The display device according to configuration 10, characterized in that, if the determination means determines that the fisheye image includes the one image on which the image processing has been performed, the updating means updates the one image currently displayed on the display means to the one image on which the image processing has been performed. (Configuration 12) The display device according to Configuration 10 or 11, characterized in that it is a head-mounted display. (Configuration 13) An image processing system comprising a display device that displays a left-eye image visible to the left eye and a right-eye image visible to the right eye, and an image processing device that is communicatively connected to the display device, The aforementioned image processing device is A first acquisition means capable of acquiring a fisheye image, which is a video captured using a fisheye lens, The system includes an image processing means for performing image processing on the aforementioned fisheye image, If the fisheye image includes the left eye image and the right eye image, the image processing means performs the image processing on one of the left eye image and the right eye image, and if the image processing on one of the images is performed within a predetermined time, it performs the image processing on the other image. The aforementioned display device is A second acquisition means for acquiring the fisheye image from the image processing device, A display means for displaying the fisheye image acquired by the second acquisition means, The system includes an update means for updating the fisheye image displayed on the display means, The image processing system is characterized in that, if the fisheye image includes one of the images on which the image processing has been performed, the updating means updates the one of the images displayed on the display means to the one of the images on which the image processing has been performed. (Method 1) A method for controlling an image processing device that is communicatively connected to a display device that displays a left-eye image visible to the left eye and a right-eye image visible to the right eye, An acquisition process that allows for the acquisition of fisheye images, which are videos captured using a fisheye lens, The process includes an image processing step that performs image processing to reduce distortion on the fisheye image, A control method for an image processing apparatus, characterized in that, in the image processing step, if the fisheye image includes the left eye image and the right eye image, the image processing is performed on one of the left eye image and the right eye image, and if the image processing on one of the images is performed within a predetermined time, the image processing is performed on the other image. (Program 1) A program characterized by causing a computer to execute the control method described in Method 1. [Explanation of symbols]

[0032] 110 Image Processing Device 100 Digital Cameras 120 HMD 205 Control Unit 235 Communications Department 1000 Image Processing Systems

Claims

1. An image processing device that is communicatively connected to a display device that displays a left-eye image visible to the left eye and a right-eye image visible to the right eye, A means for acquiring fisheye images, which are videos captured using a fisheye lens, The system includes an image processing means that performs image processing to reduce distortion on the aforementioned fisheye image, The image processing means is characterized in that, if the fisheye image includes the left eye image and the right eye image, it performs the image processing on one of the left eye image and the right eye image, and if the image processing on one of the images is performed within a predetermined time, it performs the image processing on the other image.

2. The system includes a determination means for determining whether the fisheye image acquired by the acquisition means includes the left eye image and the right eye image, The image processing apparatus according to claim 1, characterized in that, if the determination by the determination means determines that the fisheye image includes the left eye image and the right eye image, the image processing is performed on one of the images; if the image processing on one of the images is performed within a predetermined time, the image processing is performed on the other image; and if it is determined that the fisheye image does not include the left eye image and the right eye image, the image processing is omitted.

3. The image processing apparatus according to claim 2, characterized in that, if the determination by the determination means determines that the fisheye image includes the left eye image and the right eye image, the image processing means is capable of performing a process to correct the parallax between the left eye image and the right eye image for the left eye image and the right eye image.

4. The image processing apparatus according to claim 1, further comprising output means for outputting the image on which the image processing has been performed and the other image on which the image processing has been performed to the display device when the image processing has been performed on the other image.

5. The image processing apparatus according to claim 4, wherein the output means outputs the image on which the image processing has been performed to the display device if the image processing on the one image has not been performed within a predetermined time.

6. The image processing apparatus according to claim 1, characterized in that when the image processing means performs the image processing on one of the images, if the fisheye image is an even-numbered frame, the image processing is performed on the left eye image, and if the fisheye image is an odd-numbered frame, the image processing is performed on the right eye image.

7. The image processing apparatus according to claim 1, characterized in that the image processing means is capable of performing a process of converting the fisheye image based on equirectangular projection as the image processing.

8. The image processing apparatus according to claim 1, characterized in that the acquisition means acquires a circular fisheye image as the fisheye image.

9. The display device is equipped with update control means for performing control to update the fisheye image displayed on the display device, The image processing apparatus according to claim 1, characterized in that, if the fisheye image includes one of the images on which the image processing has been performed, the update control means causes the one of the images displayed on the display device to be updated with the one of the images on which the image processing has been performed.

10. A display device that is communicably connected to the image processing device described in claim 1, An acquisition means for acquiring the fisheye image from the image processing device, A display means for displaying the fisheye image acquired by the acquisition means, The system includes an update means for updating the fisheye image displayed on the display means, The display device is characterized in that, if the fisheye image includes one of the images on which the image processing has been performed, the updating means updates the one of the images currently displayed on the display means to the one of the images on which the image processing has been performed.

11. The fisheye image acquired by the acquisition means includes a determination means for determining whether or not the one image on which the image processing has been performed is included. The display device according to claim 10, wherein, as a result of the determination by the determination means, it is determined that the fisheye image includes the one image on which the image processing has been performed, the updating means updates the one image currently displayed on the display means to the one image on which the image processing has been performed.

12. The display device according to claim 10, characterized in that it is a head-mounted display.

13. An image processing system comprising a display device that displays a left-eye image visible to the left eye and a right-eye image visible to the right eye, and an image processing device that is communicatively connected to the display device, The aforementioned image processing device is A first acquisition means capable of acquiring a fisheye image, which is a video captured using a fisheye lens, The system includes an image processing means for performing image processing on the aforementioned fisheye image, If the fisheye image includes the left eye image and the right eye image, the image processing means performs the image processing on one of the left eye image and the right eye image, and if the image processing on one of the images is performed within a predetermined time, it performs the image processing on the other image. The aforementioned display device is A second acquisition means for acquiring the fisheye image from the image processing device, A display means for displaying the fisheye image acquired by the second acquisition means, The system includes an update means for updating the fisheye image displayed on the display means, The image processing system is characterized in that, if the fisheye image includes one of the images on which the image processing has been performed, the updating means updates the one of the images displayed on the display means to the one of the images on which the image processing has been performed.

14. A method for controlling an image processing device that is communicatively connected to a display device that displays a left-eye image visible to the left eye and a right-eye image visible to the right eye, An acquisition process that allows for the acquisition of fisheye images, which are videos captured using a fisheye lens, The process includes an image processing step that performs image processing to reduce distortion on the fisheye image, A control method for an image processing apparatus, characterized in that, in the image processing step, if the fisheye image includes the left eye image and the right eye image, the image processing is performed on one of the left eye image and the right eye image, and if the image processing on one of the images is performed within a predetermined time, the image processing is performed on the other image.

15. A program characterized by causing a computer to execute the control method described in claim 14.

Images