Video Processing System

Abstract

We provide a video processing system that suppresses 3D motion sickness. [Solution] The video processing system includes a camera, a video processing device, and a display. The video processing device executes the following operations: an acquisition process for acquiring an image of a first display range from the camera; a first calculation process for setting a second display range smaller than the first display range within the first display range of the image acquired in the acquisition process; a second calculation process for detecting movement within the first display range of a specific image captured in the second display range; a third calculation process for moving the position of the second display range within the first display range in accordance with the movement of the specific image; and an output process for outputting a time series of the specific image in the second display range obtained by repeating the second and third calculation processes.

Description

[Technical Field]

[0001] The present invention relates to a video processing system. [Background technology]

[0002] Patent Document 1 discloses a video system. This video system includes a 3D video player, a parallax detection unit, and a fatigue evaluation unit. The 3D video player plays back a left-eye image and a right-eye image. The parallax detection unit detects the difference in horizontal position between the left and right images as the parallax amount. The fatigue evaluation unit evaluates the fatigue level based on the parallax amount. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-355808 Summary of the Invention [Problem to be solved by the invention]

[0004] In the case of so-called first-person perspective video, a situation may arise in which the viewer's line of sight on the video is moving while the viewer's head is not moving. In this case, the viewer may feel motion sickness due to a mismatch between visual information and information obtained from the semicircular canals, etc. The video system disclosed in Patent Document 1 does not propose any method for generating images that prevent such motion sickness. Note that although a device capable of playing three-dimensional video has been given as an example, the same problem can occur with two-dimensional video. [Means for solving the problem]

[0005] An image processing system for solving the above problem comprises a camera capable of changing the shooting range in accordance with the direction of a user's head, an image processing device that processes and outputs images captured by the camera, and a display that displays the images output by the image processing device, wherein the image processing device performs the following steps: an acquisition process for acquiring an image of a first display range from the camera; a first calculation process for setting a second display range smaller than the first display range within the first display range of the image acquired in the acquisition process; a second calculation process for detecting movement within the first display range of a specific image displayed in the second display range; a third calculation process for moving the position of the second display range within the first display range in accordance with the movement of the specific image; and an output process for outputting a time series of the specific images in the second display range obtained by repeating the second calculation process and the third calculation process.

[0006] In the above configuration, the position of the second display range is moved in accordance with the movement of the specific image, resulting in an image that looks like a still image, thereby preventing the user from feeling motion sickness.

[0007] The video processing system may further include an estimation device that estimates a head velocity, which is the speed of head movement of the user, and the video processing device may be configured to, if the head velocity estimated by the estimation device is equal to or greater than a predetermined value, not perform the third calculation process, and to output, in the output process, a time series of images in the second display range set in the first calculation process, and, if the head velocity estimated by the estimation device is less than the predetermined value, to output, in the output process, a time series of the specific images in the second display range obtained by repeating the second calculation process and the third calculation process.

[0008] In the video processing system, the video processing device may be configured to execute the third calculation process when the movement speed of the specific image within the first display range is equal to or less than a predetermined first threshold.

[0009] In the video processing system, the video processing device may be configured to set the second display range to the center of the first display range in the first calculation process. In the video processing system, the video processing device may be configured, in the second calculation process, to separate a background image and a moving object image from the specific image in the second display range, and detect movement of the background image as movement of the specific image.

[0010] In the video processing system, the video processing device may be configured to perform the first calculation processing again when the edge of the second display range coincides with the edge of the first display range, or when a part of the second display range deviates outside the first display range. [Effects of the Invention]

[0011] The above technical concept can prevent users from feeling drunk. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the configuration of a video processing system. [Figure 2] FIG. 2 is a flowchart showing the video processing procedure of the first embodiment. [Figure 3] FIG. 3 is a diagram showing the setting of the second display range. [Figure 4] FIG. 4 is a diagram illustrating the tracking process. [Figure 5] FIG. 5 is a flowchart showing the video processing procedure of the second embodiment. [Figure 6] FIG. 6 is a diagram illustrating head speed and tracking. [Figure 7] FIG. 7 is a flowchart showing the video processing procedure of the third embodiment. [Figure 8] FIG. 8 is a diagram illustrating the movement speed and tracking of a specific image. [Figure 9] FIG. 9 is a flowchart showing a video processing procedure according to the modified example. DETAILED DESCRIPTION OF THE INVENTION

[0013] First Embodiment A first embodiment of a video processing system 10 will be described below with reference to FIGS. (Overall configuration of the video processing system) As shown in FIG. 1, the image processing system 10 in this embodiment includes a camera 20 that can change the shooting range by following the direction of the user's head, an image processing device 30 that processes and outputs the image captured by the camera 20, and a display 40 that displays the image output by the image processing device 30.

[0014] Camera 20 is a so-called stereoscopic camera. Specifically, although not shown, camera 20 has two lenses arranged at a fixed distance apart, two image sensors that receive light that has passed through each lens, and a signal processing circuit. These lenses, image sensors, and signal processing circuit are housed within the housing of camera 20.

[0015] Light from the subject passes through the lens of camera 20 and forms an image on the image sensor. The light that forms an image on the image sensor is converted into an electrical signal by the image sensor. The image converted into an electrical signal is then converted by a signal processing circuit into a standard for transmitting video as a digital signal. The images converted into the digital standard are one per frame. Camera 20 outputs images one frame at a time in chronological order. The frame rate, which is the unit of time at which camera 20 captures one image, is, for example, 120 fps (frames per sec). In this case, it corresponds to capturing 120 images per second. However, the frame rate is not limited to this and can be set arbitrarily.

[0016] Camera 20 is configured so that, for example, the direction of the user's head moves in conjunction with the shooting direction of camera 20. Specifically, the housing of camera 20 can be rotated around a predetermined first rotation axis by an electric motor. In addition, the housing of camera 20 can be adjusted on three axes: up / down, front / back, and left / right.

[0017] The video processing device 30 includes at least a video acquisition unit 32, a calculation unit 34, and a video output unit 36. For convenience, the video acquisition unit 32, the calculation unit 34, and the video output unit 36 ​​are described as separate functional blocks, but they do not necessarily have to be configured as separate processing circuits. The video acquisition unit 32 acquires images from the camera 20. For example, the video acquisition unit 32 is connected to the camera 20 via a digital interface cable or the like. This allows the video acquisition unit 32 to receive images of each frame from the camera 20 in chronological order. The calculation unit 34 performs first to third calculation processes, which will be described later, based on the images. The video output unit 36 ​​outputs the images after being processed by the calculation unit 34.

[0018] The display 40 is capable of displaying the image output by the video processing device 30. In this embodiment, the display 40 is a so-called head-mounted display. The display 40 outputs the images detected by the two image sensors of the camera 20 as an image for the right eye and an image for the left eye, respectively. A user wearing the display 40 can see the image captured by the camera 20 in three dimensions due to the parallax between the image for the right eye and the image for the left eye displayed on the display 40.

[0019] Although not shown, the display 40 has a built-in gyro sensor. The gyro sensor can detect the angular velocity of the display 40. The gyro sensor can detect angular velocities around two orthogonal axes. The display 40 outputs the detection results of the gyro sensor to the camera 20 described above. When a user wears the display 40 on their head, the orientation of the display 40 corresponds to the orientation of the user's head. Therefore, the detection results of the gyro sensor of the display 40 reflect the orientation of the user's head. The camera 20 changes the imaging range by driving each electric motor in response to the detection results of the gyro sensor of the display 40. In other words, the camera 20 can change the imaging range by following the orientation of the user's head.

[0020] (Processing flow of video processing device) The following describes a series of processes performed by the video processing device 30. The video processing device 30 starts a series of processes when the power of the video processing device is turned on, and then ends the series of processes when the power is turned off.

[0021] 2, first, in the acquisition process of step S11, the video acquisition unit 32 of the video processing device 30 acquires an image of the first display range 33 from the camera 20. The image here refers to an image for one frame.

[0022] 3, the image acquired by the image acquisition unit 32 from the camera 20 is configured as an image of a first display range 33. In one example, the first display range 33 corresponds to the range equivalent to the angle of view of the camera 20. For example, when a 50 mm lens and a full-size image sensor are used, the angle of view of the camera 20 is 47 degrees.

[0023] 2, after step S11, the calculation unit 34 determines in step S13 whether the second display range 35 has been set. The setting of the second display range 35 will be described in detail in step S15. If the second display range 35 has been set, the calculation unit 34 skips the first calculation process in step S15 and performs the process of step S17. If the second display range 35 has not been set, the process of the calculation unit 34 proceeds to the first calculation process in step S15.

[0024] Next, in step S15, the calculation unit 34 performs a first calculation process. The first calculation process is a process of setting a second display range 35, which is smaller than the first display range 33, within the first display range 33. As shown in FIG. 3 , the second display range 35, which is smaller than the first display range 33, is set inside the outer edge of the first display range 33. In one example, the position of the second display range 35 is the center of the first display range 33 in the vertical and horizontal directions. In other words, the geometric center of the second display range 35 coincides with the geometric center of the first display range 33. However, this is not limited to this, and the position of the second display range 35 may be offset to the left or right of the first display range 33, or may be offset to the top or bottom. In another example, the size of the second display range 35 is 70% of the size of the first display range 33 in the horizontal and vertical directions. In this case, the area of ​​the second display range 35 is 49% of the area of ​​the first display range 33. However, the present invention is not limited to this, and the size of the second display range 35 may be larger or smaller than 70% of the size of the first display range 33.

[0025] As shown in FIG. 2, after step S15, the calculation unit 34 performs a second calculation process. The second calculation process is composed of steps S17 and S19. As shown in FIG. 3, in step S17, the calculation unit 34 performs a process of separating a background image 52, which is a specific image 50, and a moving object image from the second display range 35. For example, background subtraction can separate a stationary background from a moving object. The separated stationary background and moving object can each be image data. In this case, the background image 52 corresponds to the stationary background and its image data in background subtraction.

[0026] Then, in step S19, the calculation unit 34 detects movement of the specific image 50 displayed in the second display range 35 within the first display range 33. Detecting movement of the specific image 50 within the first display range 33 means detecting movement of the background image 52 within the first display range 33. In other words, it can be said that blurring of the background image 52 is detected. For example, when the camera 20 is capturing an image in a certain direction, if the mount on which the camera 20 is placed is unstable, the background image 52 may move even if no moving object is captured. In another example, when the camera 20 is attached to a robot, if the robot is standing in an unstable location, the image captured by the camera 20 may shake even if the robot is not moving. This shaking is different from the movement of the user monitoring the displayed image and may cause motion sickness. In step S19, the calculation unit 34 separates the background image 52 and then detects the movement of the background image 52 as movement of the specific image 50. Since the detection is performed after separating the background image 52, the accuracy of the movement detection can be improved.

[0027] Next, in step S23, the calculation unit 34 performs a third calculation process. In the third calculation process, the position of the second display range 35 within the first display range 33 is moved in accordance with the movement of the specific image 50. Moving the position of the second display range 35 within the first display range 33 in accordance with the movement of the specific image 50 means that the image of the second display range 35 is extracted so that the background image 52 appears stationary by moving the second display range 35 in accordance with the movement of the background image 52.

[0028] As a specific example, as shown in FIG. 4, the calculation unit 34 processes images from frame 1 to frame 3, which are three chronologically consecutive frames. In this example, the background image 52 moves to the right in the figure from frame 1 to frame 3. In step S23, the calculation unit 34 extracts images of the second display range 35 that follow the background image 52 in each frame in accordance with the movement of the background image 52. As a result of this tracking process, the extracted chronological images of the second display range 35 appear to be still images, even though the background image 52 is moving within the chronological images of the first display range 33.

[0029] In this embodiment, the image acquisition unit 32 acquires an image within the first display range 33, and does not acquire image data outside the first display range 33. Therefore, when the edge of the second display range 35 coincides with the edge of the first display range 33, or when a part of the second display range 35 deviates from the first display range 33, the calculation unit 34 is configured to reset the second display range 35.

[0030] Specifically, after step S23, the calculation unit 34 performs the process of step S25. In step S25, the calculation unit 34 determines whether the edge of the second display range 35 coincides with the edge of the first display range 33. In this embodiment, even when a part of the second display range 35 deviates from the first display range 33, the calculation unit 34 treats this as if the edge of the second display range 35 coincides with the edge of the first display range 33. For example, as shown in FIG. 4, assume that the edge of the second display range 35 coincides with the edge of the first display range 33 in frame 3. In this case, the calculation unit 34 makes a positive determination in step S25. As shown in FIG. 2, if the determination in step S25 is positive, the process of the calculation unit 34 proceeds to step S33. Then, in step S33, the calculation unit 34 temporarily cancels the setting of the second display range 35. Then, the processing of the image acquisition unit 32 again acquires the first display range 33 in step S11, and the processing of the calculation unit 34 proceeds to steps S13 and S15 to reset the second display range 35. That is, the calculation unit 34 again performs the first calculation processing.

[0031] At this time, as shown in the example of FIG. 4, the image displayed on the display 40 appears to jump from frame 3 to frame 4 as the background image 52 is updated. In other words, the background appears to shift in an instant. Because this jump or shift is a sudden change, it is expected that the brain will not perceive it as strange. This can prevent the user from feeling sick. Note that an intermediate image may be inserted between frame 3 and frame 4.

[0032] After step S25, in step S27, the video output unit 36 ​​performs an output process of outputting the image of the second display range 35 to the display 40. After step S27, the video acquisition unit 32 performs the process of step S11 again.

[0033] In this way, the video processing device 30 repeats the processes from step S11 to step S27, and continues outputting images of the second display range 35. If the second display range 35 has already been set, step S15 is skipped based on the determination in step S13, and steps S11, S17, S19, S23, S25, and S27 are repeated. As a result, a time series of specific images 50 of the second display range 35 obtained by repeating the second calculation process and the third calculation process is output.

[0034] (Operation of the first embodiment) The operation of the first embodiment will be described with reference to FIGS. For example, suppose that surveillance is performed by displaying images acquired by camera 20 on display 40. In this case, if the movement of the camera's 20 imaging range and the movement of the user's head always match, the movement of the image from camera 20 matches the movement of the user's head sensed by the human semicircular canals, and the user is unlikely to feel motion sickness. If the movement of the camera's 20 imaging range and the movement of the user's head do not match, the user may feel motion sickness.

[0035] In this regard, in this embodiment, by moving the position of the second display range 35 within the first display range 33 in accordance with the movement of the specific image 50, the image displayed on the display 40 to which the second display range 35 is output appears to be stationary, even though the captured image is moving. In other words, even if the surveillance video is moving at a certain speed and there is no other reference video within the field of view, by outputting it to the display 40 so that it appears stationary, the person will not mistakenly believe that they are moving. This can prevent the user from feeling motion sickness.

[0036] (Effects of the first embodiment) According to the first embodiment described above in detail, the following effects are achieved. (1-1) Image processing system 10 includes camera 20 that can change the imaging range by following the direction of the user's head, image processing device 30 that processes and outputs images captured by camera 20, and display 40 that displays the images output by image processing device 30. Image processing device 30 executes the following operations: an acquisition process that acquires an image in first display range 33 from camera 20; a first calculation process that sets, within first display range 33 of the image acquired in the acquisition process, a second display range 35 that is smaller than first display range 33; a second calculation process that detects movement within first display range 33 of specific image 50 displayed in second display range 35; a third calculation process that moves the position of second display range 35 within first display range 33 in accordance with the movement of specific image 50; and an output process that outputs a time series of specific image 50 in second display range 35 obtained by repeating the second and third calculation processes.

[0037] According to this configuration, by moving the position of the second display range 35 within the first display range 33 in accordance with the movement of the specific image 50, the image displayed on the display 40 to which the second display range 35 is output appears to be still, even though the captured image is moving. In other words, even if the surveillance video is moving at a certain speed and there is no other reference video within the field of view, by outputting the surveillance video to the display 40 so that it appears to be still, the person will not mistakenly believe that they are moving. This can prevent the user from feeling motion sickness.

[0038] (1-2) In the second calculation process, the video processing device 30 separates the background image 52 and the moving object image from the specific image 50 in the second display range 35, and detects movement of the background image 52 as movement of the specific image 50. According to this configuration, after separating the background image 52, the process proceeds to step S19, where movement of the specific image 50 displayed in the second display range 35 within the first display range 33 is detected, thereby improving the accuracy of movement detection.

[0039] (1-3) The video processing device 30 performs the first arithmetic process again when the edge of the second display range 35 coincides with the edge of the first display range 33, or when a part of the second display range 35 deviates from the first display range 33. With this configuration, the image displayed on the display 40 appears to jump as the background image 52 is updated. In other words, the background appears to shift in an instant. Because this jump or shift is a sudden change, it is expected that the brain will not sense any discomfort. This can prevent the user from feeling motion sickness.

[0040] Second Embodiment An image processing system 10 according to the second embodiment will be described with reference to Figures 5 and 6. Image processing system 10 according to the second embodiment differs from image processing system 10 according to the first embodiment mainly in that whether or not to perform tracking processing is determined depending on the magnitude relationship between head velocity 60, which is the speed of the movement of the user's head, and a specified value.

[0041] As shown in Fig. 5, the video processing device 30 executes the processes of steps S11 to S19. The processes of these steps are the same as those in the first embodiment, and therefore will not be described here. In the second embodiment, the calculation unit 34 executes the process of step S21 after step S19. In the fourth calculation process of step S21, the calculation unit 34 calculates the head velocity 60 of the user based on the image of the first display range 33 acquired in the acquisition process of step S11. Then, the calculation unit 34 compares the calculated head velocity 60 with a predetermined specified value.

[0042] An example of a method for calculating head velocity 60 will be described using the example of FIG. 6. The camera 20 used in this embodiment is a camera 20 that can change the capture range by tracking the direction of the user's head. Methods for detecting the direction of the user's head include a gyro sensor, image recognition of the user's head, and image recognition of the user's eyes. In this embodiment, the calculation unit 34 functions as an estimation device that estimates head velocity 60, which is the speed of the user's line of sight movement, based on the detection results of the gyro sensor. In one example, head movement may be defined as follows: Assume that the camera 20 captures three images. Frame 1 is an image taken at time t1, and the head direction is θ1=0°. Similarly, frame 2 is an image taken at time t2, and the head direction is θ2. Frame 3 is an image taken at time t3, and the head direction is θ3. In this case, head velocity 60 from frame 1 to frame 2 is expressed as (θ2-θ1) / (t2-t1). The head velocity 60 from frame 2 to frame 3 is expressed as (θ3-θ2) / (t3-t2). In other words, the head velocity 60 at any time can be calculated as dθ / dt.

[0043] As shown in step S21 of FIG. 5, in the fourth calculation process, the calculation unit 34 compares the head velocity 60 with a preset specified value. If, in step S21, the head velocity 60 estimated by the estimation device is less than a predetermined value, the processing of the calculation unit 34 proceeds to a third calculation process in step S23. As a result, a process is performed to move the position of the second display range 35 within the first display range 33 in accordance with the movement of the specific image 50. Then, a time series of the specific image 50 in the second display range 35 obtained by repeating the second calculation process and the third calculation process is output. In this case, as in the first embodiment, in step S27, the video output unit 36 ​​outputs to the display 40 an image that appears to be stationary despite the fact that the background is moving.

[0044] In step S21, if the head velocity 60 estimated by the estimation device is equal to or greater than a predetermined value, the calculation unit 34 does not perform the third calculation process in step S23. Instead, in step S29, the calculation unit 34 performs a process that does not follow the movement of the specific image 50. In other words, the calculation unit 34 maintains the position of the second display range 35 without changing it. Thereafter, in step S31, the calculation unit 34 performs a process of outputting an image of the second display range 35 within the first display range 33. In other words, the image within the second display range 35 set in the first calculation process is output to the display 40 as is.

[0045] After step S31, the processing of the calculation unit 34 proceeds to step S33. In step S33, the calculation unit 34 cancels the setting of the second display range 35. The processing content of step S33 is the same as in the first embodiment. Then, the video acquisition unit 32 again performs the acquisition processing of the first display range 33 in step S11, and the processing of the calculation unit 34 proceeds to steps S13 and S15, where the second display range 35 is reset.

[0046] (Operation of the second embodiment) The operation of the second embodiment will be described with reference to FIGS. In this embodiment, similar to the first embodiment, when a background image 52 is set as the second display range 35 within the first display range 33, even if the background image 52 moves, the second display range 35 follows and the movement of the displayed image is stopped.

[0047] Here, if head velocity 60, which is the speed of the user's head movement, is equal to or greater than a certain specified value, the user is significantly changing the direction of their head, and the background is significantly moving accordingly. In this case, the movement of the image from camera 20 matches the movement of the human semicircular canals, so there is little chance of the user feeling sick. In such a case, it is possible to prevent the user from feeling sick without performing a process to stop the movement of second display range 35, which is background image 52.

[0048] Therefore, if head velocity 60 is equal to or greater than a certain specified value, the image displayed in second display range 35 can be output as is, thereby preventing the user from feeling sick. (Effects of the second embodiment) According to the second embodiment described above in detail, in addition to the effects of the first embodiment, the following effects are achieved.

[0049] (2-1) The image processing system 10 further includes an estimation device that estimates a head velocity 60, which is the speed of the user's head movement. If the head velocity 60 estimated by the estimation device is equal to or greater than a predetermined value, the image processing device 30 outputs, in the output process, a time series of images within the second display range 35 set in the first calculation process without performing the third calculation process; if the head velocity 60 estimated by the estimation device is less than the predetermined value, the image processing device 30 outputs, in the output process, a time series of specific images 50 within the second display range 35 obtained by repeating the second calculation process and the third calculation process. According to this configuration, when the head velocity 60, which is the speed of the user's head movement, is equal to or greater than a certain specified value, the user is significantly changing the direction of their head, and therefore, by outputting the image displayed in the second display range 35 as is, it is possible to prevent the user from feeling sick.

[0050] Third Embodiment An image processing system 10 according to the third embodiment will be described with reference to Figures 7 and 8. The image processing system 10 according to the third embodiment differs from the image processing system 10 according to the first embodiment mainly in that the image processing system 10 determines whether to perform tracking processing depending on the magnitude relationship between the moving speed 62 of a specific image 50 and the first threshold value.

[0051] As shown in Fig. 7, the video processing device 30 executes the processes of steps S11 to S17. The processes of these steps are the same as those in the first embodiment, and therefore will not be described here. In the third embodiment, the calculation unit 34 executes the process of step S19A after step S17. In the second calculation process of step S19A, the calculation unit 34 extracts the specific image 50, as in the first embodiment. Then, the calculation unit 34 compares the moving speed 62 of the specific image 50 displayed in the second display range 35 within the first display range 33 with a first threshold value.

[0052] Using the example shown in FIG. 8, a method for calculating the movement speed 62 of the specific image 50 within the first display range 33 will be described. Assume that the camera 20 captures three images, and frame 1 is an image captured at time t1, with the coordinate of the specific image 50 within the first display range 33 being x1. The coordinates may be any predetermined point, such as the center of the specific image 50 or the top left point of the specific image 50. Similarly, assume that frame 2 is an image captured at time t2, with the coordinate of the specific image 50 within the first display range 33 being x2. Assume that frame 3 is an image captured at time t3, with the coordinate of the specific image 50 within the first display range 33 being x3. In this case, the movement speed 62 of the specific image 50 from frame 1 to frame 2 is expressed as (x2-x1) / (t2-t1). The movement speed 62 of the specific image 50 from frame 2 to frame 3 is expressed as (x3-x2) / (t3-t2). That is, the moving speed 62 of any specific image 50 at any time can be calculated as dx / dt.

[0053] As shown in step S19A of Figure 7, in the second calculation process, the calculation unit 34 compares dx / dt, which is the movement speed 62 of the specific image 50 displayed in the second display range 35 within the first display range 33, with a predetermined first threshold value.

[0054] In step S19A, if the moving speed 62 of the specific image 50 within the first display range 33 is less than the predetermined first threshold, the calculation unit 34 executes a third calculation process in step S23. Then, a process is performed to move the position of the second display range 35 within the first display range 33 in accordance with the movement of the specific image 50. As a result, similar to the first embodiment described above, the video output unit 36 ​​outputs an image in which the background appears to be stationary to the display 40 in step S27.

[0055] In step S19A, if the moving speed 62 is equal to or greater than the first threshold, the processing of the calculation unit 34 does not proceed to the third calculation processing of step S23. Instead, in step S29, the calculation unit 34 performs processing that does not follow the movement of the specific image 50. In other words, the calculation unit 34 maintains the position of the second display range 35 without changing it. Thereafter, in step S31, processing is performed to output an image of the second display range 35 within the first display range 33. As shown in FIG. 8, if the moving speed 62dx / dt is equal to or greater than the first threshold, an image that is the image displayed in the second display range 35 is output to the display 40 as is. After the output of step S31, the processing of the calculation unit 34 proceeds to step S33 before acquiring the next image. In step S33, the calculation unit 34 cancels the setting of the second display range 35. Note that the processing content of step S33 is the same as in the first embodiment. Then, the image acquisition unit 32 acquires the first display range 33 again in step S11, and the processing of the calculation unit proceeds to steps S13 and S15, where the second display range is reset.

[0056] (Operation of the third embodiment) The operation of the third embodiment will be described with reference to FIGS. In this embodiment, similar to the first embodiment, when a background image 52 is set as the second display range 35 within the first display range 33, even if the background image 52 moves, the second display range 35 follows and the movement of the displayed image is stopped.

[0057] Here, if the moving speed 62 of the background image 52 exceeds a certain threshold, it can be determined that the background is not still. If the background that the user is looking at moves significantly, it is natural for the displayed image to move significantly to match the background. For such large movements, if the image is made to move significantly to match the background, it is possible to prevent the user from feeling sick.

[0058] Therefore, when the second display range 35, which is the background image 52, moves within the first display range 33, a threshold value is set for the movement speed 62 of the background image 52 to stop the movement. By setting a threshold value, it is possible to distinguish between cases where processing to stop the background is required and cases where the movement of the background is output as is. This makes it possible to specifically set the range of background movement that should suppress sickness using the threshold value of the movement speed 62.

[0059] (Effects of the third embodiment) According to the third embodiment described above in detail, in addition to the effects of the first embodiment, the following effects are achieved.

[0060] (3-1) The video processing device 30 executes the third calculation process when the moving speed 62 of the specific image 50 within the first display range 33 is equal to or less than a predetermined first threshold. With this configuration, it is possible to specifically set the range of the magnitude of background movement that should suppress sickness.

[0061] <Example of change> The above embodiment can be modified as follows: The above embodiment and the following modifications can be combined with each other within the scope of technical compatibility.

[0062] In the above embodiments, if an object is made up of multiple objects, the multiple objects may be integrated, and conversely, if an object is made up of a single object, it may be divided into multiple objects. Regardless of whether the objects are integrated or not, it is sufficient that the object of the invention can be achieved.

[0063] The camera 20 does not have to be a stereoscopic camera. In other words, the camera 20 may have a single set of a lens and an image sensor. The display 40 does not have to be a head-mounted display, but may be a stationary display installed in a room or the like.

[0064] In each embodiment, the head direction may be detected by performing image recognition of the user's head. The head direction may also be detected by performing image recognition of the user's eyeballs to detect the direction of the pupils. Note that when detecting the head direction in this manner, it is not essential that the display 40 include a gyro sensor.

[0065] The above embodiments may be integrated and implemented based on a flowchart such as that shown in Fig. 9. That is, both the process of comparing the movement speed 62 within the first display range 33 of the specific image 50 with the first threshold value (step S19A) and the process of comparing the head speed 60 with a predetermined specified value (step S21) may be executed. In this case, as shown in Fig. 9, if the determinations in both steps S19A and S21 are positive, the process of step S23 may be executed.

[0066] The process of step S25 in each embodiment is not essential. In each embodiment, when the background image 52 is set as the second display range 35, the image may be made clearer by determining the contours by overlapping frames of the background image 52. With this configuration, even if the resolution has become coarse due to digital zoom or the like, a clear image can be created by adding together multiple frames.

[0067] The setting range of the second display range 35 can be changed as needed. For example, the second display range 35 does not necessarily have to be rectangular, and may be circular, elliptical, or any polygonal shape other than a rectangle.

[0068] The background image 52 does not have to be set as the specific image 50. For example, a specific object within the second display range 35 may be set as the specific image 50. [Explanation of symbols]

[0069] 10...Video processing system 20...Camera 30...Video processing device 32...Video acquisition unit 33...First display range 34...Arithmetic section 35...Second display range 36...Video output section 40...Display 42...Head-mounted display 50...Specific image 52...Background image 60…Head speed 62…Movement speed

Claims

1. a camera capable of changing the range of photography by following the direction of the user's head; a video processing device that processes and outputs images captured by the camera; a display that displays an image output by the video processing device; Equipped with The video processing device includes: an acquisition process for acquiring an image of a first display range from the camera; a first calculation process for setting a second display range, the second display range being smaller than the first display range, within the first display range of the image acquired in the acquisition process; a second calculation process for detecting movement of the specific image displayed in the second display range within the first display range; a third calculation process for moving the position of the second display range within the first display range in accordance with the movement of the specific image; an output process of outputting the specific image of the second display range obtained by repeating the second calculation process and the third calculation process in chronological order; To execute Video processing system.

2. The method further includes an estimation device for estimating a head velocity, which is a velocity of the movement of the user's head, The video processing device includes: When the head velocity estimated by the estimation device is equal to or greater than a predetermined value, the third calculation process is not performed, and in the output process, images within the second display range set in the first calculation process are output in chronological order; When the head velocity estimated by the estimation device is less than the specified value, the specific image of the second display range obtained by repeating the second calculation process and the third calculation process is output in chronological order in the output process. The video processing system according to claim 1 .

3. the video processing device executes the third arithmetic process when a moving speed of the specific image within the first display range is equal to or less than a predetermined first threshold. The video processing system according to claim 1 .

4. In the first calculation process, the video processing device sets the second display range to the center of the first display range. The video processing system according to claim 1 .

5. In the second calculation process, the video processing device separates a background image and a moving object image from the specific image in the second display range, and detects movement of the background image as movement of the specific image. The video processing system according to claim 1 .

6. the video processing device performs the first arithmetic processing again when an edge of the second display range coincides with an edge of the first display range or when a part of the second display range deviates from the outside of the first display range. The video processing system according to claim 1 .

Images