Imaging device, display device, control method for imaging device, and imaging system

Abstract

To reduce a shift in calibration settings.SOLUTION: An imaging apparatus has: imaging means; detection means that detects a posture of the head of a user with respect to the imaging apparatus; decision means that, on the basis of the posture of the head detected by the detection means, decides a recording area cut out from an imaging range by the imaging means; determination means that determines whether to adjust a shift in a position of the recording area in the imaging range; and notification means that, when the determination means determines to adjust the shift in the position of the recording area, transmits a notification prompting the adjustment of the shift in the position of the recording area to a display unit that displays information on the shift in the position of the recording area.SELECTED DRAWING: Figure 15

Description

Technical Field

[0001] The present invention relates to an imaging device, a display device, a control method for an imaging device, and an imaging system.

Background Art

[0002] When imaging with a handheld camera, since the photographer operates such as imaging while keeping the camera directed in the imaging direction, it is difficult to use hands for other than the imaging act or to experience the on-site feeling of being present there while directing consciousness to things other than imaging.

[0003] For example, it is difficult for a photographer trying to image a child to hold the child during imaging or to image while holding the child. Also, a photographer during sports watching may not be conscious of cheering and may not remember the game content, or may concentrate on sports watching and interrupt imaging. Even during a group trip, when the photographer concentrates on imaging, he / she cannot experience the same level of感动 as other members, and when giving priority to experience, he / she neglects imaging.

[0004] Therefore, by imaging a wide range with an omnidirectional camera and cutting out and editing a desired video portion from the captured omnidirectional video, the photographer can also leave in the video the fact that he / she concentrated on the experience during the experience. However, the editing of omnidirectional video is complicated because it involves a series of operations such as image conversion work and designation of the cut-out area.

[0005] It is conceivable to detect the line-of-sight position during imaging and automatically determine the cut-out area based on the detected line-of-sight position. However, the detection of the line-of-sight position may cause errors due to individual differences or the positional relationship with the sensor. Various calibration processes have been proposed to correct the errors in the detection of the line-of-sight position.

[0006] Patent Document 1 discloses a process for displaying an AR image at a position on a display corresponding to a predetermined reference position, and performing calibration using the direction in which the AR image appears to overlap the reference position as the reference direction. Patent Document 2 discloses a process for displaying an image on a display control device equipped with a camera that detects the user's gaze, displaying multiple set positions on the image, and performing gaze calibration based on the gaze fixed on the set positions. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2016-110489 [Patent Document 2] Japanese Patent Publication No. 2015-153302 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] However, even with calibration before image acquisition, the positional relationship between the gaze detection device and the photographer may change during image acquisition, potentially causing a shift in the calibration settings. If this shift in calibration settings during image acquisition is not taken into account, it becomes difficult to properly extract the area the user was viewing from the 360-degree image.

[0009] The present invention aims to provide a technique for reducing deviations in calibration settings. [Means for solving the problem]

[0010] Book Invention First aspect teeth, An imaging device, Imaging means, A detection means for detecting the posture of the user's head relative to the imaging device, Determination means for determining a recording area cut out from an imaging range by the imaging means based on the posture of the head detected by the detection means; Determination means for determining whether or not to adjust a positional deviation of the recording area in the imaging range; Notification means for transmitting a notification prompting adjustment of the positional deviation of the recording area to a display device that displays information regarding the positional deviation of the recording area when it is determined by the determination means that the positional deviation of the recording area is to be adjusted; 、 A distance measuring sensor that measures the distance to the subject, having death, The determination means determines that the misalignment of the recording area should be adjusted when the amount of change per unit time of the measurement value by the distance sensor exceeds a predetermined threshold. being characterized by It is an imaging device. . A second aspect of the present invention is: An imaging device, Imaging means, A detection means for detecting the posture of the user's head relative to the imaging device, A determination means that determines a recording area to be cut out from the imaging range of the imaging means based on the posture of the head detected by the detection means, A determination means for determining whether or not to adjust the misalignment of the recording area in the imaging range, If the determination means determines that the misalignment of the recording area should be adjusted, the notification means transmits a notification prompting the adjustment of the misalignment of the recording area to a display device that displays information regarding the misalignment of the recording area. It has, The notification means is, The information of the head posture when a calibration process is performed to adjust the position of the recording area in the imaging range to suit the user, and the information detected by the detection means The current head posture information and the information regarding the misalignment of the recording area obtained based on that information are transmitted to the display device. When the tilt of the imaging device changes, information regarding the misalignment of the recording area is reacquired and transmitted to the display device. This imaging device is characterized by the following features.

Advantages of the Invention

[0011] According to the present invention, it is possible to reduce a deviation in calibration settings.

Brief Description of the Drawings

[0012] [Figure 1] It is an external view of a camera according to an embodiment. [Figure 2] It is a diagram showing a state in which a user wears a camera. [Figure 3]This is a view of the battery compartment from the rear of the camera. [Figure 4] This is a view of the shooting and detection unit from the left side of the camera. [Figure 5] This is an external view of a 360-degree camera. [Figure 6] This diagram illustrates how to extract a region from a 360-degree image. [Figure 7] This is an external view of the display device used for calibration processing. [Figure 8] This diagram illustrates the simplified calibration process. [Figure 9] This is a functional block diagram of the camera. [Figure 10] This diagram illustrates the hardware configuration of a camera. [Figure 11] This diagram illustrates the hardware configuration of a display device. [Figure 12] This is a flowchart illustrating the standard calibration process. [Figure 13] This flowchart illustrates the details of the standard calibration process. [Figure 14] This diagram illustrates positioning indicators. [Figure 15] This is a flowchart illustrating the calibration verification process. [Figure 16] This diagram illustrates the misalignment of the recording area. [Figure 17] This is a flowchart for a simple calibration process on a display device. [Figure 18] This is a flowchart for a simple calibration process using a camera. [Modes for carrying out the invention]

[0013] <Embodiment> Embodiments of the present invention will now be described with reference to the drawings. Figure 1 is an external view of camera 1 according to an embodiment. Figure 1 shows an external view of camera 1 as an example of an imaging device to which the present invention can be applied. In Figure 1, camera 1 comprises an imaging / detection unit 10, a battery unit 90, and a connection unit 80 that connects the imaging / detection unit 10 and the battery unit 90.

[0014] The imaging / detection unit 10 includes a face direction detection window 13, a start switch 14, a stop switch 15, an imaging lens 16, an LED 17, and microphones 19L and 19R. The face direction detection window 13 transmits infrared light and its reflected light emitted from an infrared LED lighting circuit 21 (Figure 10) built into the imaging / detection unit 10 for detecting the position of various parts of the user's (photographer's, user's) face. The start switch 14 is a switch for starting imaging. The stop switch 15 is a switch for stopping imaging.

[0015] The imaging lens 16 directs the light rays to be imaged to the solid-state image sensor 42 inside the imaging / detection unit 10 (Figure 1 This leads to 0). LED17 is an LED that indicates imaging is in progress or provides a warning. Microphones 19R and 19L are microphones that pick up ambient sounds; microphone 19L picks up sounds from the left side of the user's surroundings (right side in Figure 1), and microphone 19R picks up sounds from the right side of the user's surroundings (left side in Figure 1).

[0016] Figure 2 shows the camera 1 in a state where it is worn by a user. When the battery unit 90 is attached to the user's back and the shooting / detection unit 10 is attached to the front of the user's body, the shooting / detection unit 10 is biased and supported toward the user's chest by the connecting parts 80, which are connected to both ends near the left and right ends. As a result, the shooting / detection unit 10 is positioned in front of the user's collarbone.

[0017] When the user puts on camera 1, the face direction detection window 13 is positioned below the user's chin. Inside the face direction detection window 13 is an infrared focusing lens 26 (Figure 10). The optical axis (detection optical axis) of the infrared focusing lens 26 is oriented in a different direction from the optical axis (imaging optical axis) of the imaging lens 16, and the face direction detection unit 20 (Figure 10) detects the user's face direction from the position of various parts of the face, such as the neck, chin, and nose. As a result, the imaging unit 40 (Figure 10) is able to capture images in the direction of the user's face (observation direction).

[0018] Figure 3 is a view of the battery unit 90 from the rear of the camera 1. In Figure 3, the battery unit 90 includes a charging cable inlet 91, adjustment buttons 92L and 92R, and a spine protection cutout 93. The charging cable inlet 91 is an inlet for a charging cable (not shown), and the internal battery is charged from an external power source or power is supplied to the shooting / detection unit 10 via this charging cable.

[0019] The adjustment buttons 92L and 92R are used to adjust the length of the band sections 82L and 82R of the connection section 80. Adjustment button 92L is for adjusting the left band section 82L, and adjustment button 92R is for adjusting the side band section 82R. In the example shown in Figure 3, the band sections 82L and 82R are adjusted independently by the two adjustment buttons 92L and 92R, but they could also be adjusted with a single button.

[0020] The spine-relief cutout 93 is a cutout designed to avoid the spine so that the battery unit 90 does not come into contact with the user's spine. The spine-relief cutout 93 reduces discomfort when worn by avoiding the protruding part of the human spine and prevents the device from moving from side to side during use.

[0021] Figure 4 is a view of the shooting / detection unit 10 from the left side of the camera 1. The angle adjustment button 85L is a button provided on the angle holding unit 81L and is used when adjusting the angle of the shooting / detection unit 10. Although not shown in Figure 4, the angle holding unit 81R on the right side of the camera 1 has an angle adjustment button 85R located symmetrically to the angle adjustment button 85L. The angle holding units 81L and 81R are collectively referred to as the angle holding unit 81. The angle adjustment buttons 85L and 85R are collectively referred to as the angle adjustment button 85.

[0022] The user can change the angle between the imaging / detection unit 10 and the angle holding unit 81 by pressing the angle adjustment button 85 and operating the angle holding unit 81. The protrusion angle of the chest connection pad 18 can also be changed. Through the action of these two angle-changing components (angle adjustment button 85 and chest connection pad 18), the imaging / detection unit 10 can adjust the direction of the optical axis of the imaging lens 16 to be approximately horizontal, even if there are individual differences in the shape of the user's chest position.

[0023] Camera 1 can generate a video by extracting the region of the user's face direction detected via the face direction detection window 13 from the captured video. However, the camera 1 worn around the neck (capturing There may be a misalignment between the optical axis of lens 16) and the direction of the user's face, and individual differences are also expected. In order to properly crop the area that the user intends to image, it is preferable to perform a calibration process to adjust the position of the cropping area relative to the direction of the face before imaging.

[0024] Here, with reference to Figures 5 and 6, we will explain how to extract a region from a 360-degree image. Figure 5 illustrates a 360-degree camera 503 equipped with a lens 504 and a shooting button 505. The lens 504 is one of a pair of hemispherical fisheye lenses configured on both sides of the housing of the 360-degree camera 503, and the 360-degree camera 503 performs 360-degree photography using this pair of fisheye lenses. In other words, 360-degree photography is performed by combining the projected images from this pair of fisheye lenses.

[0025] Figure 6 illustrates the process of extracting a region from a 360-degree image captured by the 360-degree camera 503. Figures 6(b) and 6(c) show examples of extracting a portion of the image captured by the 360-degree camera 503 and converting it into a planar image.

[0026] Figure 6(a) shows an example of an image obtained by 360-degree imaging with the 360-degree camera 503, and the image includes the subjects: the photographer 606, the child 607, and the tree 608. The image shown in Figure 6(a) is an image from a hemispherical optical system obtained by combining the projection images of a pair of fisheye lenses, so the photographer 606 is greatly distorted.

[0027] The child 607, the subject that photographer 606 was trying to photograph, had its body positioned at the periphery of the hemispherical optical system, resulting in significant distortion and stretching of the body. On the other hand, the tree 608 was positioned directly in front of lens 504, and was therefore photographed without significant distortion. By cropping a portion of the image in Figure 6(a) and converting it into a planar image, it is possible to generate an image with a field of view similar to what a person normally sees.

[0028] Figure 6(b) is an image cropped from the image in Figure 6(a) that is positioned directly in front of lens 504. In the image in Figure 6(b), the tree 608 is in the center, representing a field of view similar to that of a normal human. Since the child 607, which the photographer 606 was trying to capture, is not included in Figure 6(b), the cropped area is modified. For example, by moving the cropped area to the left and 30° downward from the tree 608 in the image in Figure 6(a), the cropped area will include the child 607. Figure 6(c) shows the image after cropping the area including the child 607 and performing a planar transformation.

[0029] When cropping a desired area after imaging (hereinafter referred to as "trimming"), the photographer can focus on the immediate experience during imaging, but the burden of the trimming work after imaging increases. Therefore, camera 1 reduces the burden of the trimming work by identifying the cropping area based on the user's face direction detected via the face direction detection window 13. A calibration process is performed to adjust the position of the cropping area relative to the face direction in order to trim the area intended by the user.

[0030] Users who wear Camera 1 will have individual differences and adjustment differences, such as body size, the orientation and angle of the neck area where Camera 1 is attached, the condition of their clothing, such as the collar, when wearing it, and differences in the adjustment of the band parts 82L and 82R. Therefore, the optical axis of the imaging lens 16 of Camera 1 and the center of the field of view when the user is facing forward (hereinafter referred to as the field of view center) do not usually coincide. For the user, it is desirable to set the center of the field of view, which corresponds to the user's posture and movement, as the center of the cropping area, rather than the optical axis of the imaging lens 16 of Camera 1.

[0031] There are individual differences in the center of the field of view and the range of neck movement when the user turns their head in directions other than directly in front, such as up, down, left, right, and diagonally. Relationship between the face direction detected by the face direction detection window 13 and the center of the field of view Individual differences also exist regarding gender. To address these individual differences, users perform a calibration process that correlates face direction with the center of the field of view.

[0032] Figure 7 is an external view of the display device 700 for calibration processing. Calibration processing is a process of adjusting a cropped area (hereinafter also referred to as the recording area) identified based on the direction of the user's face from the image captured by camera 1, to match the user wearing camera 1. It is desirable that calibration processing be performed not only in front but also in all directions such as up, down, left, right, and diagonally.

[0033] In Figure 7, the display device 700 includes a power button 702, a display unit 703, a button 704, an in-camera 705, a face sensor 706, an angular velocity sensor 707, and an acceleration sensor 708. The display device 700 also includes a communication interface that enables high-speed connection with the camera 1 via wireless LAN or the like.

[0034] The power button 702 accepts ON / OFF operation of the display device 700 when pressed and held. The power button 702 also accepts instructions for various processes when pressed and short. The display unit 703 can display images captured by camera 1 or display menu screens such as calibration settings. A transparent touch sensor is provided on the top surface of the display unit 703, and the touch sensor accepts touch operations on the displayed screen (for example, the menu screen). Button 704 is a button that accepts user instructions for various functions such as calibration processing.

[0035] The in-camera 705 is a camera capable of capturing images of a person observing the display device 700. The face sensor 706 detects the face shape and face direction of a person observing the display device 700. The face sensor 706 is a variety of sensors, such as an optical sensor, a ToF sensor, or a millimeter-wave radar. The angular velocity sensor 707 is located inside the display device 700 and is therefore shown by a dotted line. The display device 700 has a calibrator function and is equipped with a 3D X, Y, Z gyro sensor (angular velocity sensor 707). The accelerometer 708 detects the orientation (tilt) of the display device 700.

[0036] The display device 700 implements the function of a calibrator by incorporating an in-camera 705 and various sensors. When the user turns their face in five directions, for example, forward, up, down, left, and right, the cropping range is determined more appropriately by performing a calibration process in each direction. The calibration process performed for multiple directions using the in-camera 705 and various sensors will hereafter be referred to as the standard calibration process.

[0037] The standard calibration process is expected to be performed when camera 1 is first started up. However, the mounting position of camera 1 on the user may shift while imaging is in progress. Since it would be cumbersome for the user to perform the calibration process again for multiple directions when a shift occurs, the display device 700 has a function to perform the calibration process simply. For example, the display device 700 has a function to display a confirmation image showing the amount of shift in the head posture, allowing the user to perform the calibration process by adjusting the position of camera 1. This calibration process is also called the simplified calibration process. The standard calibration process and the simplified calibration process are collectively referred to as the calibration process.

[0038] Referring to Figure 8, a simplified calibration process on the display device 700 will be explained. Figure 8(a) shows an example in which the display device 700 displays calibration information (information regarding the misalignment of the recording area) received from the camera 1. The misalignment of the recording area occurs when the user's head posture is misaligned relative to the camera 1. Therefore, the misalignment of the recording area This information can be expressed, for example, as a misalignment of the head's posture.

[0039] The display device 700 displays a confirmation image to adjust the positional misalignment of the recording area based on the amount of deviation (angle deviation in the three axes of X, Y, and Z) between the detected current head posture and the head posture when the calibration process was performed. In the example in Figure 8(a), the confirmation image is an image of the head with arrows indicating the amount of deviation superimposed. The confirmation image may also be a three-dimensional image.

[0040] Furthermore, the display device 700 may change the color of the head image according to the amount of displacement. For example, the color of the head image can be red if the amount of displacement is greater than or equal to a predetermined value, yellow if it is less than the predetermined value, and green when the amount of displacement is almost zero. The display device 700 may also display the head image when there is no displacement in a different color from the head image that indicates the current state.

[0041] Figure 8(b) shows the state in which camera 1 is positioned appropriately after the simplified calibration process. The user adjusts the position and tilt of camera 1 while looking at the image on the display unit 703 as illustrated in Figure 8(b), so that the arrow 710 indicating the direction of the face's front coincides with the optical axis direction (Z axis direction) of camera 1. During the simplified calibration process, if the user changes the mounting position of camera 1 and the tilt of camera 1 changes, the display device 700 receives information regarding the misalignment of the recording area reacquired by camera 1. The display device 700 updates the confirmation image based on the received information regarding the misalignment of the recording area.

[0042] The user presses button 704 to end the simplified calibration process when the arrow 710 indicating the direction of the face's front aligns with the optical axis direction of camera 1. Note that the simplified calibration process is not limited to the method illustrated in Figures 8(a) and 8(b). For example, the simplified calibration process may involve performing a standard calibration process for the front direction and correcting the amount of deviation in the up, down, left, and right directions based on the amount of deviation in the front direction. Also, the display device 700 is not limited to displaying the amount of deviation of the head posture as a confirmation image, but may also display the amount of deviation of camera 1. The user can adjust the position and tilt of camera 1 while checking the amount of deviation of camera 1.

[0043] The display device 700 is, for example, a mobile device such as a smartphone. A smartphone acting as the display device 700 can perform calibration processing by making its internal firmware compatible with the firmware in camera 1. Alternatively, the display device 700 can also perform calibration processing by making the firmware in camera 1 compatible with the smartphone's application or operating system.

[0044] The standard calibration process and the simplified calibration process may be implemented using different devices. The display device 700 for the simplified calibration process only needs to be able to display the calibration deviation information received from camera 1 along with the head image, and may be a simple device without an in-camera 705 or various sensors.

[0045] Figure 9 is a functional block diagram of camera 1. Camera 1 comprises a face direction detection unit 20, a recording area determination unit 30, an imaging unit 40, a development processing unit 50, a primary recording unit 60, and a transmission unit 70. Each functional block will be described with reference to the hardware configuration shown in Figure 10.

[0046] The face direction detection unit 20 detects the face direction using an infrared LED 22 and an infrared detection processing device 27 located inside the face direction detection window 13. The detected face direction is the posture of the user's head relative to the camera 1. The posture of the head is determined based on the positional relationship of the user's jaw base, chin, nose, etc., as detected by the infrared detection processing device 27, etc., and is used to determine the left and right position of the camera 1. The orientation (angle) of the head in both the directional and vertical directions can be obtained. The detected face orientation (user's head posture) information is output to the recording area determination unit 30.

[0047] The recording area determination unit 30 uses the face direction information acquired by the face direction detection unit 20 to determine the position and range of the recording area to be extracted from the image captured by the shooting unit 40. The range (size) of the recording area may be a predetermined fixed size, or it may be variable, for example, according to the size of the subject. The information regarding the position and range of the recording area is, for example, imaging direction and field of view information. The recording area determination unit 30 outputs the determined position and range information of the recording area to the development processing unit 50.

[0048] The imaging unit 40 converts light rays from the subject into an image and outputs the converted image to the development processing unit 50. The development processing unit 50 uses the imaging direction and field of view information from the recording area determination unit 30 to extract and develop the recording area from the image captured by the imaging unit 40. The development processing unit 50 extracts the image (recording area) in the direction the user is looking and stores it in the primary recording unit 60. The primary recording unit 60 is a built-in primary memory 103 (Figure 10) or the like that records the image information. The image information recorded in the primary recording unit 60 can be transmitted to an external device by the transmission unit 70.

[0049] The transmitting unit 70 is a communication interface for communicating with a predetermined communication partner. The transmitting unit 70 is wirelessly connected to, for example, the display device 700, the calibrator 800, and the simple display device 900, and transmits the image captured by the imaging unit 40 and information about the recording area. The camera 1 is connected to at least one of the display device 700, the calibrator 800, and the simple display device 900 to constitute an imaging system.

[0050] The display device 700 is a display device that can be connected to the camera 1's transmitter 70 via a high-speed wireless LAN (hereinafter referred to as "high-speed wireless"). High-speed wireless communication is, for example, IEEE 802. This is wireless communication compliant with communication standards such as WiFi 5 and WiFi 4, in addition to the 11ax (WiFi 6) standard. The display device 700 may be a device developed specifically for camera 1, or it may be a general-purpose smartphone, tablet, etc.

[0051] Furthermore, the display device 700 may be connected to the camera 1's transmitter 70 using low-power wireless communication. The display device 700 and the transmitter 70 may be connected using both high-speed wireless and low-power wireless communication, or switched between the two. Video footage and other data may be transmitted using high-speed wireless communication, while lighter data with less data volume than video footage, or data that does not require a long transmission time, may be transmitted using low-power wireless communication. Low-power wireless communication may be Bluetooth® or other short-range wireless communication such as NFC (Near Field Communication).

[0052] The calibrator 800 is a device for initializing and individually configuring the camera 1, and, like the display device 700, is a device that can be connected to the camera 1's transmitter 70 via high-speed wireless or low-power wireless. The display device 700 may also function as the calibrator 800 by incorporating the functions of the calibrator 800.

[0053] The simple display device 900 is connected to the camera 1 via low-power wireless communication. While the simple display device 900 is not suitable for transmitting video due to time constraints, it can be used to instruct the camera on when to start and stop imaging, or to receive images for checking composition. The simple display device 900 may be a device specifically developed for the camera 1, or it may be a smartwatch or similar device.

[0054] Figure 10 is a diagram illustrating the hardware configuration of camera 1. In Figure 10, camera 1 The system includes a control unit 101, a power switch 11, an imaging mode switch 12, a face direction detection window 13, a start switch 14, a stop switch 15, an imaging lens 16, and an LED 17.

[0055] Camera 1 comprises an infrared LED lighting circuit 21, an infrared LED 22, an infrared focusing lens 26, and an infrared detection processing device 27, which constitute a face direction detection unit 20 (Figure 9). Camera 1 comprises an imaging driver 41, a solid-state image sensor 42, and an imaging signal processing circuit 43, which constitute an imaging unit 40 (Figure 9). Camera 1 comprises a low-power wireless unit 71 and a high-speed wireless unit 72, which constitute a transmission unit 70 (Figure 9).

[0056] In the example shown in Figure 10, camera 1 has one imaging unit 40, but it may have two or more imaging units 40. By having multiple imaging units 40, camera 1 can capture 3D images (three-dimensional images), capture images with a wider angle of view than that which can be obtained with one imaging unit 40, and capture images in multiple directions.

[0057] Camera 1 is equipped with various types of memory, including a large-capacity non-volatile memory 51, an internal non-volatile memory 102, and a primary memory 103. Camera 1 is also equipped with an audio processing unit 104, a speaker 105, a vibrator 106, an angular velocity sensor 107, an acceleration sensor 108, a distance sensor 111, a touch sensor 112, and various switches 110.

[0058] The control unit 101 is, for example, a CPU, connected to the power switch 11, and controls the operation of the camera 1 to implement various functions. The control unit 101 functions as the recording area determination unit 30 and the development processing unit 50 in Figure 9. The control unit 101 also controls functions such as determining whether or not to adjust the misalignment of the recording area and notifying the user or the display device 700 to adjust the misalignment of the recording area.

[0059] The infrared LED lighting circuit 21 controls the lighting and extinguishing of the infrared LED 22 and controls the emission of infrared rays 23 from the infrared LED 22 toward the user. The face direction detection window 13 is composed of a visible light cut filter. The face direction detection window 13 does not transmit visible light very well, but transmits infrared rays 23 and reflected infrared rays 25 sufficiently, which are in the infrared region.

[0060] The infrared focusing lens 26 is a lens that focuses reflected light rays 25. The infrared detection processing device 27 has a sensor that detects the reflected light rays 25 focused by the infrared focusing lens 26. The sensor of the infrared detection processing device 27 forms an image of the focused reflected light rays 25, converts it into sensor data, and outputs it to the control unit 101.

[0061] As shown in Figure 2, when the user wears the camera 1 around their neck, the face direction detection window 13 is located below the user's chin. The infrared light 23 emitted from the infrared LED lighting circuit 21 passes through the face direction detection window 13 and is irradiated onto the infrared irradiation surface 24, which is near the user's chin, as shown in Figure 10. The infrared light 23 reflected by the infrared irradiation surface 24 becomes reflected light 25, passes through the face direction detection window 13, and is focused by the infrared condensing lens 26 to the sensor of the infrared detection processing device 27.

[0062] The various switches 110 are switches for executing functions other than those performed by the power switch 11, imaging mode switch 12, face direction detection window 13, start switch 14, and stop switch 15.

[0063] The imaging driver 41 includes a timing generator and other components, and generates various timing signals related to imaging. The imaging driver 41 outputs the generated timing signals to each part involved in imaging, driving the imaging process.

[0064] The solid-state image sensor 42 outputs a signal obtained by photoelectric conversion of the subject image projected from the imaging lens 16 to the imaging signal processing circuit 43. The imaging signal processing circuit 43 outputs imaging data generated by processing such as clamping and A / D conversion to the signal from the solid-state image sensor 42 to the control unit 101.

[0065] The built-in non-volatile memory 102 is, for example, flash memory and stores the startup program for the control unit 101 and the settings for various program modes. The built-in non-volatile memory 102 also stores settings such as changes to the observation field of view (angle of view) and the effectiveness level of vibration control.

[0066] The primary memory 103 is, for example, RAM (Random Access Memory) and temporarily stores video data being processed, calculation results from the control unit 101, calibration information, etc.

[0067] The large-capacity non-volatile memory 51 records captured image data and the like. The large-capacity non-volatile memory 51 may be a semiconductor memory without a removable mechanism, or it may be a removable recording medium such as an SD card. Furthermore, the large-capacity non-volatile memory 51 may be configured as a memory integrated with the built-in non-volatile memory 102.

[0068] The low-power wireless unit 71 transmits and receives data via low-power wireless communication between the display device 700, the calibrator 800, and the simplified display device 900. The high-speed wireless unit 72 transmits and receives data via high-speed wireless communication between the display device 700 and the calibrator 800.

[0069] The audio processing unit 104 is equipped with a right microphone 19R and a left microphone 19L that pick up external sounds (analog signals), processes the picked-up analog signals, and generates audio signals. The LED 17, speaker 105, and vibrator 106 notify or warn the user of the status of camera 1 by emitting light, sound, or vibrating.

[0070] The angular velocity sensor 107 is a gyroscope and detects the movement of the camera 1 itself. The acceleration sensor 108 detects the orientation (tilt) of the shooting / detection unit 10. The angular velocity sensor 107 and the acceleration sensor 108 are built into the shooting / detection unit 10. The distance measuring sensor 111 is, for example, a ToF sensor and measures the distance to the subject. The touch sensor 112 is, for example, built into the chest connection pad 18 and detects contact with the user.

[0071] Figure 11 is a diagram illustrating the hardware configuration of the display device 700. In Figure 11, the display device 700 includes a control unit 701, a power button 702, a display unit 703, a button 704, an in-camera 705, a face sensor 706, an angular velocity sensor 707, an acceleration sensor 708, an image signal processing circuit 709, and various switches 711.

[0072] The display device 700 also includes a built-in non-volatile memory 712, a primary memory 713, a large-capacity non-volatile memory 714, a speaker 715, a vibrator 716, an LED 717, an audio processing unit 720, a low-power wireless unit 771, and a high-speed wireless unit 772.

[0073] The control unit 701 is, for example, a CPU, which is connected to the power button 702 and the face sensor 706, and controls the operation of the display device 700.

[0074] The imaging signal processing circuit 709 provides the same functionality as the imaging driver 41, solid-state image sensor 42, and imaging signal processing circuit 43 of camera 1. The imaging data generated by the imaging signal processing circuit 709 is output to the control unit 701.

[0075] The various switches 711 are switches for executing functions other than those performed by the power button 702 and button 704.

[0076] The angular velocity sensor 707 is a gyroscope sensor that detects the movement of the display device 700 itself. The accelerometer sensor 708 detects the orientation of the display device 700. The angular velocity sensor 707 and the accelerometer sensor 708 are built into the display device 700.

[0077] The built-in non-volatile memory 712 is, for example, flash memory and stores the startup program for the control unit 701 and the settings for various program modes. The primary memory 713 is, for example, RAM and temporarily stores the video data being processed, the calculation results of the imaging signal processing circuit 709 and the control unit 701.

[0078] The high-capacity non-volatile memory 714 records captured image data or image data received from an external device. The high-capacity non-volatile memory 714 may be a removable recording medium such as an SD card, or it may be a semiconductor memory without a removable mechanism.

[0079] The speaker 715, vibrator 716, and LED 717 communicate the status of the display device 700 to the user or provide warnings by emitting sound, vibrating, or emitting light. The audio processing unit 720 is equipped with a left microphone 719L and a right microphone 719R that pick up external sound (analog signal), and processes the picked-up analog signal to generate an audio signal.

[0080] The low-power wireless unit 771 transmits and receives data to and from the camera 1 via low-power wireless communication. The high-speed wireless unit 772 transmits and receives data to and from the camera 1 via high-speed wireless communication.

[0081] The face sensor 706 includes an infrared LED lighting circuit 721, an infrared LED 722, an infrared focusing lens 726, and an infrared detection processing device 727. The infrared LED lighting circuit 721 is a circuit that has the same function as the infrared LED lighting circuit 21 in Figure 10. The infrared LED lighting circuit 721 controls the lighting and extinguishing of the infrared LED 722 and controls the emission of infrared light 723 from the infrared LED 722 toward the user.

[0082] The infrared focusing lens 726 is a lens that focuses the reflected light rays 725 of the infrared rays 723. The infrared detection processing device 727 has a sensor that detects the reflected light rays focused by the infrared focusing lens 726. The sensor of the infrared detection processing device 727 forms an image of the focused reflected light rays 725, converts it into sensor data, and outputs it to the control unit 701.

[0083] When the face sensor 706 shown in Figure 7 is pointed at the user, as shown in Figure 11, infrared light 723 emitted from the infrared LED lighting circuit 721 is irradiated onto the infrared irradiation surface 724, which is the entire face of the user. The infrared light 723 reflected by the infrared irradiation surface 724 becomes reflected light rays 725, which are then focused by the infrared condensing lens 726 onto the sensor of the infrared detection processing device 727.

[0084] Referring to Figure 12, the flow of the standard calibration process will be explained. Here, the display device 700 is assumed to be operating as a calibrator 800. The standard calibration process is performed with the user wearing the camera 1 around their neck and looking at the positioning indicator displayed on the display device 700. For example, in the standard calibration process for the front direction, the user holds the display device 700 in front of them and looks at the positioning indicator, and the camera 1 photographs the positioning indicator displayed on the display device 700. The camera 1 stores information indicating the relationship between the coordinates of the positioning indicator and the face direction (head posture) when the positioning indicator was photographed as calibration information in the primary memory 103, etc.

[0085] In S1201, the control unit 701 activates the display device 700, for example, in response to an operation by the user. It operates. The display device 700 may also be configured to be activated by an instruction from camera 1 when the user performs an operation to execute a standard calibration process on camera 1.

[0086] In S1202, the control unit 701 connects to camera 1 to exchange information used for standard calibration processing. The control unit 701 determines whether it has connected to camera 1 and communication has started. The control unit 701 waits until communication starts, and then proceeds to S1203.

[0087] In S1203, the control unit 701 starts an application for standard calibration processing (hereinafter also referred to as the app). The application for standard calibration processing may be started by the user's operation, or it may be started automatically by an instruction (notification) from camera 1.

[0088] In S1204, the control unit 701 performs a standard calibration process. The standard calibration process is performed for multiple directions. For example, the standard calibration process is performed for five directions: the front (center), the upper front, the lower front, the right front, and the left front of the imaging range.

[0089] Refer to Figure 13 for details on the standard calibration process of S1204. Figure 13 is a flowchart illustrating the details of the standard calibration process. Figure 13 describes the standard calibration process for the front of the imaging range.

[0090] In S1301, the display unit 703 displays positioning indicators and guidance indicating the direction in which to perform the standard calibration process. In the standard calibration process in the forward direction illustrated in Figure 13, the display unit 703 displays guidance to position the display device 700 so that the positioning indicators are positioned in front and to face the positioning indicators.

[0091] Figure 14 illustrates a positioning indicator 1401. The user positions the positioning indicator 1401 (mark) in front of them according to the guidance, faces the positioning indicator 1401, and presses button 704. When button 704 is pressed, camera 1 starts the standard calibration process in the forward direction.

[0092] In S1302, the control unit 701 uses the in-camera 705 to capture an image of the user. The image captured by the in-camera 705 is used to confirm whether the user is facing forward and to obtain the user's line of sight.

[0093] In S1303, the control unit 701 acquires head posture information. This head posture information is the posture of the head relative to the horizontal plane and is obtained from the face direction detected by the face direction detection unit 20 (the posture of the head relative to the camera 1) and the tilt of the shooting / detection unit 10 detected by the acceleration sensor 108.

[0094] In S1304, the control unit 701 determines whether the user is facing forward based on the position of the user's head captured by the in-camera 705, the tilt of the display device 700 detected by the acceleration sensor 708, and the information acquired from camera 1 in S1303. If the user is not facing forward, the process proceeds to S1305; if the user is facing forward, the process proceeds to S1306.

[0095] In S1305, the control unit 701 displays guidance on the display unit 703 instructing the user to correct the orientation of their face. The user changes the orientation of their face, or the display device 7 The user corrects their facial orientation by adjusting the position in which they hold the device. Once the user corrects their facial orientation, the control unit 701 returns to S1302 and continues the standard calibration in the forward direction.

[0096] In S1306, the control unit 701 acquires gaze position information from the user's video captured by the in-camera 705, and calculates a gaze correction value based on the gaze position information and the head posture information acquired from camera 1 in S1303.

[0097] In S1307, the control unit 701 corrects the user's head posture information, taking into account the gaze correction value calculated in S1306. In S1308, the control unit 701 transmits the corrected frontal head posture information from the display device 700 to the camera 1, and the camera 1 performs standard calibration processing based on the received head posture information.

[0098] In S1309, the control unit 701 determines whether the standard calibration process in the forward direction has been completed on camera 1. Upon receiving notification from camera 1 that the standard calibration process in the forward direction has been completed, the control unit 701 terminates the process shown in Figure 13.

[0099] Once the standard calibration process in the front direction is completed, the control unit 701 performs the same standard calibration process in the front upper, front lower, front right, and front left directions as in the front direction. The direction and order of the standard calibration process are arbitrary and are not limited to five directions; there may be fewer or more directions. Furthermore, the directions targeted for the standard calibration process are not limited to up, down, left, and right, but may also include diagonal upward and diagonal downward directions. In other words, in the process of S1204 in Figure 12, the control unit 701 performs the process shown in Figure 13 for each direction in which the standard calibration process is performed.

[0100] In S1205 of Figure 12, the control unit 701 notifies the user that the standard calibration process is complete in each direction. In S1206, the control unit 701 determines whether the application for standard calibration has been terminated. The application for standard calibration may be terminated, for example, by user operation, or it may be terminated automatically when the standard calibration process for each direction is completed. When the application for standard calibration is terminated, the standard calibration process is terminated.

[0101] The calibration confirmation process will be explained with reference to Figures 15 to 17. The calibration confirmation process is a process that confirms and controls the timing of the execution of the standard calibration process and the simplified calibration process.

[0102] Of the calibration processes, the standard calibration process, as described in Figures 12 to 14, is performed before the first use of camera 1 or before shooting begins. The simplified calibration process is a process to easily adjust the calibration if the mounting position of camera 1 shifts during imaging. The simplified calibration process can be performed with simpler operations than the standard calibration process.

[0103] The simplified calibration process can be performed using the same display device 700 or calibrator 800 as the standard calibration process. Alternatively, the simplified calibration process may be performed using a simplified display device 900, which has more limited functionality than the display device 700. In the following description, the simplified calibration process will be described as being performed using the display device 700.

[0104] If the mounting position of camera 1 shifts during imaging, the recording will be cropped from the imaging range of camera 1. A misalignment of the recording area occurs. If the user does not notice that the mounting position of camera 1 has shifted during imaging, the recording area intended by the user will not be extracted. In this embodiment, if camera 1 is suspected to have a misalignment of the recording area, it notifies the user to perform a simple calibration process.

[0105] Figure 15 is a flowchart illustrating the calibration confirmation process. The calibration confirmation process begins, for example, when the power to camera 1 is turned on. In S1500, the control unit 101 of camera 1 determines whether or not recording is in progress by the imaging unit 40. If recording is in progress, the process proceeds to S1501. If recording is not in progress, the process proceeds to S1510.

[0106] In S1501, the control unit 101 determines whether or not the user has given an instruction to end recording. Recording continues until an instruction to end recording is given. If an instruction to end recording is given, the process proceeds to S1502. In S1502, the control unit 101 ends the recording, and the process returns to S1500. Camera 1 enters a standby state for a shooting instruction.

[0107] In S1510, while waiting for a shooting instruction, the control unit 101 determines whether the standard calibration process has been completed. If the standard calibration process has not been completed, the process proceeds to S1511. If the standard calibration process has been completed, the process proceeds to S1520.

[0108] In S1511, the control unit 101 performs the standard calibration process described in Figures 12 and 13. Once the standard calibration process is complete, the process returns to S1500, and the camera 1 enters a waiting state for a shooting instruction.

[0109] In S1520, the control unit 101 determines whether or not to perform a simple calibration process. That is, it determines whether or not to adjust the misalignment of the recording area. If the simple calibration process is to be performed, the process proceeds to S1521. If the simple calibration process is not to be performed, the process returns to S1500, and the camera 1 enters a waiting state for a shooting instruction.

[0110] In S1520, the method for determining whether or not to perform a simplified calibration process, that is, whether or not to adjust for misalignment of the recording area, will be explained in detail. Six specific examples will be described below, but the method for determining whether or not to perform a simplified calibration process is not limited to the following methods, as long as it can determine whether or not misalignment of the recording area is possible.

[0111] The first determination method is based on the amount of deviation between the position of the recording area in the imaging range when the standard calibration process (or the simplified calibration process performed immediately before) was executed and the current position of the recording area.

[0112] Figure 16 illustrates the misalignment of the recording area. In Figure 16(a), camera 1 is facing forward, and the angle of the optical axis of camera 1 (imaging lens 16) is 0° with respect to the horizontal. The user's face direction relative to camera 1 is 90°. The face direction is defined as the direction of the line of sight relative to the vertical direction of the face direction detection window 13. In this case, the recording area is located approximately in the center of the imaging range.

[0113] In Figure 16(b), camera 1 is tilted upward relative to the front, and the angle of the optical axis of camera 1 is 25° with respect to the horizontal. Since the user remains facing forward, the user's face direction relative to camera 1 is 65°. In this case, the recording area is shifted downward to match the direction the user is actually looking.

[0114] In Figure 16(c), camera 1 is tilted further upward relative to the front, and the angle of the optical axis of camera 1 is 45° with respect to the horizontal. Since the user remains facing forward, the user's face direction relative to camera 1 is 45°. In this case, the recording area is shifted downward to match the direction the user is actually looking, and extends beyond the imaging range.

[0115] In the first determination method, the control unit 101 notifies the user to perform a simple calibration process if the amount of displacement of the recording area exceeds a predetermined threshold, in order to prevent the recording area from going outside the imaging range. The control unit 101 can, for example, use the position of the recording area as the center coordinate of the recording area, and calculate the amount of displacement as the distance between the center coordinates of the recording area at the time of calibration and the current recording area. The predetermined threshold can be determined, for example, according to the size of the imaging range and the recording area, so as to prevent the recording area from going outside the imaging range.

[0116] Camera 1 can prevent the recording area from going outside the imaging range by prompting the user to perform a simple calibration process when the amount of displacement of the recording area exceeds a predetermined threshold.

[0117] The second determination method is based on the difference between the tilt of camera 1 when the standard calibration process (or the simplified calibration process performed immediately before) was executed and the current tilt of camera 1.

[0118] In the second determination method, the control unit 101 notifies the user to perform a simple calibration process if the current tilt deviation of the camera 1 exceeds a predetermined threshold. The tilt of the camera 1 can be expressed, for example, as the angle of the optical axis of the camera 1 with respect to the horizontal direction, as shown in Figure 16.

[0119] For example, when the calibration process is performed, the tilt of camera 1 is set to 0° as shown in Figure 16(a), and a predetermined threshold is set to 25°. When the tilt of camera 1 reaches 25°, as shown in Figure 16(b), the control unit 101 prompts the user to perform a simple calibration process.

[0120] In the example in Figure 16, the tilt of camera 1 was described as a tilt in the vertical direction, but it may also be shown as a tilt (angle) in the three axes. When the tilt information of camera 1 is shown as a tilt in the three axes, the amount of displacement may be the sum of the displacement amounts in each direction. Whether or not the amount of displacement exceeds a predetermined threshold may be determined by whether or not the displacement amount in each direction exceeds the respective predetermined threshold.

[0121] When the tilt deviation of camera 1 exceeds a predetermined threshold, camera 1 prompts the user to perform a simple calibration process, thereby preventing the recording area intended by the user from falling outside the imaging range due to the positional deviation of camera 1.

[0122] The third determination method is based on the difference between the user's head posture when the standard calibration process (or the simplified calibration process performed immediately before) was executed and the user's current head posture. The head posture refers to the head posture relative to camera 1.

[0123] In the third determination method, the control unit 101 notifies the user to perform a simple calibration process if the amount of deviation in the user's current head posture exceeds a predetermined threshold. The user's head posture is, for example, similar to the face direction described in Figure 16, measured by the face direction detection window 13. This can be defined as the direction of the line of sight relative to the vertical.

[0124] For example, when the calibration process is performed, the user's head position is set to 90° as shown in Figure 16(a), and a predetermined threshold is set to 25°. As shown in Figure 16(b), when the user's head position is 65° and the deviation is 25°, the control unit 101 prompts the user to perform a simple calibration process.

[0125] In the example in Figure 16, the user's head posture information was described as vertical tilt, but it may also be shown as tilt (angle) in the three axes. When the head posture information is shown as tilt in the three axes, the amount of displacement may be the sum of the displacement amounts in each direction. Furthermore, whether or not the amount of displacement exceeds a predetermined threshold may be determined by whether or not the displacement amount in each direction exceeds the respective predetermined threshold.

[0126] When the amount of deviation in the user's head posture exceeds a predetermined threshold, camera 1 prompts the user to perform a simple calibration process, thereby preventing the recording area intended by the user from falling outside the imaging range due to changes in the positional relationship between camera 1 and the user's head.

[0127] The fourth determination method is to determine whether to perform a simple calibration process when it is detected that the user has attached camera 1. If the user removes and then reattaches camera 1, the positional relationship between camera 1 and the user's head may have changed. Therefore, when the control unit 101 detects that the user has attached camera 1, it notifies the user to perform a simple calibration process.

[0128] The control unit 101 can detect, for example, that the user has attached the camera 1 by detecting that the connection unit 80 has been connected to the shooting / detection unit 10. The control unit 101 may also determine that the user has attached the camera 1 when the angle of the shooting / detection unit 10 is adjusted using the angle adjustment buttons 85L and 85R. Furthermore, the control unit 101 may determine that the user has attached the camera 1 when the length of the band units 82L and 82R is adjusted using the adjustment buttons 92L and 92R. When the user attaches the camera 1, the camera 1 can prompt the user to perform a simple calibration process, thereby adjusting any misalignment of the recording area in a timely manner.

[0129] The fifth determination method is to determine whether to perform a simplified calibration process when the rate of change per unit time of the measurement value by the distance measuring sensor 111 exceeds a predetermined threshold. The distance measuring sensor 111 is, for example, a ToF sensor that measures the distance to the subject. If the rate of change per unit time of the distance to the subject increases, the positional relationship between the camera 1 and the subject changes, which may prevent the recording area intended by the user from being captured.

[0130] Therefore, when the amount of change per unit time of the measurement value by the distance sensor 111 exceeds a predetermined threshold, the control unit 101 notifies the user to perform a simple calibration process. The camera 1 can adjust the positional shift of the recording area in a timely manner by prompting the user to perform a simple calibration process when the positional relationship with the subject changes.

[0131] The sixth determination method is to determine whether to perform a simple calibration process when the touch sensor 112, which detects the contact state with the user when the device is attached, detects a change in the contact state with the user. The touch sensor 112 is provided, for example, on the surface where the shooting / detection unit 10 contacts the user. For example, if the user tilts their upper body forward, causing the shooting / detection unit 10 to move away from the user's body, the touch sensor 112 will no longer detect contact with the user. In other words, the positional relationship between the camera 1 and the user's head changes.

[0132] Therefore, when the control unit 101 detects a change in the contact state with the user via the touch sensor 112, it notifies the user to perform a simple calibration process. When the camera 1 changes its position relative to the user's head, it prompts the user to perform a simple calibration process, thereby allowing for timely adjustment of the recording area's positional misalignment.

[0133] As described above, the control unit 101 can determine whether or not to perform the simplified calibration process using various determination methods. These determination methods can be applied in any combination. The control unit 101 may decide to perform the simplified calibration process if any of the conditions of the determination method are met, or it may decide to perform the simplified calibration process if the conditions of multiple determination methods are met. Furthermore, the control unit 101 may adopt all or some of the six example determination methods as the method for determining whether or not to perform the simplified calibration process. If it is determined in S1520 to perform the simplified calibration process, the process proceeds to S1521.

[0134] The processes from S1521 to S1523 are the same as the first determination method, and are for checking whether the amount of displacement of the recording area is less than a predetermined threshold. If it is determined in S1520 whether or not to perform the simplified calibration process using the first determination method, the processes from S1521 to S1523 are omitted. If it is determined in S1520 whether or not to perform the simplified calibration process using a method other than the first determination method, the processes from S1521 to S1523 may be performed or omitted.

[0135] In S1521 of Figure 15, the control unit 101 acquires information on the current tilt of camera 1 and the posture of the user's head. In S1522, the control unit 101 acquires the amount of deviation between the current recording area and the recording area expected based on the settings in the calibration process, based on the information acquired in S1521.

[0136] In S1523, the control unit 101 determines whether the amount of displacement obtained in S1522 is less than a predetermined threshold. If the amount of displacement is less than the predetermined threshold, the process returns to S1500, and the camera 1 returns to the waiting state for a shooting instruction. If the amount of displacement is greater than or equal to the predetermined threshold, the process proceeds to S1524.

[0137] In S1524, the control unit 101 sends a notification to the display device 700 prompting it to perform a simplified calibration process. In S1525, the control unit 101 performs the simplified calibration process using the display device 700. Once the simplified calibration process is complete, the process returns to S1500. Camera 1 enters a waiting state for a shooting instruction.

[0138] Although Figure 15 illustrates an example where the calibration process is performed while waiting for a shooting instruction rather than during recording, the control unit 101 may also determine whether or not to perform the simplified calibration process during recording, similar to S1520.

[0139] Figure 17 is a flowchart illustrating a simplified calibration process on the display device 700. The process shown in Figure 17 is initiated when the display device 700 receives a notification from camera 1 prompting it to perform the simplified calibration process in S1524.

[0140] In S1700, the control unit 701 of the display device 700 determines whether or not it is connected to camera 1. If it is already connected to camera 1, or if it waits until it is connected to camera 1 and the connection is confirmed, the control unit 701 proceeds to S1701.

[0141] The control unit 701 may also present the user with a notification prompting them to perform a simple calibration process, such as a message or icon. The user should confirm the notification and then perform the simple calibration. An application for performing the calibration (a simple calibration confirmation application) can be launched. Note that the simple calibration confirmation application may be implemented as the same application as the standard application.

[0142] In S1701, the control unit 701 determines whether the simple calibration confirmation application has been launched. The control unit 701 waits until the simple calibration confirmation application is launched, and then proceeds to S1702. The simple calibration confirmation application is not limited to being launched by user operation; it may also be configured to be launched automatically when the control unit 701 of the display device 700 receives a notification from the camera 1 prompting it to perform the simple calibration process.

[0143] In S1702, the control unit 701 acquires information regarding the misalignment of the recording area from the camera 1. The misalignment of the recording area is caused by a shift in the head's posture (face direction). Therefore, the information regarding the misalignment of the recording area can be acquired based on the head's posture information at the time the calibration process was performed and the current head's posture information.

[0144] When the user changes the position of camera 1 and the tilt of camera 1 changes, the control unit 101 of camera 1 reacquires information regarding the misalignment of the recording area. The control unit 101 transmits the reacquired information regarding the misalignment of the recording area to the display device 700.

[0145] In S1703, the control unit 701 displays information regarding the misalignment of the recording area on the display unit 703. The control unit 701 displays, for example, as shown in Figure 8(a), an image of the head whose angle has been changed based on the amount of misalignment, and a confirmation image that includes three-axis arrows indicating the current orientation of the face.

[0146] When the control unit 701 receives information regarding the misalignment of the recording area reacquired by the camera 1 after the user changes the position of the camera 1, it updates the display content of the display unit 703 based on the received information. The user adjusts the mounting position of the camera 1 according to the misalignment information displayed on the display unit 703 and corrects the calibration error.

[0147] In S1704, the control unit 701 determines whether the simplified calibration process has been completed. The simplified calibration process is terminated when the amount of deviation is nearly eliminated or when the simplified calibration confirmation application is terminated. The simplified calibration confirmation application may be terminated by the user's operation, or it may be terminated by the control unit 701 when the amount of deviation is nearly eliminated.

[0148] If the simplified calibration process is not yet complete, the control unit 701 returns to S1702 and waits for information regarding the misalignment of the recording area to be transmitted from the camera 1. When the simplified calibration process is complete, the control unit 701 notifies the camera 1 that the simplified calibration process is complete, and the simplified calibration process on the display device 700 is terminated.

[0149] Figure 18 is a flowchart illustrating a simplified calibration process for camera 1. The simplified calibration process shown in Figure 18 details the processing on the camera 1 side at S1525 in Figure 15. The simplified calibration process is performed by the user adjusting the mounting position of camera 1 while viewing a confirmation screen on the display device 700. The simplified calibration process for camera 1 shown in Figure 18 is started at S1524 in Figure 15 when the control unit 101 of camera 1 sends a notification to the display device 700 prompting the execution of the simplified calibration process.

[0150] In S1800, the control unit 101 of camera 1 determines whether or not it is connected to the display device 700. If it is already connected to the display device 700, or if it waits until it is connected to the display device 700 and the connection is confirmed, the control unit 101 proceeds to S1801.

[0151] In S1801, the control unit 101 receives a request from the display device 700 to acquire information on the misalignment of the recording area (S1702 in Figure 17), and acquires information on the tilt of the camera 1 and the orientation of the head.

[0152] In S1802, the control unit 101 obtains the amount of displacement of the recording area based on the information of the tilt of camera 1 and the head orientation when the calibration process was performed, and the current tilt of camera 1 and head orientation information acquired in S1801.

[0153] In S1803, the control unit 101 generates information regarding the positional displacement of the recording area based on the displacement amount acquired in S1802 and transmits it to the display device 700. The information regarding the positional displacement of the recording area is generated, for example, as information regarding the displacement amount of the head posture.

[0154] In S1804, the control unit 101 determines whether the simplified calibration process has finished. The control unit 101 can determine that the simplified calibration process has finished by receiving a notification of completion of the simplified calibration process from the display device 700. If the simplified calibration process has not finished, the process returns to S1801. The control unit 101 repeatedly sends positional misalignment information to the display device 700 until the simplified calibration process is finished. If the simplified calibration process is finished, the process returns to S1500 in Figure 15, and the camera 1 enters a standby state.

[0155] According to the above embodiment, if the mounting state of camera 1 changes after the standard calibration process has been performed, the user can be prompted to perform a simplified calibration process. The user can avoid taking pictures without realizing that the position of the recording area has shifted when the mounting state of camera 1 changes. In addition, the user can adjust the position shift of the recording area by performing a calibration process with simpler operations than standard calibration.

[0156] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its gist.

[0157] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of symbols]

[0158] 1: Camera, 20: Face direction detection unit, 30: Recording area determination unit, 101: Control unit, 70: Transmission unit, 700: Display device

Claims

1. An imaging device, Imaging means, A detection means for detecting the posture of the user's head relative to the imaging device, A determination means that determines a recording area to be cut out from the imaging range of the imaging means based on the posture of the head detected by the detection means, A determination means for determining whether or not to adjust the misalignment of the recording area in the imaging range, If the determination means determines that the misalignment of the recording area should be adjusted, the notification means transmits a notification prompting the adjustment of the misalignment of the recording area to a display device that displays information regarding the misalignment of the recording area. A distance measuring sensor that measures the distance to the subject, It has, The determination means determines that the misalignment of the recording area should be adjusted when the amount of change per unit time of the measurement value by the distance sensor exceeds a predetermined threshold. An imaging device characterized by the following features.

2. The notification means transmits to the display device information regarding the misalignment of the recording area, which is obtained based on the head posture information obtained when a calibration process is performed to adjust the position of the recording area in the imaging range to suit the user, and the current head posture information detected by the detection means. The imaging apparatus according to feature 1.

3. When the tilt of the imaging device changes, the notification means reacquires information regarding the misalignment of the recording area and transmits it to the display device. The imaging device according to feature 2.

4. An imaging device, Imaging means, A detection means for detecting the posture of the user's head relative to the imaging device, A determination means that determines a recording area to be cut out from the imaging range of the imaging means based on the posture of the head detected by the detection means, A determination means for determining whether or not to adjust the misalignment of the recording area in the imaging range, If the determination means determines that the misalignment of the recording area should be adjusted, the notification means transmits a notification prompting the adjustment of the misalignment of the recording area to a display device that displays information regarding the misalignment of the recording area. It has, The notification means is, Information regarding the misalignment of the recording area, obtained based on the head posture information when a calibration process is performed to adjust the position of the recording area in the imaging range to suit the user, and the current head posture information detected by the detection means, is transmitted to the display device. When the tilt of the imaging device changes, information regarding the misalignment of the recording area is reacquired and transmitted to the display device. An imaging device characterized by the following features.

5. The determination means determines that the misalignment of the recording area should be adjusted if the amount of deviation between the position of the recording area when a calibration process is performed to adjust the position of the recording area in the imaging range to suit the user and the current position of the recording area exceeds a predetermined threshold. The imaging apparatus according to any one of claims 1 to 4.

6. The determination means determines that the positional misalignment of the recording area should be adjusted if the amount of difference between the tilt of the imaging device when a calibration process is performed to adjust the position of the recording area in the imaging range to match the user, and the current tilt of the imaging device, exceeds a predetermined threshold. The imaging apparatus according to any one of claims 1 to 5.

7. The determination means determines that the positional misalignment of the recording area should be adjusted if the amount of deviation between the head posture when a calibration process is performed to adjust the position of the recording area in the imaging range to match the user and the current head posture of the user exceeds a predetermined threshold. The imaging apparatus according to any one of claims 1 to 6.

8. The determination means determines that the misalignment of the recording area should be adjusted when it detects that the user has attached the imaging device. The imaging apparatus according to any one of claims 1 to 7.

9. The device further includes a touch sensor that detects the contact state with the user when it is attached. The determination means determines that the misalignment of the recording area should be adjusted when the touch sensor detects a change in the contact state with the user. The imaging apparatus according to any one of claims 1 to 8.

10. The information regarding the posture of the head is expressed in terms of angles in the three axial directions. The imaging apparatus according to any one of claims 2 to 9.

11. A display device for receiving the notification from the imaging device according to any one of claims 1 to 10, The notification is presented to the user, and a display shows information regarding the misalignment of the recording area. Having control means A display device characterized by the following features.

12. The display device according to claim 11, wherein the display control means receives information regarding the positional displacement of the recording area, which is obtained based on the head posture information when a calibration process is performed to adjust the position of the recording area in the imaging range to suit the user, and the current head posture information detected by the detection means, and displays a confirmation image for adjusting the positional displacement.

13. A display device that receives a notification from an imaging device, the imaging device comprising: imaging means; detection means for detecting the posture of a user's head relative to the imaging device; determination means for determining a recording area to be cut out from the imaging range by the imaging means based on the posture of the head detected by the detection means; determination means for determining whether or not to adjust the misalignment of the recording area in the imaging range; and notification means for transmitting a notification prompting adjustment of the misalignment of the recording area to a display device that displays information regarding the misalignment of the recording area, if the determination means determines that the misalignment of the recording area should be adjusted, A display control means for presenting the notification to the user and displaying information regarding the misalignment of the recording area, the display control means receiving information regarding the misalignment of the recording area obtained from the imaging device based on information about the head posture when a calibration process is performed to adjust the position of the recording area in the imaging range to match the user, and information about the current head posture detected by the detection means, and controlling to display a confirmation image for adjusting the misalignment, When the tilt of the imaging device changes, the display control means receives information regarding the positional shift of the recording area reacquired by the imaging device and updates the confirmation image. A display device characterized by the following features.

14. The aforementioned confirmation image is a three-dimensional image. The display device according to claim 12 or 13.

15. When the display control means receives the notification, it launches an application for adjusting the misalignment of the recording area. The display device according to any one of claims 11 to 14.

16. A control method for an imaging device having an imaging means and a distance measuring sensor for measuring the distance to a subject, A detection step for detecting the posture of the user's head relative to the imaging device, A determination step in which a recording area is cut out from the imaging range by the imaging means based on the posture of the head detected in the detection step, A determination step of determining whether or not to adjust the positional misalignment of the recording area in the imaging range, If the determination step determines that the misalignment of the recording area should be adjusted, a notification step is performed to send a notification prompting the adjustment of the misalignment of the recording area to a display device that displays information regarding the misalignment of the recording area. It has, In the determination step, if the amount of change per unit time of the measurement value by the distance sensor exceeds a predetermined threshold, it is determined that the misalignment of the recording area should be adjusted. A control method for an imaging device, characterized by the following:

17. A control method for an imaging device having an imaging means, A detection step for detecting the posture of the user's head relative to the imaging device, A determination step in which a recording area is cut out from the imaging range by the imaging means based on the posture of the head detected in the detection step, A determination step of determining whether or not to adjust the positional misalignment of the recording area in the imaging range, If the determination step determines that the misalignment of the recording area should be adjusted, a notification step is performed to send a notification prompting the adjustment of the misalignment of the recording area to a display device that displays information regarding the misalignment of the recording area. It has, In the aforementioned notification step, Information regarding the misalignment of the recording area, obtained based on the head posture information obtained when a calibration process is performed to adjust the position of the recording area in the imaging range to suit the user, and the current head posture information detected in the detection step, is transmitted to the display device. When the tilt of the imaging device changes, information regarding the misalignment of the recording area is reacquired and transmitted to the display device. A control method for an imaging device, characterized by the following:

18. A program for causing a computer to function as one of the means of an imaging apparatus according to any one of claims 1 to 10.

19. An imaging system having an imaging device and a display device, The imaging device is Imaging means, A detection means for detecting the posture of the user's head relative to the imaging device, A determination means that determines a recording area to be cut out from the imaging range of the imaging means based on the posture of the head detected by the detection means, A determination means for determining whether or not to adjust the misalignment of the recording area in the imaging range, If the determination means determines that the misalignment of the recording area should be adjusted, the notification means transmits a notification to the display device prompting the adjustment of the misalignment of the recording area. A distance measuring sensor that measures the distance to the subject, It has, The determination means determines that the positional misalignment of the recording area should be adjusted when the amount of change per unit time of the measurement value by the distance sensor exceeds a predetermined threshold, The aforementioned display device is The system includes a display control means that presents the notification to the user and displays information regarding the misalignment of the recording area. An imaging system characterized by the following features.

20. An imaging system having an imaging device and a display device, The imaging device is Imaging means, A detection means for detecting the posture of the user's head relative to the imaging device, A determination means that determines a recording area to be cut out from the imaging range of the imaging means based on the posture of the head detected by the detection means, A determination means for determining whether or not to adjust the misalignment of the recording area in the imaging range, If the determination means determines that the misalignment of the recording area should be adjusted, the notification means transmits a notification to the display device prompting the adjustment of the misalignment of the recording area. It has, The notification means is, Information regarding the misalignment of the recording area, obtained based on the head posture information when a calibration process is performed to adjust the position of the recording area in the imaging range to suit the user, and the current head posture information detected by the detection means, is transmitted to the display device. When the tilt of the imaging device changes, information regarding the misalignment of the recording area is reacquired and transmitted to the display device. The aforementioned display device is The system includes a display control means that presents the notification to the user and displays information regarding the misalignment of the recording area. An imaging system characterized by the following features.

Images