Light detection device

Abstract

The present disclosure relates to a light detection device capable of improving detection accuracy of a multi-IMU system. In the present invention, a multi-IMU system is stacked on an image sensor. A backside resonator is provided between a substrate and an angular velocity sensor constituting the multi-IMU system, which absorbs oscillation generated by an oscillator of the angular velocity sensor, thereby suppressing acoustic interference occurring between adjacent angular velocity sensors. The present disclosure is applicable to an imaging device.

Description

Technical Field

[0001] This disclosure relates to optical detection devices, and more specifically to optical detection devices for improving the detection accuracy of multiple IMUs. Background Technology

[0002] A multi-inertial measurement unit (IMU) approach is proposed to improve detection accuracy by integrating the detection results of multiple IMUs.

[0003] As a technique for improving the detection accuracy of multiple IMUs, a technique has been proposed that enables the appropriate combination of observations from multiple IMUs based on the noise characteristics of the multiple IMUs and the conditions of the observations (see Patent Document 1).

[0004] Reference List

[0005] Patent documents

[0006] Patent Document 1: International Publication No. 2020 / 045099 Summary of the Invention

[0007] The problem to be solved by the present invention

[0008] Meanwhile, in vibration-type IMUs using microelectromechanical systems (MEMS) in multi-IMU systems, such as the example in Patent Document 1, angular velocity is detected based on the Coriolis force generated by rotating the object while applying vibration to it.

[0009] However, since multiple IMUs generate vibrations, each IMU may experience interference due to vibrations generated by other IMUs, and beat noise caused by interference (acoustic interference) may occur.

[0010] In particular, due to the improved manufacturing accuracy of IMUs in recent years, which has reduced manufacturing deviations, and the increasing number of IMUs being manufactured with similar frequencies of vibrations generated by individual IMUs, interference is more likely to occur, and IMUs are more likely to be affected by beat noise caused by interference.

[0011] This disclosure was made in view of this situation, and in particular, it reduces the impact of beat noise caused by acoustic interference between the individual IMUs constituting a multi-IMU, and achieves high accuracy for the multi-IMU.

[0012] Solution to the problem

[0013] A light detection apparatus according to one aspect of this disclosure includes: a light detection element that detects light; and an angular velocity sensor that detects the angular velocity of the light detection element, wherein the angular velocity sensor includes a back-side resonator between the angular velocity sensor and a substrate on which the angular velocity sensor is mounted, the back-side resonator absorbing vibrations of the angular velocity sensor.

[0014] In one aspect of this disclosure, a light detection element for detecting light and an angular velocity sensor for detecting the angular velocity of the light detection element are provided, wherein a back-side resonator is provided between the angular velocity sensor and a substrate on which the angular velocity sensor is mounted, the back-side resonator absorbing vibrations of the angular velocity sensor. Attached Figure Description

[0015] Figure 1 It is a diagram used to describe multiple IMUs.

[0016] Figure 2 This is a diagram used to describe the structure of the IMU.

[0017] Figure 3 It is used to describe Figure 2 The diagram shows the circuit configuration of the IMU's readout circuit.

[0018] Figure 4 It is used to describe Figure 2 A diagram illustrating the operation of the IMU.

[0019] Figure 5 It is a diagram used to describe the operation of multiple IMUs.

[0020] Figure 6 This is a diagram used to describe interference generated by multiple IMUs.

[0021] Figure 7 This is a diagram used to describe interference generated by multiple IMUs.

[0022] Figure 8 This is a diagram used to describe interference generated by multiple IMUs.

[0023] Figure 9 This is a diagram illustrating a basic configuration example of multiple IMUs used to describe the contents of this disclosure.

[0024] Figure 10 It is used to describe Figure 9 A diagram illustrating an example of the physical arrangement of MEMS (vibration elements), demodulation circuitry, and signal processing circuitry for a multi-IMU.

[0025] Figure 11 This is a diagram illustrating an example of the arrangement of accelerometer and angular velocity sensors in a multi-IMU.

[0026] Figure 12 This is a diagram illustrating an example configuration of an accelerometer and an angular velocity sensor using a three-dimensional vibration element.

[0027] Figure 13 This is a diagram illustrating an example configuration of an accelerometer and an angular velocity sensor using three one-dimensional vibration elements.

[0028] Figure 14 This is a diagram illustrating an example configuration of a one-dimensional vibration element for a capacitive angular velocity sensor in the X and Y directions.

[0029] Figure 15 This is a diagram illustrating an example configuration of a one-dimensional vibration element for a capacitive angular velocity sensor in the Z direction.

[0030] Figure 16 This is a diagram illustrating an example of the arrangement of multiple IMUs, each consisting of three angular velocity sensors, each with three one-dimensional vibration elements.

[0031] Figure 17 This is a diagram illustrating an example configuration of a one-dimensional vibrating element for a capacitive accelerometer in the X and Y directions.

[0032] Figure 18 This is a diagram illustrating an example configuration of a one-dimensional vibration element for a capacitive accelerometer in the Z direction.

[0033] Figure 19 This is a diagram illustrating an example configuration of multiple IMUs, each consisting of three accelerometers, each with three one-dimensional vibration elements.

[0034] Figure 20 This is a diagram illustrating an example arrangement of multiple IMUs, including two accelerometers and three angular velocity sensors.

[0035] Figure 21 This is a diagram illustrating an example arrangement of a multi-IMU comprising two accelerometers and three angular velocity sensors, with adjacent angular velocity sensors tilted at 90 degrees.

[0036] Figure 22 This is a diagram illustrating an example of the arrangement of accelerometer and angular velocity sensors in a multi-IMU used in AR glasses.

[0037] Figure 23 This is a diagram illustrating an example of the arrangement of accelerometers and angular velocity sensors in a multi-IMU used in VR glasses.

[0038] Figure 24 This is a diagram illustrating an example of the arrangement of accelerometer and angular velocity sensors used in a multi-IMU for a camera device.

[0039] Figure 25 This is a diagram illustrating another example of the arrangement of accelerometers and angular velocity sensors used in multiple IMUs for VR glasses.

[0040] Figure 26 This is a diagram illustrating an example configuration of a back-side resonator applied to an angular velocity sensor.

[0041] Figure 27 This is a diagram illustrating an example configuration of a back-side resonator applied to an angular velocity sensor.

[0042] Figure 28 This is an exploded stereoscopic view of an angular velocity sensor that utilizes a back-side resonator.

[0043] Figure 29 It was applied Figure 28 Detailed exploded perspective view of the angular velocity sensor with a back-side resonator.

[0044] Figure 30 This is an enlarged top view of the back-side resonator.

[0045] Figure 31 This is a diagram used to describe the operation of a back-side resonator.

[0046] Figure 32 This is a diagram used to describe the first modification of the back-side resonator.

[0047] Figure 33 This is a diagram used to describe the first modification of the back-side resonator.

[0048] Figure 34 This is a diagram used to describe the second modification of the back-side resonator.

[0049] Figure 35 This is a diagram illustrating a configuration example for applying multiple IMUs, including an angular velocity sensor with a back-side resonator, to an image sensor for camera shake correction.

[0050] Figure 36 This is a diagram illustrating an example configuration of an imaging device that achieves camera shake correction by driving optical blocks.

[0051] Figure 37 This is a diagram illustrating an example configuration of an imaging device that achieves camera shake correction by driving an image sensor.

[0052] Figure 38 This is a diagram illustrating a detailed configuration example of an imaging device that achieves camera shake correction by driving an image sensor.

[0053] Figure 39This is a diagram illustrating the overview of the imaging apparatus described in this disclosure.

[0054] Figure 40 This is a diagram used to describe an example configuration of the imaging apparatus according to the first modification of the second embodiment.

[0055] Figure 41 It is a timing diagram used to describe the shake correction process of a camera device.

[0056] Figure 42 It is used to describe by Figure 40 The flowchart shows the imaging process performed by the imaging device.

[0057] Figure 43 It is a graph used to describe the number of IMU units and the calibration accuracy.

[0058] Figure 44 This is a diagram illustrating an example configuration of a second modified imaging apparatus according to the second embodiment.

[0059] Figure 45 This is a diagram used to describe a configuration example of a general-purpose personal computer. Detailed Implementation

[0060] Preferred embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Note that in this specification and the drawings, redundant descriptions of these components are omitted by using the same reference numerals to denote components with substantially the same functional configuration.

[0061] The following describes implementation methods for carrying out this technology. The descriptions will proceed in the following order.

[0062] 1. Overview of the contents of this disclosure

[0063] 2. First Implementation Method

[0064] 3. Second Implementation Method

[0065] 4. First modification of the second implementation method

[0066] 5. Second modification of the second embodiment

[0067] 6. Examples executed by software

[0068] <<1. Overview of the Contents of this Disclosure>>

[0069] <About Multiple IMUs>

[0070] This disclosure specifically reduces the impact of beat noise caused by interference between the individual inertial measurement units (IMUs) that make up a multi-IMU, and achieves high accuracy for the multi-IMU.

[0071] First, to outline the overview of the present disclosure, a multiple IMU will be described.

[0072] As shown in Figure 1 on the left, a single IMU 1 has the following configuration: it includes, for example, an acceleration sensor that detects translational motion, i.e., acceleration, in each of three axial directions including X, Y and Z axes, and a gyroscope sensor (angular velocity sensor) that detects rotational motion, i.e., angular velocity, and the single IMU 1 detects acceleration and angular velocity in each of the three axial directions.

[0073] Although there exist single IMUs 1 with high accuracy, generally, the higher the accuracy, the larger and more expensive the single IMU 1 is, and if higher accuracy is pursued, the size increases and the cost also increases.

[0074] Therefore, as shown in Figure 1 on the right, the multiple IMU 10 is configured such that a plurality of low-accuracy but low-cost IMUs 1 (for example, n IMUs), such as IMU 1-1 to IMU 1-n, are provided, and a combiner 2 combines the acceleration and angular velocity that are detection results of each of the IMU 1-1 to IMU 1-n, so as to reduce noise density and bias fluctuation to and improve detection accuracy, thereby achieving higher accuracy.

[0075] The device size and device cost of each of the low-accuracy and low-cost IMUs 1-1 to IMU 1-n that constitute the multiple IMU 10 shown in Figure 1 on the right can be made sufficiently smaller than the device size and device cost in the case where a single high-accuracy IMU 1 as shown in Figure 1 on the left is prepared, and lower cost can also be achieved.

[0076] Note that, hereinafter, in a case where it is not necessary to particularly distinguish IMU 1-1 to IMU 1-n, IMU 1-1 to IMU 1-n are simply referred to as IMU 1, and other configurations are abbreviated in a similar manner. In addition, in the present specification, hereinafter, it is assumed that IMU 1 is a small and low-cost IMU with relatively low accuracy, but IMU 1 may also be a large, expensive and highly accurate IMU.

[0077] <Structure of IMU>

[0078] Next, the structure of IMU 1 will be described with reference to Figure 2 .

[0079] As shown in Figure 2As shown on the right, each of the IMUs 1 constituting the multiple IMUs 10 has the following structure, which, from top to bottom in the figure, includes: a silicon-based vibrating element 11, a base 12 for fixing the vibrating element 11, and a readout circuit 13 for reading the vibration of the vibrating element 11 and outputting the angular velocity. The vibrating element 11, the base 12, and the readout circuit 13 are arranged in accordance with... Figure 2 The right side shows the sequence of attachments (joints) together, and as shown... Figure 2 As shown on the left, they are integrated together by resin molding.

[0080] <Circuit configuration of the readout circuit>

[0081] Next, we will refer to Figure 3 Describe the circuit configuration of the readout circuit 13 in IMU 1.

[0082] Note that in Figure 3 The following section will describe the configuration for detecting angular velocity within the readout circuitry that constitutes IMU 1. Regarding the configuration for acceleration detection in IMU 1, the configuration is obtained by removing the demodulation circuitry from the configuration for detecting angular velocity; therefore, the description will focus on the more complex configuration for detecting angular velocity.

[0083] The readout circuit 13 includes a drive circuit block 31, a sensing circuit block 32, and a digital output circuit block 33.

[0084] The drive circuit block 31 provides an oscillation signal with a predetermined drive frequency to the vibration element 11 and the sensing circuit block 32, which are composed of microelectromechanical systems (MEMS), and causes the vibration element 11 to vibrate based on the oscillation signal.

[0085] The sensing circuit block 32 detects the vibration generated by the Coriolis force acting on the vibrating element 11 that vibrates based on the oscillation signal as an analog signal, and outputs the analog signal to the digital output circuit block 33.

[0086] The digital output circuit block 33 converts the vibration provided by the sensing circuit block 32, which is generated by the Coriolis force acting on the vibrating element 11, from an analog signal to a digital signal, and outputs the digital signal as an angular velocity.

[0087] More specifically, the drive circuit block 31 includes an oscillator circuit 51 and an automatic gain control circuit 52.

[0088] The oscillator circuit 51 includes an RC circuit that uses the vibration provided from the vibrating element 11 as a reference signal to generate an oscillation signal, and outputs the oscillation signal to the automatic gain control circuit 52 and the phase shifting circuit 72 of the sensing circuit block 32.

[0089] The automatic gain control circuit 52 adjusts the gain of an oscillation signal having a driving frequency supplied from the oscillator circuit 51, supplies the oscillation signal to the vibrating element 11, and causes the vibrating element 11 to vibrate.

[0090] The sensing circuit block 32 includes a charge amplifier circuit 71, a phase shift circuit 72, a synchronous demodulation circuit 73 and an LPF 74.

[0091] The charge amplifier circuit 71 detects the vibration of the vibrating element 11 as a vibration signal, amplifies the vibration signal, and supplies the vibration signal to the phase shift circuit 72.

[0092] The phase shift circuit 72 adjusts the phase of the vibration signal of the vibrating element 11 detected by the charge amplifier circuit 71 based on the oscillation signal supplied from the oscillator circuit 51, and outputs the vibration signal to the synchronous demodulation circuit 73.

[0093] The synchronous demodulation circuit 73 demodulates, from the phase-adjusted vibration signal of the vibrating element 11, a waveform represented by an envelope that indicates the Coriolis force acting on the vibrating element 11, and outputs the waveform to the LPF 74.

[0094] The LPF 74 smoothes the waveform indicating the Coriolis force acting on the vibrating element 11, and outputs the waveform as information about angular velocity in the form of an analog signal to the digital output circuit block 33.

[0095] The digital output circuit block 33 includes an AD conversion circuit 91, a decimation filter 92 and a digital output circuit 93.

[0096] The AD conversion circuit 91 converts information about angular velocity in the form of an analog signal derived from the Coriolis force acting on the vibrating element 11 into a digital signal, and outputs the digital signal to the decimation filter 92.

[0097] The decimation filter 92 averages the information about angular velocity in the form of a digital signal, and outputs the information to the digital output circuit 93.

[0098] The digital output circuit 93 outputs the digitized and averaged information about angular velocity as a digital signal.

[0099] <Operation of IMU>

[0100] Next, reference will be made to Figure 4 to describe the operation of the IMU 1.

[0101] As shown Figure 4 in the upper left part of, the vibrating element 11 vibrates based on a reference signal in the form of an oscillation signal of driving frequency fb, which is oscillated by the oscillator circuit 51 and has a gain adjusted by the automatic gain control circuit 52.

[0102] At this time, when the Coriolis force acts on the vibrating element 11, amplitude modulation caused by the Coriolis force is applied, and thus, for example, as shown by waveform fbc, the waveform output from the charge amplifier circuit 71 is amplitude modulated relative to the driving frequency fb according to the Coriolis force.

[0103] The synchronous demodulation circuit 73 demodulates the amplitude modulation caused by the Coriolis force from the envelope of the waveform fbc into a waveform of an analog signal indicating the Coriolis force, i.e., the angular velocity, and outputs the waveform to the LPF 74.

[0104] In this way, the waveform of the analog signal extracted as the Coriolis force is converted into a digital signal by the digital output circuit block 33 and output as a digitized angular velocity value.

[0105] For example, such as Figure 5 As shown, multiple IMUs collect and integrate the above n IMUs 1, and the angular velocities detected by each IMU 1-1 to IMU 1-n are combined by the combiner 2 to output the angular velocity with improved accuracy.

[0106] Interference generated by multiple IMUs

[0107] Specifically, the multi-IMU 10 has, for example, Figure 6 The configuration shown.

[0108] In other words, Figure 6 The multiple IMU 10 has a configuration that sets up IMU 1-1 to IMU 1-4 on the printed circuit board 110.

[0109] Using this configuration, in Figure 6 In the multiple IMU 10, the angular velocities detected by each IMU 1-1 to IMU 1-4 are combined, and thus the detection accuracy is improved and the angular velocity is output.

[0110] Meanwhile, it is known that IMU 1 is manufactured such that the drive frequency varies by, for example, about 3% due to individual differences in manufacturing.

[0111] Therefore, if IMU 1 is designed such that, for example, the drive frequency is 20,000 kHz, the following configuration can be used: Figure 6 As shown in IMUs 1-1 to IMU 1-4, IMU 1-1 is driven at 20.000 kHz, IMU 1-2 is driven at 20.010 kHz, IMU 1-3 is driven at 19.900 kHz, and IMU 1-4 is driven at 20.020 kHz.

[0112] In this case, since the frequency difference between the driving frequencies of IMU 1-1 to IMU 1-4 is small, they interfere with each other during the vibration of the vibrating element 11.

[0113] More specifically, such as Figure 7 As shown, for example, regarding the reference signal in the form of an oscillation signal of drive frequency fb output by automatic gain control circuit 52 in predetermined IMU 1, a reference signal in the form of drive frequency fb' (≠fb) of another nearby IMU 1 becomes a disturbance (acoustic vibration), causing interference, and amplitude modulation occurs in the reference signal actually provided to the vibration element 11, and the reference signal is provided to the vibration element 11 as an amplitude modulation signal fe including the beat corresponding to the frequency difference.

[0114] Therefore, when a reference signal with a driving frequency fb is provided to the vibrating element 11 and Figure 7 When the waveform fc is detected as angular velocity, when the reference signal provided to the vibration element 11 is changed to the amplitude modulation signal fe due to the disturbance, the angular velocity is detected as the amplitude modulation signal indicated by the thick line in the figure, relative to the waveform fc that was initially detected as angular velocity, and thus the angular velocity is incorrect.

[0115] Similarly, beats are generated as mutual vibrations with frequencies corresponding to the frequency differences between each of IMU 1-1 to IMU 1-4.

[0116] In other words, such as Figure 8 As shown, regarding beat frequency, the beat frequency between IMU 1-1 and IMU 1-2, i.e., the difference between the two drive frequencies, is 10 Hz; the beat frequency between IMU 1-1 and IMU 1-3, i.e., the difference between the two drive frequencies, is 100 Hz; and the beat frequency between IMU 1-1 and IMU 1-3, i.e., the difference between the two drive frequencies, is 20 Hz.

[0117] In addition, the beat difference between IMU 1-2 and IMU 1-3, i.e., the difference between the two drive frequencies, is 110 Hz; the beat difference between IMU 1-2 and IMU 1-4, i.e., the difference between the two drive frequencies, is 10 Hz; and the beat difference between IMU 1-3 and IMU 1-4, i.e., the difference between the two drive frequencies, is 120 Hz.

[0118] Therefore, in IMU 1-1 to IMU 1-4, the error vibration of the fluctuation frequency is superimposed due to the interference generated by the mutual reference signals, and thus the angular velocity including the error is detected in IMU 1-1 to IMU 1-4. Therefore, there is a possibility that even if the angular velocities are combined, the appropriate angular velocity cannot be obtained.

[0119] <<2. First Implementation Method>>

[0120] Therefore, in the multi-IMU disclosed herein, the corresponding number and arrangement of the accelerometer and gyroscope sensors (angular velocity sensors) constituting the multi-IMU are designed according to the application, and a mechanism is provided to mechanically suppress the propagation of vibration between the vibrating elements, thereby suppressing the effect of beat (acoustic interference) caused by mutual vibration and improving detection accuracy.

[0121] <Multiple IMU configuration examples in this disclosure>

[0122] Figure 9 The basic configuration of the multiple IMUs disclosed herein is shown. Figure 9 The multi-IMU 201 includes multiple MEMS (vibration elements) 221-1 to 221-n, demodulation circuit 222, and signal processing circuit 223.

[0123] MEMS (Vibration Element) 221 has the same characteristics as... Figure 4 The configuration corresponding to the vibration element 11 detects the amplitude modulation associated with acceleration and angular velocity, and outputs the amplitude modulation to the demodulation circuit 222.

[0124] The demodulation circuit 222 demodulates each of the amplitude modulations of the multiple MEMS (vibration elements) 221 into a waveform of an analog signal and outputs the waveform to the signal processing circuit 223.

[0125] The signal processing circuit 223 converts the waveforms of the various analog signals from the multiple MEMS (vibration elements) 221 provided by the demodulation circuit into digital signals, and combines the digital signals through signal processing to generate and output various combined outputs of acceleration and angular velocity.

[0126] Furthermore, the demodulation circuit 222 and the signal processing circuit 223 are application-specific integrated circuits (ASICs), which allows for improved detection accuracy and reduced costs depending on the application.

[0127] Furthermore, regarding the number and arrangement of MEMS (vibration elements) 221-1 to 221-n, by setting the number and arrangement according to the application, the detection accuracy can be improved and the cost reduced.

[0128] Figure 10 An example of the physical arrangement of the MEMS (vibration elements) 221-1 to 221-n of the multiple IMU 201, the demodulation circuit 222, and the signal processing circuit 223 is shown.

[0129] The multi-IMU 201 includes a stacked MEMS layer 231, a demodulation circuit, and a signal processing circuit layer 232.

[0130] Multiple MEMS (vibration elements) 221-1 to 221-n are arranged in MEMS layer 231. In addition, demodulation circuit 222 and signal processing circuit 223 are formed in demodulation circuit and signal processing circuit layer 232.

[0131] A MEMS substrate (MEMS die) Zd is provided in the MEMS layer 231, and on the MEMS substrate (MEMS die) Zd, a region Za in which a MEMS 221A as an accelerometer sensor is arranged and a region Zg in which a MEMS 221G as an angular velocity sensor (gyroscope sensor) is arranged are provided, and the area of ​​each region is set according to the application, and the number of MEMS 221A and MEMS 221G is arranged according to the application.

[0132] exist Figure 10 In the left part, “ACC” is written in the area Za where the MEMS 221A, which is arranged as an acceleration sensor, and “Gyro” is written in the area Zg where the MEMS 221G, which is arranged as an angular velocity sensor (gyroscope sensor).

[0133] In addition, Figure 10 In the center of the right side, MEMS 221A, which acts as an acceleration sensor, is denoted as "A", and MEMS 221G, which acts as an angular velocity sensor (gyroscope sensor), is denoted as "G".

[0134] In other words, Figure 10 The following example is shown in the center of the right side: three MEMS 221A as accelerometers are arranged linearly in the vertical direction on the right side of the figure, and nine MEMS 221G as angular velocity sensors (gyroscope sensors) are arranged in a three-by-three array in the horizontal × vertical direction.

[0135] Note that in the following text, MEMS 221A, which is used as an acceleration sensor, will also be referred to as acceleration sensor 221A, and MEMS 221G, which is used as an angular velocity sensor (gyroscope sensor), will also be referred to as angular velocity sensor (gyroscope sensor) 221G.

[0136] Figure 10 The number and arrangement of the accelerometer 221A and angular velocity sensor 221G in the example are merely one example, and various numbers and arrangements can be adopted depending on the application.

[0137] For example, such as Figure 11 As shown in the upper left of the multi-IMU 201A, an accelerometer 221A and an angular velocity sensor 221G can be set.

[0138] In addition, such as Figure 11As shown in the upper right part of the diagram, a multi-IMU 201B, an accelerometer 221A-11 and three angular velocity sensors 221G-11 to 221g-13 can be arranged in a row along the horizontal direction.

[0139] In addition, such as Figure 11 As shown in the lower left of the multi-IMU 201C, two accelerometers 221A-21 and 221A-22 and two angular velocity sensors 221G-21 and 221G-22 can each be arranged in a row vertically in the figure.

[0140] In addition, such as Figure 11 As shown in the lower left of the image, the multi-IMU 201D can be configured as follows, which includes two accelerometers 221A-31 and 221A-32 arranged in a row and nine angular velocity sensors 221G-31 to 221G-39 arranged in a three-by-three configuration.

[0141] <Configuration Example of 3D Angular Velocity Sensor and Accelerometer>

[0142] Next, a configuration example of the three-dimensional angular velocity sensor 221G and the acceleration sensor 221A will be described.

[0143] For the three-dimensional angular velocity sensor 221G and the acceleration sensor 221A, there are the following types: one three-dimensional vibration element is provided, which can detect three-dimensional angular velocity and acceleration independently relative to the orthogonal X-axis, Y-axis and Z-axis directions; and three one-dimensional vibration elements are provided, which can detect one dimension for each of the orthogonal X-axis, Y-axis and Z-axis directions.

[0144] <Types of Three-Dimensional Vibration Elements>

[0145] More specifically, such as Figure 12 As shown, there is an angular velocity sensor 221G-3D × 1 ( Figure 12 (Left side of the image) and accelerometer 221A-3D × 1 ( Figure 12 (On the right side of the image), each of them is equipped with a three-dimensional vibration element, which can independently detect three-dimensional angular velocity and acceleration relative to the orthogonal X-axis, Y-axis and Z-axis directions.

[0146] The 3DMG vibration element of the angular velocity sensor 221G-3D × 1 and the 3DMA vibration element of the accelerometer 221A-3D × 1 are, for example, piezoelectric three-dimensional vibration elements.

[0147] Note that in Figure 12In the text, for the angular velocity sensor 221G-3D × 1, there is a black dot mark inside the dashed box, and for the accelerometer sensor 221A-3D × 1, there is a mark with a black dot inside the white circle inside the dotted-dash box, and similar symbols are used in the following text.

[0148] <Types of One-Dimensional Vibrating Elements>

[0149] On the other hand, such as Figure 13 As shown, there is an angular velocity sensor 221G-1D × 3 ( Figure 13 (Left side of the image) and accelerometer 221A-1D × 3 ( Figure 13 (Right side of the middle section) The angular velocity sensor 221G-1D × 3 is equipped with a total of three one-dimensional vibration elements 1DMG-X, 1DMG-Y and 1DMG-Z that can detect one-dimensional angular velocity in the orthogonal X-axis, Y-axis and Z-axis directions respectively. The accelerometer 221A-1D × 3 is equipped with a total of three one-dimensional vibration elements 1DMA-X, 1DMA-Y and 1DMA-Z that can detect one-dimensional acceleration in the orthogonal X-axis, Y-axis and Z-axis directions respectively.

[0150] The one-dimensional vibration elements 1DMG-X, 1DMG-Y, and 1DMG-Z of the angular velocity sensor 221G-1D × 3 and the one-dimensional vibration elements 1DMA-X, 1DMA-Y, and 1DMA-Z of the accelerometer sensor 221A-1D × 3 are, for example, capacitive one-dimensional vibration elements.

[0151] <Detailed configuration example of a one-dimensional vibration element (X and Y directions) of a capacitive angular velocity sensor>

[0152] Here, we will refer to Figure 14 This describes a detailed configuration example of a one-dimensional vibration element for a capacitive angular velocity sensor used in the X and Y directions.

[0153] Figure 14 The one-dimensional vibration element 1DMG-X,Y of the capacitive angular velocity sensor in the upper part for the X and Y directions includes a fixed part GHF and counterweights GHW1 and GHW2 within the fixed part GHF. The counterweights GHW1 and GHW2 have mutually symmetrical shapes and are configured to vibrate independently in opposite phases along the left and right directions in the figure.

[0154] like Figure 14 As shown in the lower left, the counterweights GHW1 and GHW2 repeat the following operation at a predetermined frequency: in the first cycle, they vibrate in the direction of arrows DHL1 and DHR1 to face each other, and in the second cycle, they vibrate in the direction of arrows DHL2 and DHR2 to move away from each other.

[0155] In this state, for example, as shown in the lower middle part of the figure, when a rotation (roll rotation, pitch rotation) is applied about axis Ax (X or Y) in the direction of arrow XYR, as shown by the dashed line, the counterweights GHW1 and GHW2 deform.

[0156] At the boundary between the counterweights GHW1 and GHW2 and the fixed part GHF, such as Figure 14 As shown in the lower right part, the comb-shaped electrodes EL1 and EL2, which are opposite each other, are arranged to mesh with each other at a predetermined interval, and as... Figure 14 As shown in the lower left, the capacitance generated when vibrating at a predetermined frequency is in a stable state.

[0157] However, as Figure 14 As shown in the lower middle part, when the counterweights GHW1 and GHW2 move and deform as shown by the dashed line, the distance between electrodes EL1 and EL2 changes from the predetermined distance, and the capacitance changes.

[0158] By detecting this change in capacitance, the angular velocity of rotation (roll or pitch) about the X or Y direction is detected. Therefore, the one-dimensional vibration elements 1DMG-X and Y of the angular velocity sensors for the X and Y directions, respectively, are arranged orthogonally in the XY plane.

[0159] <Detailed configuration example of a one-dimensional vibration element (Z direction) of a capacitive angular velocity sensor>

[0160] Next, we will refer to Figure 15 This describes a detailed configuration example of a one-dimensional vibration element for a capacitive angular velocity sensor in the Z direction.

[0161] Figure 15 The one-dimensional vibration element 1DMG-Z of the capacitive angular velocity sensor in the upper part of the Z-direction includes a fixed part GVF and counterweights GVW1 and GVW2 within the fixed part GVF. The counterweights GVW1 and GVW2 are configured to vibrate independently in opposite phases along the left and right directions in the figure.

[0162] like Figure 15 As shown in the lower left, the counterweights GVW1 and GVW2 repeat the following operation at a predetermined frequency: in the first cycle, they vibrate in the direction of arrows DHL1 and DHR1 to face each other, and in the second cycle, they vibrate in the direction of arrows DHL2 and DHR2 to move away from each other.

[0163] In this state, for example, as shown in the lower middle part of the figure, when a rotation (yaw rotation) is applied about axis Ax (Z) in the direction of arrow ZR, as shown by the dashed line, the counterweights GVW1 and GVW2 deform.

[0164] At the boundary between the counterweights GVW1 and GVW2 and the fixed part GVF, such as Figure 14 As shown in the lower right part, comb electrodes EL1 and EL2 are arranged opposite each other at a predetermined interval, and as... Figure 15 As shown in the lower left part, the capacitance generated during vibration is in a stable state.

[0165] However, as Figure 15 As shown in the lower right part, when a rotation (yaw rotation) is applied in the direction of arrow ZR and the movement of counterweights GVW1 and GVW2 is deformed as shown by the dashed line, the distance between electrodes EL1 and EL2 changes from the predetermined distance, and the capacitance changes.

[0166] The angular velocity in the Z direction is detected by detecting this change in capacitance.

[0167] <Example of an arrangement of angular velocity sensors, each comprising three one-dimensional vibration elements>

[0168] Next, an example of the arrangement of the angular velocity sensors 221G using three angular velocity sensors 221G-1D × 3 will be described. Each angular velocity sensor 221G-1D × 3 includes a total of three one-dimensional vibration elements 1DMG-X, 1DMG-Y and 1DMG-Z, which are capable of detecting one-dimensional angular velocities in the orthogonal X-axis direction, Y-axis direction and Z-axis direction, respectively.

[0169] As mentioned above, in Figure 16 In region Zg1, angular velocity sensors 221G-51 to 221G-53, each including one-dimensional vibration elements 1DMG-X, 1DMG-Y and 1DMG-Z, are arranged.

[0170] exist Figure 16 In the angular velocity sensor 221G-51, one-dimensional vibrating elements 1DMG-X and 1DMG-Y are arranged at 90 degrees to each other on an XY plane parallel to the paper. In contrast, one-dimensional vibrating element 1DMG-Z can be arranged relative to 1DMG-X and 1DMG-Y in any positional relationship, as long as 1DMG-Z is on the XY plane. However, in Figure 16 In this configuration, the one-dimensional vibration element 1DMG-Z is arranged parallel to the one-dimensional vibration element 1DMG-X. Note that this also applies to the angular velocity sensors 221G-52 and 221G-53.

[0171] In addition, Figure 16In this configuration, among angular velocity sensors 221G-51 to 221G-53, angular velocity sensor 221G-52 is arranged to form a 90-degree angle with angular velocity sensors 221G-51 and 221G-53. Therefore, since the positional relationship between the two sides in the XY plane is changed by 90 degrees, vibrations on both sides are eliminated and beats are suppressed. In either case, acceleration can be detected with high accuracy.

[0172] In addition, as referenced Figure 14 and Figure 15 The one-dimensional vibration elements 1DMG-X, 1DMG-Y, and 1DMG-Z constituting the angular velocity sensor 221G are configured to detect acceleration by the change in capacitance between comb electrodes EL1 and EL2, the change in capacitance being caused by the deformation of counterweights GHW and GVW according to the generated angular velocity. Therefore, as the number of comb electrodes EL1 and EL2 increases, the angular velocity sensor 221G can detect angular velocity with higher accuracy. Thus, by increasing the number of angular velocity sensors 221G (one-dimensional vibration elements 1DMG-X, 1DMG-Y, and 1DMG-Z) and increasing the number of comb electrodes EL1 and EL2, the accuracy of angular velocity detection can be improved.

[0173] <Detailed configuration example of a one-dimensional vibration element (X and Y directions) of a capacitive accelerometer>

[0174] Next, we will refer to Figure 17 This describes a detailed configuration example of a one-dimensional vibration element for a capacitive accelerometer used in the X and Y directions.

[0175] Figure 17 The one-dimensional vibration element 1DMA-X,Y of the capacitive accelerometer in the upper part for the X and Y directions includes a fixed part AHF and a counterweight AHW within the fixed part AHF. The counterweight AHW has a shape shown by the dashed line extending in the left-right direction in the figure.

[0176] The counterweight AHW is configured to be pressed by a spring (not shown) from both directions of arrows AHD1 and AHU1. Figure 17 Maintain the position of the center AHC as the reference in the vertical direction (perpendicular to X or Y) in the upper left part.

[0177] In this state, for example, as shown in the lower part of the figure, when an acceleration is applied relative to the center AHC in the direction of arrow AH, the load balance of the spring (not shown) is disrupted, and the counterweight AHW moves in the direction of arrow AHU1 from the position of the shape shown by the solid line to the position of the shape shown by the dashed line.

[0178] At the boundary between the counterweight AHW and the fixing part AHF, such as Figure 14 As shown in the lower right part, comb electrodes EL1 and EL2 are arranged opposite each other at a predetermined interval, and... Figure 17 The capacitance generated in the state shown in the upper part is in a stable state.

[0179] However, for example, such as Figure 17 As shown in the lower part, when the counterweight AHW moves upward in the figure as shown by the dashed line, the distance between electrodes EL1 and EL2 changes from a predetermined distance, and the capacitance changes.

[0180] Acceleration in the X or Y direction is detected by detecting this change in capacitance. Therefore, the one-dimensional vibration elements 1AMG-X and Y of the acceleration sensors for the X and Y directions, respectively, are arranged orthogonally in the XY plane.

[0181] <Detailed configuration example of a one-dimensional vibration element (Z direction) of a capacitive accelerometer>

[0182] Next, we will refer to Figure 18 This describes a detailed configuration example of a one-dimensional vibration element for a capacitive accelerometer 221A in the Z direction.

[0183] Figure 18 The one-dimensional vibration element 1DMA-Z of the capacitive accelerometer 221A in the upper part for the Z direction includes a fixed part AVF and a counterweight AVW within the fixed part AVF. The counterweight AVW is configured to be driven like a seesaw around the axis Ax in the figure by acceleration from the Z direction.

[0184] like Figure 18 As shown in the upper part, the counterweight AVW has a cavity H on the left side formed by hollowing out the counterweight. It has an asymmetrical shape around axis Ax in the figure and has different configurations of the counterweight on the left and right.

[0185] In this state, for example, as shown in the lower part of the figure, when an acceleration is applied in the direction of arrow DV21 in the direction perpendicular to axis Ax (Z-axis direction), the counterweight AVW has an asymmetrical shape about axis Ax and has different weights on the left and right. Therefore, the heavier right part in the figure moves from the position indicated by the shape shown by the solid line in the direction of arrow DV22, and the lighter left part, which is provided with cavity H, tilts in the direction of arrow DV21, as indicated by the shape shown by the dashed line.

[0186] At the boundary between the counterweight AVW and the fixed part AVF, such as Figure 14 As shown in the lower right part, comb electrodes EL1 and EL2 are arranged opposite each other at a predetermined interval, and as shown in the figure... Figure 18 The capacitance generated by the counterweight AVW at the top, as shown, is in a stable state when it is stationary.

[0187] However, as Figure 18 As shown in the lower part, when the counterweight AVW is tilted as shown by the dashed line, the distance between electrodes EL1 and EL2 changes from the predetermined distance, and the capacitance changes.

[0188] By detecting this change in capacitance, acceleration in the Z direction can be detected.

[0189] <Example of an arrangement of accelerometers, each comprising three one-dimensional vibration elements>

[0190] Next, an example of the arrangement of the accelerometer 221A using three accelerometers 221A-1D × 3 will be described. Each accelerometer 221A-1D × 3 includes a total of three one-dimensional vibration elements 1DMA-X, 1DMA-Y and 1DMA-Z, which are capable of detecting one-dimensional acceleration in the orthogonal X-axis, Y-axis and Z-axis directions, respectively.

[0191] exist Figure 19 In region Za1, accelerometers 221A-51 to 221A-53, each including one-dimensional vibration elements 1DMA-X, 1DMA-Y and 1DMA-Z, are arranged.

[0192] exist Figure 19 In the accelerometer sensor 221A-51, one-dimensional vibration elements 1DMA-X and 1DMA-Y are arranged at 90 degrees to each other on an XY plane parallel to the paper. In contrast, one-dimensional vibration element 1DMA-Z can be arranged relative to the one-dimensional vibration elements 1DMA-X and 1DMA-Y in any positional relationship, as long as 1DMA-Z is on the XY plane. However, in Figure 19 In this configuration, the one-dimensional vibration element 1DMA-Z is arranged parallel to the one-dimensional vibration element 1DMA-X. Note that this also applies to the accelerometers 221A-52 and 221A-53.

[0193] In addition, Figure 19 In the accelerometer sensors 221A-51 to 221A-53, accelerometer 221A-52 is arranged to form a 90-degree angle with accelerometer sensors 221A-51 and 221A-53. Therefore, since the positional relationship between the two sides changes by 90 degrees in the XY direction, the vibration on both sides is eliminated, and acceleration can be detected with high accuracy.

[0194] In addition, as referenced Figure 17 and Figure 18The one-dimensional vibration elements 1DMA-X, 1DMA-Y, and 1DMA-Z constituting the accelerometer sensor 221A are configured to detect acceleration by changes in capacitance between comb electrodes EL1 and EL2, the changes in capacitance being caused by changes in the positions of counterweights AHW and AVW according to the generated acceleration. Therefore, the larger the size (heavier the counterweights) of the counterweights AHW and AVW constituting the one-dimensional vibration elements 1DMA, the more accurately the accelerometer sensor 221A can detect acceleration. Thus, by making the arrangement area (and spatial dimensions) of the accelerometer sensor 221A (one-dimensional vibration elements 1DMA-X, 1DMA-Y, and 1DMA-Z) larger (heavier), the accuracy of acceleration detection can be improved.

[0195] <Changes in the arrangement of the accelerometer and angular velocity sensors (Part 1)>

[0196] Next, an example of the arrangement of accelerometers and angular velocity sensors in a multi-IMU will be described.

[0197] Figure 20 An example configuration of the multi-IMU 201E is shown, in which: two accelerometers 221A-101 and 221A-102, each comprising three one-dimensional vibration elements, are arranged in a three-by-three configuration, and nine angular velocity sensors 221G-101 to 221G-109, each comprising three one-dimensional vibration elements.

[0198] As described above, the accuracy of angular velocity detection is improved by increasing the number of angular velocity sensors 221G, and the accuracy of acceleration detection is improved by making the acceleration sensor 221A larger (heavier).

[0199] Therefore, in Figure 20 In the IMU 201E, two accelerometers 221A are arranged, each with a larger arrangement area than the angular velocity sensor 221G. Note that a single accelerometer 221A with dimensions corresponding to the two sensors can be arranged. However, when using a single accelerometer 221A, it is possible that acceleration cannot be detected in the event of an incorrect operating state due to some malfunction. Therefore, with two or more accelerometers, even if the detection accuracy of each sensor decreases, more stable acceleration detection can be achieved by providing redundancy.

[0200] Furthermore, regarding the angular velocity sensors 221G, although each angular velocity sensor 221G is smaller than the accelerometer 221A, nine more angular velocity sensors 221G are arranged than the number of accelerometer sensors 221A in order to increase the number of comb electrodes EL1 and EL2 used for detecting capacitance. Regarding the angular velocity sensors 221G, the individual dimensions can be made smaller, and a greater number of angular velocity sensors 221G can be arranged, allowing for an increase in the number of electrodes and enabling the detection of capacitance changes with higher accuracy. However, by making the angular velocity sensors 221G smaller, the number of electrodes per angular velocity sensor 221G is reduced, and therefore, the size must be such that the total number of electrodes is not reduced.

[0201] Furthermore, as shown in the multi-IMU 201E, due to the limited placement area, the dimensions of the angular velocity sensor 221G and the accelerometer 221A, the ratio of their respective placement areas, and the number of each sensor need to be set according to the respective detection accuracy of acceleration and angular velocity required by the installed device.

[0202] Furthermore, the counterweights GHW and GVW constituting the one-dimensional vibration element 1DMG (which constitutes the angular velocity sensor 221G) and the counterweights AHW and AVW constituting the one-dimensional vibration element 1DMA (which constitutes the accelerometer sensor 221A) are arranged in vacuum-sealed spaces within the respective housings of the one-dimensional vibration elements 1DMG and 1DMA. With this configuration, the accuracy of detection of angular velocity and acceleration based on the movement of the counterweights GHW and GVW and the counterweights AHW and AVW can be improved due to the increased vacuum and reduced air resistance.

[0203] However, higher vacuum levels come with higher costs, especially for the 221G angular velocity sensor (which can detect angular velocity with higher accuracy as the number increases). Low-cost sensors with low vacuum levels have limited accuracy improvements even when the number increases. Therefore, it is necessary to use sensors with vacuum levels that are cost-effective and set the number according to the required accuracy.

[0204] <Changes in the arrangement of the accelerometer and angular velocity sensors (Part 2)>

[0205] To date, a multi-IMU 201E has been described, in which: two accelerometers 221A-101 and 221A-102, each comprising three one-dimensional vibration elements, are arranged in a three-by-three configuration, and angular velocity sensors 221G-101 to 221G-109, each comprising three one-dimensional vibration elements.

[0206] However, in Figure 20In the multi-IMU 201E, since the angular velocity sensors 221G-101 to 221G-109 are arranged in the same direction, vibration may cause interference and beats associated with acoustic interference may occur.

[0207] Therefore, as Figure 21 As shown, the angular velocity sensor 221G can be arranged such that every other angular velocity sensor 221G is tilted to the left by only 90 degrees in both the horizontal and vertical directions, thereby increasing the distance between vibration sources, changing the direction by 90 degrees, and suppressing acoustic interference.

[0208] exist Figure 21 In the multi-IMU 201F, in Figure 20 In the multi-IMU 201E, angular velocity sensors 221G-101 to 221G-109 are replaced by angular velocity sensors 221G'-102, 221G'-104, 221G'-106, and 221G'-108.

[0209] In other words, the angular velocity sensors 221G'-102, 221G'-104, 221G'-106 and 221G'-108 each have the following configuration: the corresponding angular velocity sensors 221G-102, 221G-104, 221G-106 and 221G-108 are rotated only 90 degrees to the left in the figure.

[0210] With this arrangement, acoustic interference can be suppressed because the direction of vibration between adjacent angular velocity sensors 221G changes by 90 degrees. Note that if adjacent angular velocity sensors 221G are tilted by only 90 degrees, it is sufficient, and therefore the tilt can be 90 degrees to the left or 90 degrees to the right. Furthermore, if there are multiple angular velocity sensors 22G that need to be tilted by 90 degrees, all angular velocity sensors 22G can be tilted by 90 degrees in the same direction, or they do not need to be tilted in the same direction.

[0211] <Changes in the arrangement of the accelerometer and angular velocity sensors (Part 3)>

[0212] Next, we will refer to Figure 22 Describe the changes in the arrangement of accelerometers and angular velocity sensors in the case of multiple IMUs mounted on augmented reality (AR) glasses.

[0213] Figure 22 The arrangement of the accelerometer and angular velocity sensors of the multiple IMUs mounted on AR glasses is shown.

[0214] Installed Figure 22 The AR glasses 251 have multiple IMUs 201G, including accelerometers 221A-101b and 221A-102b and angular velocity sensors 221G-101s, 221G-103s, 221G-105s, 221G-107s, 221G-109s, 221G'-102s, 221G'-104s, 221G'-106s and 221G'-108s.

[0215] Accelerometers 221A-101b and 221A-102b have a ratio Figure 21 The accelerometers 221A-101 and 221A-102 have larger configurations.

[0216] In AR glasses 251, self-position estimation via Simultaneous Localization and Mapping (SLAM) is performed based on images captured by camera device 252 shown in the figure. In this SLAM-based self-position estimation, high-accuracy position and attitude information from low-noise accelerometer sensor 221A is necessary, thus requiring high-accuracy inertial navigation. Therefore, in the multi-IMU 201G mounted on AR glasses 251, accelerometer sensors 221A-101b and 221A-102b are configured to improve accuracy by increasing their footprint and enlarging the counterweights AHW and AVW to be larger (heavier), since greater mass results in lower noise.

[0217] The angular velocity sensors 221G-101s, 221G-103s, 221G-105s, 221G-107s, 221G-109s, 221G'-102s, 221G'-104s, 221G'-106s, and 221G'-108s are basically similar. Figure 21 The angular velocity sensors 221G-101 to 221G-109 of the multi-IMU 201F are arranged, but their size is reduced due to the enlargement of the accelerometer sensors 221A-101b and 221A-102b.

[0218] <Changes in the arrangement of the accelerometer and angular velocity sensors (Part 4)>

[0219] Next, we will refer to Figure 23 Describe the changes in the arrangement of accelerometers and angular velocity sensors when multiple IMUs are installed on virtual reality (VR) glasses.

[0220] Figure 23 The arrangement of the accelerometer and angular velocity sensors of the multiple IMUs mounted on VR glasses is shown.

[0221] Installed Figure 23The multi-IMU 201H on the VR glasses 261 includes two accelerometers 221A-101s and 221A-102s, nine angular velocity sensors 221Gs and nine angular velocity sensors 221G's. Here, regarding the nine angular velocity sensors 221Gs and nine angular velocity sensors 221G's, a total of eighteen sensors are arranged in a horizontal direction of six × three vertical direction, and are tilted alternately at 90 degrees in the horizontal and vertical directions.

[0222] Accelerometers 221A-101s and 221A-102s have a ratio Figure 21 The accelerometers 221A-101 and 221A-102 have smaller configurations.

[0223] In VR glasses 261, head tracking performed by angular velocity sensors 221Gs and 221G's requires pose estimation with small long-term bias fluctuations. As mentioned above, the angular velocity sensor 221G exhibits higher sensitivity and accuracy with an increase in the number of comb teeth used for detecting capacitance. Therefore, Figure 21 The number of angular velocity sensors in the 221G is nine, while... Figure 23 The number of IMUs 201H is doubled, with eighteen angular velocity sensors 221Gs and 221G's arranged in the array. As a result, noise is reduced and bias stability is improved, thus achieving a configuration suitable for long-term use applications such as VR glasses 261.

[0224] The angular velocity sensors 221Gs and 221G's are basically arranged as follows: Figure 21 The angular velocity sensors 221G-101 to 221G-109 of the multi-IMU 201F extend in the horizontal direction, but the size is reduced due to the doubling of the total number.

[0225] <Changes in the arrangement of the accelerometer and angular velocity sensors (Part 5)>

[0226] Next, we will refer to Figure 24 Describe the changes in the arrangement of accelerometer and angular velocity sensors when multiple IMUs are mounted on a camera device.

[0227] Figure 24 The arrangement of the accelerometer and angular velocity sensors of the multiple IMUs mounted on the camera device is shown.

[0228] Installed Figure 24The multi-IMU 201I on the camera device 271 includes nine angular velocity sensors 221Gs and nine angular velocity sensors 221G's. Here, the nine angular velocity sensors 221Gs and nine angular velocity sensors 221G's, totaling eighteen sensors, are arranged in a horizontal arrangement of six × three vertical arrangement, alternating between the horizontal and vertical directions. Additionally, an accelerometer 221A is not provided.

[0229] In camera shake correction of camera device 271, suppressing rotation of the optical axis is important, thus requiring attitude stabilization to suppress rotation on three axes (roll, pitch, and yaw). The angular velocity sensor 221G exhibits higher sensitivity and accuracy with an increase in the number of comb electrodes used for sensing capacitance. Consequently, the noise density becomes lower, and the speed responsiveness is improved, resulting in a configuration suitable for applications such as accurately tracking fast-moving camera shake correction.

[0230] The angular velocity sensors 221Gs and 221G's are basically arranged as follows: Figure 21 The angular velocity sensors 221G-101 to 221G-109 of the multi-IMU 201F extend in the horizontal direction, but the size is reduced due to the doubling of the total number.

[0231] <Changes in the arrangement of the accelerometer and angular velocity sensors (Part 6)>

[0232] Next, we will refer to Figure 25 Describe the changes in the arrangement of accelerometers and angular velocity sensors when multiple IMUs are mounted on virtual reality (VR) glasses.

[0233] Figure 25 The arrangement of the accelerometer and angular velocity sensors of the multiple IMUs mounted on VR glasses is shown.

[0234] Installed Figure 25 The VR glasses 261' feature a multi-IMU 201J that includes accelerometers 221A-101 and 221A-102 and an angular velocity sensor 221G-151. The angular velocity sensor 221G-151 comprises 10 one-dimensional vibration elements 1DMG-X, 10 one-dimensional vibration elements 1DMG-Y, and 30 one-dimensional vibration elements 1DMG-Z. In other words, in... Figure 25 In the example, the one-dimensional vibration elements 1DMG-X and 1DMG-Y, as well as the one-dimensional vibration element 1DMG-Z, which constitute the angular velocity sensor 221G-151, exist in different numbers.

[0235] Head tracking using a gyroscope in VR glasses requires pose estimation with small long-term bias fluctuations. Since fusion with an accelerometer can be used in both roll (rotation direction about the x-axis) and pitch (rotation direction about the y-axis), pose estimation with small long-term drift can be achieved by using both the angular velocity sensor 221G and the accelerometer 221A together.

[0236] On the other hand, since fusion with the accelerometer 221A cannot be performed in the yaw direction (rotation direction about the z-axis), the bias fluctuation of the angular velocity sensor 221G-151 itself is required to be small. For example... Figure 25 As shown, since more one-dimensional vibration elements 1DMG-Z are arranged than one-dimensional vibration elements 1DMG-X and 1DMG-Y, in other words, since the arrangement area Zz of one-dimensional vibration elements 1DMG-Z is set to be wider than the arrangement areas Zx and Zy of one-dimensional vibration elements 1DMG-X and 1DMG-Y, an angular velocity sensor 221G-151 with higher accuracy in the yaw direction than on other axes is realized, and VR head tracking with small long-term drift is achieved.

[0237] <Backside Resonator>

[0238] Next, the back-side resonator, which is configured to mechanically reduce acoustic interference between the angular velocity sensors 221G, will be described.

[0239] Note that the description of the back-side resonator will use the term "Multi-IMU 201K," in which the angular velocity sensors 221G are arranged in a 3x3 array, as shown below. Figure 26 The left side is shown. However, here, only the acceleration sensor 221A is omitted from the figure, although it could actually be included. Furthermore, the number and arrangement of the angular velocity sensors 221G can be other configurations, and are not limited to these. Figure 26 Multiple IMUs 201K.

[0240] like Figure 26 As shown in the left part of the multi-IMU 201K, it is known that acoustic interference occurs among the nine angular velocity sensors 221G due to the transmission of mutual vibrations between the angular velocity sensors 221G.

[0241] The back-side resonator absorbs the vibrations generated in each individual angular velocity sensor 221G, suppresses the transmission to adjacent angular velocity sensors 221G, and reduces acoustic interference.

[0242] Figure 26 The right side is a schematic side cross-sectional view of the angular velocity sensor 221G, including the back-side resonator. (See diagram below.) Figure 26As shown on the right, the angular velocity sensor 221G includes, from top to bottom, a MEMS layer 301, an anchor 302, and a back-side resonator 303.

[0243] MEMS layer 301 is a layer in which a configuration corresponding to the one-dimensional vibration elements 1DMG-X, 1DMG-Y and 1DMG-Z constituting the angular velocity sensor 221G is formed.

[0244] Anchor 302 is formed on the back resonator 303 and has a configuration that supports MEMS layer 301 at a point.

[0245] The back-side resonator 303 uses silicon and other materials to absorb vibrations generated in the MEMS layer 301 and transmitted via the anchor 302, thereby suppressing the propagation to the adjacent angular velocity sensor 221G and reducing the impact of acoustic interference.

[0246] More specifically, the back-side resonator 303 acts as a band-stop filter, which cuts off acoustic vibrations of a predetermined frequency generated by driving one-dimensional vibration elements 1DMG-X, 1DMG-Y and 1DMG-Z that constitute the angular velocity sensor 221G formed in the MEMS layer 301, and reduces acoustic interference by suppressing the transmission of vibrations to adjacent angular velocity sensors 221G.

[0247] Next, we will refer to Figures 27 to 31 Describe the details of MEMS layer 301, anchor 302 and back-side resonator 303.

[0248] Notice, Figure 27 The left side shows when... Figure 27 The cross-sectional structure obtained when the side section of the right part, with multiple IMUs at 201K units, is sliced ​​at dashed lines L1 and L2. Additionally, Figure 28 This is an exploded stereoscopic view based on the 221G angular velocity sensor. Furthermore, Figure 29 yes Figure 28 A detailed top view of each layer in the exploded 3D diagram. Additionally, Figure 30 This is an enlarged top view of the back-side resonator 303, and Figure 31 This is a diagram showing the state in which the back-side resonator 303 absorbs vibrations generated in the MEMS layer 301 and transmitted via the anchor 302.

[0249] In other words, when Figure 27 The cross-sectional structure obtained when the side cross-section, shown in the upper right corner, is sliced ​​at the dashed line L1, using multiple IMU 201K units as units, is... Figure 27 The cross-sectional structure S1 in the upper left part is a cross-sectional structure in which a MEMS layer 301 is formed, which is configured to correspond to the one-dimensional vibration elements 1DMG-X, 1DMG-Y and 1DMG-Z constituting the angular velocity sensor 221G.

[0250] In addition, when Figure 27 The cross-sectional structure obtained when the side cross-section, shown in the lower right corner, is sliced ​​at the dashed line L1, using multiple IMU 201K units as units, is... Figure 27 The cross-sectional structure S2 in the lower left part is a structure in which the back-side resonator 303 is formed.

[0251] Note that in Figure 27 In the cross-sectional structure S2 in the lower left part, the position of the anchor 302 formed on the back resonator 303 is also... Figure 27 The right side of the text indicates this.

[0252] In addition, Figure 27 In this paper, the horizontal dimension of MEMS layer 301 is represented as smaller than the dimension of back-side resonator 303. However, this is because it is an exaggerated representation of the space between adjacent MEMS layers 301. In reality, the horizontal dimension of MEMS layer 301 is only very slightly smaller than the dimension of back-side resonator 303.

[0253] In addition, Figure 28 The diagram indicates the stacking relationship between MEMS layer 301, anchor 302, and back-side resonator 303. However, to aid in understanding the stacking relationship, the corresponding dimensions are exaggerated, and the actual dimensional relationships should be referenced from [the diagram]. Figure 29 Detailed view.

[0254] Anchors 302 are positioned at five locations on the platform 303c of the back-side resonator 303, and support the MEMS layer 301 at these five points. Note that the number of anchors 302 can be more than five.

[0255] In addition, such as Figure 30 As shown, the back-side resonator 303 includes a fixing part 303a, a damper part 303b, and a platform part 303c.

[0256] The fixing part 303a is connected to the substrate 311 in the lower layer. The damper part 303b has a flexible structure that absorbs vibrations transmitted from the MEMS layer 301 via the anchor 302 provided on the platform part 303c, and suppresses the propagation of vibrations to the adjacent angular velocity sensor 221G.

[0257] For example, such as Figure 31 As shown, when the platform 303c moves in the direction of the arrow in the figure due to the vibration generated in the MEMS layer 301 being transmitted via the anchor 302 formed in the upper layer, the damper section 303b is deformed to have the following characteristics: Figure 31The damper portion 303b', as shown by the dashed line, absorbs vibrations and suppresses the propagation of vibrations toward the substrate 311, which is fixed by the fixing portion 303a. As a result, the propagation of vibrations toward the adjacent angular velocity sensor 221G formed on the substrate 311 is suppressed, acoustic interference can be suppressed, and the angular velocity detection accuracy of the angular velocity sensor 221G can be improved.

[0258] <First Modification of the Backside Resonator>

[0259] So far, an example has been described in which a damper portion 303b and a fixing portion 303a of a back-side resonator 303 are formed at the corner of the platform 303c. However, a configuration corresponding to the damper portion 303b and the fixing portion 303a can be formed on the side surface portions on all four sides of the platform 303c.

[0260] Figure 32 and Figure 33 An example configuration of an angular velocity sensor 221G' is shown, which includes a modification of a back-side resonator in which a configuration corresponding to the damper portion 303b and the fixing portion 303a is formed on the side surface portions on all four sides of the platform portion 303c.

[0261] Figure 32 The left side is a top view of the modified angular velocity sensor 221G' with a back-side resonator, and... Figure 32 The right side is along Figure 32 The side cross-section view taken by the dashed line L10 on the left side. Additionally, Figure 33 Is with Figure 27 The corresponding diagram.

[0262] In other words, Figure 33 The left side shows when... Figure 33 The cross-sectional structure obtained when the side of the right part, which is in units of multiple IMU 201K', is sliced ​​at dashed lines L11 and L12.

[0263] In other words, when Figure 33 The cross-sectional structure obtained when the side cross-section, shown in the upper right part, is sliced ​​at the dashed line L11, using multiple IMU 201K' units as the unit, is... Figure 33 The cross-sectional structure S11 in the upper left part is a cross-sectional structure in which a MEMS layer 301 is formed, which is configured to correspond to the one-dimensional vibration elements 1DMG-X, 1DMG-Y and 1DMG-Z constituting the angular velocity sensor 221G'.

[0264] In addition, when Figure 33 The cross-sectional structure obtained when the side cross-section, in units of multiple IMU 201K', shown in the lower right part, is sliced ​​at the dashed line L12 is... Figure 33The cross-sectional structure S12 in the lower left part is a structure in which the back-side resonator 303s is formed.

[0265] In other words, Figure 32 and Figure 33 In the angular velocity sensor 221G', a back-side resonator 303s is formed instead of a back-side resonator 303.

[0266] The back-side resonator 303s includes a fixed part 303as, a damper part 303bs, and a platform part 303cs.

[0267] The fixing portion 303as is formed such that it surrounds the stage portion 303cs in a rectangular shape and is connected to a substrate (corresponding to substrate 311) in a lower layer. The damper portion 303bs has a flexible structure, such as a spring-like structure, to absorb vibrations transmitted from the MEMS layer 301 via the anchor 302 (not shown) provided on the stage portion 303cs, and does not propagate vibrations to the adjacent angular velocity sensor 221G'.

[0268] This suppresses the propagation of vibration to the adjacent angular velocity sensor 221G' on the substrate (corresponding to substrate 311) formed in a layer lower than the back resonator 303s, thus suppressing acoustic interference and improving the angular velocity detection accuracy of the angular velocity sensor 221G'.

[0269] <Second Modification of Backside Resonator>

[0270] So far, an example has been described in which the damper portion 303bs and the fixing portion 303as of the back-side resonator 303 are formed on the side surface portions on the four sides of the platform portion 303cs, but the configuration corresponding to the fixing portion 303as, the damper portion 303bs and the platform portion 303cs can be formed in the same layer.

[0271] Figure 34 An example configuration of an angular velocity sensor is shown, in which the configuration corresponding to the mounting portion, damper portion, and platform portion of the back resonator is formed in the same layer.

[0272] Figure 34 The angular velocity sensor 221G'' includes a MEMS layer 301, an anchor 302, and a back-side resonator 303t.

[0273] The back-side resonator 303t includes a fixed part 303at, a damper part 303bt, and a platform part 303ct.

[0274] Note that the fixing part 303at, the damper part 303bt, and the platform part 303ct are the same as the fixing part 303as, the damper part 303bs, and the platform part 303cs in basic function, but the entire assembly including the fixing part 303at is formed in the same layer.

[0275] Similarly, in Figure 34 In the back-side resonator 303t, similar to the back-side resonators 303 and 303s, a configuration is adopted to absorb vibrations generated in the MEMS layer 301 and suppress propagation to the adjacent angular velocity sensor 221G''.

[0276] This suppresses the propagation of vibration to the adjacent angular velocity sensor 221G'' formed on the substrate (corresponding to substrate 311) in the lower layer, suppresses acoustic interference, and improves the angular velocity detection accuracy of the angular velocity sensor 221G''.

[0277] <<3. Second Implementation Method>>

[0278] So far, configurations of multiple IMUs 201 and multiple IMUs 201A to 201K for suppressing acoustic interference have been described, but the aforementioned multiple IMUs 201 (including multiple IMUs 201A to 201K) can also be applied to camera shake correction in image sensors.

[0279] Figure 35 An example configuration of a multi-IMU 201 configured to suppress acoustic interference is shown in a case where it is applied to an image sensor.

[0280] like Figure 35 As shown in the lower part, the aforementioned multiple IMU 201, configured to suppress acoustic interference, is coupled to the rear side of the imaging surface relative to the image sensor 401.

[0281] As described above, the multi-IMU 201 is configured to suppress acoustic interference and can detect angular velocity and acceleration with high accuracy.

[0282] Therefore, for each cell region 401a corresponding to the region on the image sensor 401 where multiple IMUs 201 are arranged, acceleration and angular velocity can be detected with high accuracy.

[0283] With this configuration, camera shake can be corrected with high accuracy for each unit region 401a by signal processing based on the precise acceleration and angular velocity detected in each unit region of the image captured by the imaging sensor 401.

[0284] Although Figure 35A configuration example using image sensor 401 is described, but sensors other than image sensor 401 can also be used, as long as the sensor detects light, and examples include depth sensors, LiDAR (light detection and ranging, laser imaging detection and ranging), time-of-flight (ToF) sensors, etc.

[0285] Additionally, the image sensor 401 stacked on top of the multiple IMUs 201 can be applied to devices that capture images for SLAM, such as those mounted on... Figure 22 The camera device 252 on the AR glasses 251.

[0286] With this configuration, high-accuracy self-position estimation can be achieved through SLAM based on images captured by the image sensor 401 acting as a camera device 252 and high-accuracy information indicating the position and orientation of the camera device 252 implemented by the multiple IMUs 201.

[0287] In particular, by employing stacked Figure 22 The configuration of multiple IMUs 201G in the image sensor 252, the ratio of the area occupied by the accelerometer 221A and the angular velocity sensor 221G, and the corresponding number of the accelerometer 221A and the angular velocity sensor 221G are optimized for SLAM within a limited space on the rear surface of the image sensor 401, and thus can achieve more accurate self-position estimation.

[0288] Additionally, the image sensor 401 that acts as the camera device 252 for capturing images for SLAM can be an RGB camera device, or it can be a depth sensor, LiDAR, or ToF sensor.

[0289] <<4. First Modification of the Second Implementation>>

[0290] So far, examples of applying camera shake correction by signal processing to each unit region 401a on the image sensor 401 have been described, but camera shake can be physically corrected using a drive mechanism instead of signal processing.

[0291] Note that camera shake refers to the shaking that occurs during imaging when the user holds and operates the imaging device, but here camera shake refers to general shaking that occurs during imaging. Therefore, for example, for imaging devices mounted on mobile devices such as drones or vehicles driven by motors or engines, the shaking during imaging caused by high-frequency vibrations from the operation of the motor or engine is also included in camera shake.

[0292] <Example of an imaging device configuration for image shake correction via driving optical blocks>

[0293] First, an overview of techniques for physically correcting camera shake using a drive mechanism will be described. Figure 36 This is an example of the configuration of an imaging device that achieves camera shake correction by driving optical blocks.

[0294] Figure 36 The imaging device 1001 includes an optical block 1011, a reflector plate 1012, a shutter 1013, an image sensor 1014, and a drive unit 1015.

[0295] The optical block 1011 has a configuration including a lens for focusing, etc., transmits incident light indicated by solid lines, and focuses the incident light onto the image sensor 1014 via the reflector plate 1012 and the shutter 1013. Note that the incident light focused onto the image sensor 1014 is represented by dashed lines. Additionally, the wavy portion in the transmission path of the incident light indicated by solid lines represents camera shake.

[0296] The reflector plate 1012, together with the mirror (not shown), reflects a portion of the incident light to the viewfinder F viewed by the user, and transmits the remaining incident light to the image sensor 1014 via the shutter 1013.

[0297] The shutter 1013 has a configuration including a mechanical or electrical configuration for controlling opening and closing, and adjusts the exposure time of light passing through the optical block 1011, entering and being transmitted to the image sensor 1014.

[0298] The image sensor 1014 includes CMOS, CCD, etc., and captures images including pixel signals based on the amount of light incident.

[0299] The drive unit 1015 includes an actuator and drives the optical block 1011 in a direction perpendicular to the incident direction of the incident light.

[0300] More specifically, when an IMU (not shown) or the like detects movement relative to the optical block 1011 caused by camera shake or the like, the drive unit 1015 drives the optical block 1011 to eliminate the detected movement.

[0301] In other words, Figure 36 In the imaging device 1001, the optical block 1011 is driven by the drive unit 1015 to eliminate movement caused by camera shake, etc., and thus correct for camera shake in the image captured by the image sensor 1014. Note that in Figure 36 The solid line indicating the path of the incident light after the drive unit 1015 is a straight line because it is used to indicate the camera device jitter that is corrected by the operation of the drive unit 1015.

[0302] However, due to Figure 36 The driving unit 1015 in the imaging device 1001 needs to drive optical blocks 1011, including lenses, etc., and therefore requires a relatively large configuration. In addition, since the driving unit 1015 has a large configuration, it is difficult to drive at high speed, and it is difficult, for example, to achieve a drive that follows and eliminates the high-frequency vibrations generated when a motor or engine is operating.

[0303] <Configuration example of an imaging device that achieves camera shake correction by driving an image sensor>

[0304] Therefore, in this disclosure, instead of driving unit 1015 for driving optical block 1011, driving unit for driving image sensor 1014 is provided, so that the configuration of driving unit is reduced in size and further obtains a configuration that can follow high-frequency vibration.

[0305] Figure 37 This is an example configuration of an imaging device that includes a drive unit for driving the image sensor 1014 to achieve camera shake correction. Note that in Figure 37 The imaging device 1021 has a similar Figure 36 Components with the same function as the imaging device 1001 are indicated by the same reference numerals, and descriptions of the components are omitted as appropriate.

[0306] In other words, Figure 37 Imaging device 1021 and Figure 36 The difference between the imaging device 1001 and the imaging device 1021 is that the imaging device 1021 includes a driving unit 1031 for driving the image sensor 1014, instead of a driving unit 1015 for driving the optical block 1011.

[0307] The driving unit 1031 includes an actuator and the like, and drives the image sensor 1014 in a direction perpendicular to the incident direction of the incident light.

[0308] When an IMU (not shown) or other device detects movement caused by camera shake or other factors, the drive unit 1031 drives the image sensor 1014 to eliminate the detected movement.

[0309] exist Figure 37 In the imaging device 1021, the image sensor 1014 is driven by the drive unit 1031 to eliminate movement caused by camera shake, etc., and thus correct camera shake in the image captured by the image sensor 1014.

[0310] because Figure 37The driving unit 1031 in the imaging device 1021 has a configuration for driving the image sensor 1014, which is relatively small and light compared to the optical block 1011 including lenses, etc., so that the configuration itself can be relatively small.

[0311] Furthermore, since the drive unit 1031 has a relatively small and light configuration, high-speed drive can be achieved. Therefore, for example, it is possible to achieve drive that follows and eliminates high-frequency vibrations generated by the operation of a motor or engine.

[0312] <Detailed configuration example of an imaging device that achieves camera shake correction by driving an image sensor>

[0313] Next, we will refer to Figure 38 A detailed configuration example of an imaging device 1021 that achieves camera shake correction by driving an image sensor 1014 is described.

[0314] Note that in Figure 38 The imaging device 1021 is configured to have a similar configuration to... Figure 37 Configurations with the same function in the imaging device 1021 are indicated by the same reference numerals, and descriptions of the configurations are omitted as appropriate.

[0315] In other words, Figure 38 The imaging device 1021 has an imaging device in which the imaging device 1021 has an imaging device 1021 having ... Figure 37 The imaging device 1021 in the middle has a more detailed configuration.

[0316] Apart from Figure 37 In addition to the configuration in the imaging device 1021, Figure 38 The imaging device 1021 also includes an IMU 1041, a position and posture detection unit 1042, and a drive control unit 1043.

[0317] In addition, the driving units 1031 are described as driving units 1031a-1 and 1031a-2 that drive the imaging sensor 1014 in the horizontal direction of the figure, and driving units 1031b-1 and 1031b-2 that drive the imaging sensor 1014 in the vertical direction of the figure.

[0318] IMU 1041, for example, is configured with multiple IMUs 201 (including multiple IMUs 201A to 201K) as described above, detects the acceleration and angular velocity of the main body of imaging device 1021, and outputs the acceleration and angular velocity to position and attitude detection unit 1042.

[0319] The position and attitude detection unit 1042 detects the position and attitude of the main body of the imaging device 1021 by integrating each of the acceleration and angular velocity detected by the IMU 1041, and outputs the position and attitude to the drive control unit 1043.

[0320] The drive control unit 1043, based on information indicating the position and orientation of the imaging device 1021 itself detected by the position and orientation detection unit 1042, outputs a control signal to each of the drive units 1031a-1 and 1031a-2, causing the imaging sensor 1014 to drive in a direction that eliminates the generated vibrations. In other words, the drive control unit 1043 drives the drive unit 1031 using the inertial navigation of the IMU 1041 and the position and orientation detection unit 1042, or via intermediate output signals (acceleration, velocity, angular velocity, and angle used as intermediate variables), and controls the position and orientation of the image sensor 1014.

[0321] More specifically, the movement of the image sensor 1014 is transmitted from the drive unit 1031, etc., attached to the main body of the imaging device 1021, and thus becomes a movement that follows the movement of the main body of the imaging device 1021 itself. In other words, the movement of the image sensor 1014 follows the movement of the main body of the imaging device 1021 and is delayed by a predetermined time relative to the movement of the imaging device 1021.

[0322] Therefore, the drive control unit 1043 predicts the movement of the image sensor 1014 based on the movement of the imaging device 1021 detected by the position and posture detection unit 1042, and provides control signals to drive the drive units 1031a-1 and 1031a-2 to eliminate the predicted movement of the image sensor 1014.

[0323] Therefore, the drive control unit 1043 controls the drive units 1031a-1 and 1031a-2 and 1031a-1 and 1031a-2 through feedforward control based on the detection results of the position and posture detection unit 1042 to eliminate the movement of the image sensor 1014.

[0324] Drive units 1031a-1 and 1031a-2 drive image sensor 1014 based on the direction and amount of movement of control signals provided from drive control unit 1043.

[0325] As a result, camera shake correction is achieved by driving the image sensor 1014 in the direction of eliminating camera shake according to the changes in the position and orientation of the imaging device 1021.

[0326] However, in Figure 38 In the configuration of the imaging device 1021, the IMU 1041 is located outside the range driven by the driving units 1031a-1 and 1031a-2 that drive the image sensor 1014. Therefore, although the position and orientation of the main body of the imaging device 1021 can be properly detected, the position and orientation of the image sensor 1014 cannot be properly detected.

[0327] Therefore, even if the image sensor 1014 is driven by drive units 1031a-1 and 1031a-2, it may not be able to properly correct for camera shake. Furthermore, especially in cases where high-frequency vibrations occur in the image sensor 1014, the IMU 1041 may be unable to detect changes in the position and orientation of the image sensor 1014 as high-frequency vibrations and may not be able to properly follow such movements, thus potentially failing to perform appropriate correction.

[0328] <Overview of the camera device disclosed herein>

[0329] Therefore, in this disclosure, an IMU is provided to detect the position and orientation of the image sensor 1014 itself, and the drive unit 1031 is driven based on the position and orientation changes of the image sensor 1014, in addition to the changes in the position and orientation of the main body of the imaging device.

[0330] Therefore, the drive unit 1015 can be controlled to follow the movement of the image sensor 1014 with high accuracy, thus enabling correction of camera shake, including high-frequency vibrations generated by the drive of a motor or engine.

[0331] Figure 39 An example configuration for use as an imaging device is shown, in which an IMU is provided to detect the position and orientation of the image sensor 1014 itself, and the drive unit 1031 is driven based on the position and orientation of the image sensor 1014 in addition to the position and orientation of the main body of the imaging device.

[0332] exist Figure 39 The imaging device 1061 has a similar Figure 38 Components in the imaging device 1021 that have the same function are indicated by the same reference numerals, and descriptions of the components are omitted as appropriate.

[0333] In other words, Figure 39 Imaging device 1061 and Figure 38The difference in the configuration of the imaging device 1021 is that an IMU 1081 and a position and attitude detection unit 1082 are newly set, and a drive control unit 1083 is set to replace the drive control unit 1043.

[0334] The IMU 1081 is configured to be integrated with the image sensor 1014. That is, the IMU 1081 is, for example, coupled to the image sensor 1014. Figure 35 The image sensor 401 uses a multi-IMU 201 to detect the acceleration and angular velocity of the image sensor 1014 and outputs the acceleration and angular velocity to the position and posture detection unit 1082.

[0335] The position and attitude detection unit 1082 detects the position and attitude of the image sensor 1014 based on the integral calculation of the angular velocity and acceleration of the image sensor 1014 provided from the IMU 1081, and outputs the position and attitude to the drive control unit 1083.

[0336] The drive control unit 1083 calculates the target values ​​of the control quantities of the drive units 1031a-1 and 1031a-2 for maintaining the position and orientation of the image sensor 1014 in a predetermined state, based on the information indicating the position and orientation of the main body of the imaging device 1061 provided by the position and orientation detection unit 1042 and the information indicating the position and orientation of the image sensor 1014 provided by the position and orientation detection unit 1082.

[0337] Then, the drive control unit 1083 generates a control signal based on the calculated target value of the control quantity, and drives the drive units 1031a-1 and 1031a-2.

[0338] In other words, the drive control unit 1083 controls the image sensor 1014 by inertial navigation based on information indicating the position and orientation of the main body of the imaging device 1061 provided by the position and orientation detection unit 1042 and information indicating the position and orientation of the image sensor 1014 provided by the position and orientation detection unit 1082, and by controlling the image sensor 1014 through intermediate output signals, so as to maintain the position and orientation of the image sensor 1014 in a predetermined state.

[0339] Furthermore, since a predetermined time delay occurs between the position and posture of the main body of the imaging device 1061 provided by the position and posture detection unit 1042 and the actual position and posture of the image sensor 1014, as described above, when only the position and posture of the main body of the imaging device 1061 are used, the drive unit 1031 only undergoes feedforward control.

[0340] However, the information indicating the position and orientation of the image sensor 1014 provided by the position and orientation detection unit 1082 can be considered as the current position and orientation of the image sensor 1014 generated by the driving units 1031a-1 and 1031a-2 and 1031a-1 and 1031a-2.

[0341] Therefore, it can also be said that the drive control unit 1083 simultaneously implements feedforward control based on the position and posture of the main body of the imaging device 1061 provided by the position and posture detection unit 1042 and feedback control based on the position and posture of the image sensor 1014 provided by the position and posture detection unit 1082.

[0342] With this configuration, the drive units 1031a-1 and 1031a-2 can be controlled to follow the movement (changes in position and orientation) of the imaging sensor 1014 with high accuracy, and thus the jitter caused by the operation of a motor or engine used as a power source in the case where the main body of the imaging device is mounted on a mobile device such as a drone or vehicle can be corrected.

[0343] <Example of the configuration of the imaging device according to the first modification of the second embodiment>

[0344] Next, we will refer to Figure 40 An example configuration of a first modified imaging apparatus according to a second embodiment of this disclosure is described. Note that in Figure 40 The image shows an example configuration where the imaging device 1101 is mounted on a mobile device 1100 such as a vehicle or drone, but a configuration where it is not mounted on the mobile device 1100 is also possible.

[0345] Figure 40 The imaging device 1101 includes: a main body unit 1111 that controls operations for correcting camera shake (including shake that accompanies vibrations accompanying movement of the moving device 1100); an imaging unit 1112 that includes an image sensor for capturing images; and an output unit 1113 that outputs an image as an imaging result.

[0346] The main unit 1111 includes an IMU 1131, a main body position and posture detection unit 1132, an image sensor position and posture detection unit 1133, a drive control unit 1134, a drive unit 1135, and a camera shake correction processing unit 1136.

[0347] IMU 1131 corresponds to Figure 39 The IMU 1041 detects the acceleration and angular velocity of the main body unit 1111 and outputs the acceleration and angular velocity to the main body position and attitude detection unit 1132.

[0348] The main body position and posture detection unit 1132 has the same characteristics as... Figure 39 The configuration corresponding to the position and posture detection unit 1042 includes a translational motion calculation unit 1151 and a rotational motion calculation unit 1152, which detects the position and posture of the main body unit 1111 and outputs the position and posture to the drive control unit 1134.

[0349] The translational motion calculation unit 1151 detects the position of the main unit 1111 by performing an integral operation based on the acceleration information provided by the IMU 1131, and outputs the position to the drive control unit 1134.

[0350] The rotational motion calculation unit 1152 detects the posture of the main body unit 1111 by performing an integral operation based on the information of the angular velocity provided from the IMU 1132, and outputs the posture to the drive control unit 1134.

[0351] The image sensor position and pose detection unit 1133 has a configuration substantially similar to that of the subject position and pose detection unit 1132, and has the same... Figure 39 The position and pose detection unit 1082 is configured accordingly. The image sensor position and pose detection unit 1133 includes a translational motion calculation unit 1171 and a rotational motion calculation unit 1172, detects the position and pose of the imaging unit 1112 (image sensor 1181), and outputs the position and pose to the drive control unit 1134.

[0352] The translational motion calculation unit 1171 detects the position of the image sensor 1181 by integrating information indicating the acceleration provided by the IMU 1182 of the imaging unit 1112, and outputs the position to the drive control unit 1134.

[0353] The rotational motion calculation unit 1172 detects the pose of the image sensor 1181 by integrating information indicating the angular velocity provided by the IMU 1182 of the imaging unit 1112, and outputs the pose to the drive control unit 1134.

[0354] Drive control unit 1134 corresponds to Figure 39 The drive control unit 1083 controls the drive unit 1135 based on information indicating the position and orientation of the main unit 1111 provided by the main unit position and orientation detection unit 1132 and information indicating the position and orientation of the image sensor 1181 of the imaging unit 1112 provided by the image sensor position and orientation detection unit 1133.

[0355] More specifically, the drive control unit 1134 includes a control quantity target value calculation unit 1134a, and based on information indicating the position and orientation of the main unit 1111 and information indicating the position and orientation of the image sensor 1181, the control quantity target value calculation unit 1134a calculates a control quantity target value for maintaining the position and orientation of the image sensor 1181 in a predetermined state.

[0356] Then, the drive control unit 1134 generates a control signal for driving the drive unit 1135 based on the control quantity target value calculated by the control quantity target value calculation unit 1134a, and provides the control signal to the drive unit 1135 to drive the drive unit 1135.

[0357] The drive unit 1135 has a component including and Figure 39 The position and orientation of the image sensor 1181 are driven by the configuration of actuators, etc., corresponding to the drive units 1031 (1031a-1, 1031a-2, 1031b-1 and 1031b-2) and driven by control signals from the drive control unit 1134.

[0358] The drive control unit 1134 provides information indicating changes in the position and orientation of the main unit 1111 and the image sensor 1181 to the camera shake correction processing unit 1136.

[0359] The camera shake correction processing unit 1136 includes an image frame buffer 1136a, which buffers images provided from the image sensor 1181. Based on the position and orientation of the main unit 1111 and information indicating the position and orientation of the image sensor 1181, the camera shake correction processing unit 1136 corrects the buffered image captured by the image sensor 1181 through signal processing, and outputs the corrected image to the output unit 1113.

[0360] Note that this will be referred to later. Figure 41 The details of the image camera shake correction processing of the camera shake correction processing unit 1136 are described.

[0361] The imaging unit 1112 includes an image sensor 1181 and an IMU 1182. The image sensor 1181 has a... Figure 39 The image sensor 1014 is configured to capture an image including pixel signals based on the amount of incident light, and provide the image to the camera device shake correction processing unit 1136.

[0362] IMU 1182 has the same Figure 39The configuration corresponds to the IMU 1081 and has a configuration integrated with the image sensor 1181, and thus detects the acceleration and angular velocity of the image sensor 1181, and outputs the acceleration and angular velocity to the image sensor position and pose detection unit 1133.

[0363] For example, imaging unit 1112 has wherein the aforementioned multiple IMUs 201 are coupled to relative to Figure 35 The image sensor 401 is configured on the rear side of the imaging surface.

[0364] In other words, the image sensor 1181 has the same characteristics as... Figure 35 The image sensor 401 has a corresponding configuration, and the IMU 1182 has a corresponding configuration for multiple IMUs 201.

[0365] Therefore, IMU 1182 is also configured with multiple IMU 201s to suppress acoustic interference.

[0366] Therefore, in image sensor 1181 and IMU 1182, it is also possible to target each unit region of image sensor 401 ( Figure 35 Multiple IMUs 201 are arranged in unit area 401a of the image sensor 401 to detect acceleration and angular velocity. Note that in the following text, each unit area of ​​the image sensor 401 ( Figure 35 The multiple IMUs 201 arranged in the cell region 401a) are also referred to as IMU cells.

[0367] The output unit 1113 outputs the image corrected by the camera shake correction processing unit 1136. More specifically, the output unit 1113 includes an image recording unit 1191 and a transmission unit 1192.

[0368] The image recording unit 1191 records the image corrected by the camera shake correction processing unit 1136 as data.

[0369] The transmission unit 1192 includes, for example, Ethernet, and transmits the image corrected by the camera device shake correction processing unit 1136 to an external information processing device or communication terminal via a network (not shown).

[0370] Note that the output unit 1113 may have other configurations and may include, for example, a display with display functions, and may display an image corrected by the camera device shake correction processing unit 1136.

[0371] Therefore, in Figure 40In the imaging device 1101, the drive control unit 1134 performs control by inertial navigation based on information indicating the position and posture of the subject of the imaging device 1101 provided by the subject position and posture detection unit 1132 and information indicating the position and posture of the image sensor 1181 provided by the image sensor position and posture detection unit 1133, and by intermediate output signals, so as to maintain the position and posture of the image sensor 1181 in a predetermined state.

[0372] Furthermore, since a predetermined time delay occurs between the position and orientation of the main body unit 1111 of the imaging device 1101 provided by the main body position and orientation detection unit 1132 and the actual position and orientation of the image sensor 1181, the drive unit 1135 only undergoes feedforward control when only the position and orientation of the main body of the imaging device 1101 are used.

[0373] However, the information indicating the position and orientation of the image sensor 1181 provided by the image sensor position and orientation detection unit 1133 can be considered as the current position and orientation of the image sensor 1181 generated by the driving unit 1135.

[0374] Therefore, it can also be said that the drive control unit 1134 simultaneously implements feedforward control of the drive unit 1135 based on the position and posture of the main unit 1111 of the imaging device 1101 provided by the main body position and posture detection unit 1132, and feedback control of the drive unit 1135 based on the position and posture of the image sensor 1181 provided by the image sensor position and posture detection unit 1133.

[0375] <Regarding camera shake correction>

[0376] Regarding the driving of the drive unit 1135 controlled by the drive control unit 1134, since there is a time lag between the provision of the control signal and the actual driving, it may be impossible to correct camera jitter relative to vibrations that are faster than the predetermined speed.

[0377] The camera shake correction processing unit 1136 corrects camera shake that cannot be corrected even by driving the drive unit 1135 by means of signal processing, based on the position and posture of the main unit 1111 of the imaging device 1101 provided by the main body position and posture detection unit 1132 and provided by the drive control unit 1134, and the position and posture of the image sensor 1181 provided by the image sensor position and posture detection unit 1133.

[0378] As described above, the imaging unit 1112 has, for example, the following configuration: wherein the aforementioned multiple IMUs 201 are coupled to each unit region relative to... Figure 35The rear side of the imaging surface of the image sensor 401.

[0379] Therefore, the IMU 1182 can output the same information as... Figure 35 The acceleration and angular velocity of each unit region in the image sensor 1181 corresponding to the unit region 401a of the image sensor 401.

[0380] Therefore, the image sensor position and pose detection unit 1133 obtains the indication and Figure 35 The image sensor 401's unit area 401a corresponds to the position and orientation information of the image sensor 1181's unit area, and the information is output to the drive control unit 1134.

[0381] The drive control unit 1134 acquires and stores the indications provided by the image sensor position and posture detection unit 1133. Figure 35 The image sensor 401 provides information on the position and orientation of the unit area 401a of the image sensor 401 corresponding to the unit area of ​​the image sensor 1181, and provides this information to the camera device shake correction processing unit 1136.

[0382] Furthermore, correspondingly, the image sensor 1181 outputs to the camera device shake correction processing unit 1136, including data that is coupled to the IMU 1182. Figure 35 The image sensor 401's unit region 401a corresponds to an image of a unit pixel signal. Note that in the following text, it is used in conjunction with... Figure 35 The pixel group of the cell region 401a of the image sensor 401 corresponding to the cell region on the image sensor 1181 is also referred to as a pixel unit. Therefore, the IMU unit of each cell region as the multiple IMU 201 and the pixel unit including the pixel group of the cell region are the corresponding configuration of the image sensor 1181 and IMU 1182 in the cell region.

[0383] The image sensor 1181 of the imaging unit 1112 outputs pixel signals to the camera device shake correction processing unit 1136 in units of pixels.

[0384] Camera shake correction processing unit 1136 from and Figure 35 The position and pose information of the unit region of the image sensor 1181 corresponding to the unit region 401a of the image sensor 401 are used to obtain the pixel-by-pixel motion vector, and the pixel signal provided by the corresponding image sensor 1181 in units of pixel units is corrected according to the motion vector, and the pixel signal is buffered in the image frame buffer 1136a.

[0385] In other words, the camera shake correction processing unit 1136 obtains motion vectors based on position and pose information provided in units of pixel units as unit regions, repeatedly performs processing based on the obtained motion vectors to apply camera shake correction processing to the corresponding image in units of pixel units and buffers the image, and outputs the image to the output unit 1113 when one frame has been buffered.

[0386] For example, in the image sensor 1181, there are N unit regions (pixel units #1 to #N and IMU units #1 to #N) that use IMU units and pixel units as units respectively, according to the following... Figure 41 The timing diagram shows the process executed by the camera shake correction processing unit 1136.

[0387] Notice, Figure 41 The following diagrams, from top to bottom, show the readout timing in pixel units of image sensor 1181, the readout timing in IMU 1182 for acceleration and angular velocity (position and attitude) in IMU units, the timing for performing correction processing in camera shake correction processing unit 1136, the write timing to image frame buffer 1136a, the accumulation timing in image frame buffer 1136a, and the output timing of image frames.

[0388] In other words, when the synchronization signal indicating the readout of the image frame synchronization signal n starts at the timing indicated by the frame synchronization signal SyncFn, the unit synchronization signal indicating the readout of pixel unit #1, which is the first pixel unit, is simultaneously assumed to be the unit synchronization signal SyncU#1. Note that the frame synchronization signal is, for example, 30 Hz, 60 Hz, or 120 Hz, and the unit synchronization signal is, for example, approximately 1 kHz to 10 kHz.

[0389] In the frame synchronization signal SyncFn== unit synchronization signal SyncU#1, when the reading of image frame n begins, the pixel signal of pixel unit #1 is first read in the image sensor 1181 and the pixel signal is provided to the camera device shake correction processing unit 1136.

[0390] Furthermore, the acceleration and angular velocity of the corresponding IMU unit #1 are simultaneously read from IMU 1182. Then, the image sensor position and pose detection unit 1133 detects information indicating the position and pose of the cell region corresponding to IMU unit #1 in image sensor 1181, and provides this information to the drive control unit 1134. Additionally, the drive control unit 1134 provides the information indicating the position and pose of the cell region corresponding to IMU unit #1 in image sensor 1181 to the camera shake correction processing unit 1136.

[0391] At timing t1, which is the next timing, the camera device shake correction processing unit 1136 obtains a motion vector based on information indicating the position and orientation of the cell region corresponding to the IMU cell #1, performs camera device shake correction processing on the pixel signal of the corresponding pixel cell #1 by using the obtained motion vector, and stores the pixel signal in the image frame buffer 1136a.

[0392] Subsequently, in the unit synchronization signal SyncU#2, the pixel signal of pixel unit #2 is read by the image sensor 1181 and provided to the camera device shake correction processing unit 1136.

[0393] Additionally, the acceleration and angular velocity of the corresponding IMU unit #2 are simultaneously read from IMU 1182. Then, the image sensor position and pose detection unit 1133 detects information indicating the position and pose of the cell region corresponding to IMU unit #2 in image sensor 1181 and provides this information to the drive control unit 1134. Furthermore, the drive control unit 1134 provides the information indicating the position and pose of the cell region corresponding to IMU unit #2 in image sensor 1181 to the camera shake correction processing unit 1136.

[0394] Then, at the next timing t2, the camera device shake correction processing unit 1136 obtains a motion vector based on information indicating the position and orientation of the cell region corresponding to the IMU cell #2, performs camera device shake correction processing on the pixel signal of the corresponding pixel cell #2 by using the obtained motion vector, and stores the pixel signal in the image frame buffer 1136a.

[0395] Subsequently, similar processing is repeated until pixel unit #N and IMU unit #N, and when the image data of a frame that has undergone camera shake correction processing is buffered in image frame buffer 1136a, camera shake correction processing unit 1136 outputs the image signal of frame n buffered in image frame buffer 1136a to output unit 1113 with frame synchronization signal SyncF(n+1) == unit synchronization signal SyncU#1 as the readout timing of the next frame (n+1).

[0396] Note that in the calculation of the target value of the control quantity of the drive unit 1135 used to drive the position and orientation of the image sensor 1181, the position and orientation obtained for each cell region (i.e., for each IMU cell) can be used to enable the control performed by the drive unit 1135 to be executed at a high frequency.

[0397] In addition, in the calculation of the target value of the control quantity, the target value of the control quantity can be obtained by using information such as the average value obtained from the position and pose statistics of each IMU unit for a frame, or by using information indicating the position and pose of a specific unit region.

[0398] <Imaging Processing>

[0399] Next, we will refer to Figure 42 The flowchart description is provided by Figure 40 The imaging processing is performed by the imaging device 1101.

[0400] In step S401, IMU 1131 detects the acceleration and angular velocity of the main body unit 1111 and outputs the acceleration and angular velocity to the main body position and attitude detection unit 1132.

[0401] In step S402, the translational motion calculation unit 1151 of the main body position and posture detection unit 1132 detects the position of the main body unit 1111 by performing integration calculation based on the acceleration information provided from the IMU 1131, and outputs the position to the drive control unit 1134. The rotational motion calculation unit 1152 of the main body position and posture detection unit 1132 detects the posture of the main body unit 1111 by performing integration calculation based on the angular velocity information provided from the IMU 1132, and outputs the posture to the drive control unit 1134.

[0402] In step S403, image sensor 1181 captures an image.

[0403] In step S404, the image sensor 1181 and IMU 1182 set the unprocessed cell regions in the cell regions corresponding to the pixel units and IMU units, respectively, as the target cell regions.

[0404] In step S405, the image sensor 1181 reads the pixel signal of the pixel unit corresponding to the target unit area and outputs the pixel signal to the camera device shake correction processing unit 1136.

[0405] In step S406, IMU 1182 detects the acceleration and angular velocity of the image sensor 1181 of the IMU unit corresponding to the target unit region, and outputs the acceleration and angular velocity to the image sensor position and pose detection unit 1133.

[0406] In step S407, the translational motion calculation unit 1171 of the image sensor position and pose detection unit 1133 detects the position of the target cell region of the image sensor 1181 by integrating information indicating the acceleration of the IMU cell corresponding to the target cell region provided by the IMU 1182, and outputs the position to the drive control unit 1134. The rotational motion calculation unit 1172 of the image sensor position and pose detection unit 1133 detects the pose of the target cell region of the image sensor 1181 by integrating information indicating the angular velocity of the IMU cell corresponding to the target cell region provided by the IMU 1182 of the imaging unit 1112, and outputs the pose to the drive control unit 1134.

[0407] The drive control unit 1134 provides information indicating the position and orientation of the main unit 1111 and information indicating the position and orientation of the target unit area of ​​the image sensor 1181 to the camera shake correction processing unit 1136.

[0408] In step S408, the camera shake correction processing unit 1136 obtains motion vectors pixel by pixel in the target unit region based on information indicating the position and orientation of the main body unit 1111 and information indicating the position and orientation of the target unit region of the image sensor 1181, and applies camera shake correction processing using the obtained motion vectors pixel by pixel in the target unit region.

[0409] In step S409, the camera shake correction processing unit 1136 buffers the pixel signals of the target unit region that have undergone camera shake correction processing in the image frame buffer 1136a.

[0410] In step S410, the image sensor 1181 and the IMU 1182 determine whether there are unprocessed cell regions in the cell regions corresponding to the pixel cell and the IMU cell, respectively.

[0411] In step S410, if there are unprocessed unit regions, the processing returns to step S404.

[0412] That is, the processing of steps S404 to S410 is repeated, and the processing of performing camera shake correction processing on each unit area and performing buffering in the image frame buffer 1136a is repeated, until camera shake correction processing is performed on all unit areas.

[0413] Then, after performing camera shake correction on all unit areas and determining in step S410 that there are no unprocessed unit areas, the process proceeds to step S411.

[0414] In step S411, the camera shake correction processing unit 1136 reads an image of a frame that has undergone camera shake correction processing and is buffered in the image frame buffer 1136a, and outputs these images to the output unit 1113.

[0415] In step S412, the drive control unit 1134 generates a control signal for controlling the drive unit 1135 based on information indicating the position and posture of the main unit 1111 provided by the main unit position and posture detection unit 1132 and information indicating the position and posture of the image sensor 1181 of the imaging unit 1112 provided by the image sensor position and posture detection unit 1133, and outputs the control signal to the drive unit 1135.

[0416] More specifically, the drive control unit 1134 controls the control quantity target value calculation unit 1134a, and based on the information indicating the position and posture of the main body unit 1111 and the information indicating the position and posture of the image sensor 1181, the control quantity target value calculation unit 1134a calculates the control quantity target value used by the drive unit 1135 to bring the position and posture of the image sensor 1181 into a predetermined state.

[0417] In step S413, the drive control unit 1134 generates a control signal for driving the drive unit 1135 based on the control quantity target value calculated by the control quantity target value calculation unit 1134a, provides the control signal to the drive unit 1135, and controls the drive.

[0418] In step S414, it is determined whether the imaging process has been terminated, and if no termination has been indicated, the process returns to step S401.

[0419] That is, repeat steps S401 to S414 until the imaging process is terminated.

[0420] Then, in step S414, the process ends when an instruction to terminate the imaging process is given.

[0421] Through the above processing, in addition to the information on the position and posture of the main unit 1111, the driving unit 1135 also controls the position and posture of the image sensor 1181 based on the information on the position and posture of the image sensor 1181. Therefore, the camera device shake correction of the image sensor 1181 can be achieved with high accuracy and high speed.

[0422] In particular, since the IMU 1182 is provided integrated with the image sensor 1181, the position and posture of the image sensor 1181 detected by the IMU 1182 can be appropriately detected, so the image capturing apparatus 1101 mounted on the mobile device 1100 such as a drone or a vehicle can also correct image capturing apparatus shake caused by high-frequency vibration from a drive motor, engine, or the like of the mobile device 1100 (shake caused by high-frequency vibration from a motor, engine, or the like).

[0423] Furthermore, an image captured by the image sensor 1181 can be corrected through signal processing on the basis of information indicating the position and posture detected for each unit area with an image unit corresponding to an IMU unit being taken as a unit, and thus image capturing apparatus shake can be corrected with higher accuracy.

[0424] Note that, although the above describes an example where the shake correction processing is implemented by obtaining a motion vector on the basis of information indicating the position and posture of the main body unit 1111 and information indicating the position and posture of the image sensor 1181, it can also be considered that high-frequency vibration has little influence on the position and posture of the main body unit 1111, and therefore the shake correction processing through signal processing can be implemented by obtaining a motion vector only in accordance with information indicating the position and posture of the image sensor 1181.

[0425] <Number of IMU Units and Accuracy of Image Capturing Apparatus Shake Correction>

[0426] Note that the above describes an example where N IMU units each including a plurality of IMUs 201 are provided for one image sensor 1181, and N only needs to be 1 or more.

[0427] Therefore, for example, as Figure 43 shown on the left side, when N is 1, the plurality of IMUs 201B1 that configure an IMU unit can be used, as Figure 43 shown in the middle, when N is 4, the plurality of IMUs 201B4 that configure an IMU unit can be used, as Figure 43 shown on the right side, when N is 16, the plurality of IMUs 201B16 that configure an IMU unit can be used, or a larger number of IMUs may be used.

[0428] Furthermore, regarding the number N of the plurality of IMUs 201 that configure an IMU unit, the larger the value of N is, the more accurate image capturing apparatus shake correction can be achieved, but as the number increases, the processing load and power consumption also increase, and the cost also increases, therefore there is a trade-off between the accuracy of the image capturing apparatus shake correction processing, the processing load, power consumption, and cost. Accordingly, it is desirable to determine the number of the plurality of IMUs according to the accuracy and cost required for the purpose.

[0429] <<5. Second Modification of the Second Implementation>>

[0430] Note that, although the above describes the relationship between... Figure 35 The image sensor 401 and the multiple IMU 201 shown similarly integrate the imaging unit 1112 of the image sensor 1181 and IMU 1182, but other configurations can also be used, as long as the image sensor 1181 and IMU 1182 are configured to be in contact with each other, and the position and orientation of the image sensor 1181 can be obtained by the IMU 1182.

[0431] For example, such as Figure 44 As shown, a single IMU 1182 can be configured to be in contact with the side surface portion of the image sensor 1181.

[0432] Notice, Figure 44 An example configuration is also shown where the drive unit 1135 is also integrated around the image sensor 1181. That is, as... Figure 44 As shown, the imaging unit 1112 may have a configuration in which the image sensor 1181, IMU 1182 and driving unit 1135 are integrated, and may, for example, form a package structure of an imaging element that integrates these components.

[0433] Furthermore, the imaging unit 1112 may have the following configuration: wherein the driving unit 1135 is further added thereto, integrating a reference. Figure 35 The image sensor 401 and the multiple IMU 201 described are integrated in a configuration and together with the configuration, and, for example, an imaging element package structure integrating these components can be formed.

[0434] <<6. Examples executed by software>>

[0435] Furthermore, the aforementioned series of processes can be performed by hardware, but also by software. In the case where the series of processes are performed by software, the program constituting the software is installed from a recording medium in a computer incorporated into dedicated hardware, or from a recording medium in a general-purpose computer, for example, which can perform various functions by installing various programs.

[0436] Figure 45 An example configuration of a general-purpose computer is shown. This personal computer includes a central processing unit (CPU) 11001. An input / output interface 11005 is connected to the CPU 11001 via a bus 11004. Read-only memory (ROM) 11002 and random access memory (RAM) 11003 are connected to the bus 11004.

[0437] Connected to the input / output interface 11005 are: an input unit 11006, which includes input devices such as a keyboard and mouse for users to input operation commands; an output unit 11007, which outputs the processed operation screen and the image of the processing result to a display device; a storage unit 11008, which includes a hard disk drive storing programs and various types of data; and a communication unit 11009, which includes a local area network (LAN) adapter and performs communication processing via a network such as the Internet. Furthermore, a drive 11010 is connected to read data from and write data to a removable storage medium 11011, such as a disk (including a floppy disk), optical disk (including a CD-ROM and a DVD), magneto-optical disk (including a miniature optical disk (MD)), or semiconductor memory.

[0438] The CPU 11001 performs various types of processing based on programs stored in the ROM 11002 or read from a removable storage medium 11011 such as a disk, optical disk, magneto-optical disk, or semiconductor memory and installed in the storage unit 11008, and programs loaded from the storage unit 11008 into the RAM 11003. Furthermore, data required by the CPU 11001 to perform various types of processing is appropriately stored in the RAM 11003.

[0439] In the computer 1000 configured as described above, for example, the CPU 11001 loads the program stored in the storage unit 11008 into the RAM 11003 via the input / output interface 11005 and the bus 11004 and executes the program to perform the series of processes described above.

[0440] For example, the program executed by the computer (CPU 11001) can be provided by recording it in a removable recording medium 11011, such as a packaging medium. Alternatively, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0441] In a computer, by installing the removable storage medium 11011 onto the drive 11010, a program can be installed in the storage unit 11008 via the input / output interface 11005. Furthermore, the program can be received by the communication unit 11009 via a wired or wireless transmission medium and installed in the storage unit 11008. Alternatively, the program can be pre-installed in the ROM 11002 or the storage unit 11008.

[0442] Note that a program executed by a computer may be a program that performs processing sequentially in the order described in this specification, or it may be a program that performs processing in parallel or at necessary time intervals (such as when a call is made).

[0443] Notice, Figure 45 CPU 11001 implementation Figure 40 The functions of the drive control unit 1134 and the camera shake correction processing unit 1136.

[0444] Furthermore, in this specification, a system refers to a collection of multiple constituent elements (devices, modules (components), etc.), and it is not important whether all constituent elements are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a device in which multiple modules are housed in one housing, are both systems.

[0445] Furthermore, the implementation methods of this disclosure are not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of this disclosure.

[0446] This disclosure can also be configured as follows.

[0447] <1> A light detection device, comprising:

[0448] A light detection element that detects light; and

[0449] An angular velocity sensor detects the angular velocity of the optical detection element, wherein...

[0450] The angular velocity sensor includes a back-side resonator between the angular velocity sensor and the substrate on which the angular velocity sensor is mounted, the back-side resonator absorbing the vibration of the angular velocity sensor.

[0451] <2> according to <1> The aforementioned optical detection device, wherein,

[0452] The angular velocity sensor includes a vibrating element that detects angular velocities in three dimensions, and detects the angular velocities in the three dimensions from the vibrating element.

[0453] <3> according to <1> The aforementioned optical detection device, wherein,

[0454] The angular velocity sensor includes three vibrating elements that detect the angular velocity in one of the three orthogonal dimensions, and detects the angular velocity in all three dimensions from the three vibrating elements.

[0455] <4> according to <3> The aforementioned optical detection device, wherein,

[0456] The three vibration elements in the angular velocity sensor are arranged in a predetermined positional relationship.

[0457] <5> according to <4> The aforementioned optical detection device, wherein,

[0458] Of the three vibration elements in the angular velocity sensor, at least two vibration elements that detect angular velocities in orthogonal dimensions are arranged orthogonally.

[0459] <6> according to <4> The aforementioned optical detection device, wherein,

[0460] A predetermined number of the angular velocity sensors are arranged in a two-dimensional array.

[0461] <7> according to <6> The aforementioned optical detection device, wherein,

[0462] The predetermined number of angular velocity sensors arranged in a two-dimensional array are all arranged in the same direction.

[0463] <8> according to <6> The aforementioned optical detection device, wherein,

[0464] The predetermined number of angular velocity sensors arranged in a two-dimensional array are arranged with each sensor rotated 90 degrees in each dimension.

[0465] <9> according to <6> The aforementioned optical detection device further includes:

[0466] An accelerometer sensor detects the acceleration of the photodetector element, wherein...

[0467] The acceleration sensor includes a vibration element that detects acceleration in the three dimensions, and detects acceleration in the three dimensions from the vibration element.

[0468] <10> according to <6> The aforementioned optical detection device further includes:

[0469] An accelerometer sensor detects the acceleration of the photodetector element, wherein...

[0470] The accelerometer includes three vibration elements that detect acceleration in one of the three orthogonal dimensions, and detects acceleration in the three dimensions from the three vibration elements.

[0471] <11> according to <10> The aforementioned optical detection device, wherein,

[0472] The three vibration elements in the acceleration sensor are arranged in a predetermined positional relationship.

[0473] <12> according to <11> The aforementioned optical detection device, wherein,

[0474] Of the three vibration elements in the acceleration sensor, at least two vibration elements that detect acceleration in orthogonal dimensions are arranged orthogonally.

[0475] <13> according to <12> The aforementioned optical detection device, wherein,

[0476] A predetermined number of the acceleration sensors are arranged in a two-dimensional array.

[0477] <14> according to <13> The aforementioned optical detection device, wherein,

[0478] The predetermined number of accelerometers arranged in a two-dimensional array are all arranged in the same direction.

[0479] <15> according to <13> The aforementioned optical detection device, wherein,

[0480] The predetermined number of accelerometers arranged in a two-dimensional array are arranged with each sensor rotated 90 degrees in each dimension.

[0481] <16> according to <13> The aforementioned optical detection device, wherein,

[0482] In a device equipped with the aforementioned optical detection device, when comparing the acceleration and the angular velocity, the detection of the acceleration is prioritized, and the area where the acceleration sensor is arranged is made larger than the area where the angular velocity sensor is arranged.

[0483] <17> according to <16> The aforementioned optical detection device, wherein,

[0484] This makes the counterweight of the vibration element included in the acceleration sensor larger and heavier.

[0485] <18> according to <13> The aforementioned optical detection device, wherein,

[0486] In a device equipped with the aforementioned optical detection device, when comparing the acceleration and the angular velocity, the detection of the angular velocity is emphasized, and the area where the angular velocity sensor is arranged is made larger than the area where the acceleration sensor is arranged.

[0487] <19> according to <18> The aforementioned optical detection device, wherein,

[0488] The angular velocity sensors are arranged in a large number.

[0489] <20> according to <3> The aforementioned optical detection device, wherein,

[0490] In a device equipped with the optical detection device, when the importance of each of the three dimensions of angular velocity is different, the vibration elements in the angular velocity sensor that detect the angular velocity of one of the three dimensions are arranged in a larger number of ways to detect the angular velocity of the dimension with higher importance, and in a smaller number of ways to detect the angular velocity of the dimension with lower importance.

[0491] <21> according to <1> The aforementioned optical detection device, wherein,

[0492] The light detection element is an RGB camera device, a depth sensor, a LiDAR (light detection and ranging, laser imaging detection and ranging) or a time-of-flight (ToF) sensor.

[0493] List of reference numerals

[0494] 201, 201A to 201K Multiple Inertial Measurement Units (IMUs)

[0495] 221 MEMS

[0496] 221A Accelerometer

[0497] 221G angular velocity sensor

[0498] 223 Signal Processing Circuit

[0499] 301 MEMS layer

[0500] 302 Anchor

[0501] 303, 303s back-side resonator

[0502] 303a, 303as Fixing Part

[0503] 303b, 303bs Damper Section

[0504] 303c, 303cs Taiwan

[0505] 401 Image Sensor

[0506] 1100 mobile devices

[0507] 1101 Imaging Device

[0508] 1111 Main Unit

[0509] 1112 Imaging Unit

[0510] 1113 Output Unit

[0511] 1131 IMU

[0512] 1132 Main body position and posture detection unit

[0513] 1133 Image Sensor Position and Pose Detection Unit

[0514] 1134 Drive Control Unit

[0515] 1135 drive unit

[0516] 1136 Camera Shake Correction Processing Unit

[0517] 1151 Translational Motion Calculation Unit

[0518] 1152 Rotational Motion Calculation Unit

[0519] 1171 Translational Motion Calculation Unit

[0520] 1172 Rotational Motion Calculation Unit

[0521] 1181 Image Sensor

[0522] 1182 IMU

[0523] 1191 Image Recording Unit

[0524] 1192 Transmission Unit

Claims

1. A light detection device, comprising: A light detection element, wherein the light detection element detects light; as well as An angular velocity sensor detects the angular velocity of the optical detection element, wherein... The angular velocity sensor includes a back-side resonator between the angular velocity sensor and the substrate on which the angular velocity sensor is mounted, the back-side resonator absorbing the vibration of the angular velocity sensor.

2. The optical detection device according to claim 1, wherein, The angular velocity sensor includes a vibrating element that detects angular velocities in three dimensions, and detects the angular velocities in the three dimensions from the vibrating element.

3. The optical detection device according to claim 1, wherein, The angular velocity sensor includes three vibrating elements that detect the angular velocity in one of the three orthogonal dimensions, and detects the angular velocity in all three dimensions from the three vibrating elements.

4. The optical detection device according to claim 3, wherein, The three vibration elements in the angular velocity sensor are arranged in a predetermined positional relationship.

5. The optical detection device according to claim 4, wherein, Of the three vibration elements in the angular velocity sensor, at least two vibration elements that detect angular velocities in orthogonal dimensions are arranged orthogonally.

6. The optical detection device according to claim 4, wherein, A predetermined number of the angular velocity sensors are arranged in a two-dimensional array.

7. The optical detection device according to claim 6, wherein, The predetermined number of angular velocity sensors arranged in a two-dimensional array are all arranged in the same direction.

8. The optical detection device according to claim 6, wherein, The predetermined number of angular velocity sensors, arranged in a two-dimensional array, are arranged with each sensor rotated 90 degrees in each dimension.

9. The optical detection device according to claim 6, further comprising: An accelerometer sensor detects the acceleration of the photodetector element, wherein... The acceleration sensor includes a vibration element that detects acceleration in the three dimensions, and detects acceleration in the three dimensions from the vibration element.

10. The optical detection device according to claim 6, further comprising: An accelerometer sensor detects the acceleration of the photodetector element, wherein... The accelerometer includes three vibration elements that detect acceleration in one of the three orthogonal dimensions, and detects acceleration in the three dimensions from the three vibration elements.

11. The optical detection device according to claim 10, wherein, The three vibration elements in the acceleration sensor are arranged in a predetermined positional relationship.

12. The optical detection device according to claim 11, wherein, Of the three vibration elements in the acceleration sensor, at least two vibration elements that detect acceleration in orthogonal dimensions are arranged orthogonally.

13. The optical detection device according to claim 12, wherein, A predetermined number of the acceleration sensors are arranged in a two-dimensional array.

14. The optical detection device according to claim 13, wherein, The predetermined number of accelerometers arranged in a two-dimensional array are all arranged in the same direction.

15. The optical detection device according to claim 13, wherein, The predetermined number of accelerometers arranged in a two-dimensional array are arranged with each sensor rotated 90 degrees in each dimension.

16. The optical detection device according to claim 13, wherein, In a device equipped with the aforementioned optical detection device, when comparing the acceleration and the angular velocity, the area where the acceleration sensor is arranged is made larger than the area where the angular velocity sensor is arranged.

17. The optical detection device according to claim 16, wherein, This makes the counterweight of the vibration element included in the acceleration sensor larger and heavier.

18. The optical detection device according to claim 13, wherein, In a device equipped with the aforementioned optical detection device, when comparing the acceleration and the angular velocity, the detection of the angular velocity is emphasized, and the area where the angular velocity sensor is arranged is made larger than the area where the acceleration sensor is arranged.

19. The optical detection device according to claim 18, wherein, The angular velocity sensors are arranged in a large number.

20. The optical detection device according to claim 3, wherein, In a device equipped with the optical detection device, when the importance of each of the three dimensions of angular velocity is different, the vibration elements in the angular velocity sensor that detect the angular velocity of one of the three dimensions are arranged in a larger number of ways to detect the angular velocity of the dimension with higher importance, and in a smaller number of ways to detect the angular velocity of the dimension with lower importance.

21. The optical detection device according to claim 1, wherein, The light detection element is an RGB camera device, a depth sensor, a LiDAR (light detection and ranging, laser imaging detection and ranging) or a time-of-flight (ToF) sensor.

Images