Electronic device, control method, and program

Abstract

To provide an electronic apparatus that can perform highly accurate imaging state determination processing.SOLUTION: An electronic apparatus has: an acquisition unit that acquires first information on a shake; an operation unit that inputs the first information to a machine learning model and outputs second information on the type of the shake; and a first shake correction control unit that performs control of shake correction by using the second information. The first shake correction control unit performs the control of shake correction after a still image photographing instruction is issued by using the second information output from the operation unit based on the first information before the still image photographing instruction is issued.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an electronic device used in an imaging system having a shake correction function, and a control method thereof. [Background technology]

[0002] Electronic devices with a photographing function, such as cameras and smartphones, may be equipped with a shake correction function that suppresses image blur caused by camera shake, etc. As a control for this shake correction function, a technique is known in which a photographing state is determined from an angular velocity signal, an acceleration signal, etc., and a shake correction signal for performing shake correction is calculated according to the result.

[0003] Patent Document 1 describes a shake correction device that distinguishes between a still shooting state and a walking shooting state based on the time that has elapsed since an angular velocity signal exceeded a threshold value. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5743838 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the image stabilization device disclosed in Patent Document 1, there is room for further improvement in the accuracy of determining the shooting state.

[0006] SUMMARY OF THE PRESENT EMBODIMENTS An object of the present invention is to provide an electronic device that enables highly accurate determination of the shooting state. [Means for solving the problem]

[0007] According to one aspect of the present invention for solving the above-mentioned problems, there is provided an electronic device comprising: an acquisition unit that acquires first information regarding shake; a calculation unit that outputs second information by inputting the first information into a machine learning model; and a first shake correction control unit that controls shake correction using the second information; the calculation unit outputs the second information based on time series data of a first length as the first information, and the second information based on time series data of a second length longer than the first length as the first information; The first shake correction control unit uses the second information output from the calculation unit based on the first information before the instruction to shoot a still image is given to control shake correction after the instruction to shoot a still image is given, and the second information is information on a type of shake selected from a plurality of shake types including at least two of a walking state, a tripod state, a panning state, and a holding state, and the first shake correction control unit performs the control different from each other depending on the at least two shake types.

[0008] According to another aspect of the present invention, there is provided an electronic device capable of communicating with another device, the electronic device including: an acquisition unit that acquires first information related to a shake occurring in at least one of the other device and the electronic device; a calculation unit that outputs second information by inputting the first information into a machine learning model; and a communication unit that transmits the second information corresponding to the first information before an instruction to capture a still image is given to the other device; the calculation unit outputs the second information based on time series data of a first length as the first information, and the second information based on time series data of a second length longer than the first length as the first information; The other device uses the second information transmitted by the communication unit to control shake correction after an instruction to shoot the still image is given, and the second information is information on a type of shake selected from a plurality of shake types including at least two of a walking state, a tripod state, a panning state, and a holding state, and the other device performs the control different from each other depending on the at least two shake types.

[0009] A control method according to another aspect of the present invention includes a first step of acquiring first information related to shake, a second step of inputting the first information into a machine learning model to output second information, and a third step of controlling shake correction using the second information, In the second step, the second information based on time series data of a first length as the first information and the second information based on time series data of a second length longer than the first length as the first information are output;In the third step, the second information output in the second step based on the first information before the instruction to shoot a still image is used to control shake correction after the instruction to shoot a still image is given, and the second information is information on a type of shake selected from a plurality of shake types including at least two of a walking state, a tripod state, a panning state, and a holding state, and the third step is characterized in that the control different from each other is performed depending on the at least two shake types. Effect of the Invention

[0010] According to the present invention, it is possible to provide an electronic device that enables highly accurate determination processing of the shooting state. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing an imaging system. [Diagram 2] 2 is a block diagram showing functions of an MPU included in the imaging device. FIG. [Diagram 3] 4 is a flowchart showing a photographing operation in the first embodiment. [Figure 4] FIG. 1 is a conceptual diagram showing a machine learning model. [Diagram 5] 11A and 11B are diagrams illustrating the accuracy of determining the type of shake. [Figure 6] 10 is a flowchart showing a process for determining a type of shake. [Figure 7] 13 is a flowchart showing a shake correction process. [Figure 8] 13 is a flowchart showing a photographing operation in the second embodiment. [Figure 9] 13 is a flowchart showing a photographing operation in the third embodiment. [Figure 10] 13 is a flowchart showing a photographing operation in the fourth embodiment. [Figure 11] FIG. 2 is a block diagram showing functions of an MPU included in the lens device. [Figure 12]13 is a flowchart showing a photographing operation in the fifth embodiment. [Figure 13] FIG. 1 illustrates a smartphone. [Figure 14] FIG. 1 is a block diagram of an imaging unit and an MPU included in a smartphone. [Figure 15] FIG. 2 is a block diagram showing functions of an MPU included in a smartphone. [Figure 16] FIG. 13 is a diagram showing a display unit of a smartphone when a photography application is started. [Figure 17] 20 is a flowchart showing a photographing operation in the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0013] [Example 1] 1 is a block diagram of an imaging system including an imaging device 200 and a lens device 100 capable of communicating with the imaging device 200. In this embodiment, the imaging device 200 has a function as the electronic device of the present invention.

[0014] The imaging device 200 has an imaging element 201, a signal processing circuit 202, a recording processing unit 203, a display unit 204, an operation unit 205, a camera MPU 206, and a camera communication unit 207. The imaging device 200 also has a camera angular velocity sensor 208, a camera acceleration sensor 209, an imaging element drive motor 210, a camera shake correction control unit 211, and an imaging element encoder 212.

[0015] The image sensor 201 photoelectrically converts an object image (optical image) formed by the imaging optical system in the lens device 100 and outputs an electrical signal (analog signal). The image sensor 201 is configured by, for example, a CMOS sensor or a CCD sensor. The analog signal output from the image sensor 201 is converted into a digital signal by an A / D conversion circuit (not shown).

[0016] The signal processing circuit 202 performs various image processing on the digital signal from the A / D conversion circuit (not shown) to generate a video signal. The signal processing circuit 202 also generates focus information indicating the focus state of the imaging optical system and luminance information indicating the exposure state. The signal processing circuit 202 also outputs the video signal to a display unit 204, which displays the video signal as a live view image used to check the composition, focus state, etc. The signal processing circuit 202 also outputs the video signal to a recording processing unit 203. The recording processing unit 203 stores the video signal as image data in an external memory or the like.

[0017] The operation unit 205 is composed of a mode dial for setting various imaging modes, a release button for instructing imaging preparation operations and the start of imaging, etc. When a first release signal (Sw1) is generated by the photographer pressing the release button halfway and input to the camera MPU 206, imaging preparation operations such as autofocus are performed in the lens device 100. When the photographer presses the release button all the way down, a second release signal (Sw2) is generated. A still image (image signal) is generated and recorded by inputting the second release signal to the camera MPU 206. In other words, pressing the release button all the way down corresponds to an instruction to shoot.

[0018] The camera MPU 206 controls the imaging device 200. The camera MPU 206 also communicates with the lens device 100 via a camera communication unit 207 to exchange data. In this embodiment, the camera MPU 206 is a microprocessing unit (MPU), but various processors or circuits such as a central processing unit (CPU) may also be used.

[0019] The camera angular velocity sensor 208 outputs an angular velocity signal related to the angular velocity of the imaging device 200. The camera acceleration sensor 209 outputs an acceleration signal related to the acceleration of the imaging device 200. The angular velocity signal output from the camera angular velocity sensor 208 and the acceleration signal output from the camera acceleration sensor 209 are input to the camera MPU 206 as a camera shake detection signal. The camera MPU 206 calculates a drive target signal for driving the imaging element 201 using the camera shake detection signal. A shake correction signal corresponding to the difference between this drive target signal and a position signal of the imaging element 201 output from the imaging element encoder 212 is output to the camera shake correction control unit 211. The camera shake correction control unit 211 drives the imaging element drive motor 210 based on the shake correction signal to move the imaging element 201 in a direction perpendicular to the optical axis of the imaging optical system in the lens device 100. This movement of the imaging element 201 suppresses image shake caused by the shake of the imaging system.

[0020] The imaging device 200 may have only one of the camera angular velocity sensor 208 and the camera acceleration sensor 209. However, when the imaging device 200 has both the camera angular velocity sensor 208 and the camera acceleration sensor 209, a larger amount of information can be obtained from the camera shake detection signal compared to when only one of them is included. As a result, a configuration having both the camera angular velocity sensor 208 and the camera acceleration sensor 209 is preferable because it enables more accurate shake correction and determination of the type of shake.

[0021] The lens device 100 includes an imaging optical system, drive motors for driving the imaging optical system, and control circuits for controlling the drive motors. The lens device 100 also includes a lens MPU 101, a lens communication unit 102, a lens angular velocity sensor 103, a lens acceleration sensor 104, and a shake correction lens encoder 105.

[0022] The imaging optical system includes a shake correction lens 106 and a focus lens 107. Moreover, the drive motors that drive the imaging optical system include a shake correction lens drive motor 108 and a focus lens drive motor 109. Moreover, each control circuit that controls each drive motor includes a lens shake correction control unit 110 and a focus lens control unit 111.

[0023] The lens MPU 101 controls the operation of each unit in the lens device 100. The lens MPU 101 also communicates with the imaging device 200 via a lens communication unit 102 to exchange data. In this embodiment, the lens MPU 101 is a microprocessing unit (MPU), but various processors or circuits such as a central processing unit (CPU) may also be used.

[0024] The lens angular velocity sensor 103 outputs an angular velocity signal related to the angular velocity of the lens device 100. The lens acceleration sensor 104 outputs an acceleration signal related to the acceleration of the lens device 100. The angular velocity signal output from the lens angular velocity sensor 103 and the acceleration signal output from the lens acceleration sensor 104 are input to the lens MPU 101 as a lens shake detection signal. The lens MPU 101 calculates a drive target signal for driving the shake correction lens 106 using the lens shake detection signal. A shake correction signal corresponding to the difference between this drive target signal and a position signal of the shake correction lens 106 output from the shake correction lens encoder 105 is output to the lens shake correction control unit 110. The lens shake correction control unit 110 drives the shake correction lens drive motor 108 based on the shake correction signal to move the shake correction lens 106, which is a part of the imaging optical system, in a direction perpendicular to the optical axis of the imaging optical system. This movement of the shake correction lens 106 suppresses image shake caused by the shake of the imaging system.

[0025] The focus lens control unit 111 adjusts the focus by moving the focus lens 107 in the optical axis direction using a control signal received from the lens MPU 101 and the focus lens drive motor 109. The focus lens control unit 111 includes a circuit for driving the focus lens 107 and a focus encoder that outputs a zone pattern signal or a pulse signal according to the movement of the focus lens 107. The subject distance can be detected based on the output of this focus encoder.

[0026] Instead of the camera shake detection signal, a lens shake detection signal output from at least one of the lens angular velocity sensor 103 and the lens acceleration sensor 104 may be input to the camera MPU 206. The lens shake detection signal is acquired by the camera MPU 206 from the lens MPU 101 via the camera communication unit 207. In that case, the imaging device 200 does not need to be provided with the camera angular velocity sensor 208 and the camera acceleration sensor 209.

[0027] Next, the function of the camera MPU 206 will be described with reference to Fig. 2. In this embodiment, the function of the camera MPU 206 determines the type of shake using information related to the camera shake detection signal. In addition, the type of shake is determined using a machine learning model in order to improve the accuracy of the shake type determination.

[0028] The camera MPU 206 includes a camera acquisition unit 206a and a camera calculation unit 206b.

[0029] The camera acquisition unit 206a acquires first information related to shake. In this embodiment, as the first information, a feature amount of a pre-processed camera shake detection signal is acquired.

[0030] The camera calculation unit 206b inputs the first information acquired by the camera acquisition unit 206a to a machine learning model, which will be described later, and outputs information relating to the type of shake as second information.

[0031] However, in general, processing using a machine learning model requires a high computation load. Therefore, when a machine learning model is used to determine the type of shake, it may take longer than before for the camera calculation unit 206b to perform the process of determining the type of shake. In this case, there is a risk of a delay occurring between when the user issues a shooting instruction via the operation unit 205 during still image shooting and when shooting actually starts. This is because the camera shake correction control unit 211 needs to perform shake correction after waiting for the completion of the shake type determination process by the camera calculation unit 206b. Therefore, in this embodiment, information on the type of shake determined by the camera calculation unit 206b before the user issues a shooting instruction via the operation unit 205 is used for shake correction after the user issues a shooting instruction. This eliminates the need for the camera shake correction control unit 211 to wait for the completion of the shake type determination process by the camera calculation unit 206b after the shooting instruction is issued. As a result, it is possible to reduce delays during still image shooting while enabling highly accurate shake type determination using a machine learning model even during still image shooting.

[0032] Next, the shooting operation of the first embodiment will be described with reference to the flowchart of Fig. 3. Fig. 3 starts when the power of the imaging device 200 is turned on. In this embodiment, the imaging device 200 performs shake correction and determines the type of shake. Therefore, the lens device 100 does not need to be provided with the shake correction lens encoder 105, the shake correction lens 106, the shake correction lens drive motor 108, and the lens shake correction control unit 110.

[0033] First, in step S101, the imaging device 200 performs initial communication with the lens apparatus 100 to identify the model of the lens apparatus 100. At this time, the lens apparatus 100 identifies the model of the imaging device 200.

[0034] In step S102, the camera MPU 206 communicates status with the lens MPU 101 via the camera communication unit 207. The camera MPU 206 also transmits the camera status (aperture value, shutter speed, etc. set by the user) to the lens MPU 101. The camera MPU 206 also receives the lens status (current focal length, aperture status, focus lens drive status, etc.) from the lens MPU 101.

[0035] In step S103, the camera MPU 206 determines whether or not Sw1 has been input to the camera MPU 206 from the operation unit 205. If it is determined that Sw1 has been input, the process proceeds to step S104. On the other hand, if it is determined that Sw1 has not been input, the process returns to step S102.

[0036] In step S104, the camera calculation unit 206b starts determining the type of shake using a machine learning model. The camera calculation unit 206b records information on the determined type of shake in the camera MPU 206. The camera calculation unit 206b continues to determine the type of shake using machine learning until step S109. Details of step S104 will be described later.

[0037] In step S105, the camera MPU 206 performs distance measurement to focus on the subject, and calculates the drive amount of the focus lens 107. The camera MPU 206 transmits the calculated drive amount of the focus lens 107 to the focus lens control unit 111.

[0038] In step S106, the camera MPU 206 performs distance measurement again, and if it is determined that the distance is within the focal depth, the process proceeds to step S107. If the distance is outside the focal depth, the process returns to step S105.

[0039] In step S107, the camera MPU 206 determines whether or not Sw2 has been input to the camera MPU 206 from the operation unit 205. If it is determined that Sw2 has been input, the process proceeds to step S108. On the other hand, if it is determined that Sw2 has not been input, the process returns to step S105.

[0040] In step S108, camera shake correction control unit 211 performs shake correction. The shake correction is performed based on the shake type determination result determined by camera calculation unit 206b before Sw2 is input to camera MPU 206. Therefore, it is possible to shorten the time from when Sw2 is input to camera MPU 206 until shake correction by camera shake correction control unit 211 is started. Also, in step S108, driving of an aperture (not shown) included in lens device 100 and driving of a shutter (not shown) included in imaging device 200 are performed. Details of the process related to shake correction based on the shake type determination result in step S108 will be described later.

[0041] In step S109, the signal processing circuit 202 reads out a signal from the image sensor 201. The read out signal is converted into digital data and stored in an external memory (not shown) by the recording processing unit 203. The camera calculation unit 206b also stops the process of determining the type of shake.

[0042] Next, the machine learning model will be described. Fig. 4 is a conceptual diagram showing the relationship between the input and output of the machine learning model in this embodiment. The machine learning model used in this embodiment is an algorithm for outputting the class to which input data belongs, based on the input data.

[0043] The machine learning model has a learning phase and an estimation phase. In the learning phase, parameters in the machine learning model are updated to improve the accuracy of class determination for input data. A class is associated with a type of shake and is a predetermined number of categories defined in advance. In addition, data that combines input data and a correct answer label indicating the class of the input data is called teacher data. Learning is performed by the machine learning model using the teacher data as input to perform class determination, comparing the obtained determined class with the class indicated by the correct answer label, and updating the parameters in the machine learning model to minimize the difference (loss) between the two. In the learning phase, learning is performed a predetermined number of times. Or, learning is repeated until a termination condition, such as reaching a predetermined determination accuracy, is met. The camera calculation unit 206b can determine the type of shake from the input data with high accuracy by using the trained machine learning model obtained in this way in the estimation phase.

[0044] In this embodiment, a random forest is used as a machine learning model. A random forest is an algorithm that uses a feature amount as input data. Therefore, the input data as the first information in this embodiment is a feature amount of a camera shake detection signal that has been preprocessed. In addition, the algorithm of the machine learning model may be a logistic regression, a support vector machine, a naive Bayes, a neural network, a state space model, or the like. In addition, the input data as the first information may be appropriately changed depending on the algorithm. In that case, the input data as the first information may be a camera shake detection signal that has only been preprocessed, a camera shake detection signal that has not been preprocessed, or a feature amount of a camera shake detection signal that has not been preprocessed. In addition, information on a motion vector obtained from the image sensor 201 may be used as the first information instead of the camera shake detection signal.

[0045] For example, when an algorithm that does not require features such as a neural network is used as a machine learning model, the feature is not generated and the camera shake detection signal is used as the first information. In this case, pre-processing of the camera shake detection signal is not essential.

[0046] Next, the pre-processing and feature quantities of the camera shake detection signal in this embodiment will be described. Note that the contents of the pre-processing and feature quantities may be appropriately changed depending on the learning environment.

[0047] In the pre-processing, the camera shake detection signal is standardized, the unit of the camera shake detection signal is converted, and missing values ​​and outliers in the camera shake detection signal are removed.

[0048] Standardization is a process that converts the data so that its average is 0 and its variance is 1. Furthermore, unit conversion is a process that aligns the resolution of the angular velocity signal and acceleration signal, which are the camera shake detection signals. Standardization and unit conversion are processes that improve the accuracy of learning by aligning the scale of the camera shake detection signals.

[0049] A missing value is empty data that occurs due to a detection error of the camera angular velocity sensor 208 or the camera acceleration sensor 209. An outlier is data in the camera shake detection signal that has an extremely large difference in value from other data. Outliers and missing values ​​are factors that reduce the accuracy of learning. For this reason, the camera MPU 206 detects and removes outliers and missing values ​​from the shake detection signal to prevent them from being input to the machine learning model.

[0050] The feature quantities include the average value, maximum value, minimum value, sum of squares, variance, standard deviation, kurtosis, skewness, and the like of the camera shake detection signal.

[0051] The output data as the second information in this embodiment is a type of shake, which may be a walking state, a tripod state, a panning state, a holding state, or any other state. The class output from the machine learning model may be a walking state, a tripod state, a panning state, or the like, and the result may be used as the type of shake as it is. In addition, the classes to be defined may be large shake, vertical shake, horizontal shake, and minute shake, and the large shake may be associated with the walking state, the vertical shake and horizontal shake with the panning state, and the minute shake with the tripod state. Note that the type of shake to be determined is not limited to the above-mentioned five types, and shake types may be added or changed.

[0052] The learning phase of the machine learning model of this embodiment will be described in detail below. As input data during learning, photographic records for a walking state, a tripod state, a panning state, a holding state, and other states were used.

[0053] In the walking state, the subjects were photographed in four different directions: forward, backward, left, right, and so on. The duration of the photographing process differed depending on the photographer.

[0054] In the tripod condition, with the imaging device mounted on a tripod, we took pictures for 2 minutes and 30 seconds while changing the combination of three shooting angles (upward, horizontal, and downward) and two imaging device orientations (portrait and landscape).

[0055] In the panning state, shooting was performed for one minute while changing the combination of two directions (left and right), two orientations of the imaging device 200 (portrait and landscape), and two ways of holding the imaging device (handheld and tripod).

[0056] In the hold state, the photographer took two shots for two minutes in two different postures, standing and resting his elbows, and in different combinations of two shooting methods, live view shooting and shooting while looking through the viewfinder. The hold state refers to the photographer holding the camera firmly, resulting in relatively little shake even though the shooting was handheld.

[0057] In the other states, the imaging device was placed on a vibration table (not shown) and images were taken for 2 minutes and 30 seconds while vibration was being generated. Note that the other states refer to states that do not fall under any of the walking state, tripod state, panning state, and holding state.

[0058] The input data is divided into training data and test data. The test data is used to verify the judgment accuracy of the trained machine learning model that was trained using the training data. In this case, data from the same subject is always assigned to only one of the two. This is to prevent the judgment accuracy from being unreasonably high, since data from the same subject is likely to be correct.

[0059] Next, the accuracy of shake type determination using the machine learning model of this embodiment will be described with reference to Fig. 5. The accuracy of shake type determination using the machine learning model was 100% in walking state, 49% in tripod state, 88% in panning state, 99% in holding state, and 100% in other states. The accuracy of shake type determination showed a very high accuracy rate, especially for walking state, panning state, holding state, and other states.

[0060] Next, a method of determining the type of shake using a machine learning model in step S104 in FIG. 3 will be described with reference to the flowchart in FIG.

[0061] In step S 201 , the camera MPU 206 acquires a camera shake detection signal from the camera angular velocity sensor 208 and the camera acceleration sensor 209 .

[0062] In step S202, the camera MPU 206 reads the sensor offset from a non-volatile memory (not shown), and subtracts the offset from the camera shake detection signal.

[0063] In step S203, the camera MPU 206 sets information on the camera shake detection signal from which the offset has been removed so that it is input to the machine learning model as time-series data of a predetermined length. An appropriate length of time-series data is preferably 0.1 seconds or more and 5.0 seconds or less. If the time-series data exceeds 5.0 seconds, it is not preferable because it takes too long to acquire data used to determine the type of shake. If the time-series data is less than 0.1 seconds, it is not preferable because sufficient accuracy cannot be obtained. Moreover, in order to determine the type of shake within the shooting time, a more preferable length of time-series data is 0.1 seconds or more and 0.4 seconds or less.

[0064] In step S204, the camera MPU 206 performs pre-processing on the camera shake detection signal.

[0065] In step S205, the camera MPU 206 generates a feature amount of the pre-processed camera shake detection signal. The camera acquisition unit 206a acquires the feature amount of the pre-processed camera shake detection signal.

[0066] In step S206, the camera calculation unit 206b inputs the feature amount of the preprocessed camera shake detection signal as first information to the machine learning model.

[0067] In step S207, the camera calculation unit 206b determines the type of shake as the second information, based on the class output by the machine learning model.

[0068] Next, a process for correcting shake according to the result of determining the type of shake by the machine learning model in step S108 in FIG. 3 will be described with reference to the flowchart in FIG.

[0069] In step S 401 , the camera MPU 206 acquires a camera shake detection signal from the camera angular velocity sensor 208 and the camera acceleration sensor 209 .

[0070] In step S402, the camera MPU 206 switches the processing of the camera shake detection signal depending on the result of the shake type determination. If the camera is walking, the process proceeds to step S403, if the camera is on a tripod, the process proceeds to step S404, if the camera is panning, the process proceeds to step S405, if the camera is holding, the process proceeds to step S406, and if the camera is in any other state, the process proceeds to step S407.

[0071] In step S403, the camera MPU 206 sets the cutoff frequency in the low-pass filter operation on the camera shake detection signal so that it is higher when the value of the camera shake detection signal is larger than in other states. This is because larger image blur occurs when walking compared to other states.

[0072] In step S404, the camera MPU 206 sets the cutoff frequency in the high-pass filter calculation for the camera shake detection signal to be higher than in other states. This is because low-frequency shake is unlikely to occur in the tripod state, and erroneous shake correction due to sensor offset is prevented.

[0073] In step S405, camera MPU 206 transmits a drive signal to camera shake correction control unit 211 for driving image sensor 201 to gradually return to the optical axis center position over a predetermined time. As a result, the movement of image sensor 201 for shake correction can be stopped. This is to prevent the photographer's intentional movement from being corrected as shake. Camera shake correction control unit 211 may also drive image sensor 201 so that shake correction in the panning direction is stopped and shake correction in directions other than the panning direction is performed.

[0074] In step S406, the camera MPU 206 detects a signal relating to low-frequency shake from the vector information obtained from the image sensor 201, and adds it to the position signal of the image sensor 201 output by the image sensor encoder 212. When the hold state is determined, the accuracy of correction of low-frequency shake is improved more than in other states.

[0075] In step S407, camera MPU 206 sets the cutoff frequency in the low-pass filter operation on the camera shake detection signal so that it is higher when the value of the camera shake detection signal is small than when the user is walking.

[0076] In step S408, the camera MPU 206 generates a shake correction signal. The camera MPU 206 calculates a drive target signal for driving the image sensor 201 using the camera shake detection signal that has been processed in any one of steps S403 to S407. A shake correction signal is generated according to the difference between this drive target signal and the position signal of the image sensor 201 output by the image sensor encoder 212.

[0077] In step S409, camera shake correction control unit 211 acquires a shake correction signal from camera MPU 206 and performs shake correction.

[0078] As described above, the photographing operation described in the first embodiment of the present invention makes it possible to determine the type of shake with high accuracy using a machine learning model and reduce delays when photographing a still image.

[0079] In this embodiment, the image capture device 200 performs shake correction and determines the type of shake. However, in this embodiment, the lens device 100 may perform shake correction and determine the type of shake. In that case, instead of the camera MPU 206, the lens MPU 101 is provided with a function for determining the type of shake. Moreover, instead of the camera shake correction control unit 211 for controlling shake correction, the lens device 100 uses the lens shake correction control unit 110. Moreover, instead of the image capture element 201 as a driven member that corrects shake by moving in the optical axis direction of the imaging optical system, the lens device 100 uses the shake correction lens 106. Moreover, instead of the image capture element drive motor 210 for driving the image capture element 201, the lens device 100 uses the shake correction lens drive motor 108 for driving the shake correction lens 106. Moreover, instead of the image capture element encoder 212 for outputting a position signal of the image capture element 201, the lens device 100 uses the shake correction lens encoder 105 for outputting a position signal of the shake correction lens 106.

[0080] In this embodiment, the type of shake may be determined by inputting the time series data of the first length and the time series data of the second length, which is longer than the first length, into the machine learning model. In this case, the type of shake based on the time series data of the first length and the type of shake based on the time series data of the second length are determined by the camera calculation unit 206b. Then, the camera shake correction control unit 211 performs shake correction using at least one of the type of shake based on the time series data of the first length and the type of shake based on the time series data of the second length.

[0081] Here, the determination of the type of shake based on the time series data of the first length is superior in that it takes a short time to obtain the data used for the determination, but the accuracy of the determination of the type of shake is relatively low. On the other hand, the determination of the type of shake based on the time series data of the second length is relatively accurate in the estimation of the type of shake, but it takes time to obtain the data used for the determination. Thus, there is a trade-off between the advantages of the determination of the type of shake based on the time series data of the first length and the advantages of the determination of the type of shake based on the time series data of the second length.

[0082] For this reason, it is preferable to determine the type of shake based on the time series data of the first length and the time series data of the second length. This allows the type of shake to be determined based on the time series data of the first length during the acquisition period of the time series data of the second length while enabling highly accurate determination of the type of shake. However, the type of shake based on the time series data of the first length may be used for shake correction even after the type of shake based on the time series data of the second length is output. This is preferable for instantaneously switching the type of shake used for shake correction when the type of shake changes. In addition, the first length is preferably 0.1 to 0.4 seconds, and the second length is preferably more than 0.4 seconds and 5.0 seconds. In addition, the time series data of the first length and the time series data of the second length may be input into different machine learning models to determine the type of shake. In this case, it is preferable to use a machine learning model that has been trained to be suitable for inputting the time series data of the first length and a machine learning model that has been trained to be suitable for inputting the time series data of the second length.

[0083] In this embodiment, optical shake correction is performed by driving the image sensor 201. However, in this embodiment, electronic shake correction may be performed instead of optical shake correction. In that case, the image sensor 200 does not need to have the image sensor drive motor 210, which is effective in saving space in the image sensor 200.

[0084] Electronic shake correction is performed using a camera shake detection signal, similar to optical shake correction. Here, instead of the camera shake detection signal, the camera MPU 206 may acquire a lens shake detection signal from the lens angular velocity sensor 103 and the lens acceleration sensor 104. The camera MPU 206 calculates the shake correction amount based on the camera shake detection signal. Then, the shake correction control unit 211 performs shake correction by cutting out a frame image based on the shake correction amount from the image displayed on the display unit 204 or the image recorded in an external memory (not shown).

[0085] In this embodiment, the shake correction may be performed using vector information indicating the moving direction and moving speed of the subject. The vector information is calculated from the image sensor 201. Then, the camera shake correction control unit 211 performs the shake correction by moving the image sensor 201 in a direction perpendicular to the optical axis of the imaging optical system in the lens device 100 based on the vector information.

[0086] [Example 2] Next, a second embodiment will be described.

[0087] In the first embodiment, the imaging device 200 determines the type of shake and performs shake correction. In contrast, in the second embodiment, the imaging device 200 determines the type of shake, and the lens device 100 performs shake correction. That is, the first embodiment differs in that the same electronic device performs shake type determination and shake correction, whereas the second embodiment performs these functions in different electronic devices. Therefore, the imaging device 200 of this embodiment does not need to be provided with the imaging element encoder 212, the imaging element drive motor 210, and the camera shake correction control unit 211.

[0088] Next, the photographing operation in the second embodiment will be described with reference to the flowchart in Fig. 8. Steps S501 to S504 in Fig. 8 are omitted because they perform the same processes as steps S101 to S104 in Fig. 3. Also, steps S506 to S508 in Fig. 8 are omitted because they perform the same processes as steps S105 to S107 in Fig. 3. Also, step S510 in Fig. 8 is omitted because it performs the same processes as step S109 in Fig. 3.

[0089] In step S505, the camera MPU 206 transmits information relating to the type of shake determined by the camera calculation unit 206b to the lens MPU 101. At this time, the determination result of the type of shake is recorded in the lens MPU 101. In addition, in step S510, the transmission of information relating to the type of shake from the camera MPU 206 to the lens MPU 101 is continuously performed up to step S510.

[0090] In step S509, the camera MPU 206 drives a shutter (not shown) included in the imaging device 200. At this time, an aperture (not shown) included in the lens device 100 is driven. In step S509, the lens shake correction control unit 110 also performs shake correction. The shake correction is performed based on the result of the shake type determined by the camera calculation unit 206b before Sw2 is input to the camera MPU 206.

[0091] As described above, according to the shooting operation in which shake correction and shake type determination are performed by different electronic devices as described in the second embodiment of the present invention, it is possible to determine the type of shake with high accuracy using a machine learning model and reduce delays when shooting a still image.

[0092] In this embodiment, the imaging device 200 determines the type of shake, and the lens device 100 performs shake correction. However, in this embodiment, the lens device 100 may determine the type of shake, and the imaging device 200 may perform shake correction. In that case, instead of the camera MPU 206, the lens MPU 101 is provided with a function for determining the type of shake. Also, instead of the lens shake correction control unit 110 for controlling shake correction, the imaging device 200 uses a camera shake correction control unit 211. Also, instead of the shake correction lens 106 that corrects shake by moving in the optical axis direction of the imaging optical system, the imaging device 200 uses an imaging element 201. Also, instead of the shake correction lens drive motor 108 for driving the shake correction lens, an imaging element drive motor 210 for driving the imaging element 201 is used. Also, instead of the shake correction lens encoder 105 for outputting a position signal of the shake correction lens 106, an imaging element encoder 212 for outputting a position signal of the imaging element 201 is used.

[0093] [Example 3] Next, a third embodiment will be described.

[0094] In the first embodiment, even after the user issues an instruction to shoot, a large computation load is allocated to the determination of the type of shake in the camera calculation unit 206b. In contrast, in the third embodiment, the camera calculation unit 206b stops determining the type of shake after the user issues an instruction to shoot a still image. Therefore, in this embodiment, the computation load after the user issues an instruction to shoot can be reduced. As a result, in this embodiment, the delay from when an instruction to shoot a still image is issued due to the determination of the type of shake until the image is actually shot can be reduced more than in the first embodiment.

[0095] In this embodiment, the imaging device 200 performs shake correction and determines the type of shake. Therefore, the lens device 100 does not need to be provided with the shake correction lens encoder 105, the shake correction lens 106, the shake correction lens drive motor 108, and the lens shake correction control unit 110.

[0096] Next, the shooting operation in the third embodiment of the present invention will be described with reference to the flowchart in Fig. 9. Steps S601 to S603 in Fig. 9 are omitted because they perform the same processes as steps S101 to S103 in Fig. 3. Also, steps S605 to S607 in Fig. 9 are omitted because they perform the same processes as steps S105 to S107 in Fig. 3. Also, steps S609 to S610 in Fig. 9 are omitted because they perform the same processes as steps S108 to S109 in Fig. 3.

[0097] In step S604, the camera calculation unit 206b starts determining the type of shake using a machine learning model. The camera calculation unit 206b records information on the determined type of shake in the camera MPU 206. The determination of the type of shake using machine learning in the camera calculation unit 206b is continuously performed until it is stopped in step S608.

[0098] In step S608, the camera calculation unit 206b stops determining the type of shake, thereby reducing the calculation load on the camera calculation unit 206b.

[0099] According to the photographing operation described in the third embodiment of the present invention as described above, by stopping the determination of the type of shake after a photographing instruction is issued, it is possible to reduce the delay during still image photographing more than in the first embodiment.

[0100] In this embodiment, the image capture device 200 performs shake correction and determines the type of shake. However, in this embodiment, the lens device 100 may perform shake correction and determine the type of shake. In that case, instead of the camera MPU 206, the lens MPU 101 is provided with a function for determining the type of shake. In step S607, the lens MPU 101 acquires information that a shooting instruction has been issued from the camera MPU 206 and stops determining the type of shake. In addition, instead of the camera shake correction control unit 211 for controlling shake correction, the lens device 100 uses the lens shake correction control unit 110. In addition, instead of the image capture element 201 as a driven member that corrects shake by moving in the optical axis direction of the image capture optical system, the lens device 100 uses the shake correction lens 106. In addition, instead of the image capture element drive motor 210 for driving the image capture element 201, the lens device 100 uses the shake correction lens drive motor 108 for driving the shake correction lens 106. Furthermore, instead of the image sensor encoder 212 for outputting a position signal of the image sensor 201 , the lens device 100 uses a shake correction lens encoder 105 for outputting a position signal of the shake correction lens 106 .

[0101] This embodiment may be applied to a configuration in which the determination of the type of shake and the shake correction as in the second embodiment are performed by different electronic devices.

[0102] [Example 4] Next, a fourth embodiment will be described.

[0103] In the fourth embodiment, the camera MPU 206 determines the type of shake, and the image capture device 200 and the lens device 100 perform shake correction according to the determination result. In this embodiment, by using a correction ratio between the shake correction performed by the lens device 100 and the shake correction performed by the image capture device 200, it is possible to perform shake correction with higher accuracy than in the first embodiment.

[0104] Next, the shooting operation in the fourth embodiment will be described with reference to the flowchart in Fig. 10. Steps S701 to S704 in Fig. 10 are omitted because they perform the same processes as steps S101 to S104 in Fig. 3. Also, steps S706 to S708 in Fig. 10 are omitted because they perform the same processes as steps S105 to S107 in Fig. 3. Also, step S711 in Fig. 10 is omitted because it performs the same processes as step S109 in Fig. 3.

[0105] In step S705, the camera MPU 206 transmits information about the determined type of shake to the lens MPU 101.

[0106] In step S709, the camera MPU 206 acquires the correction ratio of the shake correction by the camera shake correction control unit 211 from the lens MPU 101. The correction ratio is the ratio between the shake correction amount by the camera shake correction control unit 211 and the shake correction amount by the lens shake correction control unit 110. The correction ratio is calculated by the lens MPU 101.

[0107] In step S710, camera shake correction control unit 211 performs shake correction. At this time, shake correction is also performed by lens shake correction control unit 110. The shake correction by camera shake correction control unit 211 and lens shake correction control unit 110 is performed based on the correction ratio and the determination result of the type of shake determined before Sw2 is input to camera MPU 206. Also, in step S710, driving of an aperture (not shown) included in lens device 100 and driving of a shutter (not shown) included in imaging device 200 are performed.

[0108] As described above, the shooting operation in which shake correction is performed using multiple shake correction control units described in the fourth embodiment of the present invention makes it possible to determine the type of shake with high accuracy using a machine learning model and reduce delays when shooting a still image.

[0109] In this embodiment, the determination of the type of shake and the calculation of the correction ratio are performed by different electronic devices, but they may be performed by the same electronic device. However, when performed by different electronic devices, the determination of the type of shake and the calculation of the correction ratio can be shared between the imaging device 200 and the lens device 100, enabling efficient processing. Therefore, it is preferable to configure the determination of the type of shake and the calculation of the correction ratio to be performed by different electronic devices.

[0110] In this embodiment, the type of shake is determined by the imaging device 200. However, in this embodiment, the type of shake may be determined by the lens device 100. In that case, instead of the camera MPU 206, the lens MPU 101 is provided with a function for determining the type of shake.

[0111] Furthermore, in this embodiment, the correction ratio is calculated by the lens device 100. However, in this embodiment, the correction ratio may be calculated by the imaging device 200. In that case, in step S709, the camera MPU 206 transmits the correction ratio of the shake correction by the lens shake correction control unit 110 to the lens MPU 101.

[0112] In this embodiment, the camera calculation unit 206b may stop determining the type of shake after a shooting instruction is issued, similarly to the third embodiment.

[0113] [Example 5] Next, a fifth embodiment will be described.

[0114] In the fifth embodiment, the imaging device 200 and the lens device 100 determine the type of shake, and the imaging device 200 performs shake correction. If the respective determination results of the imaging device 200 and the lens device 100 match, the result is highly reliable, and if they do not match, the result can be determined to be unreliable. In this embodiment, if the determination result of the shake type of the imaging device 200 (second information) and the determination result of the shake type of the lens device 100 (third information) match, shake correction is performed according to the determination result, and if they do not match, shake correction is performed according to other conditions. As a result, shake correction according to the shake type can be performed only when the reliability of the determination result of the shake type is high. In addition, deterioration of correction accuracy caused by shake correction being performed according to an erroneous determination result of the shake type can be reduced.

[0115] In this embodiment, the imaging device 200 performs shake correction. Therefore, the lens device 100 does not need to include the shake correction lens encoder 105, the shake correction lens 106, the shake correction lens drive motor 108, and the lens shake correction control unit 110.

[0116] 11 is a block diagram showing the functions of the lens MPU 101. The type of shake in the lens apparatus 100 in this embodiment is determined using the functions shown in FIG.

[0117] The lens MPU 101 includes a lens acquisition unit 101a and a lens calculation unit 101b. The lens acquisition unit 101a has the same function as the camera acquisition unit 206a. The lens calculation unit 101b has the same function as the camera calculation unit 206b.

[0118] Next, the shooting operation in the fifth embodiment will be described with reference to the flowchart in Fig. 12. Steps S801 to S803 in Fig. 12 are omitted because they perform the same processes as steps S101 to S103 in Fig. 3. Steps S805 to S807 in Fig. 12 are omitted because they perform the same processes as steps S105 to S107 in Fig. 3. Steps S811 to S812 in Fig. 12 are omitted because they perform the same processes as steps S108 to S109 in Fig. 3.

[0119] In step S804, the camera calculation unit 206b starts determining the type of shake using a machine learning model. The camera calculation unit 206b records information on the determined type of shake in the camera MPU 206. At this time, the lens calculation unit 101b also starts determining the type of shake in the same manner as the camera calculation unit 206b. In addition, information on the type of shake determined by the lens calculation unit 101b is recorded in the lens MPU 101. The determination of the type of shake using the machine learning model of the camera calculation unit 206b and the lens calculation unit 101b is continuously performed until step S811.

[0120] In step S808, the camera MPU 206 acquires the shake type determination result from the lens MPU 101.

[0121] In step S809, the camera MPU 206 compares the shake type determination result performed by the camera calculation unit 206b with the shake type determination result performed by the lens calculation unit 101b. If the shake type determination results are not the same, the process proceeds to step S810. On the other hand, if the shake type determination results are the same, the process proceeds to step S811.

[0122] In step S810, the camera shake correction control unit 211 performs shake correction according to other conditions. Therefore, it is possible to prevent shake correction from being performed according to a judgment result of a shake type with low reliability. In addition, it is possible to reduce deterioration in the accuracy of shake correction based on an erroneous judgment result of a shake type.

[0123] According to the photographing operation described above in the fifth embodiment of the present invention, shake correction can be performed according to the type of shake based on the reliability of the shake type determination result.

[0124] In this embodiment, the image pickup device 200 performs the shake correction. However, in this embodiment, the lens device 100 may perform the shake correction. In that case, instead of the camera shake correction control unit 211 for controlling the shake correction, the lens device 100 uses the lens shake correction control unit 110. Also, instead of the image pickup element 201 as a driven member that corrects shake by moving in the optical axis direction of the image pickup optical system, the lens device 100 uses the shake correction lens 106. Also, instead of the image pickup element drive motor 210 for driving the image pickup element 201, the lens device 100 uses the shake correction lens drive motor 108 for driving the shake correction lens 106. Also, instead of the image pickup element encoder 212 for outputting a position signal of the image pickup element 201, the lens device 100 uses the shake correction lens encoder 105 for outputting a position signal of the shake correction lens 106.

[0125] In this embodiment, the camera calculation unit 206b may stop determining the type of shake after a shooting instruction is issued in the same manner as in embodiment 3. At this time, the lens calculation unit 101b may also stop determining the type of shake.

[0126] In this embodiment, the shake correction may be performed using both the camera shake correction control unit 211 and the lens shake correction control unit 110 as in the fourth embodiment.

[0127] Furthermore, in this embodiment, the camera MPU 206 compares the shake type determination results of the camera calculation unit 206b and the lens calculation unit 101b. However, the lens MPU 101 may compare the shake type determination results of the camera calculation unit 206b and the lens calculation unit 101b instead of the camera MPU 206. In that case, in step S808, the camera MPU 206 transmits the shake type determination result to the lens MPU 101.

[0128] [Example 6] Next, a smartphone according to a sixth embodiment will be described.

[0129] The smartphone of Example 6 is different from the imaging device 200 of Example 1 in that it has a shooting function but does not have a release button. FIG. 13 is a diagram showing a smartphone 400. FIG. 13(a) shows the front side of the smartphone 400. Also, FIG. 13(b) shows the back side of the smartphone 400. Also, the smartphone 400 does not include a release button like the operation unit 205 of FIG. 1. Therefore, in Example 6, there is a difference in the processing caused by Sw1 and Sw2 generated by the operation unit 205 in Example 1.

[0130] The smartphone 400 includes a display unit 401 , an operation unit 402 , and an imaging unit 403 .

[0131] The display unit 401 is configured with a liquid crystal panel or the like, and has a touch sensor. That is, the user can operate the smartphone 400 by touching the display unit 401 in addition to the operation unit 402.

[0132] The imaging unit 403 is used to capture an image of a subject.

[0133] FIG. 14 is a block diagram showing the image capturing unit 403 and the MPU 404.

[0134] The imaging unit 403 has an imaging element 501, a focus lens 502, an imaging element drive motor 503, a focus lens drive motor 504, a shake correction control unit 505, and a focus lens control unit 506. The imaging unit 403 also has an angular velocity sensor 507, an imaging element encoder 508, a signal processing circuit 509, and a recording processing unit 510.

[0135] The MPU 404 controls the smartphone 400 in response to inputs made from the display unit 401 and the operation unit 402 .

[0136] Fig. 15 is a block diagram showing the functions of the MPU 404. In this embodiment, the type of shaking of the smartphone 400 is determined using the functions shown in Fig. 15.

[0137] The MPU 404 includes an acquisition unit 404a and a calculation unit 404b. The acquisition unit 404a has the same function as the camera acquisition unit 206a. The calculation unit 404b has the same function as the camera calculation unit 206b.

[0138] FIG. 16 shows the state of the display unit 401 when a photography application is started in the smartphone 400.

[0139] When a photography application is started, a first area 601 for displaying an image of a subject and a second area 602 which is a virtual button for issuing a photography instruction are displayed on the display unit 401 of the smartphone 400.

[0140] In this embodiment, the result of the shake type determination performed by the calculation unit 404b before a shooting instruction is given by operating the second area 602 in shooting a still image is used for shake correction after the shooting instruction is given. As a result, similar to the first embodiment, it is possible to highly accurately determine the shake type using a machine learning model in still image shooting and reduce the delay in processing from the shooting instruction to the shooting itself.

[0141] Next, a sixth embodiment of the present invention will be described with reference to the flowchart of Fig. 17. Fig. 17 starts with the smartphone 400 starting up an application for photographing.

[0142] In step S901, the calculation unit 404b starts determining the type of shake using a machine learning model. The input data to the machine learning model uses the feature amount of the shake detection signal that has been preprocessed and acquired by the acquisition unit 404a. The calculation unit 404b records information on the determined type of shake in the MPU 404. The calculation unit 404b continues to determine the type of shake using machine learning until step S907.

[0143] In step S 902 , the MPU 404 measures the distance to bring the object into focus, and calculates the amount of drive of the focus lens 502 .

[0144] In step S903, the MPU 404 performs distance measurement again, and if it is determined that the focal depth is within the focal depth, the process proceeds to step S904. If the focal depth is outside the focal depth, the process returns to step S902 within a predetermined time or a predetermined number of times.

[0145] In step S904, the MPU 404 determines whether or not a shooting instruction has been issued by the user by touching the second area 602. If a shooting instruction has been issued, the process proceeds to step S905. On the other hand, if a shooting instruction has not been issued, the process returns to step S902.

[0146] In step S905, the shake correction control unit 505 performs shake correction. The shake correction is performed based on the shake type determined by the calculation unit 404b before a shooting instruction is given by the user's operation of touching the second area 602. Therefore, it is possible to shorten the time from when a shooting instruction is given until shake correction is started by the shake correction control unit 505.

[0147] In step S 906 , the signal processing circuit 509 reads out the signal from the image sensor 501 , and stores the captured image in the recording processing unit 510 .

[0148] As described above, the photographing operation of the smartphone described in the sixth embodiment of the present invention makes it possible to determine the type of shake with high accuracy using a machine learning model and reduce delays when photographing a still image.

[0149] In this embodiment, the shake correction of the smartphone 400 is performed by driving the image sensor 501. However, if the image capturing unit 403 includes a shake correction lens, the shake correction of the smartphone 400 may be performed by driving the shake correction lens. Moreover, the shake correction of the smartphone 400 may be performed electronically.

[0150] In this embodiment, the calculation unit 404b may stop determining the type of shake after a shooting instruction is issued to the camera MPU 206, similarly to the third embodiment.

[0151] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0152] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and variations are possible within the scope of the present invention. [Explanation of symbols]

[0153] 100 Lens device (electronic device) 110 Lens shake correction control unit (shake correction control unit) 200 Imaging devices (electronic devices) 211 Camera shake correction control unit (shake correction control unit) 101a Lens acquisition unit (acquisition unit) 101b Lens calculation unit (calculation unit) 206a Camera acquisition unit (acquisition unit) 206b Camera calculation unit (calculation unit) 400 Smartphones (electronic devices) 404a Acquisition Department 404b Arithmetic unit 505 Image stabilizer control unit

Claims

1. An acquisition unit that acquires first information regarding shake; A calculation unit that outputs second information by inputting the first information into a machine learning model; a first image stabilization control unit that controls image stabilization using the second information; the calculation unit outputs the second information based on time series data of a first length as the first information, and the second information based on time series data of a second length longer than the first length as the first information; the first shake correction control unit controls shake correction after an instruction to capture a still image is given, using the second information output from the calculation unit based on the first information before an instruction to capture a still image is given; the second information is information on a type of shaking selected from a plurality of types of shaking including at least two of a walking state, a tripod state, a panning state, and a hold state; The electronic device, wherein the first shake correction control unit performs different controls depending on the at least two types of shake.

2. The electronic device according to claim 1 , wherein the calculation unit stops the process using the machine learning model in response to the shooting instruction.

3. 3. The electronic device according to claim 1, wherein the second length is greater than 0.4 seconds and not greater than 5.0 seconds.

4. 4. The electronic device according to claim 3, wherein the first length is equal to or greater than 0.1 seconds and equal to or less than 0.4 seconds.

5. The electronic device according to any one of claims 1 to 4, characterized in that the first shake correction control unit performs the control using the second information based on the time series data of the first length before the second information based on the time series data of the second length is output from the calculation unit.

6. The sensor has at least one of an angular velocity sensor that outputs an angular velocity signal and an acceleration sensor that outputs an acceleration signal, 6. The electronic device according to claim 1, wherein the first information is data based on a shake detection signal related to at least one of the angular velocity signal and the acceleration signal.

7. 7. The electronic device according to claim 6, wherein the first information is data based on the vibration detection signal that has been subjected to at least one of standardization processing, removal of missing values, removal of outliers, and unit conversion processing.

8. 8. The electronic device according to claim 6, wherein the first information includes at least one of an average value, a maximum value, a minimum value, a sum of squares, a variance, a standard deviation, a kurtosis, and a skewness of the shake detection signal.

9. 9. The electronic device according to claim 6, wherein the first shake correction control unit performs the control by using a shake correction signal based on the shake detection signal and the second information.

10. Further comprising an imaging element, 10. The electronic device according to claim 1, wherein the first image blur correction control unit performs the control by driving the image sensor.

11. Further comprising a lens, 10. The electronic device according to claim 1, wherein the first image blur correction control unit performs the control by driving the lens.

12. The second image stabilization control unit is capable of communicating with another device that includes the second image stabilization control unit for performing the control. The electronic device described in any one of claims 1 to 11, characterized in that the first shake correction control unit obtains a correction ratio between the shake correction by the first shake correction control unit and the shake correction by the second shake correction control unit, and performs the control using the correction ratio and the second information.

13. 13. The electronic device according to claim 12, wherein the correction ratio is information calculated by the other device.

14. 12. The electronic device according to claim 1, wherein the first shake correction control unit performs the control based on the second information and third information relating to a type of shake acquired from another device.

15. 15 . The electronic device according to claim 14 , wherein the first image stabilization control unit performs the control using the second information when the second information and the third information match.

16. An electronic device capable of communicating with other devices, an acquisition unit that acquires first information related to a vibration occurring in at least one of the other device and the electronic device; A calculation unit that outputs second information by inputting the first information into a machine learning model; a communication unit that transmits the second information corresponding to the first information before a still image capture instruction is given to the other device, the calculation unit outputs the second information based on time series data of a first length as the first information, and the second information based on time series data of a second length longer than the first length as the first information; the other device uses the second information transmitted by the communication unit to control image stabilization after a shooting instruction for the still image is given; and the second information is information on a type of shaking selected from a plurality of types of shaking including at least two of a walking state, a tripod state, a panning state, and a hold state; The other device performs different controls depending on the at least two types of vibration.

17. A first step of obtaining first information regarding runout; a second step of inputting the first information into a machine learning model to output second information; and a third step of controlling shake correction using the second information, In the second step, the second information based on time series data of a first length as the first information and the second information based on time series data of a second length longer than the first length as the first information are output; In the third step, the second information output in the second step is used based on the first information before the instruction to shoot a still image is given, to control image stabilization after the instruction to shoot a still image is given; the second information is information on a type of shaking selected from a plurality of types of shaking including at least two of a walking state, a tripod state, a panning state, and a hold state; The control method is characterized in that in the third step, different controls are performed depending on the at least two types of vibration.

18. A program causing a computer to execute the control method according to claim 17.

Images