Information Processing Apparatus, Information Processing Method, and Program

Abstract

The present disclosure relates to an information processing apparatus, an information processing method, and a program for achieving simple and appropriate calibration. When a pen-type device including an inertial measurement unit (e.g., IMU) moves and displays an image corresponding to a drawing of a trajectory, calibration is performed based on the measured values of the inertial measurement device in one stationary posture and the latitude and absolute azimuth angle of the inertial measurement device when the pen-type device is inserted into a pen holder to be fixed in a predetermined direction. It can be applied to a device including an IMU.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program, and more particularly, to an information processing apparatus, an information processing method, and a program that can be appropriately calibrated with a simple operation. Background Art

[0002] There is a drawing technology for drawing an image on a display unit of a tablet using a stylus pen provided with a pressure sensor, wherein the stylus pen is brought into contact with a display surface of a tablet or the like to perform drawing.

[0003] In this case, the pressure sensor of the stylus pen needs to be calibrated at predetermined intervals.

[0004] Therefore, a technique for calibrating a pressure sensor by configuring such that, when a touch screen pen is placed upright on a predetermined pen holder, a predetermined force acts on a pressure sensor at the front end of the pen (see Patent Document 1) has been proposed.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2016-157322 Summary of the Invention

[0008] Problems to be solved by the present invention

[0009] Incidentally, in the above-mentioned drawing technology, drawing is achieved by contact between the stylus pen and the tablet computer, but, for example, it is considered to provide a gyro sensor instead of the pressure sensor in the stylus pen so that drawing can be performed according to the movement of the pen tip part in a non-contact state with the tablet computer.

[0010] In this case, it is conceivable to perform calibration using angular velocity when the pen is placed upright on a predetermined pen holder, but calibration cannot be performed appropriately using only angular velocity obtained in one type of stationary posture because the earth's automatic rotation component and deviation component are included.

[0011] In order to implement calibration using angular velocity, calibration cannot be properly performed unless angular velocities obtained in at least two different types of resting postures are used, and the user needs to position the pen so as to adopt at least two types of resting postures. As a result, operations related to calibration become cumbersome.

[0012] The present disclosure has been made in view of such circumstances, and in particular to achieve appropriate calibration through simple operations.

[0013] Solution to the problem

[0014] An information processing apparatus and program according to one aspect of the present disclosure are an information processing apparatus and program, the information processing apparatus including: a control unit that calibrates an inertial measurement unit based on a static attitude measurement value that is a measurement value detected by the inertial measurement unit and information related to the arrangement of the inertial measurement unit in a static attitude.

[0015] An information processing method according to one aspect of the present disclosure is an information processing method including: calibrating an inertial measurement unit based on a static attitude measurement value that is a measurement value detected by the inertial measurement unit and information related to the arrangement of the inertial measurement unit in a static attitude.

[0016] In one aspect of the present disclosure, in a static attitude, an inertial measurement unit is calibrated based on a static attitude measurement value that is a measurement value detected by the inertial measurement unit and information related to the arrangement of the inertial measurement unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a view illustrating a configuration example of a pen-type drawing system to which the technology of the present disclosure is applied.

[0018] Figure 2 is a view illustrating a detailed external configuration example of a pen and a pen case.

[0019] Figure 3 is a view illustrating a detailed configuration example of a pen.

[0020] Figure 4 is a view illustrating a detailed configuration example of a pen case.

[0021] Figure 5 is a view illustrating a configuration example of a recess in a lid of a pen case.

[0022] Figure 6 is a view illustrating an outline of operations of a pen-type drawing system.

[0023] Figure 7 is a view illustrating an outline of operations of a pen-type drawing system.

[0024] Figure 8 is a view illustrating an outline of operations of a pen-type drawing system.

[0025] Figure 9 is a flowchart illustrating an overall process of a pen-type drawing system.

[0026] Figure 10 is a diagram illustrating a hardware configuration example of a pen constituting a pen-type drawing system.

[0027] Figure 11 is a diagram illustrating functions implemented by a control unit of a pen.

[0028] Figure 12 is a diagram that describes the functions implemented by the inertial navigation processing unit in Figure 11 .

[0029] Figure 13 is a diagram that describes the relationship between the global coordinate system and the sensor coordinate system.

[0030] Figure 14 is a diagram that describes the state transition related to the operation of the pen plotter system.

[0031] Figure 15 is a flowchart that describes the pen control processing in Figure 11 .

[0032] Figure 16 is a flowchart that describes Figure 15 the state transition processing in

[0033] Figure 17 is a flowchart that describes Figure 15 the initial gravity estimation processing in

[0034] Figure 18 is a flowchart that describes Figure 15 the deviation estimation processing in

[0035] Figure 19 is a flowchart that describes the drawing processing shown in Figure 15 .

[0036] Figure 20 is a view that describes Modification Example 1 of the external configuration of the pen and the pen holder.

[0037] Figure 21 is a view that describes Modification Example 2 of the external configuration of the pen and the pen holder.

[0038] Figure 22 is a view that describes Modification Example 3 of the external configuration of the pen and the pen holder.

[0039] Figure 23 is a view that describes an application example of the pen plotter system.

[0040] Figure 24 is a view that describes the azimuth difference between the pen and the pen holder.

[0041] Figure 25 is a diagram that describes an example of updating the absolute azimuth angle after obtaining the absolute azimuth angle of the pen holder.

[0042] Figure 26 is a diagram that describes an example of the hardware configuration of the pen and the pen holder in an application example of the pen plotter system.

[0043] Figure 27 It is a diagram depicting the functions implemented by the control unit of the pen holder.

[0044] Figure 28 It is a flowchart depicting the collaborative processing between the pen and the pen holder.

[0045] Figure 29 It is a flowchart depicting a modified example of the collaborative processing between the pen and the pen holder.

[0046] Figure 30 It is a diagram showing a configuration example of a general personal computer. Detailed implementation

[0047] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in the description and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0048] Hereinafter, the modes for executing the present technology will be described. The description will be made in the following order.

[0049] 1. Preferred embodiments

[0050] 2. Application examples

[0051] 3. Examples executed by software

[0052] <<1. Preferred embodiments>>

[0053] <Pen-type drawing system>

[0054] Specifically, the present disclosure can achieve appropriate calibration through simple operations.

[0055] Figure 1 It is a view depicting the external configuration of a pen-type drawing system to which the technology of the present disclosure is applied.

[0056] Figure 1 The pen-type drawing system 11 in [[ ]] includes a pen 31, a pen holder 32, a personal computer (PC) 33, and a display device 34.

[0057] When the user 21 holds the pen 31 and the user 21 moves the tip of the pen to correspond to the desired shape to be drawn, the corresponding information of the trajectory of the tip of the pen is detected and transmitted to the PC 33 through communication such as Wi-Fi.

[0058] More specifically, the pen 31 includes an inertial measurement unit (IMU), which includes an acceleration sensor and a gyroscope sensor, obtains the trajectory of the position of the tip of the pen when moved by the user 21 through inertial navigation, and transmits the trajectory to the PC 33.

[0059] The PC 33 generates a drawing image that draws a trajectory corresponding to the movement of the tip of the pen 31 based on the trajectory information of the tip of the pen 31 transmitted from the pen 31, and causes a display device 34 including a liquid crystal display (LCD), organic electroluminescence (EL), etc. to display the drawn image.

[0060] The pen holder 32 is formed of a transparent acrylic plate or the like, is provided with a hole 32a having substantially the same shape as the tip of the pen 31, and the pen holder 32 is configured such that the tip of the pen 31 is inserted into the hole 32a to vertically hold the pen 31.

[0061] In addition, the pen holder 32 and the pen 31 are configured such that their relative positional relationship does not change, and the pen 31 is configured such that when vertically placed on the pen holder 32, the absolute azimuth angle on the earth becomes a constant azimuth angle.

[0062] With such a configuration, the pen 31 can obtain an automatic rotation component by acquiring the absolute azimuth angle and latitude, so that calibration can be achieved by specifying a deviation component based on the angular velocity detected in a stationary posture of being vertically placed on the pen holder 32.

[0063] Therefore, in Figure 1 the pen-type drawing system 11, the user 21 takes out the pen 31 from the pen holder 32 and moves the tip to draw a desired shape, so that the display device 34 can display the desired shape as a drawn image.

[0064] At this time, the pen 31 can take a stationary posture by a simple operation of being vertically placed on the pen holder 32, and the combined IMU can be appropriately calibrated.

[0065] It should be noted that although Figure 1 an example in which a drawing image corresponding to the drawing operation of the pen 31 is displayed on the display device 34 is described, any other configuration can be used as long as it can perform display, and for example, a head-mounted display (HMD) or the like can be used.

[0066] <External configuration examples of the pen and the pen holder>

[0067] Figure 2 External configuration examples of the pen 31 and the pen holder 32 are shown.

[0068] The pen 31 includes a pressure-sensitive sensor 51 and a light-emitting unit 52.

[0069] The pressure-sensitive sensor 51 is a button that the user presses when trying to draw on the display device 34 by operating the button. By operating this button, the position of the tip changes while maintaining the state.

[0070] That is, when the pressure-sensitive sensor 51 is in a pressed state, the information on the trajectory of the tip of the inertial navigation detection pen 31 is detected and sent to the PC 33.

[0071] At this time, based on the information on the trajectory sent from the pen 31, the PC 33 causes the display device 34 to display the drawn image as the tip of the pen 31 moves.

[0072] When the pen 31 is placed upright on the pen holder 32 and calibration is completed, the light-emitting unit 52 emits light; when the time elapsed since the pen 31 is taken out of the pen holder 32 and a drawing operation is performed exceeds a predetermined time, it is turned off; and when the pen is placed upright on the pen holder 32 again and calibration is completed, the light-emitting unit 52 emits light.

[0073] Thereafter, the operation in which the user 21 takes out the pen 31 from the pen holder 32, presses the pressure-sensitive sensor 51, and the tip of the pen 31 moves to draw a shape desired by the user 21 is called a drawing operation.

[0074] Therefore, when performing a drawing operation, the pen 31 detects the information on the trajectory of the tip by inertial navigation and transmits the information to the PC 33, and the PC 33, based on the transmitted trajectory information, causes the display device 34 to display the drawn image corresponding to the trajectory drawn as the tip of the pen 31 moves.

[0075] In addition, as Figure 3 shown, the pen 31 includes: a circuit board 61 that constitutes various control units including the pressure-sensitive sensor 51 and the light-emitting unit 52; a battery 62 that provides driving power for the pen 31; and a multi-IMU 63 that includes IMUs 63a to 63d.

[0076] The pressure-sensitive sensor 51, the light-emitting unit 52, and the multi-IMU 63 are provided on the circuit board 61, and their operations are further controlled by a control unit 101 ( Figure 10 ) provided on the circuit board 61.

[0077] The multi-IMU 63 includes a plurality of IMUs 63a to 63d, detects position information based on the angular velocity and acceleration obtained by each of the IMUs 63a to 63d by inertial navigation, and outputs the position information to the control unit 101.

[0078] Note that for the purpose of improving detection accuracy, the multi-IMU 63 includes four IMUs 63a to 63d, but it can be any number as long as the number is two or more, and if a reduction in detection accuracy is allowed, it can be configured by one IMU instead of the multi-IMU 63.

[0079] As Figure 4As shown, the pen holder 32 is formed of transparent acrylic resin and includes an inner diameter portion 86, an outer peripheral portion 85, and a lid 84 that form a hole 32a having substantially the same shape as the tip of the pen 31.

[0080] As Figure 4 shown in the central portion of, the outer peripheral portion 85 is fixed by fitting the bottom to a stepped portion at the bottom of the inner diameter portion 86 and fitting the upper portion to the lid 84 via O-rings 82 and 83.

[0081] In addition, a hollow portion is formed between the inner diameter portion 86 and the outer peripheral portion 85, and the hollow portion is filled with a colored liquid 81 having light-transmitting transparency.

[0082] With this configuration, by inserting the tip of the pen 31 into the hole 32a, the pen can be vertically placed on the pen holder 32.

[0083] Furthermore, since the liquid 81 is in a state of being filled by the inner diameter portion 86, the outer peripheral portion 85, and the lid 84 formed of transparent acrylic resin, the light emitted by the light-emitting unit 52 at the tip of the pen 31 causes the liquid 81 to emit light, so that even when the pen 31 is vertically placed on the pen holder 32, the user can visually recognize the light-emitting state of the light-emitting unit 52.

[0084] In addition, as Figure 5 shown, a recess 84a having a shape corresponding to the pressure-sensitive sensor 51 having a convex shape is formed in the lid 84 that constitutes the outer shape of the hole 32a, and when the pen is vertically placed by inserting the tip into the hole 32a, the pen 31 cannot be vertically placed unless the pressure-sensitive sensor 51 having a convex shape and the recess 84a are in a position facing each other and mating.

[0085] With this configuration, the pen 31 is always vertically placed with respect to the pen holder 32 while maintaining a specific positional relationship.

[0086] Note that in the following description, it is assumed that the pen holder 32 does not move or rotate once it is erected. Thus, when the pen 31 is vertically placed on the pen holder 32, it is assumed that the pen 31 is always vertically placed at the same absolute azimuth angle.

[0087] <Overview of the operation of the pen-type drawing system>

[0088] Next, an overview of the operation of the pen-type drawing system 11 will be described with reference to Figures 6 to 8 description Figure 1 of.

[0089] First, for the initial calibration of the pen 31, the tip of the pen 31 is inserted into the hole 32a of the pen holder 32 and vertically placed as Figure 6 shown in the left and central portions of.

[0090] Note that, in the following description, unless otherwise specified, the case where the pen 31 is placed upright on the pen container 32 refers to the state in which the pen tip of the pen 31 is inserted into the hole 32a of the pen container 32 in a fitted state with the pressure-sensitive sensor 51 having a convex shape of the pen 31 and the concave portion 84a provided in the cap 84 facing each other as described above.

[0091] Here, the pen 31 calculates the automatic rotation component included in the angular velocity detected by the multi-IMU 63 by acquiring the position information (latitude) on the earth and the information on the absolute azimuth angle, obtains the deviation component based on the automatic rotation component, and performs calibration.

[0092] When the calibration is completed, as shown in the right part of Figure 6 , the pen 31 controls the light-emitting unit 52 at the pen tip to emit light to indicate the completion of the calibration. Along with this, in the pen container 32, since the inner diameter portion 86, the outer peripheral portion 85, and the cap 84 are formed of transparent acrylic resin and the hollow portion between the inner diameter portion 86 and the outer peripheral portion 85 is filled with the colored liquid 81, when the light-emitting portion 52 at the pen tip of the pen 31 emits light, it can be visually recognized that the entire pen container 32 emits light.

[0093] The user 21 visually recognizes that the light-emitting unit 52 at the pen tip of the pen 31 is emitting light through the pen container 32 to recognize the completion of the calibration.

[0094] Next, as shown in the left part of Figure 7 (the same as the right part of Figure 6 ), when the completion of the calibration is recognized via the pen container 32 by the color change of the pen tip of the pen 31, the user 21 takes out the pen 31 from the pen container 32 at the moment when the user wants to draw, as shown in the right part of Figure 7

[0095] Figure 7 Figure 8 As shown in the right part of , the user 21 takes out the pen 31, and as shown in the left part of

[0096] , while pressing the pressure-sensitive sensor 51 provided on the side surface portion of the pen 31, moves the pen tip to draw a trajectory of a desired shape.

[0096] Then, when the drawing button 31a is pressed, the pen 31 recognizes that the user 21 has the intention to draw, obtains the trajectory (displacement of the position information) as the pen tip of the pen 31 moves through inertial navigation, and sends the trajectory as the trajectory information of the pen tip of the pen 31 to the PC 33.

[0097] Based on the trajectory information of the displacement indicating the position of the tip of the pen 31 transmitted from the pen 31, for example, the PC 33 generates a drawing image as if the drawing image is drawn with the pen as the tip of the pen 31 moves, and causes the display device 33 to display the drawn image, as Figure 8 shown in the right part of

[0098] Note that as time elapses since the pen 31 is taken out of the pen container 32, an error accumulates in the position information of the tip obtained by the IMU. Therefore, when the elapsed time after taking out from the pen container 32 exceeds a predetermined time (regular 10 seconds), and an error greater than the predetermined error accumulates and calibration becomes necessary, the pen 31 turns off the light emitting unit 52 at the tip.

[0099] When the user closes the light emitting unit 52 at the tip of the pen 31 in this way and recognizes that calibration is required, the user inserts the pen 31 into the hole 32a of the pen container 32 to place the pen 31 upright, as Figure 6 shown in

[0100] By repeating this series of operations, while calibrating the error of the IMU of the pen 31, when the calibration is completed, the user 21 takes out the pen 31 from the pen container 32, moves the tip while pressing the pressure sensitive sensor 51 to trace the desired shape, and thus can have a drawing image for drawing the desired shape displayed on the display device 34.

[0101] In addition, during calibration, since the pen 31 can obtain the automatic rotation component by acquiring its own position information (latitude) on the earth and the information of the absolute attitude when standing upright on the pen container 32, the calibration can be achieved only by adopting a stationary attitude in which the pen 31 stands upright on the pen container 32 in order to obtain the deviation component.

[0102] As a result, when using the pen 31, the user can make the pen 31 take a stationary attitude required for calibration only by performing the operation of standing the pen upright on the pen container 32, so that easy and highly accurate calibration can be achieved.

[0103] <Overall Processing>

[0104] Next, the overall operation of the pen-type drawing system 11 Figure 9 will be described with reference to the flowchart of Figure 1

[0105] In step S11, the pen 31 determines whether the on / off button (not shown) has been operated and the power has been turned on.

[0106] If it is determined in step S11 that the power is not turned on, similar processing is repeated.

[0107] When it is assumed that the power is turned on in step S11, the process proceeds to step S12.

[0108] In step S12, the pen 31 determines whether it is in a state of being upright on the pen holder 32, that is, whether to perform drawing based on the presence or absence of its own movement.

[0109] When it is determined in step S12 that there is no movement of the pen 31 and the pen 31 is in a state of being upright on the pen holder 32, the process proceeds to step S13.

[0110] In step S13, the pen 31 performs a calibration process and calculates the automatic rotation component and deviation component required to correct the multi-IMU 63.

[0111] In step S14, the pen 31 causes the light emitting unit 52 to emit light to indicate that the calibration has been completed.

[0112] In step S15, it is determined whether the user 21 indicates to draw a trajectory corresponding to the movement of the tip of the pen 31 based on whether the pen 31 moves and is taken out of the pen holder 32 and the pressure-sensitive sensor 51 is pressed.

[0113] When it is determined in step S15 that there is no indication of drawing, the same process is repeated.

[0114] Then, when it is determined in step S15 that the pen moves and is taken out of the pen holder 32 and the pressure-sensitive sensor 51 is pressed and the user 21 has indicated to draw a trajectory corresponding to the movement of the tip of the pen 31, the process proceeds to step S16.

[0115] In step S16, the pen 31 corrects the attitude rotation matrix based on the automatic rotation component and deviation component, converts the acceleration in the sensor coordinate system into the acceleration in the global coordinate system in time series based on the corrected attitude rotation matrix, and adds the accelerations in sequence, thereby outputting the position information in the global coordinate system of the trajectory of the pen 31 as inertial navigation to the PC 33, and causing the display device 34 to draw the trajectory of the tip of the pen 31 according to the position change.

[0116] In step S17, the pen 31 determines whether a predetermined time (for example, about 10 seconds) has passed since the pen 31 was taken out of the pen holder 32 and an instruction for drawing has been given, the error caused by inertial navigation has been accumulated, and thus the reliability of the drawing position has become lower than the reliability in a predetermined state.

[0117] In the case where it is determined in step S17 that the predetermined time has not elapsed and the reliability of the drawing position is not lower than that in the predetermined state, that is, in the case where sufficient reliability is determined in the drawing, the process returns to step S15. That is, until the predetermined time has passed, the processes in steps S15 to S17 are repeated, and as long as the state where the pen 31 is taken out from the pen container 32 and presses the pressure-sensitive sensor 51 continues, a locus corresponding to the movement of the tip of the pen 31 is continuously drawn on the display device 34.

[0118] Then, in the case where it is assumed in step S17 that the predetermined time has passed and the reliability of the drawing position has become lower than the predetermined state, the process proceeds to step S18.

[0119] In step S18, the pen 31 turns off the light-emitting unit 52 to indicate that calibration is required.

[0120] In step S19, the pen 31 determines whether the power has been turned off and an end has been instructed, and in the case where the power has not been turned off and no end has been instructed, the process returns to step S12.

[0121] Note that in the case where it is determined in step S12 that the pen 31 is not placed upright in the pen container 32, the processes in steps S13 to S17 are skipped, and the process proceeds to step S18.

[0122] That is, through the process in step S18, it is shown that calibration is necessary by turning off the light-emitting unit 52, and the user returns the pen 31 to the pen container 32 to stand upright, so that the processes in steps S12 to S19 are repeated.

[0123] Then, in step S19, when it is assumed that the power is turned off and an end instruction is given, the process ends.

[0124] Through the above processing, when the pen 31 is placed upright on the pen container 32 and calibrated, the light-emitting unit 52 emits light to indicate that the calibration is completed. Then, within a predetermined time after the calibration is completed and drawing is instructed, in a state where a drawing operation is performed by pressing the pressure-sensitive sensor 51, a locus corresponding to the movement of the tip of the pen 31 is drawn on the display device 34.

[0125] When the predetermined time has passed, the light-emitting unit 52 is turned off to indicate that calibration must be performed, and along with this, the pen 31 is placed upright on the pen container 32 to perform calibration. Then, when the calibration is completed, the light-emitting unit 52 is turned on again, and similar processing is repeated, so that a locus corresponding to the movement of the tip of the pen 31 is displayed as a drawn image on the display device 34.

[0126] Since the pressure-sensitive sensor 51 having a protruding shape is placed upright in the hole 32a of the pen holder 32 to fit into the recess 84a, the pen 31 can be properly calibrated by a simple operation in a stationary posture.

[0127] <Example of the hardware configuration of a pen-type drawing system>

[0128] Next, with reference to Figure 10 , an example of the hardware configuration of a pen-type drawing system will be described.

[0129] The pen 31 includes a control unit 101, an acquisition unit 102, a communication unit 103, a multi-IMU 63, a light-emitting unit 52, and a pressure-sensitive sensor 51, and is electrically connected via a bus 111 such as a serial bus to enable mutual communication.

[0130] The control unit 101 includes a processor, a memory, etc., and controls the overall operation of the pen 31. [[ID=E16]]

[0131] In addition, by executing a predetermined application program, the control unit 101 performs calibration based on the position information (latitude) on the earth and the absolute azimuth angle acquired in advance by the acquisition unit 102 and the acceleration and angular velocity detected by the multi-IMU 63, implements the inertial navigation process described later, and calculates the trajectory of the tip of the pen 31.

[0132] In addition, the control unit 101 controls the light-emitting unit 52 at the end of the correction to emit light indicating the end of the correction, and turns off the light-emitting unit 52 when a predetermined time has elapsed since the end of the correction to indicate that correction is required.

[0133] In addition, when the pressure-sensitive sensor 51 is pressed and a drawing operation is indicated, the control unit 101 obtains position information indicating the trajectory of the tip of the pen 31 through the inertial navigation process, controls the communication unit 103 to transmit the position information to the PC 33, and causes the display device 34 to display the position information.

[0134] The acquisition unit 102 receives and acquires the operation input of the user using a keyboard, operation buttons, etc. (not shown) or the input of the absolute azimuth angle and latitude on the earth provided from GNSS, etc., and provides the input to the control unit 101.

[0135] Note that the absolute azimuth angle and latitude on the earth acquired by the acquisition unit 102 are used as substantially fixed values once acquired, so the control unit 101 stores and uses the information of the absolute azimuth angle and latitude acquired by the acquisition unit 102.

[0136] The communication unit 103 is controlled by the control unit 101, communicates with the PC 33 through, for example, Wi-Fi, etc., and transmits and receives various data and programs.

[0137] <Function implemented by the control unit of the pen in Figure 10 >

[0138] Next, with reference to Figure 11 will be described the function implemented by the control unit 101 of the pen 31 in Figure 10 .

[0139] The control unit 101 includes an inertial navigation processing unit 131, a light emission control unit 132, a pressure-sensitive sensor processing unit 133, and an external interface (I / F) 134.

[0140] In addition, the light emission control unit 132 is controlled by the inertial navigation processing unit 131. When the calibration is completed, it provides a control signal to the control IC 141 that controls the light emission unit 52, causing the LED 142 to emit light, indicating that the calibration is completed.

[0141] Furthermore, when calibration is required, the light emission control unit 132 provides a control signal for instructing the control IC 141 that controls the light emission unit 52 to turn off the LED 142, indicating that calibration is required.

[0142] The pressure-sensitive sensor processing unit 133 determines whether it is pressed based on the pressure value provided by the pressure-sensitive sensor 51 to indicate a drawing operation, and when the pressure value is higher than a predetermined value, it outputs a signal indicating the drawing operation to the inertial navigation processing unit 131 via the external I / F 134.

[0143] When the pen 31 is placed upright on the pen holder 32 and in a stationary posture state, the inertial navigation processing unit 131 obtains the initial gravity direction, the auto-rotation component, and the deviation component based on the acceleration and angular velocity detected by the multi-IMU 63 and the absolute azimuth angle and latitude information obtained by the acquisition unit 102, thereby achieving calibration.

[0144] The inertial navigation processing unit 131 corrects the angular velocity obtained by the multi-IMU 63 through the auto-rotation component and the deviation component to update the attitude rotation matrix, thereby achieving calibration. By integrating the acceleration obtained by the multi-IMU 63 through inertial navigation, it obtains the time-series position information representing the tip trajectory of the pen 31.

[0145] The inertial navigation processing unit 131 transmits the position information, which is the time-series position information indicating the trajectory of the tip of the pen 31, from the communication unit 103 to the PC 33 via the external I / F 134, and causes the display device 34 to display the corresponding drawn image.

[0146] < Figure 11 Function implemented by the inertial navigation processing unit in

[0147] Next, with reference to Figure 12Describe a configuration example for implementing the function of the inertial navigation processing unit 131 in Figure 11 The inertial navigation processing unit 131 has a position and attitude processing unit 151, an initial attitude estimation unit 152, and a stationary determination unit 153.

[0148] The stationary determination unit 153 determines whether the pen 31 is in a stationary state based on the information of acceleration and angular velocity provided by the multi-IMU 63, that is, whether the pen 31 is in a stationary state of being uprightly placed on the pen holder 32, and outputs the determination result to the initial attitude estimation unit 152 and the position and attitude processing unit 151.

[0149] When it is determined that the pen 31 is in a stationary state, the position and attitude processing unit 151 and the initial attitude estimation unit 152 assume that it is in a state to perform calibration (initial attitude estimation state), and when the determination result is a non-stationary state, assume that it is in a drawing state and switch the operation.

[0150] In the initial attitude estimation state, the position and attitude processing unit 151 estimates the automatic rotation component based on the information of the initial gravity direction, absolute azimuth angle, and latitude provided by the initial attitude estimation unit 152, and provides the estimation result to the initial attitude estimation unit 152.

[0151] The initial attitude estimation unit 152 detects the information of the roll and pitch of the pen 31 as the information of the gravity direction based on the acceleration information provided by the multi-IMU 63, and provides this information to the position and attitude processing unit 151.

[0152] In addition, the initial attitude estimation unit 152 estimates the deviation component based on the automatic rotation component provided by the position and attitude processing unit 151, and provides the deviation component as the estimation result to the position and attitude processing unit 151.

[0153] When in the drawing state, the position and attitude processing unit 151 corrects the attitude rotation matrix based on the automatic rotation component and the deviation component, and obtains the absolute attitude of the pen 31 based on the corrected attitude rotation matrix and the detection result of the multi-IMU 63.

[0154] In addition, the position and attitude processing unit 151 integrates the absolute attitude of the pen 31 to obtain the position and velocity indicating the trajectory of the tip of the pen 31, and outputs the position and velocity.

[0155] More specifically, the position and attitude processing unit 151 includes an attitude estimation unit 171, an automatic rotation estimation unit 172, and an acceleration integration unit 173.

[0156]

[0157] ​The attitude estimation unit 171 supplies the information of the absolute azimuth and latitude acquired by the acquisition unit 102 to the automatic rotation estimation unit 172 to estimate (calculate) the automatic rotation component.

[0158] <Estimation of Autorotation Component>

[0159] Here, a method of estimating (calculating) the auto-rotation component by the auto-rotation estimating unit 172 will be described.

[0160] For example, Figure 13 As shown, when a global coordinate system (xg, yg, zg) which is a coordinate system representing a position on the earth with latitude lat and a sensor coordinate system (xs, ys, zs) which is a coordinate system of the multi-IMU 63 are defined, the spin component represented in the global coordinate system is expressed by the following expression (1).

[0161] [Expression 1]

[0162]

[0163] Here, ω er is the Earth's rotational velocity (angular velocity) and is a fixed value of 15deg / h, lat is the latitude on Earth, and ω er_g_x 、ω er_g_y and ω er_g_z Respectively expressed in Figure 13 In the global coordinate system shown in g Axis direction, y g Axis direction and z g Angular velocity component along the axis.

[0164] That is, as shown in expression (1), y g Component ω in the axial direction er_g_y is zero, x g Component ω in the axial direction er_g_x Yes er cos(lat), and z g Component ω in the axial direction er_g_z Yes er sin(lat).

[0165] When the autorotation component expressed in such a global coordinate system is converted into the sensor coordinate system, it is expressed by the following expression (2).

[0166]

[0167] Here, ω er_s_x 、ω er_s_y , and ω er_s_z respectively in Figure 13 In the sensor coordinate system shown ins the x-axis direction, y s the y-axis direction, and z s the angular velocity components in the z-axis direction. In addition, f(azimuth, lat) is a function represented by the absolute azimuth angle and the latitude (lat), and indicates that the angular velocity components of the sensor coordinate system are represented by multiplying the automatic rotation components of the global coordinate system by the inverse matrix M s→g of the attitude rotation matrix M g→s and the attitude rotation matrix M s→g is used to convert the global coordinate system into the sensor coordinate system.

[0168] This attitude rotation matrix M s→g (and the inverse matrix M g→s ) is uniquely set based on the gravity direction and the absolute azimuth angle.

[0169] Therefore, the automatic rotation estimation unit 172 sets the inverse matrix M according to the information on the gravity direction of the global coordinate system provided by the initial gravity estimation unit 181 of the initial attitude estimation unit 152 and the absolute azimuth angle g→s , and calculates the above expression (2) based on the angular velocity of the sensor coordinate system detected by the multi-IMU 63 to estimate (calculate) the automatic rotation components of the sensor coordinate system.

[0170] The attitude estimation unit 171 provides the automatic rotation components estimated (calculated) by the automatic rotation estimation unit 172 to the bias estimation unit 182 of the initial attitude estimation unit 152.

[0171] The bias estimation unit 182 estimates the bias components by subtracting the automatic rotation components from the average value of the angular velocities detected by the multi-IMU 63 within a predetermined time, and provides the bias components to the attitude estimation unit 171 of the position and attitude processing unit 151.

[0172] The attitude estimation unit 171 corrects the detection results of the multi-IMU 63 by subtracting the automatic rotation components and the bias components from the angular velocities detected by the multi-IMU 63, and updates the attitude rotation matrix M by using the attitude calculation of the corrected angular velocities s→g .

[0173] Using the updated attitude rotation matrix M s→g , the attitude estimation unit 171 converts the acceleration in the sensor coordinate system, which is the detection result of the multi-IMU 63, into the acceleration in the global coordinate system, and obtains the acceleration as the absolute attitude.

[0174] The acceleration integration unit 173 integrates the acceleration that is the absolute attitude estimated by the attitude estimation unit 171, updates the velocity and position, and outputs the updated velocity and position together with the movement of the tip of the pen 31 as the position information of the trajectory.

[0175] When the information of the trajectory of the movement of the tip of the pen 31 obtained in this way is sent to the PC 33, the PC 33 generates a drawn image based on the trajectory and causes the display device 34 to display the drawn image.

[0176] <State transition>

[0177] Next, the state transition related to the operation of the pen-type drawing system 11 will be described with reference to Figure 14 the following.

[0178] As Figure 14 shown, there are five types of states related to the operation of the pen-type drawing system 11, and the states transition under predetermined conditions.

[0179] Specifically, there are five states, including: initial state St0, initial attitude estimation state St1, attitude determination state St2, drawing state St3, and re-attitude estimation state St4.

[0180] The initial state St0 is the state immediately after the power is turned on or the state in which a predetermined time has elapsed since the start of the drawing operation in the drawing state, and the detection accuracy of the multi-IMU 63 has become lower than the predetermined state and calibration is necessary.

[0181] The initial attitude estimation state St1 is the state in which the pen 31 is placed upright on the pen holder 32.

[0182] The attitude determination state St2 is the state in which the initial gravity direction and deviation components are obtained and calibration is completed.

[0183] The drawing state St3 is the state in which the user 21 takes out the pen 31 from the pen holder 32 and turns on the pen 31 by pressing the pressure-sensitive sensor 51, and drawing is indicated.

[0184] The re-attitude estimation state St4 is the drawing state St3, which is the state in which no drawing has been performed from the indication of drawing until a predetermined time has elapsed.

[0185] In the initial state St0, when it is determined to be a stationary state, the state transitions to the initial attitude estimation state St1, as shown by the arrow tr01 in Figure 14 the following.

[0186] In the initial attitude estimation state St1, when the initial gravity direction and deviation components are obtained and calibration is completed in a stationary state, the state transitions to as shown in Figure 14the posture determination state St2 indicated by the arrow tr12 in

[0187] In addition, in the initial posture estimation state St1, when it is determined that the device is not stationary until calibration is completed, the state returns to the initial state St0, as Figure 14 indicated by the arrow tr10 in

[0188] In the posture determination state St2, when it is determined that the device is not in a stationary state, it is assumed that the pen 31 is taken out of the pen holder 32, and as Figure 14 indicated by the arrow tr23 in , the state transitions to the drawing state St3.

[0189] In the drawing state St3, drawing can be performed, and a trajectory corresponding to the movement of the tip of the pen 31 is obtained through inertial navigation.

[0190] Note that in the drawing state, it is sufficient if the pen 31 is taken out of the pen holder 32, that is, if an error due to the multi-IMU 63 occurs through inertial navigation, and thus it does not matter whether the pressure-sensitive sensor 51 is pressed and an actual drawing operation is performed.

[0191] In the drawing state St3, when it is determined that the pen 31 returns to the pen holder 32 and is in a stationary state until a predetermined time has passed since the timing when the state was set to the drawing state St3, as Figure 14 indicated by the arrow tr34 in , the state transitions to the re-posture estimation state St4.

[0192] In the re-posture estimation state St4, when it is determined that the pen 31 is taken out of the pen holder 32 and is not stationary until a predetermined time has passed since the timing when the state was set to the drawing state St3, the state returns to the drawing state St3.

[0193] In addition, in the drawing state St3, when a predetermined time has passed since the timing when the state was set to the drawing state St3, the state transitions to the initial state St0.

[0194] In addition, in the re-posture estimation state St4, when a predetermined time has passed since the timing when the state was set to the drawing state St3, the stationary state (i.e., the state where the pen 31 is upright on the pen holder 32) continues. Therefore, the initial gravity direction and deviation components are obtained as they are, calibration is completed, and the state transitions to the posture determination state St2 as Figure 14 indicated by the arrow tr42 in .

[0195] That is, when the pen 31 is powered on, the state becomes the initial state St0, and in addition, when it is assumed that the pen is upright on the pen holder 32 and stationary, the state of the pen 31 transitions to the initial posture estimation state St1. [[ID=3,6]]

[0196] When the state transitions to the initial attitude estimation state St1, the initial gravity direction and deviation components are set and calibration is performed. When the calibration is completed, the state transitions to the attitude determination state St2.

[0197] At this time, the light-emitting unit 52 emits light, indicating the completion of calibration, and the user can recognize that the drawing operation is possible.

[0198] Then, when the pen 31 is taken out of the pen holder 32 and is assumed to be not stationary, the state transitions to the drawing state St3.

[0199] When the elapsed time since the transition to the drawing state St3 is shorter than a predetermined time (e.g., 10 seconds), if the pen 31 returns to the pen holder 32 and is assumed to be in a stationary state, the state transitions to the re-attitude estimation state St4.

[0200] Here, in the re-attitude estimation state St4, in a state where the elapsed time since the transition to the drawing state St3 is shorter than a predetermined time (e.g., 10 seconds), when the pen 31 is taken out of the pen holder 32 again and is assumed to be not stationary, the state transitions to the drawing state St3 again.

[0201] In addition, in the drawing state St3, when the elapsed time since the transition to the drawing state St3 is shorter than a predetermined time (e.g., 10 seconds) and the pen 31 returns to the pen holder 32 in a state where it is assumed to be in a stationary state, the state transitions to the re-attitude estimation state St4 again.

[0202] That is, in the drawing state St3, if the elapsed time since the transition to the drawing state St3 is shorter than a predetermined time (e.g., 10 seconds), if the pen 31 is taken out of the pen holder 32, then it becomes the drawing state St3, and if the pen returns to the pen holder 32, then it becomes the re-attitude estimation state St4.

[0203] Then, in the drawing state St3, after the elapsed time since the transition to the drawing state St3 has passed a predetermined time (e.g., 10 seconds), the light-emitting unit 52 is turned off, the state returns to the initial state St0, and subsequent state transitions continue.

[0204] In addition, in the re-attitude estimation state St4, after the elapsed time since the transition to the drawing state St3 has passed a predetermined time (e.g., 10 seconds), the pen 31 is in a state of being vertically placed on the pen holder 32 and is thus in the same state as the initial attitude estimation state St1. Accordingly, the initial gravity direction and deviation components are set as they are, calibration is performed, and when the calibration is completed, the state transitions to the attitude determination state St2.

[0205] The control process of the pen 31 is a process corresponding to the state transition as Figure 14 shown.

[0206] <Control processing>

[0207] Next, the control processing of pen 31 will be described with reference to Figure 15 the flowchart of

[0208] In step S51, acquisition unit 102 acquires information on the absolute azimuth angle and latitude of pen 31, and provides this information to inertial navigation processing unit 131 via external I / F 134. Attitude estimation unit 171 in position and attitude processing unit 151 of inertial navigation processing unit 131 holds the information on the absolute azimuth angle and latitude provided from acquisition unit 102.

[0209] Here, for example, the information on the absolute azimuth angle and latitude of pen 31 acquired by acquisition unit 102 can be acquired as information input by a user's operation on a keyboard or the like. In addition, regarding the information on latitude, information detected based on signals from satellites (not shown) can be acquired by a global navigation satellite system (GNSS) device or the like. Note that as long as it is assumed that the position of pen holder 32, cloves, etc. do not change, the information on the absolute azimuth angle and latitude only needs to be given once, so the processing in step S51 can be skipped after the first acquisition. In addition, the information can be acquired and stored in control unit 101 in advance separately from the processing.

[0210] In step S52, position and attitude processing unit 151 acquires and buffers information on one sample of the angular velocity and acceleration provided from multi-IMU 63.

[0211] In step S53, position and attitude processing unit 151 performs state transition processing, and determines and transitions the state in the control processing of pen 31.

[0212] It should be noted that the state transition processing will be described in detail later with reference to Figure 16 the flowchart of

[0213] In step S54, position and attitude processing unit 151 determines whether the current state is the initial state St0.

[0214] In step S54, if the current state is not the initial state St0, the processing proceeds to step S55.

[0215] In step S55, position and attitude processing unit 151 determines whether the current state is the drawing state St3.

[0216] In step S55, if it is determined that the current state is not the drawing state, the processing proceeds to step S56.

[0217] In step S56, the initial gravity estimation unit 181 of the initial attitude estimation unit 152 performs an initial gravity estimation process for estimating the initial gravity direction.

[0218] Note that the initial gravity estimation process will be described in detail later with reference to Figure 17 the flowchart.

[0219] In step S57, the deviation estimation unit 182 of the initial attitude estimation unit 152 performs a deviation estimation process for estimating the deviation component.

[0220] Note that the deviation estimation process will be described in detail later with reference to Figure 18 the flowchart.

[0221] Furthermore, when it is determined in step S55 that the current state is the drawing state, the process proceeds to step S58.

[0222] In step S58, the position and attitude processing unit 151 performs a drawing process and provides the PC 33 with position information indicating a trajectory corresponding to the movement of the tip of the pen 31. When the position information corresponding to the trajectory is acquired, the PC 33 generates a drawing image corresponding to the position information and causes the display device 34 to display the drawing image.

[0223] Note that the drawing process will be described in detail later with reference to Figure 19 the flowchart in

[0224] In addition, when it is determined in step S54 that the current state is not the initial state, the processes in steps S55 to S58 are skipped.

[0225] In step S59, the position and attitude processing unit 151 determines whether the end of the process has been indicated.

[0226] In step S59, when the end of the process has not been indicated, the process returns to step S52.

[0227] That is, the processes in steps S52 to S59 are repeated until the end of the process is indicated.

[0228] Then, in step S59, when the end of the process is indicated, the process ends.

[0229] [[ID=4,2]]By performing the above series of processes, the operation of the pen 31 is controlled, and the user 21 grasps the pen 31 and moves the tip so that a trajectory corresponding to the movement of the tip is obtained, and a drawing image corresponding to the trajectory can be displayed on the display device 34.

[0230] It should be noted that the automatic rotation component and the deviation component are obtained through the initial gravity estimation process and the deviation estimation process in steps S56 and S57, and basically achieve reference Figure 9The calibration process described by the flowchart.

[0231] That is, the gravity direction is obtained through the initial gravity estimation process described later, and the automatic rotation component of the global coordinate system is obtained from the latitude information through the deviation estimation process described later. The attitude rotation matrix M s→g The inverse matrix M of g→s Based on the gravity direction and the absolute azimuth angle, the automatic rotation component of the global coordinate system is obtained by the inverse matrix M g→s Converted into the automatic rotation component of the sensor coordinate system, and the deviation component of the sensor coordinate system is obtained from the difference between the average angular velocity detected by the multi-IMU 63 in the stationary attitude and the automatic rotation component of the sensor coordinate system.

[0232] Here, since the automatic rotation component of the sensor coordinate system is obtained through the latitude, gravity direction, and absolute direction, the information detected by the multi-IMU 63 in the stationary attitude is not required. In addition, since the deviation component is obtained from the information detected by the multi-IMU 63 in the stationary attitude and the automatic rotation component, calibration can be achieved in one type of stationary attitude.

[0233] <State transition process>

[0234] Next, reference will be made to Figure 16 The flowchart of to describe the state transition process.

[0235] In step S71, the stationary determination unit 153 determines whether the information on the angular velocity and acceleration (which is the detection result of the multi-IMU 63) has been buffered for a predetermined time (for example, one second).

[0236] In the case where it is determined in step S71 that the information on the angular velocity and acceleration as the detection result of the multi-IMU 63 has been buffered for a predetermined time, the process proceeds to step S72.

[0237] It should be noted that in the case where it is determined in step S71 that the information on the angular velocity and acceleration as the detection result of the multi-IMU 63 has not been buffered for the predetermined time, the state transition process ends.

[0238] In step S72, the stationary determination unit 153 determines whether the pen 31 is stationary based on the buffered information on the angular velocity and acceleration as the detection result of the multi-IMU 63.

[0239] In the case where it is determined to be stationary in step S72, the process proceeds to step S73.

[0240] In step S73, the attitude estimation unit 171 determines whether the current state is the initial state St0.

[0241] When it is determined in step S73 that the current state is not the initial state St0, the process proceeds to step S74.

[0242] In step S74, the attitude estimation unit 171 determines whether the current state is the drawing state St3.

[0243] When it is determined in step S74 that the current state is the drawing state St3, the process proceeds to step S75.

[0244] In step S75, the attitude estimation unit 171 Figure 14 transfers the current state to the re-attitude estimation state St4 as shown by the arrow tr34 in

[0245] Note that when it is determined in step S74 that the current state is not the drawing state St3, the process in step S75 is skipped.

[0246] Furthermore, when it is determined in step S73 that the current state is the initial state St0, the process proceeds to step S76.

[0247] In step S76, the attitude estimation unit 171 transitions to the initial attitude estimation state St1, as shown by the arrow tr01 in Figure 14

[0248] On the other hand, in step S72, when it is not stationary and the pen 31 is taken out of the pen holder 32 and some movement is assumed, the process proceeds to step S77.

[0249] In step S77, the attitude estimation unit 171 sets the initial gravity determination flag to zero, indicating that the information on the initial gravity direction has not been obtained.

[0250] In step S78, the attitude estimation unit 171 determines whether the current state is the initial attitude estimation state St1.

[0251] When it is determined in step S78 that the current state is not the initial attitude estimation state St1, the process proceeds to step S79.

[0252] In step S79, the attitude estimation unit 171 determines whether the current state is the attitude determination state St2 or the re-attitude estimation state St4.

[0253] In step S79, when the current state is the attitude determination state St2 or the re-attitude estimation state St4, the process enters step S80.

[0254] In step S80, the attitude estimation unit 171 Figure 14 transfers the current state to the drawing state St3 as shown by the arrow tr23 or tr43 in​

[0255] In addition, in step S79, when the current state is neither the attitude determination state St2 nor the re-attitude estimation state St4, the process ends.

[0256] Furthermore, when it is determined in step S78 that the current state is the initial attitude estimation state St1, the process proceeds to step S80.

[0257] In step S80, as shown by the arrow tr10 in Figure 14 , the attitude estimation unit 171 transfers the current state to the initial state St0.

[0258] Through the above series of processes, the state transition is controlled with reference to Figure 14 the description.

[0259] <Initial gravity estimation process>

[0260] Next, the initial gravity estimation process will be described with reference to the flowchart of Figure 17 .

[0261] In step S101, the initial gravity estimation unit 181 of the initial attitude estimation unit 152 determines whether the acceleration as the detection result of the multi-IMU 63 has been buffered for a predetermined time (e.g., one second).

[0262] When it is determined in step S101 that the accelerations of the three axes as the detection results of the multi-IMU 63 have been buffered for a predetermined time (e.g., one second), the process proceeds to step S102.

[0263] In step S102, the initial gravity estimation unit 181 calculates the average value of the buffered accelerations of the three axes as the gravity information.

[0264] In step S103, the initial gravity estimation unit 181 provides the calculated initial gravity information to the attitude estimation unit 171 of the position and attitude processing unit 151.

[0265] In step S104, the attitude estimation unit 171 sets the initial gravity determination flag to 1 as the information indicating that the initial gravity has been set.

[0266] It should be noted that in step S101, when it is determined that the accelerations of the three axes as the detection results of the multi-IMU 63 are not buffered within a predetermined time (e.g., one second) and the initial gravity cannot be obtained, the process proceeds to step S105.

[0267] In step S105, the attitude estimation unit 171 sets the initial gravity determination flag to zero as the information indicating that the initial gravity has not been set.

[0268] Through the above processing, the initial gravity is set, and an initial gravity determination flag is set according to whether the initial gravity is set.

[0269] <Deviation Estimation Processing>

[0270] Next, the deviation estimation processing will be described with reference to Figure 18 the flowchart of

[0271] In step S121, the deviation estimation unit 182 of the initial attitude estimation unit 152 queries the attitude estimation unit 171 of the position and attitude processing unit 151 to determine whether the initial gravity determination flag is 1 and to determine the initial gravity.

[0272] In the case where it is determined in step S121 that the initial gravity determination flag is set to 1 and the initial gravity direction is determined, the process proceeds to step S122.

[0273] In step S122, it is determined whether the angular velocity information of the three axes detected by the gyroscope sensor in the detection results provided by the multi-IMU 63 has been buffered for a predetermined time, for example, 10 seconds.

[0274] In the case where it is determined in step S122 that the information of the angular velocity of the three axes has been buffered for a predetermined time (for example, 10 seconds), the process proceeds to step S123.

[0275] In step S123, the deviation estimation unit 182 calculates the 10-second average value of the angular velocity of the three axes as the sum of the deviation component and the auto-rotation component. That is, since the pen 31 is stationary here, the average value of the angular velocity of the three axes is calculated as the average value of the sum of the deviation component and the auto-rotation component.

[0276] In step S124, the attitude estimation unit 171 sets the attitude rotation matrix M s→g based on the obtained initial gravity direction and the held absolute azimuth angle, and sets the inverse matrix M g→s of g→s , and supplies the inverse matrix M

[0277] In step S125, as described in reference expression (1), the auto-rotation estimation unit 172 estimates (calculates) the auto-rotation component of the global coordinate system based on the held latitude information. Then, the auto-rotation estimation unit 172 converts the auto-rotation component of the global coordinate system into the auto-rotation component of the sensor coordinate system by referring to Figure 13 and the method described in expression (2) by using the information of the inverse matrix M g→s , and supplies the auto-rotation component to the attitude estimation unit 171.

[0278] The attitude estimation unit 171 supplies the auto-rotation component estimated (calculated) by the auto-rotation estimation unit 172 to the deviation estimation unit 182 of the initial attitude estimation unit 152.

[0279] In step S126, the deviation estimation unit 182 estimates (calculates) the deviation component of the sensor coordinate system by subtracting the auto-rotation component of the sensor coordinate system from the average value of the angular velocities of the three axes, and outputs the deviation component to the attitude estimation unit 171.

[0280] In step S127, the attitude estimation unit 171 changes the current state to the attitude determination state St2.

[0281] In step S128, the attitude estimation unit 171 notifies the light emission control unit 132 of the information indicating that the current state is the attitude determination state St2. Through this notification, the light emission control unit 132 causes the LED 142 of the light emission unit 52 to emit light, thereby indicating that the calibration has been completed by the user and is in the drawable state.

[0282] In step S129, the attitude estimation unit 171 sets the drawing time to zero.

[0283] On the other hand, in the case where it is determined in step S121 that the initial gravity determination flag is set to zero and the initial gravity has not been determined, or in the case where it is determined in step S122 that the information on the angular velocity of the axis has not been cached for a predetermined time (for example, 10 seconds), the process proceeds to step S130.

[0284] In step S130, the attitude estimation unit 171 notifies the light emission control unit 132 of the information indicating that the current state is not the attitude determination state St2. Through this notification, the light emission control unit 132 turns off the LED 142 of the light emission unit 52 to indicate that the calibration has not been completed by the user and is not in the drawable state.

[0285] Through the above processing, the deviation component of the sensor coordinate system is estimated (calculated) after setting the initial gravity, and the light emission unit 52 emits light when the deviation component is estimated, so that when the deviation component of the sensor coordinate system is estimated and the calibration is completed, it can be presented to the user that it is in the drawable state.

[0286] <Drawing process>

[0287] Next, the drawing process will be described with reference to Figure 19 the flowchart.

[0288] In step S151, the attitude estimation unit 171 corrects the angular velocity by subtracting the auto-rotation component and the deviation component of the sensor coordinate system from the angular velocities of the three axes detected by the multi-IMU 63.

[0289] In step S152, the attitude estimation unit 171 updates the attitude rotation matrix M based on the corrected angular velocity information through attitude calculation s→g .

[0290] In step S153, the attitude estimation unit 171 converts the acceleration detected by the multi-IMU 63 into the acceleration in the global coordinate system through the updated attitude rotation matrix M, obtains the absolute attitude of the pen 31, and outputs the absolute attitude to the acceleration integration unit 173. s→g

[0291] In step S154, the acceleration integration unit 173 calculates the time difference between a previous sampling time and the current time.

[0292] In step S155, the acceleration integration unit 173 updates the velocity by adding the change in velocity obtained by multiplying the current acceleration by the time difference to the current velocity.

[0293] In step S156, the acceleration integration unit 173 updates the position by adding the change in position obtained by multiplying the current velocity by the time difference to the current position.

[0294] In step S157, the acceleration integration unit 173 sends the information of the position and velocity to the PC 33 via the external I / F 134 as the position information indicating the trajectory of the tip of the pen 31. The PC 33 generates a drawn image based on the position information of the trajectory of the tip of the pen 31 including the information of the position and velocity, and causes the display device 34 to display the drawn image.

[0295] In step S158, the attitude estimation unit 171 updates the drawing time by increasing the drawing time by a predetermined time.

[0296] In step S159, the attitude estimation unit 171 determines whether the drawing time has passed a predetermined time (e.g., 10 seconds), whether the cumulative error of the acceleration integration unit 173 has increased, and whether the position accuracy is lower than a predetermined state and correction is required.

[0297] In the case where it is determined in step S159 that the drawing time has passed the predetermined time, the integration error of the acceleration integration unit 173 has increased, the position accuracy has become lower than the predetermined state, and calibration is necessary, the process proceeds to step S160.

[0298] In step S160, the attitude estimation unit 171 changes the current state to the initial state St0.

[0299] In step S161, the attitude estimation unit 171 initializes the drawing time to zero. ​

[0300] In step S162, the attitude estimation unit 171 notifies the light emission control unit 132 of information indicating that the current state is the initial state St0. Through this notification, the light emission control unit 132 turns off the LED 142 of the light emission unit 52, indicating to the user that calibration is not completed and drawing cannot be performed.

[0301] It should be noted that in the case where it is determined in step S159 that the drawing time has not elapsed for a predetermined time (e.g., 10 seconds) and calibration is not required, the processes in steps S160 to S162 are skipped, and the light-emitting light-emitting unit 52 continuously indicates that it is in a state where drawing is possible.

[0302] According to the above processing, the automatic rotation component that causes errors in the multi-IMU 63 is calculated based on the information of the absolute azimuth angle and latitude, and the deviation component is obtained by using the obtained automatic rotation component and a type of stationary attitude in which the pen 31 is vertically placed on the pen container 32, so that easy and appropriate calibration can be achieved.

[0303] <Modification Example 1>

[0304] In the above description, an example has been described in which the pen 31 is vertically placed on the pen container 32, the light-emitting unit 52 emits light when calibration is completed, the pen 31 is taken out from the pen container 32, and when a predetermined time has elapsed since the start of the drawing operation and the reliability of the drawing position becomes lower than a predetermined state through the integration process of acceleration, the light-emitting unit 52 is turned off.

[0305] That is, an example has been described in which it is indicated that calibration has been completed or the reliability of the drawing position has become lower than a predetermined state according to the light emission state of the light-emitting unit 52.

[0306] However, similar information can be presented in the drawing image that can be displayed on the display device 34.

[0307] For example, as Figure 20 shown, when performing a drawing operation with the pen 31 to draw from the starting point P in the arrow direction on the display surface of the display device 34, drawing can be performed by using the depth corresponding to the reliability of the drawing position and the elapsed time of the drawing operation.

[0308] That is, in Figure 20 , the drawing near the starting point P is displayed in a dark color, but as the drawing operation progresses, the color can become lighter and gradually blurred as time elapses (i.e., as the reliability decreases), and finally reaches a state where no color is added and drawing cannot be performed.

[0309] In this way, the user can recognize the degree of reduction in reliability while viewing the drawn image displayed on the display device 34, and can recognize that the reliability of the drawing position is lower than a predetermined state and calibration is required through the state of no color addition and no drawing being executable.

[0310] <Modification 2>

[0311] In the above description, an example has been described above in which the positional relationship between the pen container 32 and the pen 31 is configured as a constant relationship by placing the pen 31 upright so that the pressure-sensitive sensor 51 of the pen 31 faces the concave portion 84a of the lid 84 of the pen container 32.

[0312] However, any other configuration can be used as long as the positional relationship between the pen 31 and the pen container 32 can be kept constant.

[0313] For example, as shown in the left part of Figure 21 a pen 31' provided with a protruding portion 31a' and a pen container 32' provided with a hole 32a' having a cutout portion at a corresponding position can be used.

[0314] That is, when the cross-sectional shape of the protruding portion 31a' of the pen 31' and the cross-sectional shape of the hole 32a' of the pen container 32' match, the pen 31' can be placed upright on the pen container 32', and thus the mutual positional relationship can be kept constant.

[0315] It should be noted that Figure 21 is a top view of the hole 32a' with the pen 31' placed upright on the pen container 32'.

[0316] In addition, as shown in the central part of Figure 21 a polygonal pen 31” can be formed and a protruding portion 31a” can be provided, and a hole 32a” having a cutout portion can be provided at a corresponding position of the pen container 32”.

[0317] That is, when the cross-sectional shape of the protruding portion 31a” of the pen 31” and the cross-sectional shape of the hole 32a” of the pen container 32” match, the pen 31” can be placed upright on the pen container 32”, and thus the mutual positional relationship can be kept constant.

[0318] In addition, as shown in the right part of Figure 21 a pen 31”' having an asymmetrical shape can be used, and a corresponding asymmetrical hole 32a”' can be provided in the pen container 32”'.

[0319] That is, when the cross-sectional shape of the protruding portion 31a”' of the pen 31”' and the cross-sectional shape of the hole 32a”' of the pen container 32”' match, the pen 31”' can be placed upright on the pen container 32”', and thus the mutual positional relationship can be kept constant.

[0320] <Modified Example 3>

[0321] In the above description, a configuration has been described in which the pen holder 32 is formed of a transparent acrylic resin having a hollow portion, and the hollow portion is filled with a colored liquid so that the light emission state of the light emitting unit 52 of the pen 31 can be visually recognized.

[0322] However, as long as the light emission state of the light emitting unit 52 of the pen 31 can be visually recognized by the user, any other configuration may be adopted. For example, as Figure 22 shown in the pen holder 201, the separately provided light emitting unit 212 can emit light or turn off according to the light emission or off state of the light emitting unit 52 of the pen 31.

[0323] At this time, for example, as Figure 22 shown, the light emitting unit 212 may include a plurality of LEDs 212a to 212c that emit light of different colors, and the color of the emitted light may be switched according to the number of times the switching switch 211 is pressed.

[0324] For example, among the LEDs 212a to 212c shown in the left part of Figure 22 , the LED 212a is configured to emit light when the switching switch 211 is pressed once, and thus emit light in the color represented by the pen holder 201'.

[0325] Then, as Figure 21 shown in the right part of, it may be configured such that when the switching switch 211 is pressed twice and the pen 31 is next placed upright, the LED 212b is configured to emit light instead of the LED 212a. In this case, when the pen 31 is placed upright on the pen holder 201', as shown by the pen holder 201'', the light may be emitted in a color different from the color of the pen holder 201'.

[0326] In this case, it may be notified by short-range wireless communication such as Bluetooth (registered trademark) that the colors of the pen holder 201 and the pen 31 have been switched by the switching switch 211 of the pen holder 201, and the color of the drawing line displayed on the display device 34 may be changed according to the color of the light emitted from the LEDs 212a to 212c.

[0327] <<2. Application Example>>

[0328] In the above description, examples of obtaining absolute azimuth angle information by using GNSS or user input or the like have been described. However, multiple IMUs may be built into the pen holder so that the pen holder can detect the absolute azimuth angle. When the pen is placed upright on the pen holder, the pen can obtain the information of the absolute azimuth angle from the pen holder, and calibration can be achieved through cooperative processing between the pen and the pen holder.

[0329] That is, the pen container 301 with multiple IMUs inside and capable of detecting the absolute azimuth angle detects the absolute azimuth angle based on two types of static postures, and provides the detected absolute azimuth angle to the pen 331.

[0330] More specifically, as Figure 23 shown in the leftmost part of, after a predetermined time in the first stationary posture, for example, when the pen container 301 rotates 90 degrees and after a predetermined time in the second stationary posture, as Figure 23 shown in the second part from the left in, the pen container 301 obtains two types of detection results through multiple IMUs in two types of stationary postures, and obtains the absolute azimuth angle based on the two types of detection results obtained.

[0331] Because the pen container 301 is usually moved less frequently once it is placed, the pen container 301 is placed so that the user takes two stationary postures, so that the absolute azimuth angle is detected when the pen container is first placed.

[0332] After that, as shown in the second part from the right in Figure 23 , when the pen 331 is placed upright on the pen container 301, for example, the obtained absolute azimuth angle is transmitted from the pen container 301 to the pen 331 through communication such as Wi-Fi.

[0333] After that, as Figure 23 shown in the rightmost part of, the calibration of the pen 331 is achieved by a method similar to the above method.

[0334] <Azimuth difference between the pen container and the pen>

[0335] During calibration, the absolute azimuth angle provided from the pen container 301 is based on the value of the front direction of the pen container 301 and is different from the front direction of the pen 331.

[0336] Therefore, the pen 331 pre-obtains the azimuth difference between the pen container 301 and the pen 331, corrects the absolute azimuth angle provided from the pen container 301, and performs calibration.

[0337] More specifically, as Figure 24 shown, the difference between the front direction D1 of the pen container 301 and the front direction D2 of the position of the pressure-sensitive sensor 51 of the pen 331 is the azimuth angle difference θ.

[0338] Thus, the pen 331 subtracts the azimuth angle difference θ from the absolute azimuth angle provided from the pen container 301 as the correction to the absolute azimuth angle of the pen 331, and uses this absolute azimuth angle for calibration. Note that this azimuth angle difference θ is a fixed value, so it can be obtained from the outside relative to the pen 331 in advance.

[0339] <Update of the absolute azimuth angle of the pen container>

[0340] Once the pen holder 301 is placed, it basically moves at a low frequency. However, in the case where a change is added such that the direction rotates 90 degrees after obtaining the absolute azimuth angle (as shown by the pen holder 301' shown), it can be updated by acceleration integration, or the absolute azimuth angle can be obtained again by using calibration of two types of static postures. Figure 25 The following will describe a hardware configuration example of a pen-type drawing system that achieves calibration through collaborative processing between the pen and the pen holder.

[0341] <Hardware Configuration Example of a Pen-Type Drawing System that Achieves Calibration through Collaborative Processing between the Pen and the Pen Holder>

[0342] Next, a hardware configuration example of a pen-type drawing system that achieves calibration through collaborative processing between the pen and the pen holder will be described with reference to Figure 26 Note that in

[0343] Note that the configurations of the PC 33 and the display device 34 are similar to the configurations described in the reference Figure 26 and thus their descriptions are omitted. Figure 10 The pen 331 includes a control unit 351, an acquisition unit 352, a communication unit 353, a multi-IMU 354, a light-emitting unit 355, and a pressure-sensitive sensor 356, which are electrically connected through a bus 361 and can communicate with each other.

[0344] Note that the configurations of the control unit 351, the acquisition unit 352, the communication unit 353, the multi-IMU 354, the light-emitting unit 355, and the pressure-sensitive sensor 356 are basically similar to the configurations of the control unit 101, the acquisition unit 102, the communication unit 103, the multi-IMU 63, the light-emitting unit 52, and the pressure-sensitive sensor 51 in

[0345] and thus their descriptions are omitted. Figure 10 However, the acquisition unit 352 pre-acquires and holds information on the azimuth angle difference between the latitude input by the user 21 operating the keyboard, etc. and the pen holder 301. In addition, the acquisition unit 352 controls the communication unit 353 to communicate with the pen holder 301 to obtain information on the absolute azimuth angle, and the control unit 351 uses this information for calibration. In these aspects, the control unit 351 and the acquisition unit 352 are different from the control unit 101 and the acquisition unit 102 in

[0346] The pen holder 301 includes a control unit 381, a communication unit 382, and a multi-IMU 383 that are electrically connected through a bus 391 and can communicate with each other. Figure 10 Note that the configurations of the communication unit 382 and the multi-IMU 383 are basically similar to the configurations of the communication unit 103 and the multi-IMU 63 of the pen 31, and thus their descriptions are omitted.

[0347]

[0348] Note that the configurations of the communication unit 382 and the multi-IMU 383 are basically similar to the configurations of the communication unit 103 and the multi-IMU 63 of the pen 31, and thus their descriptions are omitted.

[0349] The basic configuration of the control unit 381 is similar to that of the control unit 101 of the pen 31, but the difference is that the control unit obtains the absolute azimuth angle of the pen barrel 301 based on the detection results of the multi-IMU 383 in two types of static postures, controls the communication unit 382, and sends the absolute azimuth angle to the communication unit 353 of the pen 331.

[0350] <by Figure 26 Functions implemented by the control unit of the pen barrel in

[0351] Next, reference will be made to Figure 27 to describe the functions implemented by the control unit 381 of the pen barrel 301 in Figure 26

[0352] The control unit 381 includes an inertial navigation processing unit 401 and an external I / F 402. It should be noted that the inertial navigation processing unit 401 and the external I / F 402 are basically the same as the inertial navigation processing unit 131 and the external I / F 134 in Figure 11

[0353] In addition, Figure 11 the difference of the inertial navigation processing unit 131 in

[0354] is that the acquisition unit 102 acquires information on the absolute azimuth angle and latitude and performs calibration based on the detection results provided by the multi-IMU 63 in one type of static posture, while the inertial navigation processing unit 401 is only given information on latitude and obtains the absolute azimuth angle based on the detection results provided by the multi-IMU 383 in two types of static postures.

[0355] Specifically, the inertial navigation processing unit 401 includes a position and attitude processing unit 411, an initial attitude estimation unit 412, and a stationary determination unit 413. Figure 12 The position and attitude processing unit 411, the initial attitude estimation unit 412, and the stationary determination unit 413 basically have functions similar to those of the position and attitude processing unit 151, the initial attitude estimation unit 152, and the stationary determination unit 153 in

[0356] However, since the absolute azimuth angle is not given, the initial attitude estimation unit 412 obtains the absolute azimuth angle and the deviation component based on the detection results detected by the multi-IMU 383 in two types of static postures, and provides the absolute azimuth angle and the deviation component to the position and attitude processing unit 411.

[0357] The calibration process performed by the position and attitude processing unit 411 based on the information of the absolute azimuth angle provided by the initial attitude estimation unit 412 and the latitude provided by the external I / F 402 is similar to the process of the position and attitude processing unit 151 in Figure 12 Herein.

[0358] More specifically, the position and attitude processing unit 411 includes an attitude estimation unit 431 and an acceleration integration unit 432.

[0359] It should be noted that the attitude estimation unit 431 and the acceleration integration unit 432 have the same functions as the attitude estimation unit 171 and the acceleration integration unit 173 in Figure 12 Herein, and thus their descriptions are omitted.

[0360] However, since the automatic rotation component is obtained by the initial attitude estimation unit 412, there is no configuration corresponding to the automatic rotation estimation unit 172 in Figure 12 Herein, and thus the processing of the automatic rotation component is omitted in the processing of the attitude estimation unit 431.

[0361] The initial attitude estimation unit 412 includes an initial gravity estimation unit 451 and an initial azimuth deviation estimation unit 452.

[0362] It should be noted that the initial gravity estimation unit 451 is similar to the initial gravity estimation unit 181 in Figure 12 Herein, and thus its description is omitted.

[0363] The initial azimuth deviation estimation unit 452 obtains the automatic rotation component and the deviation component based on the detection results of the multi-IMU 383 in two types of static postures, further obtains the initial azimuth angle from the automatic rotation component, and provides the initial azimuth angle to the attitude estimation unit 431 of the position and attitude processing unit 411.

[0364] Note that the method for obtaining the automatic rotation component and the deviation component and the method for obtaining the initial azimuth from the automatic rotation component based on the detection results of the multi-IMU 383 in two types of static postures are general processes, and thus their descriptions are omitted.

[0365] <Cooperative processing between the pen and the pen holder>

[0366] Next, the cooperative processing between the pen 331 and the pen holder 301 will be described with reference to the flowchart of Figure 28 Herein.

[0367] Note that herein, in the pen holder 301, the description is only for the process of providing the absolute azimuth angle to the pen 331, and the process of obtaining the attitude in the pen holder 301 is omitted, but as in the pen 31, the process of obtaining its own attitude is also performed.

[0368] In step S211, the initial azimuth deviation estimation unit 452 of the initial attitude estimation unit 412 calculates the deviation component and the auto-rotation component based on the acceleration and angular velocity detected by the multi-IMU 383 in two types of static postures, calculates the absolute azimuth based on the auto-rotation component, and provides the information of the absolute azimuth to the attitude estimation unit 431 of the position and attitude processing unit 411.

[0369] In step S212, the attitude estimation unit 431 determines whether the information of the absolute azimuth is provided from the initial azimuth deviation estimation unit 452.

[0370] In step S212, the processes in steps S211 and S212 are repeated until it is assumed that the information of the absolute azimuth is provided from the initial azimuth deviation estimation unit 452.

[0371] Then, when it is determined in step S212 that the information of the absolute azimuth is provided from the initial azimuth deviation estimation unit 452, the process proceeds to step S213.

[0372] In step S213, the attitude estimation unit 431 stores the information of the absolute azimuth obtained from the initial azimuth deviation estimation unit 452.

[0373] In step S214, the attitude estimation unit 431 controls the communication unit 382 to send the absolute azimuth information to the pen 331 via the external I / F 402.

[0374] Through the processing so far, the information of the initial absolute azimuth is sent to the pen 331.

[0375] In step S215, the attitude estimation unit 431 controls the stationary determination unit 413 to determine whether it is stationary based on the acceleration and angular velocity detected by the multi-IMU 383, and determines whether it is not stationary, that is, whether it moves, rotates the pen container 301, etc.

[0376] When it is determined in step S215 that the pen container 301 is not stationary and has moved, rotated, etc., the process advances to step S216.

[0377] In step S216, the attitude estimation unit 431 provides the stored information of the absolute azimuth to the acceleration integration unit 432 to be updated based on the integration result of the acceleration, and then acquires and stores the information.

[0378] In step S217, the attitude estimation unit 431 controls the communication unit 382 to send the updated absolute azimuth information to the pen 331 via the external I / F 402.

[0379] Note that when it is determined to be stationary in step S215, the processes in steps S216 and S217 are skipped.

[0380] In step S218, the attitude estimation unit 431 determines whether the end of the process is indicated. If the end is not indicated, the process returns to step S215.

[0381] That is, the processes in steps S215 to S218 are repeated until the end of the process is indicated. When the pen holder 301 is moved or rotated, the absolute attitude is updated and sent to the pen 331.

[0382] On the other hand, in the pen 331, in step S231, the control unit 351 of the pen 331 controls the acquisition unit 352 to acquire information on the latitude and azimuth difference.

[0383] In step S232, the control unit 351 controls the communication unit 353 to determine whether the absolute azimuth has been sent from the pen holder 301.

[0384] In step S232, when the absolute azimuth is sent from the pen holder 301 through the process in step S214 or S217 above, the process proceeds to step S233.

[0385] In step S233, the control unit 351 controls the communication unit 353 to acquire and store the information on the absolute azimuth sent from the pen holder 301.

[0386] In step S234, the control unit 351 subtracts the azimuth difference from the acquired absolute azimuth of the pen holder 301 as a correction to the absolute azimuth of the pen 331.

[0387] Thereafter, the control process described in the flowchart of the reference is executed using the information on the absolute azimuth of the pen 331 obtained by correcting the updated new absolute azimuth of the pen holder 301 by using the differential azimuth. However, in this case, the process is executed by replacing the absolute azimuth acquired in the process of step S51 in Figure 15 with the absolute difference obtained by correcting the absolute azimuth provided from the pen holder 301 by the azimuth difference. Figure 15

[0388] In step S235, the control unit 351 determines whether the end of the process is indicated. If the end is not indicated, the process returns to step S232.

[0389] Then, when the end is indicated in step S235, the process ends.

[0390] As described above, since calibration is performed through the cooperative processing between the pen 331 and the pen holder 301, the control processing of the pen 331 is performed based on the absolute azimuth angle provided by the pen holder 301. Therefore, the user does not need to provide information on the absolute azimuth angle.

[0391] In addition, since the information on the absolute azimuth angle is continuously updated each time the pen holder 301 is moved or rotated, the user can freely change the position and orientation of the pen holder 301 without being aware of calibration.

[0392] In addition, in the above description, an example has been described in which the update of the absolute azimuth angle performed each time the pen holder 301 is moved or rotated is updated by the acceleration integration unit 432 using the integration value. However, the absolute azimuth angle can be updated by repeatedly performing the processing in steps S211 to S214.

[0393] <Variation of the cooperative processing between the pen and the pen holder>

[0394] In the above description, an example has been described in which the absolute azimuth angle is sent to the pen 331 each time the absolute azimuth angle is obtained in the pen holder 301. However, the absolute azimuth angle can be sent to the pen 331 when the pen 331 is vertically placed on the pen holder 301.

[0395] Therefore, with reference to Figure 29 the flowchart of, a variation of the cooperative processing between the pen 331 and the pen holder 301 will be described, in which the absolute azimuth angle is sent to the pen 331 when the pen 331 is vertically placed on the pen holder 301.

[0396] It should be noted that the processing in steps S261 to S263 in the Figure 29 flowchart is similar to the processing in steps S211 to S213 in the Figure 28 flowchart. Therefore, its description is omitted.

[0397] When the absolute azimuth angle is obtained through the processing in steps SS261 to S263, the processing proceeds to step S264.

[0398] In step S264, the attitude estimation unit 431 controls the communication unit 382 to determine whether the pen 331 has notified that it is vertically placed on the pen holder 301. When the pen 331 has notified that it is vertically placed on the pen holder 301, the processing proceeds to step S265.

[0399] In step S265, the attitude estimation unit 431 controls the communication unit 382 to send the information on the absolute azimuth angle to the pen 331 via the external I / F 402.

[0400] In step S266, the attitude estimation unit 431 controls the stillness determination unit 413 to determine whether it is still based on the acceleration and angular velocity detected by the multi-IMU 383, and determines whether it is not still, that is, whether it moves, rotates the pen container 301, etc.

[0401] In the case where it is determined in step S266 that the pen container 301 is not still and has moved, rotated, etc., the process proceeds to step S267.

[0402] In step S267, the attitude estimation unit 431 provides the stored information of the absolute azimuth angle to the acceleration integration unit, and causes the acceleration integration unit 432 to update, obtain, and store the information of the absolute azimuth angle based on the integration result of the acceleration.

[0403] Note that in the case where it is determined to be still in step S266, the process in step S267 is skipped.

[0404] In step S268, the attitude estimation unit 431 determines whether the end of the process is indicated. If the end is not indicated, the process returns to step S264.

[0405] That is, the processes in steps S264 to S268 are repeated until the end of the process is indicated, and when the pen container 301 is moved or rotated, the absolute azimuth angle is updated. Then, when the pen 331 notifies that it is placed upright on the pen container 301, the absolute azimuth angle is sent from the pen container 301 to the pen 331.

[0406] On the other hand, in the pen 331, in step S281, the control unit 351 of the pen 331 controls the acquisition unit 352 to acquire the information of the latitude and azimuth angle difference.

[0407] In step S282, the control unit 351 determines whether it is still based on the acceleration and angular velocity detected by the multi-IMU 354, that is, whether the pen 331 is placed upright on the pen container 301 and whether it is still.

[0408] In the case where it is determined in step S282 that the pen 331 is still (that is, the pen 331 is placed upright on the pen container 301 and is still), the process proceeds to step S283.

[0409] In step S283, the control unit 351 controls the communication unit 353 to notify the pen container 301 that the pen is placed upright on the pen container 301.

[0410] In step S284, the control unit 351 controls the communication unit 353 to determine whether the absolute azimuth angle has been sent from the pen container 301.

[0411] That is, through the processing in step S283, it is determined in step S264 that the pen 331 has been placed upright on the pen barrel 301, and the absolute azimuth is transmitted through the processing in step S265.

[0412] Therefore, in step S284, since the absolute azimuth angle is transmitted from the pen holder 301, the process proceeds to step S285.

[0413] In step S285 , the control unit 351 controls the communication unit 353 to acquire and store the information of the absolute azimuth angle transmitted from the pen barrel 301 .

[0414] In step S286 , the control unit 351 subtracts the azimuth difference from the acquired absolute azimuth of the pen barrel 301 as correction to the absolute azimuth of the pen 331 .

[0415] Thereafter, a reference is performed using information of the absolute azimuth angle of the pen 331 obtained by correcting the updated new absolute azimuth angle of the pen barrel 301 using the difference azimuth angle. Figure 15 However, in this case, by replacing the absolute difference obtained by correcting the absolute azimuth angle provided from the pen barrel 301 with the azimuth angle difference, the absolute difference in the azimuth angle is obtained. Figure 15 The processing is performed based on the absolute azimuth angle obtained in the processing of step S51 in .

[0416] In step S287, the control unit 351 determines whether the end of the process is instructed, and returns to step S282 if the end is not instructed.

[0417] Then, if the end has been instructed in step S287, the processing ends.

[0418] Note that, in the case where it is determined in step S282 that it is not stationary and the pen 331 is not placed upright on the pen barrel 301, or in the case where the absolute azimuth angle is not transmitted in step S284, the process proceeds to step S287.

[0419] Through the above-described processing, the control processing of the pen 331 is performed based on the absolute azimuth angle provided from the pen barrel 301, and therefore, the user does not need to provide information on the absolute azimuth angle.

[0420] In addition, since the absolute azimuth is sent from the pen barrel 301 to the pen 331 whenever the pen 331 is placed upright on the pen barrel 301, even if the information of the absolute azimuth is updated whenever the pen barrel 301 moves or rotates, the absolute azimuth is not sent until the pen 331 is placed upright on the pen barrel 301, and is sent only when the pen 331 is placed upright.

[0421] In addition, in the above description, an example has been described in which the update of the absolute azimuth angle performed each time the pen container 301 moves or rotates is updated by the integral value of the acceleration integration unit 432, but the processes in steps S261 to S263 may be repeatedly executed.

[0422] <<3. Examples Executed by Software>>

[0423] Figure 30 An example of the configuration of a general-purpose computer is shown. The personal computer includes a central processing unit (CPU) 1001. The input-output interface 1005 is connected to the CPU 1001 via the bus 1004. The read-only memory (ROM) 1002 and the random access memory (RAM) 1003 are connected to the bus 1004.

[0424] The input unit 1006, the output unit 1007, the storage unit 1008, and the communication unit 1009 are connected to the input-output interface 1005. The input unit 1006 includes input devices such as a keyboard and a mouse, and the user inputs operation commands through the input devices. The output unit 1007 outputs images of the processing operation screen and the processing result to the display device. The storage unit 1008 includes a hard disk drive and the like and stores programs and various data. The communication unit 1009 includes a local area network (LAN) adapter and the like and performs communication processing via a network represented by the Internet. In addition, a drive 1010 for reading data from and writing data to a removable storage medium 1011 such as a magnetic disk (including a floppy disk), an optical disk (including a compact disc read-only memory (CD-ROM), a digital versatile disc (DVD)), a magneto-optical disk (including a mini disc (MD)), or a semiconductor memory is connected.

[0425] The CPU 1001 executes various processes according to a program stored in the ROM 1002 or a program read from a removable storage medium 1011 (such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory) installed in the storage unit 1008 and loaded from the storage unit 1008 to the RAM 1003. The RAM 1003 also appropriately stores data and the like required for the CPU 1001 to execute various processes.

[0426] In the computer configured as described above, for example, the CPU 1001 loads the program stored in the storage unit 1008 into the RAM 1003 via the input-output interface 1005 and the bus 1004 and executes the program, thereby executing the above-described series of processes.

[0427] For example, a program executed by a computer (CPU 1001) can be provided by being recorded on a removable storage medium 1011 such as an encapsulated medium or the like. In addition, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0428] In a computer, by installing the removable storage medium 1011 into the drive 1010, the program can be installed in the storage unit 1008 via the input-output interface 1005. In addition, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. In addition, the program can be pre-installed in the ROM 1002 or the storage unit 1008.

[0429] It should be noted that the program executed by the computer can be a program for performing in time series in the order described in this specification, or a program for parallel processing or for processing at a necessary timing such as when making a call.

[0430] It should be noted that in Figure 30 the CPU 1001 in Figure 10 implements the functions of the control unit 101 in Figure 25 and the control units 351, 381 in

[0431] In addition, in this specification, a system means a group of multiple components (devices, modules (components), etc.), and it does not matter whether all the components are in the same housing. Therefore, multiple devices accommodated in separate housings and connected via a network and a single device in which multiple modules are accommodated in one housing are both systems.

[0432] It should be noted that the embodiments of the present disclosure are not limited to the above embodiments, and various modifications are possible without departing from the scope of the present disclosure.

[0433] For example, the present disclosure can have a cloud computing configuration, in which a function is shared by multiple devices via a network and processing is cooperatively performed.

[0434] In addition, each step described in the above flowchart can be executed by one device, or can be executed by multiple devices in a shared manner.

[0435] In addition, in the case where a step includes multiple processes, in addition to being executed by one device, the multiple processes included in one step can also be executed by multiple devices in a shared manner.

[0436] It should be noted that the present disclosure can also have the following configuration.

[0437] <1> An information processing apparatus, comprising:

[0438] The control unit calibrates the inertial measurement unit in a stationary attitude based on the stationary attitude measurement values that are measurement values detected by the inertial measurement unit and information related to the arrangement of the inertial measurement unit.

[0439] <2>The information processing device according to <1>, wherein

[0440] The control unit calibrates the inertial measurement unit based on the stationary attitude measurement values and information on the latitude and absolute direction of the inertial measurement unit on the Earth.

[0441] <3>The information processing device according to <2>, wherein

[0442] The control unit calibrates the inertial measurement unit by calculating, as the auto-rotation component in the sensor coordinate system with the inertial measurement unit as the reference, the component related to the Earth's auto-rotation in the angular velocity of the stationary attitude measurement values based on the information on the latitude and absolute direction.

[0443] <4>The information processing device according to <3>, wherein

[0444] The control unit calibrates the inertial measurement unit by calculating the gravity direction based on the acceleration information in the stationary attitude measurement values and calculating the auto-rotation component of the sensor coordinate system based on the gravity direction and the absolute direction.

[0445] <5>The information processing device according to <4>, wherein

[0446] The control unit sets the inverse matrix of the attitude rotation matrix for converting the attitude information in the global coordinate system into the attitude information in the sensor coordinate system with the inertial measurement unit as the reference based on the gravity direction and the absolute direction, and calibrates the inertial measurement unit by converting the auto-rotation component of the global coordinate system based on the information on the latitude and absolute direction into the auto-rotation component of the sensor coordinate system through the inverse matrix.

[0447] <6>The information processing device according to <5>, wherein

[0448] The control unit calibrates the inertial measurement unit by calculating the deviation component of the sensor coordinate system from the stationary attitude measurement values based on the stationary attitude measurement values and the auto-rotation component of the sensor coordinate system.

[0449] <7>The information processing device according to <6>, wherein

[0450] The control unit calibrates the inertial measurement unit by calculating the deviation component of the sensor coordinate system by subtracting the auto-rotation component of the sensor coordinate system from the average value of the angular velocity in the stationary attitude measurement values.

[0451] <8>The information processing device according to <7>, wherein

[0452] The control unit corrects the measured values measured by the inertial measurement unit based on the automatic rotation component and the deviation component obtained through calibration.

[0453] <9>The information processing apparatus according to <8>, wherein

[0454] The control unit corrects the measured values by subtracting the automatic rotation component and the deviation component from the measured values measured by the inertial measurement unit.

[0455] <10>The information processing apparatus according to <9>, wherein

[0456] The control unit updates the attitude rotation matrix based on the angular velocity of the measured values corrected by subtracting the automatic rotation component and the deviation component, and converts the acceleration of the measured values into the acceleration in the global coordinate system through the updated attitude rotation matrix.

[0457] <11>The information processing apparatus according to <10>, wherein

[0458] The inertial measurement unit is built in a pen-shaped device that moves according to the shape desired by the user; and

[0459] The control unit obtains the position information of the trajectory of the tip of the pen-shaped device based on the acceleration in the global coordinate system, and displays the drawn image corresponding to the trajectory on the display device based on the position information of the trajectory.

[0460] <12>The information processing apparatus according to <11>, further comprising a pen holder, the pen holder including a hole through which the pen-shaped device can be fixed by inserting the pen-shaped device in a predetermined direction, wherein

[0461] A stationary attitude is the attitude in a state where the pen-shaped device is inserted and fixed to the pen holder.

[0462] <13>The information processing apparatus according to <12>, wherein

[0463] The cross-sectional shape of the pen-shaped device and the cross-sectional shape of the hole of the pen holder match each other in a predetermined direction; and

[0464] A stationary attitude is the attitude in a state where the pen-shaped device is inserted in a state where the cross-sectional shape of the pen-shaped device and the cross-sectional shape of the hole of the pen holder match each other.

[0465] <14>The information processing apparatus according to <12> or <13>, wherein

[0466] The pen-shaped device includes a light-emitting unit, and the light emission of the light-emitting unit is controlled by the control unit; and

[0467] When the automatic rotation component and the deviation component are obtained by calibration in a stationary posture in which the pen-shaped device is inserted into the pen container, the control unit causes the light-emitting unit to emit light.

[0468] <15>The information processing apparatus according to <14>, wherein,

[0469] After calibration obtains the automatic rotation component and the deviation component and causes the light-emitting unit to emit light, the pen-shaped device is taken out of the pen container and moved according to the shape desired by the user, and when position information of the tip locus is obtained based on the acceleration in the global coordinate system, when a predetermined time has elapsed since the pen-shaped device was taken out of the pen container, the control unit turns off the light-emitting unit.

[0470] <16>The information processing apparatus according to <15>, wherein,

[0471] When the pen-shaped device is taken out of the pen container and moved according to the shape desired by the user to obtain position information of the locus of the tip of the pen based on the acceleration in the global coordinate system, as time elapses since the pen was taken out of the pen container, the control unit causes an image corresponding to the drawing of the locus displayed on the display device to be displayed in a gradually fading color based on the position information of the locus.

[0472] <17>The information processing apparatus according to <15>, wherein,

[0473] The pen container includes an inertial measurement unit that detects the absolute azimuth angle of the pen container and provides the absolute azimuth angle to the pen-shaped device; and

[0474] The control unit acquires the absolute azimuth angle of the pen container and calibrates the inertial measurement unit based on the absolute azimuth angle of the pen container, the latitude information on the earth of the inertial measurement unit, and the stationary posture measurement value.

[0475] <18>The information processing apparatus according to <17>, wherein,

[0476] The control unit acquires the differential azimuth information between the pen container and the control unit itself, acquires the absolute azimuth angle of the pen container, obtains the absolute azimuth angle of the control unit itself from the absolute azimuth angle of the pen container and the differential azimuth, and calibrates the inertial measurement unit based on the absolute azimuth angle of the control unit itself, the latitude information on the earth of the inertial measurement unit, and the stationary posture measurement value.

[0477] <19>An information processing method, comprising:

[0478] In a stationary posture, calibrate the inertial measurement unit based on the stationary posture measurement value that is a measurement value detected by the inertial measurement unit and information related to the arrangement of the inertial measurement unit.

[0479] <20>A program for causing a computer to function as the following components:

[0480] The control unit calibrates the inertial measurement unit in a stationary posture based on the stationary posture measurement values, which are the measurement values detected by the inertial measurement unit, and information related to the arrangement of the inertial measurement unit.

[0481] List of reference numerals

[0482] 11 Pen-type drawing system

[0483] 31, 31', 31", 31''' Pen

[0484] 32, 32', 32", 32''' Pen holder

[0485] 32a, 32a', 32a", 32a''' Hole

[0486] 33 PC

[0487] 34 Display device

[0488] 51 Pressure-sensitive sensor

[0489] 52 Light-emitting unit

[0490] 61 Substrate

[0491] 62 Battery

[0492] 63 Multi-IMU (Inertial Measurement Unit)

[0493] 63a to 63d IMU

[0494] 81 Liquid

[0495] 82, 83 O-ring

[0496] 84 Lid

[0497] 84 Recess

[0498] 85 Outer peripheral portion

[0499] 86 Inner diameter portion

[0500] 101 Control unit

[0501] 102 Acquisition unit

[0502] 103 Communication unit

[0503] 131 Inertial navigation processing unit

[0504] 132 Light-emitting control unit

[0505] 133 Pressure-sensitive sensor processing unit

[0506] 134 External I / F

[0507] 141 Control IC

[0508] 142 LED

[0509] 151 Position and Attitude Processing Unit

[0510] 152 Initial Attitude Estimation Unit

[0511] 153 Stationary Determination Unit

[0512] 171 Attitude Estimation Unit

[0513] 172 Automatic Rotation Estimation Unit

[0514] 173 Acceleration Integration Unit

[0515] 181 Initial Gravity Estimation Unit

[0516] 182 Deviation Estimation Unit

[0517] 201 Pen Holder

[0518] 211 Switch

[0519] 212 Light Emitting Unit

[0520] 212a to 212c LED

[0521] 301, 301' Light Emitting Unit

[0522] 331 Pen

[0523] 331a Pressure Sensitive Sensor

[0524] 351 Control Unit

[0525] 352 Acquisition Unit

[0526] 353 Communication Unit

[0527] 354 Multi-IMU

[0528] 355 Light Emitting Unit

[0529] 356 Pressure Sensitive Sensor

[0530] 381 Control Unit

[0531] 382 Communication Unit

[0532] 383 Multi-IMU

[0533] 411 Position and Attitude Estimation Unit

[0534] 412 Initial Attitude Estimation Unit

[0535] 413 Stationary Determination Unit

[0536] 431 Attitude Estimation Unit

[0537] 432 Acceleration Integration Unit

[0538] 451 Initial Gravity Estimation Unit

[0539] 452 Absolute Azimuth Angle Deviation Estimation Unit.

Claims

1. An information processing apparatus, comprising: a control unit that, in a stationary posture, calculates a gravity direction based on acceleration information in a stationary posture measurement value, where the stationary posture measurement value is a measurement value detected by an inertial measurement unit; calculates a component related to the automatic rotation of the Earth in the angular velocity of the stationary posture measurement value based on the gravity direction, the latitude of the inertial measurement unit on the Earth, and information on an absolute direction, the component related to the automatic rotation of the Earth being an automatic rotation component of a sensor coordinate system with the inertial measurement unit as a reference; obtains a deviation component of the sensor coordinate system based on the automatic rotation component; and calibrates the inertial measurement unit based on the automatic rotation component and the deviation component.

2. The information processing apparatus according to claim 1, wherein the control unit sets an inverse matrix of an attitude rotation matrix for converting attitude information in a global coordinate system into attitude information in a sensor coordinate system with the inertial measurement unit as a reference based on the gravity direction and the absolute direction, and calibrates the inertial measurement unit by converting the automatic rotation component of the global coordinate system based on the information on the latitude and the absolute direction into the automatic rotation component of the sensor coordinate system through the inverse matrix.

3. The information processing apparatus according to claim 2, wherein the control unit calibrates the inertial measurement unit by calculating the deviation component of the sensor coordinate system from the stationary posture measurement value based on the stationary posture measurement value and the automatic rotation component of the sensor coordinate system.

4. The information processing apparatus according to claim 3, wherein the control unit calibrates the inertial measurement unit by calculating the deviation component of the sensor coordinate system by subtracting the automatic rotation component of the sensor coordinate system from an average value of the angular velocity in the stationary posture measurement value.

5. The information processing apparatus according to claim 4, wherein the control unit corrects a measurement value measured by the inertial measurement unit based on the automatic rotation component and the deviation component obtained through the calibration.

6. The information processing apparatus according to claim 5, wherein the control unit corrects the measurement value by subtracting the automatic rotation component and the deviation component from the measurement value measured by the inertial measurement unit.

7. The information processing apparatus according to claim 6, wherein the control unit updates the attitude rotation matrix based on the angular velocity of the measurement value corrected by subtracting the automatic rotation component and the deviation component, and converts the acceleration of the measurement value into the acceleration of the global coordinate system through the updated attitude rotation matrix.

8. The information processing apparatus according to claim 7, wherein the inertial measurement unit is built in a pen-shaped device that moves in a shape desired by a user; and the control unit obtains position information of a trajectory of a nib of the pen-shaped device based on the acceleration of the global coordinate system, and displays a drawn image corresponding to the trajectory on a display device that displays an image.

9. The information processing apparatus according to claim 8, further comprising a pen holder, the pen holder including a hole, the hole being capable of fixing the pen-shaped device by inserting the pen-shaped device into the hole in a predetermined direction, wherein, the one stationary posture is the posture in a state where the pen-shaped device is inserted and fixed to the pen holder.

10. The information processing apparatus according to claim 9, wherein, a cross-sectional shape of the pen-shaped device and a cross-sectional shape of the hole of the pen holder match each other in a predetermined direction; and the one stationary posture is the posture in a state where the pen-shaped device is inserted in a state where the cross-sectional shape of the pen-shaped device matches the cross-sectional shape of the hole of the pen holder.

11. The information processing apparatus according to claim 9, wherein, the pen-shaped device includes a light-emitting unit, and emission of the light-emitting unit is controlled by the control unit; and when the automatic rotation component and the deviation component are obtained by calibration in a stationary posture where the pen-shaped device is inserted into the pen holder, the control unit causes the light-emitting unit to emit light.

12. The information processing apparatus according to claim 11, wherein, after obtaining the automatic rotation component and the deviation component by calibration and causing the light-emitting unit to emit light, when the pen-shaped device is taken out of the pen holder and moved according to a shape desired by a user to obtain position information of a tip trajectory of the pen-shaped device based on an acceleration in the global coordinate system, the control unit turns off the light-emitting unit when a predetermined time has elapsed since the pen-shaped device was taken out of the pen holder.

13. The information processing apparatus according to claim 12, wherein, when the pen-shaped device is taken out of the pen holder and moved according to a shape desired by the user to obtain position information of a trajectory of a tip of the pen-shaped device based on an acceleration in the global coordinate system, the control unit causes a drawing image corresponding to the trajectory displayed on the display device to be displayed in a gradually fading color as time elapses since the pen-shaped device was taken out of the pen holder.

14. The information processing apparatus according to claim 12, wherein, the pen holder includes the inertial measurement unit that detects an absolute azimuth angle of the pen holder and provides the absolute azimuth angle to the pen-shaped device; and the control unit acquires the absolute azimuth angle of the pen holder and calibrates the inertial measurement unit based on the absolute azimuth angle of the pen holder, latitude information of the inertial measurement unit on the earth, and a stationary posture measurement value.

15. The information processing apparatus according to claim 14, wherein, the control unit acquires differential azimuth angle information between the pen holder and the control unit itself, acquires the absolute azimuth angle of the pen holder, obtains the absolute azimuth angle of the control unit itself from the absolute azimuth angle of the pen holder and the differential azimuth angle, and calibrates the inertial measurement unit based on the absolute azimuth angle of the control unit itself, the latitude information of the inertial measurement unit on the earth, and a stationary posture measurement value.

16. An information processing method, comprising: In a stationary attitude, calculate the direction of gravity based on the acceleration information in the stationary attitude measurement values, where the stationary attitude measurement values are the measurement values detected by an inertial measurement unit; based on the direction of gravity, the latitude of the inertial measurement unit on the earth, and the information of the absolute direction, calculate the component related to the automatic rotation of the earth in the angular velocity of the stationary attitude measurement values, and the component related to the automatic rotation of the earth is used as the automatic rotation component of the sensor coordinate system with the inertial measurement unit as the reference; obtain the deviation component of the sensor coordinate system based on the automatic rotation component; and calibrate the inertial measurement unit based on the automatic rotation component and the deviation component.

17. A computer-readable storage medium, on which a program is stored for causing a computer to function as the following components: A control unit, in a stationary attitude, calculates a gravity direction according to acceleration information in the stationary attitude measurement values, wherein, The stationary attitude measurement values are the measurement values detected by an inertial measurement unit; Based on the direction of gravity, the latitude of the inertial measurement unit on the earth, and the information of the absolute direction, calculate the component related to the automatic rotation of the earth in the angular velocity of the stationary attitude measurement values, and the component related to the automatic rotation of the earth is used as the automatic rotation component of the sensor coordinate system with the inertial measurement unit as the reference; Obtain the deviation component of the sensor coordinate system based on the automatic rotation component; Calibrate the inertial measurement unit based on the automatic rotation component and the deviation component.

Images