Electronic device, behavior estimation method, and program

Abstract

To prevent the accuracy of behavior estimation from decreasing even when a function of vibrating a vibrator is provided.SOLUTION: Electronic equipment 100 includes a detection unit 130, a vibration unit 140, and a control unit 110 that executes a behavior estimation process to estimate the user's behavior during a target period on the basis of the detected value acquired by the detection unit 130, and outputs a behavior estimation result. When the behavior estimation result during the target period including the vibration period in which the vibration unit 140 vibrates is output, the control unit 110 outputs a behavior estimation result estimated on the basis of the detected value other than the detected values detected during the vibration period.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004] ,

[0006] , , , ,

[0005] , , , ,

[0001] The present invention relates to an electronic device, a behavior estimation method, and a program.

Background Art

[0002] There is a known technique for estimating a user's behavior based on the position, speed, acceleration, etc. detected by a portable device or a wearable device. For example, Patent Document 1 discloses a technique for detecting a user's motion and the surrounding environment by a sensor unit including an acceleration sensor, a gyro sensor, a magnetic sensor, and a pressure sensor, and estimating the user's behavior. [[ID=)]

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] [[ID=)]Such behavior estimation techniques are often installed in portable devices such as smartphones and smartwatches and wearable devices. However, such devices may vibrate a built-in vibrator when there is a phone call or when an alarm function is executed. In that case, the acceleration sensor and the gyro sensor may be affected by the vibration of the vibrator, and it may become impossible to detect the accurate motion of the user, and there is a risk that the accuracy of behavior estimation may decrease.

[0005] In view of the above circumstances, the present invention has been made, and an object thereof is to provide an electronic device, a behavior estimation method, and a program that can prevent a decrease in the accuracy of behavior estimation even when having a function of vibrating a vibrator.

Means for Solving the Problems

[0006] To achieve the above object, one aspect of the electronic device according to the present invention is Detection unit, The vibrating part, Based on the detection values ​​acquired by the detection unit, the control unit performs an action estimation process to estimate the user's actions during the target period and outputs the action estimation results. Equipped with, When the control unit outputs the behavior estimation result for the target period which includes the vibration period during which the vibrating part vibrated, it outputs the behavior estimation result estimated based on the detected values ​​other than the detected values ​​detected during the vibration period. death , When the vibrating part is vibrated repeatedly in a short-period operation with a predetermined pause in between, the frequency of acquiring the detected value is increased while the short-period operation is being performed. The behavior estimation process is also performed using the detected values ​​obtained during the predetermined pause time. It is characterized by the following: [Effects of the Invention]

[0007] According to the present invention, even if the device is equipped with a function to vibrate the vibrator, it is possible to prevent a decrease in the accuracy of behavior estimation. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of the functional configuration of an electronic device according to Embodiment 1. [Figure 2] This is an example of a screen displayed on the display unit when behavior estimation is performed using the electronic device according to Embodiment 1. [Figure 3] This diagram illustrates a case where incorrect action estimation occurs due to vibration in the vibrating part. [Figure 4] This diagram illustrates the process used in Embodiment 1 to prevent incorrect behavior estimation even when the vibrating part vibrates. [Figure 5] This diagram illustrates a case where incorrect behavior estimation does not occur even when the vibrating part vibrates. [Figure 6] This is a flowchart of the vibration-aware estimation process according to Embodiment 1. [Figure 7] This diagram illustrates the process used in Embodiment 2 to prevent incorrect action estimation even when the vibrating part vibrates. [Figure 8]It is a flowchart of the vibration consideration estimation process according to Embodiment 2. [Figure 9] It is a diagram for explaining a process for preventing incorrect action estimation even when the vibrating part vibrates in Embodiment 3. [Figure 10] It is a diagram for explaining a process for preventing incorrect action estimation even when the vibrating part vibrates in Embodiment 4. [Figure 11] It is a flowchart of the vibration consideration estimation process according to Embodiment 4.

Mode for Carrying Out the Invention

[0009] An electronic device or the like according to an embodiment will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

[0010] (Embodiment 1) The electronic device according to Embodiment 1 is a smartwatch equipped with a function for estimating a user's action. This electronic device includes various sensors (detection units) such as an acceleration sensor and a gyro sensor, and by inputting detection values such as acceleration and angular velocity detected by these sensors into an action estimation algorithm and executing an action estimation process, the result of action estimation can be obtained. The action estimation algorithm estimates the user's action using, for example, a machine learning model that obtains an action estimation result from a plurality of detection values. Further, this electronic device includes a vibrator (vibrating part), and has a function of vibrating the vibrator when a call or message arrives or when the alarm time comes. Note that this electronic device is not limited to a smartwatch, and any electronic device (for example, a smartphone, a wearable device, a tablet, etc.) may be used as long as it has a function for estimating a user's action and a function of vibrating a vibrator.

[0011] As shown in FIG. 1, the electronic device 100 according to Embodiment 1 includes a control unit 110, a storage unit 120, a detection unit 130, a vibration unit 140, a display unit 150, and an operation unit 160.

[0012] The control unit 110 is composed of a processor such as a CPU (Central Processing Unit). The control unit 110 executes processes for realizing various functions of the electronic device 100, processes for estimating user actions, vibration consideration estimation processes described later, etc. according to the programs stored in the storage unit 120.

[0013] The storage unit 120 stores programs executed by the control unit 110 and necessary data. The storage unit 120 may include a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, etc., but is not limited thereto. Note that the storage unit 120 may be provided inside the control unit 110.

[0014] The detection unit 130 includes devices for detecting data used when estimating user actions, such as an acceleration sensor and a gyro sensor. The control unit 110 can acquire the values detected by each device included in the detection unit 130 as detection values at an arbitrary timing.

[0015] The acceleration sensor detects the acceleration in each of the three orthogonal axes (X-axis, Y-axis, Z-axis) of the electronic device 100. The value of the acceleration detected by the acceleration sensor is three-dimensional data consisting of the values of the acceleration in the X-axis direction, the acceleration in the Y-axis direction, and the acceleration in the Z-axis direction. Also, the gyro sensor detects the angular velocity of rotation with each of the three orthogonal axes (X-axis, Y-axis, Z-axis) of the electronic device 100 as the rotation axis. The value of the angular velocity detected by the gyro sensor is three-dimensional data consisting of the values of the angular velocity of X-axis rotation, the angular velocity of Y-axis rotation, and the angular velocity of Z-axis rotation.

[0016] Also, the detection unit 130 may include devices other than the acceleration sensor and the gyro sensor. For example, the detection unit 130 may include a microphone for detecting ambient sound, a temperature sensor for measuring ambient temperature, a pressure sensor for measuring ambient pressure, a GPS (Global Positioning System) device for acquiring the current position (e.g., three-dimensional data of latitude, longitude, and altitude), etc.

[0017] Therefore, the devices provided by the detection unit 130 include devices that are affected by vibrations of the electronic device 100 (where vibrations cause the detected value to change and errors to occur), such as acceleration sensors, gyro sensors, and microphones, and devices that are not affected by vibrations of the electronic device 100 (where vibrations do not change the detected value).

[0018] If it is necessary to distinguish between detection units 130 that are affected by vibrations from the electronic device 100 and those that are not, the unit affected by vibrations from the electronic device 100 is called the dynamic detection unit, and the unit not affected by vibrations is called the static detection unit. In other words, the dynamic detection unit is equipped with devices that are affected by vibrations, such as an acceleration sensor, a gyroscope, and a microphone, while the static detection unit is equipped with devices that are not affected by vibrations, such as a temperature sensor, a pressure sensor, and a GPS device.

[0019] The vibrating unit 140 is a so-called vibrator, a device that vibrates the electronic device 100. The vibrating unit 140 vibrates the main body of the electronic device 100 by, for example, rotating an object whose center of gravity is located offset from the axis of rotation.

[0020] The display unit 150 is equipped with a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays a display screen or operation screen that provides the functions of the electronic device 100.

[0021] The operation unit 160 is a user interface such as a push-button switch or a touch panel, and accepts user input. If the operation unit 160 is equipped with a touch panel, it may be a touch panel integrated with the display of the display unit 150.

[0022] Typically, the electronic device 100 is worn on the user's wrist, and the control unit 110 estimates the user's actions based on each detected value detected by the detection unit 130. Any algorithm can be used for the process of estimating the user's actions (action estimation process), but in this embodiment 1, a machine learning model is used that has been trained to output an action estimation result when various detected values ​​from the detection unit 130 at multiple timings are input.

[0023] For example, suppose that at a certain time, the detection unit 130 obtains 10 detection values ​​(e.g., a 3D acceleration value from an accelerometer, a 3D angular velocity value from a gyroscope, a 3D position value from a GPS device, and a pressure value from a barometric pressure sensor). Then, this machine learning model is such that, for example, if it takes the detection values ​​from the last 5 timings (50 data points in total) as input, it outputs an action estimation result (e.g., standing still, walking, running, riding a bicycle or other vehicle). However, the number of detection values ​​obtained at each timing, the number of data points input to the machine learning model, and the number of types of action estimation results can be set to any number.

[0024] Furthermore, the electronic device 100 is equipped with a clock alarm function, and when the user specifies a time, the control unit 110 executes a vibration process by vibrating the vibration unit 140, causing the main body of the electronic device 100 to vibrate and notify the user that the specified time has arrived. In addition, the electronic device 100 is equipped with a notification function that notifies the user of incoming phone calls, emails, SNS (Social Networking Service) messages, etc. When the control unit 110 receives such a notification, it executes a vibration process by vibrating the vibration unit 140, causing the main body of the electronic device 100 to vibrate and notify the user of the incoming notification.

[0025] When a user puts on the electronic device 100 and activates the behavior estimation function, the control unit 110 performs behavior estimation processing to estimate the user's behavior during the target period based on the detection values ​​acquired by the detection unit 130. Here, the target period is the period that is the target for estimating the user's behavior. The control unit 110 can continuously estimate the user's behavior by performing behavior estimation processing every target period (for example, every 5 seconds). Then, as shown in Figure 2, for example, the display unit 150 displays the results of the user's behavior estimation performed by the control unit 110 at intervals of the display cycle (for example, every 5 seconds). The example in Figure 2 shows the behavior estimation result when the user initially remains still for about 15 seconds, then starts walking, and then starts running about 5 seconds after starting to walk.

[0026] In this example, as shown in Figure 2, the control unit 110 correctly estimates the user's behavior. However, when the vibration unit 140 vibrates, the detection unit 130 is affected by the vibration caused by the vibration unit 140. As a result, even if the user is stationary, the detection unit 130 will detect acceleration etc. due to the vibration of the vibration unit 140. Therefore, if the detection values ​​from the detection unit 130 are used directly for behavior estimation, the estimation result may be incorrect (for example, the system may estimate that the user is walking when they are stationary).

[0027] For example, even if the user takes action as shown in Figure 2, if the vibration unit 140 (vibrator) vibrates between 0:07 and 0:08 due to an alarm or incoming call, the control unit 110 may, as shown in Figure 3, assume that the action during that display cycle period (0:05 to 0:10) was "walking".

[0028] Therefore, in this embodiment 1, as shown in Figure 4, the control unit 110 does not perform the action estimation process based on the detected value during the period when the vibration unit 140 is vibrating (vibration period), and uses the action estimation result from the previous period for the action estimation result during that vibration period. In this way, the action estimation result during the vibration period will be the same as the previous estimation result (in the example of Figure 4, "still"). Consequently, as shown in Figure 5, even when the vibration unit 140 (vibrator) is operating, the control unit 110 can display the same correct estimation result as in Figure 2. Note that the control unit 110 may output the action estimation result during the vibration period in a different manner (shading, different color, etc.) than during periods other than the vibration period (periods when the vibration unit 140 is not vibrating), as shown in Figure 5.

[0029] Note that Figure 4 shows what appears to be an action estimation algorithm where detection values ​​acquired at six consecutive timings are input; however, the number of detection values ​​input to the action estimation algorithm is arbitrary. Typically, the number of input detection values ​​matches the number of inputs used to train the machine learning model.

[0030] A process (vibration-aware estimation process according to Embodiment 1) that enables the control unit 110 to display the correct behavior estimation result even when the vibration unit 140 vibrates will be explained with reference to Figure 6. This vibration-aware estimation process is activated periodically (for example, once every 0.1 seconds) when the user activates the behavior estimation function of the electronic device 100. The result of behavior estimation by the vibration-aware estimation process is recorded at a specific address in the memory of the storage unit 120. An application program that wants to refer to the behavior estimation result can obtain the behavior estimation result by referring to this specific address. However, when the electronic device 100 is started, the counter variable used in this vibration-aware estimation process is initialized to 0, the flag variable is initialized to False, and the buffer that stores the most recent detected value input to the behavior estimation algorithm is also cleared. In addition, a variable (here referred to as a discriminant variable; the value is "non-vibration period" or "vibration period") may be prepared to distinguish between behavior estimation results during periods when the vibration unit 140 is not vibrating (non-vibration period) and behavior estimation results during vibration periods. In this case, the discriminant variable is initialized to "non-vibration period" when the electronic device 100 is started.

[0031] First, the control unit 110 acquires the detected value from the detection unit 130 (step S101) and adds 1 to the counter variable (step S102).

[0032] Then, the control unit 110 determines whether the flag variable is True or not (step S103). If the flag variable is True (step S103; Yes), the process proceeds to step S107.

[0033] If the flag variable is False (step S103; No), the control unit 110 determines whether the vibrating unit 140 is vibrating or not (step S104). Since the control unit 110 controls the vibration of the vibrating unit 140, the control unit 110 can easily make this determination.

[0034] If the vibrating unit 140 is vibrating (step S104; Yes), the control unit 110 sets the flag variable to True (step S106) and proceeds to step S107. If the vibrating unit 140 is not vibrating (step S104; No), the control unit 110 saves the detected value obtained in step S101 to a buffer (step S105) and proceeds to step S107.

[0035] In step S107, the control unit 110 determines whether the counter variable has reached a predetermined number of times (for example, 10 times). This predetermined number is basically the number of input values ​​of the detection value required by the action estimation algorithm, and the period during which the control unit 110 performs action estimation is determined by this predetermined number.

[0036] If the counter variable has not reached the specified number of times (step S107; No), the control unit 110 terminates the vibration-aware estimation process.

[0037] If the counter variable has reached a specified number of times (step S107; Yes), the control unit 110 determines whether the flag variable is True or not (step S108). If the flag variable is True (step S108; Yes), the control unit 110 adopts the action estimation result previously stored in memory in step S110 as the current action estimation result (step S111) and proceeds to step S112. If a discriminant variable is provided, the control unit 110 substitutes "oscillation period" for the discriminant variable in step S111.

[0038] If the flag variable is False (step S108; No), the control unit 110 performs behavior estimation processing using the detected values ​​stored in the buffer for a specified number of times up to that point (step S109). Then, the control unit 110 saves the result of the behavior estimation processing to memory (step S110) and proceeds to step S112. If a discriminant variable is provided, the control unit 110 substitutes "non-oscillating period" into the discriminant variable in step S110.

[0039] In step S112, the control unit 110 initializes the counter variable and flag variable (sets the counter variable to 0 and the flag variable to False), clears the buffer, and terminates the vibration-aware estimation process.

[0040] After the vibration-aware estimation process is completed, the control unit 110 will restart the vibration-aware estimation process at the next startup timing (for example, 0.1 seconds later). In Figure 6 and the above description, the control unit 110 performs the process of acquiring the detected value from the detection unit 130 in step S101, but the acquisition of the detected value does not necessarily have to be done in step S101. For example, the control unit 110 may omit the process in step S101, acquire the detected value if the determination in step S104 is No, and save it to the buffer in step S105. By doing so, the control unit 110 does not need to acquire the detected value from the detection unit 130 while the vibration unit 140 is vibrating.

[0041] Through the vibration-aware estimation process described above, when the vibration unit 140 vibrates, the control unit 110 does not perform the behavior estimation process based on the detected values ​​during that period, but instead adopts the result of the most recent behavior estimation process as the current behavior estimation result. In other words, if the period targeted for behavior estimation includes the vibration period, the control unit 110 outputs a behavior estimation result estimated based on detected values ​​other than those detected during the vibration period, thereby preventing a decrease in the accuracy of behavior estimation due to vibration of the vibration unit 140.

[0042] Furthermore, in Embodiment 1, compared to Embodiment 2 described later, the behavior estimation results can be output at a fixed interval, which has the advantage of not adversely affecting other processes of the electronic device 100 (for example, processes where problems occur if the interval is disrupted).

[0043] (Embodiment 2) In the above-described Embodiment 1, when the vibration unit 140 vibrated, the control unit 110 did not perform action estimation processing on the detected values ​​during that period, but instead adopted the result of the most recent action estimation processing as the action estimation result for that period, thereby enabling the output of action estimation results at a predetermined period (for example, a 5-second period). However, even if only one detected value was affected by the vibration of the vibration unit 140 within the period, the action estimation processing for that period would not be performed, which sometimes resulted in the waste of detected values ​​that were not affected by the vibration. Therefore, Embodiment 2, which uses detected values ​​that are not affected by vibration for action estimation without wasting them, will be described.

[0044] The functional configuration of the electronic device 100 according to Embodiment 2 is the same as that of the electronic device 100 according to Embodiment 1, as shown in Figure 1. However, in Embodiment 2, as shown in Figure 7, the control unit 110 ignores (or does not acquire) the detected values ​​while the vibration unit 140 is vibrating, and acquires additional detected values ​​to compensate for the ignored values ​​before executing the action estimation process. In this way, the action estimation process can be executed using only the detected values ​​when the vibration unit 140 is not vibrating. However, because the detected values ​​that would have been acquired while the vibration unit 140 was vibrating must now be acquired after the vibration unit 140 has stopped vibrating, the period in which the execution result of the action estimation process is output will be disrupted (shifted slightly backward), as shown in Figure 7.

[0045] The vibration-aware estimation process according to Embodiment 2 will be described with reference to Figure 8. Similar to the vibration-aware estimation process according to Embodiment 1, this vibration-aware estimation process is activated periodically (for example, once every 0.1 seconds) when the user activates the behavior estimation function. Also, similar to Embodiment 1, when the electronic device 100 is activated, the counter variable is initialized to 0, the flag variable to False, and the buffer that stores the most recent detected value input to the behavior estimation algorithm is also cleared. Furthermore, if a discriminant variable is used, the discriminant variable is also initialized to the "non-vibration period".

[0046] First, the control unit 110 acquires the detected value from the detection unit 130 (step S201). Then, the control unit 110 determines whether the flag variable is True or not (step S202).

[0047] If the flag variable is False (step S202; No), the control unit 110 determines whether the vibrating unit 140 is vibrating or not (step S203). If the vibrating unit 140 is vibrating (step S203; Yes), the control unit 110 sets the flag variable to True (step 204) and proceeds to step S209.

[0048] If the vibrating unit 140 is not vibrating (step S203; No), the control unit 110 stores the detected value acquired in step S201 in a buffer (step S205), adds 1 to the counter (step S206), and proceeds to step S209.

[0049] On the other hand, if the flag variable is True in step S202 (step S202; Yes), the control unit 110 determines whether the vibration unit 140 is stopped (not vibrating) or not (step S207). If the vibration unit 140 is vibrating (step S207; No), the process proceeds to step S209. If the vibration unit 140 is stopped (step S207; Yes), the control unit 110 initializes the flag variable to False (step S208) and proceeds to step S205.

[0050] The processing in steps S209 to S214 is the same as the processing in steps S107 to S112 of the vibration-considering estimation process in Embodiment 1 (Figure 6), so the explanation is omitted. In Embodiment 2, as in Embodiment 1, the processing in step S201 may be omitted for the process in which the control unit 110 acquires the detected value from the detection unit 130, and the detected value may be acquired from the detection unit 130 immediately before step S205 and saved in the buffer in step S205.

[0051] Through the vibration-aware estimation process described above, the control unit 110 performs the behavior estimation process using the detected values ​​when the vibration unit 140 is not vibrating, rather than using the detected values ​​while the vibration unit 140 is vibrating. In other words, if the period for behavior estimation includes the vibration period, the control unit 110 outputs the behavior estimation result estimated based on detected values ​​other than those detected during the vibration period, thereby preventing a decrease in the accuracy of behavior estimation due to the vibration of the vibration unit 140.

[0052] In Embodiment 2, unlike Embodiment 1, the period for outputting the behavior estimation results may be disrupted, but it has the advantage that once the vibration of the vibration unit 140 stops, the results of the behavior estimation process can be output as quickly as possible afterward.

[0053] (Embodiment 3) The electronic device 100 may have variations in the way the vibrating unit 140 vibrates so that the user can understand what kind of notification it is based on the type of vibration. For example, the following variations in vibration are possible.

[0054] - Vibration during alarm When the alarm time is reached, the device will vibrate once for a relatively long period (for example, 1 second) and repeat this action every 3 seconds. (This type of operation, where the pause time between vibrations is relatively long (2 seconds in this example), will be referred to as a "long-period operation.")

[0055] - Vibration when an email is received When an email is received, the device will vibrate only once for an extended period (for example, 2 seconds). (Hereinafter referred to as "single action.")

[0056] - Vibration when receiving a phone call When a phone call is received, the device vibrates three times in a row for a predetermined duration (for example, vibrate for 0.2 seconds, pause for 0.2 seconds, vibrate for 0.2 seconds, pause for 0.2 seconds, vibrate for 0.2 seconds), repeating this sequence every two seconds. (Such a short pause between vibrations (0.2 seconds in this example) will be referred to as a "short-period operation.")

[0057] Of the operations described above, a long-period operation refers to a periodic operation with a long pause time (2 seconds in the above example) during which the action estimation process can be executed at least once in each of the embodiments described above. A short-period operation refers to a periodic operation with a short pause time (0.2 seconds in the above example) during which the action estimation process cannot be executed even once in each of the embodiments described above. In other words, if the short-period threshold is defined as the minimum time required to execute the action estimation process (the time required to obtain the number of detection values ​​required for the action estimation algorithm), then a periodic operation with a pause time less than the short-period threshold is a short-period operation, and a periodic operation with a pause time greater than or equal to the short-period threshold is a long-period operation.

[0058] In these operations, the pause time in long-period operations is long, so in each of the embodiments described above, the control unit 110 can perform action estimation processing during this pause time. Also, in single-shot operations, after the single vibration has finished, the control unit 110 can perform action estimation processing without any problems in each of the embodiments described above.

[0059] In short-period operation, the number of detected values ​​that can be acquired during a single pause is limited due to the short pause time. However, since multiple short pauses occur periodically during short-period operation, the control unit 110 can collect the detected values ​​that can be acquired during these short pauses (a pause of 0.2 seconds in the above example), thereby enabling it to effectively utilize the detected values ​​during short-period operation to perform action estimation processing. This embodiment 3 will now be described.

[0060] The functional configuration of the electronic device 100 according to Embodiment 3 is the same as that of the electronic device 100 according to Embodiment 1, as shown in Figure 1. However, in Embodiment 3, as shown in Figure 9, when the control unit 110 causes the vibration unit 140 to perform short-period operation, the frequency of acquiring detected values ​​from the detection unit 130 is increased. The extent to which the acquisition frequency is increased is arbitrary, but for example, if it is increased according to the duty cycle ratio of the vibration time and rest time of the vibration unit 140 during short-period operation (for example, if the vibration time is a and the rest time is b, then (a+b) / b), it becomes possible to perform the behavior estimation process at the same period as when the vibration unit 140 is not vibrating. In the example shown in Figure 9, since the ratio of vibration time and rest time is 1:1, the control unit 110 is able to perform the behavior estimation process at the same period as when the vibration unit 140 is not vibrating by doubling the frequency of acquiring detected values.

[0061] The vibration-aware estimation process according to Embodiment 3 is the same as the vibration-aware estimation process according to Embodiment 2 (Figure 8). However, in Embodiment 2, the period during which the vibration-aware estimation process was periodically activated (activation period) was constant (for example, 0.1 seconds), whereas in Embodiment 3, the control unit 110 increases the frequency of acquiring detected values ​​by shortening the activation period of the vibration-aware estimation process while the vibration unit 140 is operating at a short period (to increase the frequency by n times, the activation period is reduced to 1 / n). For example, in the example shown in Figure 9, the activation period is halved in order to double the frequency of acquiring detected values.

[0062] By controlling the startup cycle and executing the vibration-aware estimation process in this way, the control unit 110 can save the detected values ​​for a specified number of times when the vibration unit 140 is not vibrating during the specified cycle. This prevents the accuracy of the behavior estimation from decreasing due to the vibration of the vibration unit 140, and allows the behavior estimation results to be output at the same cycle. In Embodiment 3 as well, if the period for behavior estimation includes a vibration period, the control unit 110 will output the behavior estimation result estimated based on detected values ​​other than those detected during the vibration period (detected values ​​when the vibration unit 140 is not vibrating).

[0063] (Embodiment 4) In Embodiment 1, when the vibrating unit 140 vibrates, the control unit 110 does not perform action estimation based on the detected values ​​during that period, but instead adopts the most recent action estimation result to prevent a decrease in the accuracy of action estimation. However, in cases where real-time action estimation is not required (or where a slight delay is acceptable), the immediately following action estimation result may be adopted instead of the most recent one. Embodiment 4, which describes this, will be explained.

[0064] The functional configuration of the electronic device 100 according to Embodiment 4 is the same as that of the electronic device 100 according to Embodiment 1, as shown in Figure 1. However, in Embodiment 4, as shown in Figure 10, the control unit 110 does not perform the behavior estimation processing based on the detected value while the vibration unit 140 is vibrating, and uses the behavior estimation result immediately afterward for the behavior estimation result during that period. As a result, there is a time lag until the behavior estimation result is obtained (a delay of the amount of time from when the vibration of the vibration unit 140 ends until the execution result of the behavior estimation processing is obtained). However, if the user wearing the electronic device 100 is the type of user who will immediately change their behavior due to the vibration of the vibration unit 140, this type of processing can better prevent a decrease in the accuracy of behavior estimation.

[0065] The vibration-aware estimation process according to Embodiment 4 is also activated periodically (for example, once every 0.1 seconds) when the user activates the behavior estimation function, similar to the vibration-aware estimation process according to Embodiment 1. In this vibration-aware estimation process, the behavior estimation results for each period are stored in the memory corresponding to that period. For example, the behavior estimation results for the period from 0:00 to 0:05 are stored at address 0, the behavior estimation results for the period from 0:05 to 0:10 are stored at address 1, and so on, with each period being stored at a different address in memory.

[0066] Furthermore, similar to Embodiment 1, when the electronic device 100 is started, the counter variable is initialized to 0 and the flag variable to False, and the buffer that stores the most recent detected value input to the action estimation algorithm is also cleared. In addition, as a buffer unique to Embodiment 4, a display standby buffer is provided to store the memory address that stores the action estimation results for the period when real-time action estimation was not possible (the period when the vibration unit 140 was vibrating), and this display standby buffer is also cleared when the electronic device 100 is started.

[0067] The vibration-aware estimation process according to Embodiment 4 will be described with reference to Figure 11. Of this process, steps S301 to S310 and step S313 are the same as steps S101 to S110 and step S112 of the vibration-aware estimation process of Embodiment 1 (Figure 6), so their explanation will be omitted.

[0068] In step S311, the control unit 110 stores the address of the memory where the action estimation results for that period are stored in the display wait buffer. Then, in step S312, the control unit 110 also stores the action estimation results in the addresses stored in the display wait buffer and clears the display wait buffer. If the vibration continues for a long period of time, multiple addresses may be stored in the display wait buffer. In that case, in step S312, the control unit 110 stores the action estimation results in all of those multiple addresses. Furthermore, if it is necessary to distinguish between the action estimation results during periods when the vibration unit 140 is not vibrating (non-vibration period) and the action estimation results during the vibration period, the control unit 110 also stores information indicating that they are the action estimation results during the vibration period in each address stored in the display wait buffer in step S312.

[0069] In the vibration-aware estimation process according to Embodiment 4, if the vibration unit 140 is vibrating during the period in which the behavior estimation is performed, the control unit 110 does not save the behavior estimation result to memory at that time, but instead saves the behavior estimation result after the vibration stops to the memory address corresponding to each period. In other words, if the period to be estimated for behavior estimation includes the vibration period, the control unit 110 outputs the behavior estimation result estimated based on detection values ​​other than the detection values ​​detected during the vibration period (detection values ​​when the vibration unit 140 is not vibrating), thereby preventing the accuracy of behavior estimation from decreasing due to the vibration of the vibration unit 140.

[0070] (Variation 1) The embodiments described above may be combined. For example, Embodiment 1 and Embodiment 4 may be combined, and during the period when the vibrating unit 140 is vibrating, the behavior estimation result may be output only when the immediately preceding behavior estimation result matches the immediately following behavior estimation result. If they do not match, both may be displayed as the behavior estimation result (for example, "Stationary or Walking") or it may be indicated that the estimation could not be performed (for example, "Unknown").

[0071] Furthermore, if the estimated behavior result immediately preceding and immediately following differs, the estimated behavior results from before and after those may also be obtained, and the final estimated behavior result may be determined by a majority vote of all obtained behavior estimate results.

[0072] By combining multiple embodiments in this way, it is possible to further prevent a decrease in the accuracy of behavior estimation due to vibration of the vibrating unit 140.

[0073] (Modification 2) In the above-described embodiment, the control unit 110 uses a machine learning model that has been trained to output an action estimation result when it receives a detection value from the detection unit 130. However, as mentioned above, the detection unit 130 can be divided into a dynamic detection unit that is affected by the vibration of the vibration unit 140 and a static detection unit that is not affected by the vibration. If action estimation can be performed using only the detection value from the static detection unit, then action estimation can be performed without any problems even if the vibration unit 140 is vibrating.

[0074] Therefore, as machine learning models used for behavior estimation, a full data model is prepared that is trained to estimate behavior by inputting detection values ​​from all devices provided by the detection unit 130, and a limited data model is prepared that is trained to estimate behavior by inputting detection values ​​only from the devices provided by the static detection unit. The control unit 110 normally uses the full data model to estimate behavior, but may use the limited data model to estimate behavior while the vibration unit 140 is vibrating.

[0075] By performing behavior estimation using a limited data model, it is possible to utilize the detected values ​​when the vibration unit 140 is vibrating, thereby further preventing a decrease in the accuracy of behavior estimation due to vibration of the vibration unit 140.

[0076] (Variation 3) As a machine learning model used for behavior estimation, a vibration data model is prepared that is trained to estimate behavior by inputting detected values ​​from the detection unit 130 while the vibration unit 140 is vibrating. The control unit 110 normally performs behavior estimation using the entire data model, but may perform behavior estimation using only the vibration data model while the vibration unit 140 is vibrating.

[0077] By performing behavior estimation using a vibration data model, behavior estimation can be performed with a certain degree of accuracy even when the vibration unit 140 is vibrating, thus further preventing a decrease in the accuracy of behavior estimation due to vibration of the vibration unit 140.

[0078] (Modification 4) Alternatively, the detection values ​​from the detection unit 130 while the vibration unit 140 is vibrating may be acquired in advance as correction data, and the control unit 110 may correct the detection values ​​from the detection unit 130 with the correction data while the vibration unit 140 is vibrating before inputting them into the machine learning model to perform behavior estimation.

[0079] By correcting the detected values ​​with correction data, it is possible to estimate the behavior with a certain degree of accuracy even when the vibrating part 140 is vibrating, thereby further preventing a decrease in the accuracy of behavior estimation due to the vibration of the vibrating part 140.

[0080] (Variation 5) Furthermore, Modification 1 may be combined with Modifications 2 to 4. That is, while the vibration unit 140 is vibrating, the control unit 110 may combine the action estimation results according to each of the above embodiments with the action estimation results from the limited data model or vibration data model, or the action estimation results after correction with correction data, and output a final action estimation result. This makes it possible to further prevent a decrease in the accuracy of action estimation due to the vibration of the vibration unit 140.

[0081] (Other variations) The electronic device 100 can also be implemented using a computer such as a tablet or PC equipped with a detection unit 130 and a vibration unit 140. Specifically, in the above embodiment, it was described that the program for vibration-aware estimation processing executed by the control unit 110 is pre-stored in the storage unit 120. However, the program may be stored and distributed on a non-temporary computer-readable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), MO (Magneto-Optical disc), memory card, or USB memory, and a computer capable of executing the above-mentioned processes may be configured by loading and installing the program into the computer.

[0082] Furthermore, the program can be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program could be posted and distributed on a bulletin board system (BBS) on a communication network. This program could then be launched and executed under the control of the operating system (OS), just like any other application program, to perform the aforementioned processes.

[0083] Furthermore, the control unit 110 may consist of any single processor, such as a single processor, multi-processor, or multi-core processor, or it may be configured by combining any of these processors with processing circuits such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

[0084] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these specific embodiments, and the present invention includes the invention described in the claims and its equivalents. The invention described in the original claims of this application is listed below.

[0085] (Note 1) Detection unit, The vibrating part, Based on the detection values ​​acquired by the detection unit, the control unit performs an action estimation process to estimate the user's actions during the target period and outputs the action estimation results. Equipped with, When the control unit outputs the behavior estimation result for the target period which includes the vibration period during which the vibrating part vibrated, it outputs the behavior estimation result estimated based on the detected values ​​other than the detected values ​​detected during the vibration period. An electronic device characterized by the following features.

[0086] (Note 2) The control unit, When performing vibration processing to vibrate the vibrating part, the behavior estimation result before or after the vibration processing is output as the behavior estimation result for the target period including the vibration period. The electronic device described in Appendix 1, characterized by the features described herein.

[0087] (Note 3) The control unit, When performing the vibration process that vibrates the vibrating part, the detection value detected during the vibration period is not used, but rather the detection value after the vibration of the vibrating part has stopped is used to perform the action estimation process. The electronic device described in Appendix 1, characterized by the features described herein.

[0088] (Note 4) The control unit, When the vibrating part is vibrated repeatedly in a short-period operation with a predetermined pause in between, the frequency of acquiring the detected value is increased while the short-period operation is being performed. The behavior estimation process is also performed using the detected values ​​obtained during the predetermined pause time. The electronic device described in Appendix 1, characterized by the features described herein.

[0089] (Note 5) The control unit, The frequency of acquiring the detected value is increased according to the duty cycle ratio of the predetermined rest time during the short-period operation and the vibration time of other predetermined times. The electronic device described in Appendix 4, characterized by the features described herein.

[0090] (Note 6) The control unit, The behavior estimation results during the aforementioned vibration period are output in a different manner than the behavior estimation results during periods other than the aforementioned vibration period. An electronic device as described in any one of the notes 1 through 5.

[0091] (Note 7) The control unit of an electronic device comprising a detection unit, a vibration unit, and a control unit, Based on the detection values ​​acquired by the detection unit, the unit performs an action estimation process to estimate the user's behavior during the target period and outputs the action estimation results. When outputting behavior estimation results for the target period including the period during which the vibrating part vibrated, the output will be behavior estimation results estimated based on the detected values ​​other than the detected values ​​detected during the vibration period. Our method for estimating behavior.

[0092] (Note 8) In the control unit of an electronic device comprising a detection unit, a vibration unit, and a control unit, Based on the detection values ​​acquired by the detection unit, the unit performs an action estimation process to estimate the user's behavior during the target period and outputs the action estimation results. When outputting behavior estimation results for the target period including the period during which the vibrating part vibrated, the output will be behavior estimation results estimated based on the detected values ​​other than the detected values ​​detected during the vibration period. A program characterized by executing a process. [Explanation of Symbols]

[0093] 100...Electronic equipment, 110...Control unit, 120...Storage unit, 130...Detection unit, 140...Vibration unit, 150...Display unit, 160...Operation unit

Claims

1. Detection unit, The vibrating part, Based on the detection values ​​acquired by the detection unit, the control unit performs an action estimation process to estimate the user's actions during the target period and outputs the action estimation results. Equipped with, When the control unit outputs the behavior estimation result for the target period which includes the vibration period during which the vibrating part vibrated, it outputs the behavior estimation result estimated based on the detected values ​​other than the detected values ​​detected during the vibration period. When the vibrating part is vibrated repeatedly in a short-period operation with a predetermined pause in between, the frequency of acquiring the detected value is increased while the short-period operation is being performed. The behavior estimation process is also performed using the detected values ​​obtained during the predetermined pause time. An electronic device characterized by the following features.

2. A detection unit, The vibrating part, Based on the detection values ​​acquired by the detection unit, the control unit performs an action estimation process to estimate the user's actions during the target period and outputs the action estimation results. Equipped with, When the control unit outputs the behavior estimation result for the target period which includes the vibration period during which the vibrating part vibrated, it outputs the behavior estimation result estimated based on the detected values ​​other than the detected values ​​detected during the vibration period. The behavior estimation results during the aforementioned vibration period are output in a different manner than the behavior estimation results during periods other than the aforementioned vibration period. An electronic device characterized by the following features.

3. The control unit, When performing vibration processing to vibrate the vibrating part, the behavior estimation result before or after the vibration processing is output as the behavior estimation result for the target period including the vibration period. The electronic device according to feature 1.

4. The control unit is When performing vibration processing to vibrate the vibrating part, the behavior estimation result before or after the vibration processing is output as the behavior estimation result for the target period including the vibration period. The electronic device according to feature 2.

5. The control unit, When performing the vibration process that vibrates the vibrating part, the detection value detected during the vibration period is not used, but rather the detection value after the vibration of the vibrating part has stopped is used to perform the action estimation process. The electronic device according to feature 1.

6. The control unit is When performing the vibration process that vibrates the vibrating part, the detection value detected during the vibration period is not used, but rather the detection value after the vibration of the vibrating part has stopped is used to perform the action estimation process. The electronic device according to feature 2.

7. The control unit, The frequency of acquiring the detected value is increased according to the duty cycle ratio of the predetermined rest time during the short-period operation and the vibration time of other predetermined times. The electronic device according to feature 1.

8. The control unit, The behavior estimation results during the aforementioned vibration period are output in a different manner than the behavior estimation results during periods other than the aforementioned vibration period. The electronic device according to any one of claims 1, 3, 5, or 7.

9. The control unit of an electronic device comprising a detection unit, a vibration unit, and a control unit, Based on the detection values ​​acquired by the detection unit, the unit performs an action estimation process to estimate the user's behavior during the target period and outputs the action estimation results. When outputting the behavior estimation results for the target period including the vibration period during which the vibrating part vibrated, the behavior estimation results estimated based on the detected values ​​other than the detected values ​​detected during the vibration period are output. When the vibrating part is vibrated repeatedly in a short-period operation with a predetermined pause in between, the frequency of acquiring the detected value is increased while the short-period operation is being performed. The behavior estimation process is also performed using the detected values ​​obtained during the predetermined pause time. Our method for estimating behavior.

10. The control unit of an electronic device comprising a detection unit, a vibration unit and a control unit, Based on the detection values ​​acquired by the detection unit, the unit performs an action estimation process to estimate the user's behavior during the target period and outputs the action estimation results. When outputting the behavior estimation results for the target period including the vibration period during which the vibrating part vibrated, the behavior estimation results estimated based on the detected values ​​other than the detected values ​​detected during the vibration period are output. The behavior estimation results during the aforementioned vibration period are output in a different manner than the behavior estimation results during periods other than the aforementioned vibration period. Our method for estimating behavior.

11. In the control unit of an electronic device comprising a detection unit, a vibration unit, and a control unit, Based on the detection values ​​acquired by the detection unit, the unit performs an action estimation process to estimate the user's behavior during the target period and outputs the action estimation results. When outputting the behavior estimation results for the target period including the vibration period during which the vibrating part vibrated, the behavior estimation results estimated based on the detected values ​​other than the detected values ​​detected during the vibration period are output. When the vibrating part is vibrated repeatedly in a short-period operation with a predetermined pause in between, the frequency of acquiring the detected value is increased while the short-period operation is being performed. The behavior estimation process is also performed using the detected values ​​obtained during the predetermined pause time. A program characterized by executing a process.

12. The control unit of an electronic device comprising a detection unit, a vibration unit and a control unit, Based on the detection values ​​acquired by the detection unit, the unit performs an action estimation process to estimate the user's behavior during the target period and outputs the action estimation results. When outputting the behavior estimation results for the target period including the vibration period during which the vibrating part vibrated, the behavior estimation results estimated based on the detected values ​​other than the detected values ​​detected during the vibration period are output. The behavior estimation results during the aforementioned vibration period are output in a different manner than the behavior estimation results during periods other than the aforementioned vibration period. A program characterized by executing a process.

Images