Electronic apparatus and control method of the same

Abstract

To provide an electronic apparatus capable of turning on / off a reporting operation, according to the state of a user.SOLUTION: An electronic apparatus comprises: image acquisition means for acquiring an image of a subject person; posture estimation means for estimating the posture of the subject person from the image; posture reporting means for performing reporting corresponding to posture information; vital data acquisition means for acquiring vital data on the subject person; and reporting interval control means for changing a reporting interval of the reporting of the posture. The reporting interval control means changes the reporting interval of the posture reporting means in accordance with the vital data when the posture information does not meet a predetermined condition.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an electronic device and a control method thereof, and particularly to a technique for notifying regarding a user's posture.

Background Art

[0002] There is a technique for determining whether a user's posture deviates from an ideal posture. Patent Document 1 discloses that, when a user takes a picture while in a predetermined posture, it is determined whether the user's posture is the target posture regardless of the positional relationship between the user and the electronic device, and a notification is made.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the method of Patent Document 1, even when a user intentionally changes their posture during rest or the like, it is determined whether the posture is the ideal posture, and a posture notification is made.

[0005] In view of the above problems, an object of the present invention is to provide an electronic device capable of turning on / off a notification operation according to a user's state.

Means for Solving the Problems

[0006] In order to achieve the above object, the electronic device of the present invention includes an image acquisition means for acquiring an image of a person as a subject, a posture estimation means for estimating the posture of the person as a subject from the image, a posture notification means for performing a notification according to the posture information, a vital data acquisition means for acquiring vital data of the person as a subject, and a notification interval control means for changing the notification interval of the posture notification. The notification interval control means is characterized in that when the posture information does not satisfy a predetermined condition, the notification interval of the posture notification means is changed according to the vital data.

Advantages of the Invention

[0007] According to the present invention, it is possible to turn ON / OFF the notification operation according to the user.

Brief Description of the Drawings

[0008]

Fig. 1a

Fig. 1b

Fig. 1c

Fig. 1d

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6a

Fig. 6b

Fig. 6c

Fig. 7a

Fig. 7b

Fig. 8

Best Mode for Carrying Out the Invention

[0009] [First Embodiment] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0010] FIGS. 1a, 1b, and 1c are external views of a notification robot 101 as an example of the imaging control device of the present invention.

[0011] FIG. 1a is a front perspective view, FIG. 1b is a left side view, and FIG. 1c is a rear perspective view.

[0012] In FIG. 1a, the exterior 105 of the notification robot 101 is composed of a resin molded product having a substantially spherical shape at the lower part.

[0013] The distance measuring sensor 102 is provided at the upper part of the front surface of the notification robot 101, and measures the distance from the notification robot 101 to an arbitrary part of the user.

[0014] In FIG. 1b, the distance measuring sensor 102 is configured to have an elevation angle α so that it can photograph the upper body of the user when the notification robot 101 is placed on a desk and used. Here, α is desirably 20° to 50°.

[0015] The light notification unit 103 is a semi-transmissive part of the main body 101, and transmits the light of the internal light source so that it can be visually recognized by the user.

[0016] In FIG. 1c, the power button 104 is provided on the back surface of the main body 101, and by pressing it by the user, the power of the notification robot 100 can be switched on / off.

[0017] FIG. 1d is a top view for explaining the installation position during the use of the notification robot 101.

[0018] The size of the notification robot 101 is approximately 6 cm in diameter and approximately 8 cm in height, and it can be installed at a free position on the desk 106. For example, when there is a monitor of a PC (personal computer) on the desk 106, it can be placed in front of the user and below the monitor (position P), or when using a notebook PC or when it cannot be placed in the front, it can be installed diagonally forward (position Q or position R).

[0019] FIG. 2 is a block diagram showing a configuration example of the notification robot 101 in the present embodiment.

[0020] The lens 201 is a lens group composed of a plurality of lenses, but here it is simply shown as a single lens for simplicity. The light condensed by the lens 201 is guided to the distance measurement sensor 102. The distance measurement sensor 102 is an element that converts an optical image into an electrical signal, and transfers the converted electrical signal (data) to the system control unit 202. The data transferred includes distance information from the distance measurement sensor 102 to the user 211. Also, the timing at which the distance measurement sensor 102 converts an optical image into an electrical signal, etc. is controlled by the system control unit 202.

[0021] The non-volatile memory 203 is an electrically erasable and recordable memory, and for example, a Flash-ROM or the like is used. In the non-volatile memory 203, constants, programs, etc. for the operation of the system control unit 202 are stored. The program here refers to a program for executing various flowcharts described later in the present embodiment.

[0022] The system control unit 202 is a control unit composed of at least one processor or circuit, and controls the entire notification robot 101. By executing the program recorded in the aforementioned non-volatile memory 203, each process of the present embodiment described below is realized. For the system memory 204, for example, RAM is used, and constants, variables, programs read from the non-volatile memory 203, etc. for the operation of the system control unit 202 are expanded. Based on the data obtained from the distance measurement sensor 102, the system control unit 202 performs human detection and extracts distance information to characteristic points of a person such as a face, right shoulder, left shoulder, and chest. Further, the system control unit 202 performs calculations using the extracted distance information and determines the posture information of the user 211. At this time, the extracted distance information and the determined posture information are recorded in the non-volatile memory 203.

[0023] The system timer 205 is a timing unit that measures the time used for various controls and the time of the built-in clock.

[0024] The power button 104 is an operation member for switching the power of the notification robot 101 on and off. The power control unit 206 is composed of a DCDC converter, a switch circuit for switching the energized block, etc. The power control unit 206 controls the DCDC converter based on the instruction of the system control unit 202 and supplies the necessary voltage to the appropriate place in the notification robot 101 for the necessary period. The power supply unit 207 consists of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, etc.

[0025] The communication I / F unit 208 is an interface that connects to an external device via a wireless or wired cable and performs data transmission and reception.

[0026] The light notification unit 103 is composed of a light-emitting element such as an LED controlled by the system control unit 202. Based on the determined attitude information, the system control unit 202 controls the light notification unit 103 to notify the user 211 of the attitude determination result. Alternatively, when the system control unit 202 is acquiring the reference attitude of the user 211 (during calibration), it controls the light notification unit 103 to notify the user 211.

[0027] The vibration unit 209 is composed of an actuator or the like, is a mechanism for physically vibrating the notification robot 101, and is controlled by the vibration control unit 210.

[0028] The vibration control unit 210 is composed of a motor driver or the like and is controlled by the system control unit 202. By controlling the presence / absence, timing, and amount of energization to the vibration unit 209, the vibration control unit 210 enables the notification robot 101 to vibrate with an arbitrary amplitude and an arbitrary period.

[0029] The above is the description of the block diagram in the embodiment of the present invention, but it is not limited to this.

[0030] Next, the notification process in this embodiment will be described with reference to the flowchart of FIG. 3. This process is realized by expanding the program recorded in the non-volatile memory 203 into the system memory 204 and executed by the system control unit 202. Note that this process starts when the power button 104 of the notification robot 101 is pressed to turn on the power and the distance measurement sensor 102 becomes operable, and the distance measurement sensor starts shooting.

[0031] In S301, the system control unit 202 starts shooting by the distance measurement sensor 102.

[0032] In S302, the system control unit 202 determines whether the reference attitude data is recorded in the non-volatile memory 203. If the reference attitude data is recorded, the process proceeds to S304; otherwise, the process proceeds to S303.

[0033] In S303, the system control unit 202 acquires reference posture data. The acquisition of the reference posture data will be described later using the flowchart of the reference data acquisition process in FIG. 4. Here, the reference posture data is the posture taken by the user after notifying the user to take a good posture, and in the processes described later, it is determined whether the current posture of the user is a good posture based on the reference posture data. By acquiring the reference posture data, for a user who originally has the habit of shrugging their shoulders, it is possible to determine whether they are shrugging their shoulders more than their original habit. If the reference posture data is used to determine whether the posture is good or bad without acquiring the reference posture data from the target user, the possibility of frequent notifications in the following cases can be reduced: when the original user has a habit, or when the user is in a posture that is convenient for work.

[0034] In S304, the system control unit 202 acquires a captured image using the distance measurement sensor 102.

[0035] In S305, the system control unit 202 detects a person from the captured image acquired in the immediately preceding S304. If multiple people are detected, the information of the person closest or largest in the image is used in S308 described later.

[0036] In S306, the system control unit 202 determines whether a person has been detected in the captured image acquired in S305. If it is determined that a person has been detected, the process proceeds to S307; otherwise, the processes from S304 to S306 are repeated.

[0037] In S307, the system control unit 202 sets a specified value Th1_d to a threshold Th1 of the posture notification time, which is used to determine whether the state where the posture of the subject satisfies specific conditions has continued for a certain period of time.

[0038] In S308, the system control unit 202 acquires the distances of the head, both shoulders, and chest of the subject detected in S306. The distances of the head, both shoulders, and chest of the subject will be described with reference to FIGS. 6a to 6c. FIGS. 6a to 6c are top views of the notification robot 101 and the subject 600. Let the actual position of the notification robot 101 in FIG. 6a be 101_01, and the position when it is virtually facing forward be 101_02. The distance of the head is the distance 614 from the notification robot 101_01 to the head 603 of the subject. The shoulder distances are the distance 610 from the notification robot 101_01 to the left shoulder 605 and the distance 611 from the notification robot 101_01 to the right shoulder 606. The chest distance is the distance 615 from the notification robot 101_01 to the chest 604.

[0039] In S309, the system control unit 202 normalizes the distances of the face, both shoulders, and chest of the subject acquired in S308 to the posture when the notification robot 101 is directly in front of the user's face, which is regarded as the front. It is converted into the data when the notification robot 101 is placed at a specific position in the direction perpendicular to the line connecting both shoulders 605 and 606. Let the specific position be 101_02 in FIG. 6a. The head distance is the distance 616 from the virtual notification robot 101_01 position to the head 603 of the subject. The head distance is converted into the distance 616 from the virtual notification robot 101_01 position to the head 603 of the subject.

[0040] Here, a specific conversion method for normalization will be described with reference to FIG. 6b. It is necessary to perform corrections regarding distance and angle, and correction of the rotation of the subject. First, the correction regarding distance will be described.

[0041] The virtual subject 630 in FIG. 6b is an image obtained by imaging the subject 600 with the virtual notification robot 101_02 in FIG. 6a, but it is an image without performing correction on the left shoulder 605 and the right shoulder 606. The virtual notification robot 101_03 in FIG. 6b is a device obtained by virtually overlapping the notification robot 101 in FIG. 6a and the virtual notification robot 101_02 in FIG. 6a.

[0042] In Fig. 6b, the distance 614 to the head 603 of the subject 600 and the distance 616 to the head 633 of the subject 630 form an angle 631. Taking the direction in which the imaging plane of the virtual notification robot 101_03 is orthogonal to the head 633 of the subject 630 as the z-axis and the direction orthogonal to the z-axis as the x-axis, with the position of the virtual notification robot 101_03 as the origin, if the z-coordinate of the left shoulder 606 is coordinate 606_z and the x-coordinate is coordinate 606_x, then the z-coordinate 636_z and the x-coordinate 636_x of the left shoulder 636 of the virtual subject 630 are (A) coordinate 636_z = (distance 616 / distance 614) × coordinate 606_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 606_x × sin(angle 631), (B) coordinate 636_x = (distance 616 / distance 614) × coordinate 606_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 606_x × cos(angle 631), and can be obtained as such.

[0043] Similarly, in Fig. 6b, with the position of the virtual notification robot 101_03 as the origin, if the z-coordinate of the right shoulder 605 is coordinate 605_z and the x-coordinate is coordinate 605_x, then the z-coordinate 635_z and the x-coordinate 635_x of the left shoulder 635 of the virtual subject 630 are (C) coordinate 635_z = (distance 616 / distance 614) × coordinate 605_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 605_x × sin(angle 631), (D) coordinate 635_x = (distance 616 / distance 614) × coordinate 605_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 605_x × cos(angle 631), and can be obtained as such.

[0044] Next, corrections regarding the rotation of the subject will be described using Fig. 6c.

[0045] The virtual subject 650 is the virtual subject after performing rotation correction on the virtual subject 630.

[0046] The line segment connecting the left shoulder 655 and the right shoulder 656 of the virtual subject 650 and the line segment connecting the left shoulder 635 and the right shoulder 636 of the virtual subject 630 have an angle 651. Assuming the position of the virtual notification robot 101_03 as the origin, the z - coordinate 656_z and the x - coordinate 656_x of the left shoulder 656 of the virtual subject 650 are (E) Coordinate 656_z = Coordinate 636_z × cos(angle 651) - (Coordinate 636_x - Distance 616) × sin(angle 651), (F) Coordinate 656_x = Coordinate 636_z × sin(angle 651) + (Coordinate 636_x - Distance 616) × cos(angle 651)+Distance 616, and can be obtained by these formulas.

[0047] Similarly, when the position of the virtual notification robot 101_03 in Fig. 6c is taken as the origin, the z - coordinate 655_z and the x - coordinate 656_x of the left shoulder 655 of the virtual subject 650 are (G) Coordinate 655_z = Coordinate 636_z × cos(angle 651) - (Coordinate 635_x - Distance 616) × sin(angle 651), (H) Coordinate 655_x = Coordinate 636_z × sin(angle 651) + (Coordinate 635_x - Distance 616) × cos(angle 651)+Distance 616, and can be obtained by these formulas.

[0048] The order of the correction of the distance and angle and the correction of rotation described above does not matter. Also, the above example is a two - dimensional correction when viewed from the top surface, and the correction is performed using a two - dimensional rotation matrix and distance. When correction is required for the height of the subject, correction using a three - dimensional rotation matrix and distance with the addition of height - direction correction to the above formula is required.

[0049] In S310, based on the values normalized in S309, the system control unit 202 calculates the difference value Zh between the head and the chest and the difference value Zc between the midpoint of the line connecting both shoulders and the chest, and records them in the system memory 204. As shown in FIG. 7a, it is the difference value Zhs between the head 711 and the chest 712 in the reference posture, and an example of the posture at the time of notification processing is the difference value Zh between the head 713 and the chest 714. The coordinate axis 701 is set such that the direction away from the notification robot 101 is positive. When Zh is smaller than Zhs (negative value), that is, when the head comes forward more than the chest, it is determined that the posture is a forward tilt posture. When Zh is larger than Zhs, it is determined that the posture is a backward tilt (slumping) posture. As shown in FIG. 7b, it is the difference value Zcs between the midpoint 715 of the line connecting both shoulders in the reference posture and the chest 716, and an example at the time of notification processing is the difference value Zc between the midpoint 717 of the line connecting both shoulders and the chest 718. When Zc is smaller than Zcs (negative value), it is determined that the posture is a hunched shoulder posture.

[0050] In S311, the system control unit 202 compares the difference value Zhs between the head and the chest in the reference posture obtained in S303, the difference value Zcs between the midpoint of the line connecting both shoulders and the chest, and Zh and Zc obtained in S310, respectively. That is, it compares Zhs with Zh and Zcs with Zc.

[0051] In S312, the system control unit 202 determines whether either (I) Zh n - Zh s < x1 or (II) Zh n - Zh s > x2 holds, or neither holds. That is, in (I), it is possible to determine whether it is a forward leaning posture. Note that x1 is a value such as -6 cm, -8 cm, -10 cm. In (II), it is possible to determine whether it is a backward leaning posture (slouching). Note that x2 is a value such as 2 cm, 4 cm, 6 cm. If it is determined that either (I) or (II) holds, the process proceeds to S313. When the state where either (I) or (II) holds continues for a threshold value Th1 or more of the posture notification time, the process moves to the processing flow for performing posture notification. Otherwise, the process proceeds to S317. Note that x1 and x2 may be arbitrarily set by the user, or may be automatically changed according to the input user profile (age, gender, build). For example, when the shoulder width is detected and the shoulder width is large, x1 and x2 are changed to larger values. Alternatively, when the gender is female, x1 and x2 are changed to smaller values.

[0052] In S313, the communication I / F unit 208 acquires the heart rate Hr1 and the heart rate variability Hrv1 of the subject measured by the external device.

[0053] In S314, the system control unit 202 determines whether either (i) the heart rate Hr1 ≤ y1 or (ii) the heart rate variability Hrv1 ≥ y2 holds, or neither holds. If (i) holds, it can be determined that the subject is in a resting state. Note that y1 is a value such as 60 bpm, 75 bpm, 90 bpm. If (ii) holds, it can be determined that the subject is in a relaxed state. Note that y2 is a value such as 700 ms, 750 ms, 800 ms. If either (i) or (ii) holds, since the subject is in a resting or relaxed state, it is determined that the subject is in a relaxed state such as during a break, and the process proceeds to S315. Otherwise, it is determined that the subject is in a tense state such as during work, and the process proceeds to S316. Note that y1 and y2 may be arbitrarily set by the user, or may be automatically changed according to the previously measured Hr1 and Hrv1.

[0054] In S315, the system control unit 202 increases the threshold Th1 of the posture notification time from the specified value Th1_d to Th1_r. Note that Th1_d and Th1_r are values such as 5 minutes, 30 minutes, and 60 minutes. Note that Th1_d and Th1_r may be arbitrarily set by the user.

[0055] In S316, the system control unit 202 measures the count T1 of the forward and backward tilt postures. T1 indicates the time during which the user's posture is in a state of leaning forward from the reference posture or in a state of slumping. If the measurement of T1 has already started, the measurement continues in S316.

[0056] In S318, the system control unit 202 determines whether the count T1 of the forward and backward tilt postures has elapsed for a time equal to or longer than the threshold Th1 of the posture notification time. If it is determined that T1 has elapsed for a time equal to or longer than Th1, the process proceeds to S319; otherwise, the process proceeds to S320.

[0057] In S319, the system control unit 202 notifies the user that the posture is in a forward and backward tilt state. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period. Note that the notification method may be changed depending on whether the state of leaning forward continues or the state of slumping continues.

[0058] In S317, the system control unit 202 resets the forward and backward tilt posture time T1 to 0. That is, if the user was temporarily in a state of leaning forward or slumping but returned to a posture close to the reference posture before the threshold Th1 of the posture notification time elapsed, T1 is reset.

[0059] In S320, the system control unit 202 determines whether (III) Zcn - Zcs < x3. That is, in (III), it is possible to determine whether it is a hunched shoulder. Note that x3 is a value such as -1.0 cm, -1.5 cm, -2.0 cm. x3 may be arbitrarily set by the user, or may be automatically changed according to the input user profile (age, gender, build). For example, when the shoulder width is detected and the shoulder width is large, x3 is changed to a large value. Or, when the gender is female, x3 is changed to a small value. Thus, since Zhn, Zhs, Zcn, and Zcs all use the z - coordinate of the chest, it is necessary to detect the position of the user's chest from the notification robot 101. Therefore, the angle 651 in FIG. 6c, that is, the angle formed by the z - axis plane of the notification robot 101 and the left and right shoulders of the user who is the subject, is desirably in the range of approximately -80° to +80°. If it is within that range, the notification robot 101 can be installed at a free position with respect to the user. If it is determined that (III) holds, the process proceeds to S321; otherwise, it proceeds to S322.

[0060] In S321, the system control unit 202 measures the count T2 of the hunched - shoulder posture. T2 indicates the time during which the user's posture is in a hunched state continuously more than the reference posture. If the measurement of T2 has already started, in S321, the measurement continues.

[0061] In S323, the system control unit 202 determines whether the count T2 of the hunched - shoulder posture has elapsed for a time equal to or more than the threshold Th1 of the posture notification time. If it is determined that T2 has elapsed for Th1 or more, the process proceeds to S324; otherwise, it proceeds to S325.

[0062] In S324, the system control unit 202 notifies the user that the posture is in a hunched - shoulder state. The method of notification is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0063] In S322, the system control unit 202 resets the shoulder-rolling posture time T2 to 0. That is, when the user was temporarily in the shoulder-rolling state but returned to a posture close to the reference posture before the threshold Th1 of the posture notification time elapsed, T2 is reset.

[0064] In S325, by setting the prescribed value Th1_d to the threshold Th1 of the posture notification time, the notification interval is returned to the prescribed state.

[0065] In S326, the system control unit 202 determines whether the user has stood up. If it is determined that the user has stood up, the process proceeds to S328; otherwise, the process proceeds to S327.

[0066] In S328, the system control unit 202 resets the seating time T3, which will be described later.

[0067] In S327, the system control unit 202 measures or continues to measure the seating time T3. The seating time T3 is the time for detecting that the user is continuously sitting. Note that the measurement of T1, T2, and T3 is performed by the system timer 205.

[0068] In S329, the system control unit 202 determines whether the seating time T3 has reached 30 minutes or more. If it is determined that T3 has reached 30 minutes or more, the process proceeds to S330; otherwise, the process proceeds to S331.

[0069] In S330, the system control unit 202 notifies the user that they have been sitting too long. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0070] Here, various notification methods of the notification robot 101 will be described with reference to FIG. 5.

[0071] In FIG. 5, the states of the notification robot 101 are set as A (correct position), B (left swing), and C (right swing).

[0072] In notification pattern 1, the state transitions as A → B → C → A → stop for 1 second → B → C → A → stop. By swaying left and right, even when the notification robot 101 is placed on the desk, the user can easily confirm the notification.

[0073] In notification pattern 2, the state transitions as A → B → C → B → C → B → C → A → stop.

[0074] In notification pattern 3, with the state remaining A, the light notification unit 103 blinks twice at 2 Hz.

[0075] In notification pattern 4, with the state remaining A, the light notification unit 103 lights up once for 2 seconds.

[0076] In notification pattern 5, with the state remaining A, the light notification unit 103 blinks three times at 1 Hz.

[0077] In notification pattern 6, with the state remaining A, the light notification unit 103 lights up brightly once for 1 second.

[0078] Notification pattern 1 is used for the posture deterioration notification S319 and the hunched shoulder notification S324 in FIGS. 3 and 8. Notification pattern 2 is used for the sitting too long notification S330.

[0079] Notification pattern 3 is used as the notification when the power button 104 is pressed and the shooting of the notification robot 101 is started (S301).

[0080] Notification pattern 4 is used as the notification when a person is detected (S305). This is also used in the same way when a person is detected again after the user stands up after being detected of sitting too long in the sitting too long detection S330 (S326).

[0081] In S331, the system control unit 202 determines whether the power of the notification robot 101 has been turned off. If it is determined that the power has been turned off, the process of FIG. 3 ends; otherwise, the process returns to S304. Note that in S331, it may be determined as Yes when the power of the PC (personal computer) used by the user is turned off, or it may be determined as Yes according to a predetermined time.

[0082] Next, with reference to FIG. 4, a flowchart for obtaining reference posture data will be described. Note that this process starts when proceeding to S303 in FIG. 3.

[0083] In S401, the system control unit 202 controls the optical notification unit 103 to notify the user to assume the reference posture. At this time, the notification is made using notification pattern 5.

[0084] In S402, it waits for a signal indicating that the user is ready. The user is determined to be ready when the power button 104 is long-pressed. The user's signal of OK may also be a gesture such as the user raising both hands as a signal of OK.

[0085] In S403, the system control unit 202 acquires the distances of the head, both shoulders, and chest as the reference posture of the detected subject. At this time, the success of acquiring the reference posture is notified using notification pattern 6.

[0086] In S404, when the notification robot 101 is in front of the user's face, the system control unit 202 normalizes the distances of the reference face, both shoulders, and chest of the subject acquired in S403 to the posture when it is in the front.

[0087] In S405, based on the values normalized in S404, the system control unit 202 calculates the difference value Zhs between the line connecting both shoulders and the head, and Zcs between the line connecting both shoulders and the chest, and records them in the volatile memory 203.

[0088] That is, the notification robot 101 can estimate whether the subject is relaxed, such as during a break, from the heart rate and heart rate variability, and change the interval of posture notification according to the estimation result.

[0089] According to the embodiment described above, when the user intentionally breaks the posture during a break or the like, it becomes possible to provide a posture notification robot that does not perform unnecessary notifications by increasing the posture notification interval.

[0090] [Second Embodiment] Next, the second embodiment will be described. In this embodiment, unlike the first embodiment, the operation interval of posture estimation is changed according to the heart rate and heart rate variability of the subject, instead of the notification interval. By changing the operation interval of posture estimation, the notification interval is also changed in conjunction. Therefore, the process of changing the threshold value of the posture notification time according to the heart rate and heart rate variability of the subject, which was performed in the first embodiment, is not performed.

[0091] The notification process in the second embodiment will be described using the flowchart of FIG. 8. Note that the same numbers are assigned to the common processes in FIGS. 3 and 8.

[0092] The processes from S301 to S307 are the same as the processes from S301 to S307 in the first embodiment, so the description is omitted.

[0093] After the process of S307, the system control unit 202 performs the processes of S313 and S314 in the first embodiment, obtains the heart rate Hr1 and heart rate variability Hrv1 of the subject, and determines whether the subject is in a relaxed state. If it is determined that the subject is in a relaxed state, the process proceeds to S332; otherwise, the process proceeds to S333.

[0094] In S332, the system control unit 202 increases the operation interval TI1 of posture estimation from the specified value TI1_d to TI1_r. Note that TI1_d and TI1_r are values such as 5 minutes, 30 minutes, and 60 minutes. Note that TI1_d and TI1_r may be arbitrarily set by the user.

[0095] In S333, the operation interval of the attitude estimation is restored to the specified state by setting the specified value TI1_d to the operation interval TI1 of the attitude estimation.

[0096] In S334, the system control unit 202 measures the count T4 of the attitude estimation time. If the measurement of T4 has already started, the measurement continues in S334. The measurement of T4 is performed by the system timer 205.

[0097] In S335, the system control unit 202 determines whether or not the count T4 of the attitude estimation time has elapsed by an amount equal to or greater than the operation interval TI1 of the attitude estimation. If it is determined that T4 has elapsed by an amount equal to or greater than TI1, the process proceeds to S336; otherwise, the process returns to S304.

[0098] In S336, the system control unit 202 resets the attitude estimation time T4.

[0099] After the process of S336, the system control unit 202 performs the same processes as S308 to S312 in the first embodiment.

[0100] After the process of S312, if the system control unit 202 determines that either (I) or (II) holds, the system control unit 202 does not perform the processes of S313, S314, and S315 in the first embodiment and proceeds to S316 in the first embodiment; otherwise, the system control unit 202 proceeds to S317 in the first embodiment.

[0101] Since the processes from S316 to S324 are the same as the processes from S316 to S324 in the first embodiment, the description thereof is omitted.

[0102] After any of the processes of S322, S323, and S324, the system control unit 202 does not perform the process of S325 in the first embodiment and proceeds to the process of S326 in the first embodiment.

[0103] For the processes from S326 to S331, since they are the same as the processes from S326 to S331 in the first embodiment, the description thereof will be omitted.

[0104] That is, the notification robot 101 can estimate whether the subject is relaxed such as during a break from the heart rate and heart rate variability, and change the intervals of posture estimation and posture notification according to the estimation result.

[0105] According to the embodiment described above, when the user unintentionally relaxes the posture during a break or the like, it becomes possible to provide a posture notification robot that does not give unnecessary notifications by increasing the operation interval of the posture estimation.

[0106] Note that the heart rate and heart rate variability acquired from an external device in S313 may be estimated from the imaging video obtained by the visible light imaging sensor or the distance measurement sensor 102.

[0107] Note that although it has been described that the user is notified by the operation of the notification robot 101 in S319, S324, and S330, the notification may be sent to the application in the user's smartphone or PC corresponding using the communication I / F unit 208. Furthermore, information regarding the user's posture may be recorded in the application, and by graphing and numericalizing it to show, it may be made easier for the user to grasp the tendency and habit of the user's posture. For example, if it is found that the notifications of the backward leaning posture increase towards Friday when Monday to Friday are working days, the user can consciously correct the posture.

[0108] Note that although the method of notification has been described as being notified by the movement or light of the notification robot 101 itself, in the case of a smartphone, the display unit may be blinked or a push notification for warning may be given.

[0109] Also, the notification robot 101 in the first and second embodiments detects the posture based on the distance measurement sensor 102, but the posture may be detected by a visible light imaging sensor that does not measure the distance. In that case, the distance information is estimated from the imaging video obtained by the imaging sensor.

[0110] In addition, the various controls described above as being performed by the system control unit 101 may be performed by one piece of hardware, or the control of the entire apparatus may be performed by a plurality of pieces of hardware sharing the processing.

[0111] Furthermore, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope without departing from the gist of the present invention are also included in the present invention. Moreover, each of the above-described embodiments merely shows one embodiment of the present invention, and it is also possible to appropriately combine the embodiments.

[0112] Also, in the above-described embodiment, the case where the present invention is applied to the notification robot 101 has been described as an example, but this is not limited to this example, and it is applicable to any electronic device capable of acquiring information regarding the positional relationship and posture between the user and the apparatus. That is, the present invention is applicable to a mobile phone terminal, a portable image viewer, a printer apparatus provided with a finder, a digital photo frame, a music player, a game machine, an electronic book reader, and the like.

[0113] (Other Embodiments) The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to a system or an apparatus via a network or various recording media, and a computer (or a CPU, MPU, etc.) of the system or apparatus reads out and executes the program code. In this case, the program and the recording medium storing the program constitute the present invention.

Description of Reference Numerals

[0114] 101 Notification robot 102 Distance measurement sensor 103 Light notification unit 104 Power button 106 Desk 201 Lens 202 System control unit 203 Non-volatile memory 204 System memory 205 System timer

Claims

1. Video acquisition means for acquiring a video of a person to be a subject, Posture estimation means for estimating the posture of the person to be a subject from the video, Posture notification means for performing notification according to the posture of the person to be a subject, Vital data acquisition means for acquiring vital data of the person to be a subject, It has notification interval control means for changing the notification interval of the posture notification, The notification interval control means changes the notification interval of the posture notification means according to the vital data when the posture information does not satisfy a predetermined condition. An electronic device characterized by that.

2. In the electronic device according to claim 1, further, The posture estimation means is characterized by changing the estimation interval of the posture. An electronic device.

3. In the electronic device according to any one of claims 1 to 2, further, The vital data acquired by the vital data acquisition means includes at least either the heart rate of the subject or the heart rate variability. An electronic device characterized by that.

4. In the electronic device according to any one of claims 1 to 3, further, The vital data acquisition means is characterized by receiving vital data from an external device. An electronic device.

5. In the electronic device according to any one of claims 1 to 4, further, The vital data acquisition means is characterized by estimating and acquiring the vital data from the video. An electronic device.

6. A video acquisition step of acquiring a video of a person to be a subject, A posture estimation step of estimating the posture of the person to be a subject from the video, A posture notification step of performing notification according to the posture information, A vital data acquisition step of acquiring vital data of the person to be a subject, It has a notification interval control step for changing the notification interval of the posture notification, The notification interval control step changes the notification interval of the posture notification step according to the vital data when the posture information does not satisfy a predetermined condition. A control method for an electronic device characterized by that.

7. A program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 5.

Images