Electronic apparatus and method for controlling the same

Abstract

To provide an electronic apparatus that notifies a user of whether a subject gets closer to a target posture.SOLUTION: An electronic apparatus has: subject video acquisition means that acquires video of a subject; posture information acquisition means that acquires posture information of the subject from a subject video at a predetermined period; reference posture holding means that holds reference posture information related to a posture targeted by the subject; posture deviation amount calculation means that calculates the amount of deviation of the posture information acquired by the posture information acquisition means from the reference posture information; posture deviation change determination means that determines the change over time of the posture deviation amount; and notification means that performs a predetermined notification to a user. When the posture deviation amount calculated by the posture deviation amount calculation means is larger than a predetermined amount, the posture deviation change determination means determines whether the current posture gets closer to a reference posture than the previous posture from the change over time of the posture deviation amount, and the notification means switches the notification according to a result of the determination.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an electronic device and a control method thereof, and particularly to a technique for notifying regarding a user's posture.

Background Art

[0002] There is a technique for determining whether a user's posture deviates from an ideal posture. Patent Document 1 discloses that by normalizing user posture information according to the positional relationship with the user, it is possible to determine whether the user's posture is the target posture regardless of the posture relationship with the user, and to notify. Further, Patent Document 2 discloses that shooting is performed while the user takes a predetermined posture, and it is determined whether the user's posture is the target posture and a notification is made.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the methods of Patent Document 1 and Patent Document 2, when it is notified that the posture is not the target posture, it may not be possible to determine whether the posture change after the notification approaches the target posture.

[0005] In view of the above problems, an object of the present invention is to provide an electronic device capable of determining a temporal change in a user's posture and notifying whether the user's posture is approaching a target posture when the posture is not the target posture.

Means for Solving the Problems

[0006] In order to achieve the above object, the electronic device of the present invention includes a subject image acquisition means for acquiring an image of a subject, a posture information acquisition means for acquiring posture information of the subject from the subject image at a predetermined cycle, a reference posture holding means for holding reference posture information regarding a target posture of the subject, a posture deviation amount calculation means for calculating a deviation amount between the posture information acquired by the posture information acquisition means and the reference posture information, a posture deviation change determination means for determining a temporal change of the posture deviation amount, and a notification means for giving a predetermined notification to a user. The posture deviation change determination means determines whether the temporal change of the posture deviation amount approaches the reference posture when the posture deviation amount is larger than a predetermined amount, and the previous notification means switches the notification according to the determination result.

Effect of the Invention

[0007] According to the present invention, since it is possible to notify a change with respect to a target posture, a user can easily confirm whether the posture is being corrected.

Brief Description of the Drawings

[0008]

Figure 1a

Figure 1b

Figure 1c

Figure 1d

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6a

Figure 6b

Figure 6c

Figure 7a

Figure 7b

Figure 8

Modes for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0010] FIGS. 1a, 1b, and 1c are external views of a notification robot 101 as an example of an imaging control device of the present invention.

[0011] FIG. 1a is a front perspective view, FIG. 1b is a left side view, and FIG. 1c is a rear perspective view.

[0012] In FIG. 1a, the exterior 105 of the notification robot 101 is composed of a resin molded product having a substantially spherical shape at the lower part.

[0013] The distance measuring sensor 102 is provided at the upper part of the front surface of the notification robot 101, and measures the distance from the notification robot 101 to an arbitrary part of the user.

[0014] In FIG. 1b, the distance measuring sensor 102 is configured to have an elevation angle α so that it can photograph the upper body of the user when the notification robot 101 is placed on a desk and used. Here, α is desirably 20° to 50°.

[0015] The light notification unit 103 is a semi-transmissive part of the main body 101, and transmits the light of the internal light source so that it can be visually recognized by the user.

[0016] In FIG. 1c, the power button 104 is provided on the back surface of the main body 101, and by the user pushing it, the power of the notification robot 100 can be switched on / off.

[0017] FIG. 1d is a top view for explaining the installation position during use of the notification robot 101.

[0018] The size of the notification robot 101 is approximately 6 cm in diameter and approximately 8 cm in height, and it can be installed at a free position on the desk 106. For example, when there is a monitor of a PC (personal computer) on the desk 106, it can be placed in front of the user and below the monitor (position P), or when using a notebook PC or when it cannot be placed in the front, it can be installed diagonally forward (position Q or position R).

[0019] FIG. 2 is a block diagram showing a configuration example of the notification robot 101 in the present embodiment.

[0020] The lens 201 is a lens group composed of a plurality of lenses, but here it is shown simply as a single lens for simplicity. The light condensed by the lens 201 is guided to the distance measurement sensor 102. The distance measurement sensor 102 is an element that converts an optical image into an electrical signal, and transfers the converted electrical signal (data) to the system control unit 202. Among the data transferred, distance information from the distance measurement sensor 102 to the user 211 is included. Also, the timing when the distance measurement sensor 102 converts an optical image into an electrical signal, etc. is controlled by the system control unit 202.

[0021] The non-volatile memory 203 is an electrically erasable and recordable memory, and for example, a Flash-ROM or the like is used. In the non-volatile memory 203, constants for the operation of the system control unit 202, programs, etc. are stored. Here, the program refers to the program for executing various flowcharts described later in this embodiment.

[0022] The system control unit 202 is a control unit composed of at least one processor or circuit, and controls the entire notification robot 101. By executing the program recorded in the aforementioned non-volatile memory 203, each process of this embodiment described later is realized. For the system memory 204, for example, RAM is used, and constants for the operation of the system control unit 202, variables, programs read from the non-volatile memory 203, etc. are expanded. The system control unit 202 performs person detection based on the data obtained from the distance measurement sensor 102, and extracts distance information to feature points of a person such as the face, right shoulder, left shoulder, chest, etc. Further, the system control unit 202 performs calculations using the extracted distance information to determine the posture information of the user 211. At this time, the extracted distance information and the determined posture information are recorded in the non-volatile memory 203.

[0023] The system timer 205 is a timing unit that measures the time used for various controls and the time of the built-in clock.

[0024] The power button 104 is an operation member for switching the power of the notification robot 101 on and off. The power control unit 206 is composed of a DCDC converter, a switch circuit for switching the energized block, etc. The power control unit 206 controls the DCDC converter based on an instruction from the system control unit 202, and supplies the necessary voltage to the appropriate place in the notification robot 101 for the necessary period. The power supply unit 207 consists of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, a Li battery, an AC adapter, etc.

[0025] The communication I / F unit 208 is an interface that connects to an external device via a wireless or wired cable and performs data transmission and reception.

[0026] The optical notification unit 103 is composed of a light-emitting element such as an LED controlled by the system control unit 202. Based on the determined attitude information, the system control unit 202 controls the optical notification unit 103 to notify the user 211 of the attitude determination result. Alternatively, when the system control unit 202 is acquiring the reference attitude (during calibration) of the user 211, it controls the optical notification unit 103 to notify the user 211.

[0027] The vibration unit 209 is composed of an actuator or the like, is a mechanism for physically vibrating the notification robot 101, and is controlled by the vibration control unit 210.

[0028] The vibration control unit 210 is composed of a motor driver or the like and is controlled by the system control unit 202. By controlling the presence / absence, timing, and amount of energization to the vibration unit 209, the vibration control unit 210 enables the notification robot 101 to vibrate with an arbitrary amplitude and an arbitrary period.

[0029] The above is the description of the block diagram in the embodiment of the present invention, but it is not limited to this.

[0030] Next, the notification process in this embodiment will be described with reference to the flowcharts of FIGS. 3 and 4. This process is realized by expanding the program recorded in the non-volatile memory 203 into the system memory 204 and executed by the system control unit 202. Note that this process starts when the power button 104 of the notification robot 101 is pressed to turn on the power and the distance measurement sensor 102 becomes operable, and the distance measurement sensor starts shooting.

[0031] In S301, the system control unit 202 starts shooting by the distance measurement sensor 102.

[0032] In S302, the system control unit 202 determines whether the reference posture data is recorded in the non-volatile memory 203. If the reference posture data is recorded, the process proceeds to S304; otherwise, it proceeds to S303.

[0033] In S303, the system control unit 202 acquires the reference posture data. The acquisition of the reference posture data will be described later using the flowchart of the reference data acquisition process in FIG. 4. Here, the reference posture data is the posture taken by the user after notifying the user to take a good posture, and in the processes described later, it is determined whether the current user's posture is a good posture based on the reference posture data. By acquiring the reference posture data, for a user who originally has the habit of shrugging their shoulders, it is possible to determine whether they are shrugging their shoulders more than before due to their original habit. If the reference posture data is used to determine whether the posture is good or bad without acquiring the reference posture data from the target user, the possibility of frequent notifications in the following processes can be reduced when the original user has a habit or in a posture that is convenient for the user to work.

[0034] In S304, the system control unit 202 acquires a photographed image using the distance measurement sensor 102.

[0035] In S305, the system control unit 202 detects a person from the photographed image acquired in the immediately preceding S304. If multiple people are detected, the information of the person closest or largest in the image is used in S307 described later.

[0036] In S306, the system control unit 202 determines whether a person is detected in the photographed image acquired in S305. If it is determined that a person is detected, the process proceeds to S307; otherwise, the processes from S304 to S306 are repeated.

[0037] In S307, the system control unit 202 acquires the distances of the head, both shoulders, and chest of the subject detected in S306. The distances of the head, both shoulders, and chest of the subject will be described with reference to FIGS. 6a to 6c. FIGS. 6a to 6c are top views of the notification robot 101 and the subject 600. Note that the actual position of the notification robot 101 in FIG. 6a is 101_01, and the position when it is virtually made to face the front is 101_02. The distance of the head is the distance 614 from the notification robot 101_01 to the head 603 of the subject. The distances of the shoulders are the distance 610 from the notification robot 101_01 to the left shoulder 605 and the distance 611 from the notification robot 101_01 to the right shoulder 606. The distance of the chest is the distance 615 from the notification robot 101_01 to the chest 604.

[0038] In S308, the system control unit 202 normalizes the distances of the face, both shoulders, and chest of the subject acquired in S307 to the posture when the notification robot 101 is in the front of the user's face. It is converted into the data when the notification robot 101 is placed at a specific position in the direction perpendicular to the line connecting both shoulders 605 and 606. The specific position is 101_02 in FIG. 6a. The distance of the head is the distance 616 from the virtual notification robot 101_01 position to the head 603 of the subject. The distance of the head is converted into the distance 616 from the virtual notification robot 101_01 position to the head 603 of the subject.

[0039] Here, a specific conversion method for normalization will be described with reference to FIG. 6b. It is necessary to perform correction regarding distance and angle, and correction of the rotation of the subject. First, the correction regarding distance will be described.

[0040] The virtual subject 630 in FIG. 6b is an image obtained by imaging the subject 600 with the virtual notification robot 101_02 in FIG. 6a, but it is an image without performing correction on the left shoulder 605 and the right shoulder 606. The virtual notification robot 101_03 in FIG. 6b is a device obtained by virtually superimposing the notification robot 101 in FIG. 6a and the virtual notification robot 101_02 in FIG. 6a.

[0041] In Fig. 6b, the distance 614 to the head 603 of the subject 600 and the distance 616 to the head 633 of the subject 630 form an angle 631. With respect to the direction in which the imaging plane of the virtual notification robot 101_03 is orthogonal to the head 633 of the subject 630 as the z-axis, and the direction orthogonal to the z-axis as the x-axis, taking the position of the virtual notification robot 101_03 as the origin, if the z-coordinate of the left shoulder 606 is coordinate 606_z and the x-coordinate is coordinate 606_x, then the z-coordinate 636_z and the x-coordinate 636_x of the left shoulder 636 of the virtual subject 630 are (A) Coordinate 636_z = (distance 616 / distance 614) × coordinate 606_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 606_x × sin(angle 631), (B) Coordinate 636_x = (distance 616 / distance 614) × coordinate 606_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 606_x × cos(angle 631), and can be obtained as such.

[0042] Similarly, in Fig. 6b, taking the position of the virtual notification robot 101_03 as the origin, if the z-coordinate of the right shoulder 605 is coordinate 605_z and the x-coordinate is coordinate 605_x, then the z-coordinate 635_z and the x-coordinate 635_x of the left shoulder 635 of the virtual subject 630 are (C) Coordinate 635_z = (distance 616 / distance 614) × coordinate 605_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 605_x × sin(angle 631), (D) Coordinate 635_x = (distance 616 / distance 614) × coordinate 605_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 605_x × cos(angle 631), and can be obtained as such.

[0043] Next, corrections regarding the rotation of the subject will be described using Fig. 6c.

[0044] The virtual subject 650 is the virtual subject after performing rotation correction on the virtual subject 630.

[0045] The line segment connecting the left shoulder 655 and the right shoulder 656 of the virtual object 650 and the line segment connecting the left shoulder 635 and the right shoulder 636 of the virtual object 630 have an angle 651. Assuming the position of the virtual notification robot 101_03 as the origin, the z coordinate 656_z and the x coordinate 656_x of the left shoulder 656 of the virtual object 650 are (E) coordinate 656_z = coordinate 636_z × cos(angle 651) - (coordinate 636_x - distance 616) × sin(angle 651), (F) coordinate 656_x = coordinate 636_z × sin(angle 651) + (coordinate 636_x - distance 616) × cos(angle 651) + distance 616, and can be obtained thereby.

[0046] Similarly, assuming the position of the virtual notification robot 101_03 in Fig. 6c as the origin, the z coordinate 655_z and the x coordinate 656_x of the left shoulder 655 of the virtual object 650 are (G) coordinate 655_z = coordinate 636_z × cos(angle 651) - (coordinate 635_x - distance 616) × sin(angle 651), (H) coordinate 655_x = coordinate 636_z × sin(angle 651) + (coordinate 635_x - distance 616) × cos(angle 651) + distance 616, and can be obtained thereby.

[0047] The order of the correction of the distance and angle and the correction of rotation described above does not matter. Also, the above example is a two-dimensional correction when viewed from the top surface, and the correction is performed using a two-dimensional rotation matrix and distance. When correction is required for the height of the object, correction using a three-dimensional rotation matrix and distance with the addition of height direction correction to the above formula is required.

[0048] In S309, the system control unit 202 calculates the difference value Zh between the head and the chest and the difference value Zc between the midpoint of the line connecting both shoulders and the chest based on the values normalized in S308, and records them in the system memory 204.

[0049] As shown in FIG. 7a, it is the difference value Zhs between the head 711 and the chest 712 in the reference posture, and an example of the posture during the notification process is the difference value Zhn between the head 713 and the chest 714. The coordinate axis 701 is positive in the direction away from the notification robot 101. When Zhn is smaller than Zhs (negative value), that is, when the head comes forward more than the chest, it is determined that the posture is a forward-leaning posture. Also, when Zhn is larger than Zhs, it is determined that the posture is a backward-leaning (slumping) posture. As shown in FIG. 7b, it is the difference value Zcs between the midpoint 715 connecting both shoulders in the reference posture and the chest 716, and an example of the notification process is the difference value Zcn between the midpoint 717 connecting both shoulders and the chest 718. When Zcn is smaller than Zcs (negative value), it is determined that the posture is a hunched-shoulder posture.

[0050] In S310, the system control unit 202 compares the difference value Zhs between the head and the chest in the reference posture acquired in S303, the difference value Zcs between the midpoint of the line connecting both shoulders and the chest, and Zhn and Zcn acquired in S309, respectively. That is, it compares Zhs with Zhn and Zcs with Zcn.

[0051] In S311, the system control unit 202 determines whether either (I) Zh n - Zh s < x1 or (II) Zh n - Zh s > x2 holds, or neither holds. That is, in (I), it can be determined whether it is a forward-leaning posture. Note that x1 is a value such as -6 cm, -8 cm, -10 cm, etc. In (II), it can be determined whether it is a backward-leaning (slumping) posture. Note that x2 is a value such as 2 cm, 4 cm, 6 cm, etc.

[0052] If it is determined that either (I) or (II) holds, the process proceeds to S312; otherwise, it proceeds to S313. Note that x1 and x2 may be arbitrarily set by the user, or may be automatically changed according to the input user profile (age, gender, physique). For example, when the shoulder width is detected and the shoulder width is large, x1 and x2 are changed to larger values. Or, when the gender is female, x1 and x2 are changed to smaller values.

[0053] In S312, the system control unit 202 starts counting the elapsed time T1 of the forward / backward tilt posture. T1 indicates the time during which the user's posture is in a state of leaning forward from the reference posture or in a slouching state. If the counting of T1 has already started, the counting continues in S312.

[0054] In S313, the system control unit 202 determines whether the elapsed time T1 of the forward / backward tilt posture has exceeded 5 minutes. If it is determined that T1 has exceeded 5 minutes, the process proceeds to S327; otherwise, it proceeds to S316.

[0055] In S327, the system control unit 202 determines whether the posture deviation amount Zhcomp for comparing the head and chest stored in the system memory 204 is reset, that is, whether it is 0. If it is determined that the posture deviation amount Zhcomp for comparing the head and chest is 0, the process proceeds to S314; otherwise, it proceeds to S328.

[0056] In S314, the system control unit 202 notifies the user that the posture is in a forward / backward tilt state. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period. Note that the notification method may be changed depending on whether the forward-leaning state continues or the slouching state continues.

[0057] In S328, the system control unit 202 compares the posture deviation amount Zhcomp for comparing the head and chest with the current posture deviation amount Zh_n - Zh_s between the head and chest, and switches the notification method based on the comparison result. The notification of posture changes will be described later using the flowchart in FIG. 8.

[0058] In S329, the system control unit 202 stores the posture deviation amount Zhcomp for comparing the head and chest, Zhcomp = Zh_n - Zh_s, in the system memory 204. The stored posture deviation amount Zhcomp for comparing the head and chest is used as a comparison target for posture changes in S330 and subsequent steps.

[0059] In S315, the system control unit 202 resets the forward and backward tilt posture time T1 to 0.

[0060] In S330, the system control unit 202 resets the posture deviation amount Zhcomp for head and chest stored in the system memory 204 to 0.

[0061] That is, if the user was temporarily in a forward-leaning or slouching state but returned to a posture close to the reference posture before 5 minutes elapsed, T1 and the posture deviation amount Zhcomp for head and chest are reset. In S316, the system control unit 202 determines whether (III) Zcn - Zcs < x3. That is, in (III), it is possible to determine whether the user has hunched shoulders. Note that x3 is a value such as -1.0 cm, -1.5 cm, -2.0 cm. x3 may be set arbitrarily by the user, or may be automatically changed according to the input user profile (age, gender, build). For example, when the shoulder width is detected and the shoulder width is large, x3 is changed to a large value. Or, when the gender is female, x3 is changed to a small value.

[0062] Thus, since both Zhn, Zhs, Zcn, and Zcs use the z-coordinate of the chest, it is necessary to detect the position of the user's chest from the notification robot 101. Therefore, the angle 651 in Fig. 6c, that is, the angle formed by the z-axis plane of the notification robot 101 and the left and right shoulders of the user who is the subject, is desirably in the range of approximately -80° to +80°. Within that range, the notification robot 101 can be installed at a free position with respect to the user.

[0063] If it is determined that (III) holds, proceed to S319; otherwise, proceed to S321.

[0064] In S317, the system control unit 202 starts counting the hunch-shoulder posture count T2. T2 indicates the time during which the user's posture continues to be in a hunched state compared to the reference posture. If the counting of T2 has already started, the counting continues in S317.

[0065] In S318, the system control unit 202 determines whether or not the count T2 of the hunched shoulder posture has elapsed for 5 minutes or more. If it is determined that T2 has elapsed for 5 minutes or more, the process proceeds to S331; otherwise, the process proceeds to S321.

[0066] In S331, the system control unit 202 determines whether or not the posture deviation amount Zhcomp for comparison between the midpoint of the line connecting both shoulders and the chest, which is stored in the system memory 204, has been reset, that is, whether it is 0. If it is determined that the posture deviation amount Zhcomp for comparison between both shoulders and the chest is 0, the process proceeds to S319; otherwise, the process proceeds to S332.

[0067] In S319, the system control unit 202 notifies the user that the posture is in the hunched shoulder state. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0068] In S332, the system control unit 202 compares the posture deviation amount Zccomp for comparison between the midpoint of the line connecting both shoulders and the chest with the current posture deviation amount Zcn - Zcs between the midpoint of the line connecting both shoulders and the chest, and switches the notification method based on the comparison result. Similar to S328, the notification of the posture change will be described later using the flowchart of FIG. 8.

[0069] In S333, the system control unit 202 stores the posture deviation amount Zccomp = Zcn - Zcs for comparison between the midpoint of the line connecting both shoulders and the chest in the system memory 204. The stored posture deviation amount Zccomp for comparison between the midpoint of the line connecting both shoulders and the chest is used as a comparison target for the posture change in S330 and subsequent steps.

[0070] In S320, the system control unit 202 resets the hunched shoulder posture time T2 to 0.

[0071] In S334, the system control unit 202 resets the posture deviation amount Zccomp for comparison between the midpoint of the line connecting both shoulders and the chest, which is stored in the system memory 204, to 0.

[0072] That is, if the user was in a hunched - shoulder state temporarily but returned to a posture close to the reference posture before 5 minutes elapsed, the posture deviation amount Zccomp for comparison between the midpoint of the line connecting both shoulders and the chest at T2 is reset.

[0073] In S321, the system control unit 202 determines whether the user has stood up. If it is determined that the user has stood up, the process proceeds to S322; otherwise, the process proceeds to S323.

[0074] In S322, the system control unit 202 resets the sitting time T3, which will be described later.

[0075] In S323, the system control unit 202 starts or continues to measure the sitting time T3. The sitting time T3 is the time for detecting that the user is continuously sitting. Note that the measurement of T1, T2, and T3 is performed by the system timer 205.

[0076] In S324, the system control unit 202 determines whether the sitting time T3 has reached 30 minutes or more. If it is determined that T3 has reached 30 minutes or more, the process proceeds to S325; otherwise, the process proceeds to S326.

[0077] In S325, the system control unit 202 notifies the user that they have been sitting too long. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0078] Here, various notification methods of the notification robot 101 will be described with reference to FIG. 5.

[0079] In FIG. 5, the states of the notification robot 101 are set as A (correct position), B (left swing), and C (right swing).

[0080] In notification pattern 1, the state transitions as A → B → C → A → stop for 1 second → B → C → A → stop. By swaying left and right, even when the notification robot 101 is placed on the desk, the user can easily confirm the notification.

[0081] In notification pattern 2, the state transitions as A → B → C → B → C → B → C → A → stop.

[0082] In notification pattern 3, while in state A, the light notification unit 103 blinks twice at 2 Hz.

[0083] In notification pattern 4, while in state A, the light notification unit 103 lights up once for 2 seconds.

[0084] In notification pattern 5, while in state A, the light notification unit 103 blinks three times at 1 Hz.

[0085] In notification pattern 6, while in state A, the light notification unit 103 lights up brightly once for 1 second.

[0086] In notification pattern 7, the state transitions as A → B → C → A → stop for 1 second → B → C → A → stop, and when stopped in state A, the light notification unit 103 lights up brightly once for 1 second.

[0087] In notification pattern 8, the state transitions as A → B → C → A → stop for 1 second → B → C → A → stop, and when stopped in state A, the light notification unit 103 lights up dimly once for 1 second.

[0088] Notification pattern 1 is used for the poor posture notification S314 and the hunched shoulder notification S319 in FIG. 3. Notification pattern 2 is used for the sitting too long notification S325.

[0089] Notification pattern 3 is used as the notification when the power button 104 is pressed and the shooting of the notification robot 101 is started (S301).

[0090] Notification pattern 4 is used as the notification when a person is detected (S305). This is also used when a person is detected again after it has been detected (S321) that the user has stood up during the sitting-too-long detection S325.

[0091] In S326, the system control unit 202 determines whether the power of the notification robot 101 has been turned off. If it is determined that the power has been turned off, the process of FIG. 3 is terminated; otherwise, the process returns to S304. Note that in S326, it may be determined Yes when the power of the PC (personal computer) used by the user is turned off, or it may be determined Yes according to a predetermined time.

[0092] Next, with reference to FIG. 4, a flowchart for obtaining reference posture data will be described. Note that this process starts when proceeding to S303 in FIG. 3.

[0093] In S401, the system control unit 202 controls the optical notification unit 103 to notify the user to assume the reference posture. At this time, the notification is made using notification pattern 5.

[0094] In S402, it waits for a signal indicating that the user is ready. The user is determined to be ready when the power button 104 is long-pressed. The user's signal of OK may be a gesture such as the user raising both hands as a signal of OK.

[0095] In S403, the system control unit 202 acquires the distances of the head, both shoulders, and chest as the reference posture of the detected subject. At this time, the success of obtaining the reference posture is notified using notification pattern 6.

[0096] In S404, the system control unit 202 normalizes the distances of the reference face, both shoulders, and chest of the subject acquired in S403 to the posture when the notification robot 101 is in front of the user's face directly.

[0097] In S405, based on the values normalized in S404, the system control unit 202 calculates the difference value Zhs between the line connecting both shoulders and the head, and the Zcs between the line connecting both shoulders and the chest, and records them in the volatile memory 203.

[0098] Next, with reference to FIG. 8, the flowchart of the posture change notification will be described. This process starts when proceeding to S328 and S332 in FIG. 3. Note that the definition is switched according to the posture determined in this flowchart. When determining forward slump and lumbar slip, the comparison posture deviation amount Zcomp for comparison is the comparison posture deviation amount Zhcomp between the head and the chest, the current difference value Zn is the difference value Zhn between the head and the chest, and the difference value Zs of the reference posture is the difference value Zhs between the head and the chest of the reference posture. Also, when determining hunched shoulders, the comparison posture deviation amount Zcomp for comparison is the comparison posture deviation amount Zccomp between the midpoint of the line connecting both shoulders and the chest, the current difference value Zn is the difference value Zcn between the midpoint of the line connecting both shoulders and the chest, and the difference value Zs of the reference posture is the difference value Zcs between the midpoint of the line connecting both shoulders and the chest of the reference posture.

[0099] In S801, the system control unit 202 compares the comparison posture deviation amount Zcomp stored in the system memory 204 with the posture deviation amount Zn - Zs calculated in S311 or S316, and determines whether the posture deviation amount is smaller than the previous time. If the posture deviation amount is smaller than the previous time, it proceeds to S802; otherwise, it proceeds to S803.

[0100] In S802, the system control unit 202 controls the light notification unit 103 and the vibration control unit 210 to notify in notification pattern 7 that the posture has improved compared to the previous time.

[0101] In S803, the system control unit 202 controls the light notification unit 103 and the vibration control unit 210 to notify in notification pattern 8 that the posture has deteriorated compared to the previous time.

[0102] When notifying of a posture change, in S802 and S803, the system control unit 202 determines the light brightness of the light notification unit 103, and the amplitude and period of the vibration control unit 210 according to the amount of posture deviation. For example, the system control unit 202 compares the current amount of posture deviation with the comparison-use posture deviation amount acquired previously. When it is determined that the current amount of posture deviation is smaller than the comparison-use posture deviation amount (Zn - Zs < Zcomp) and the posture has been improved, the system control unit 202 makes the light notification unit 103 brighter. When it is determined that the current amount of posture deviation is equal to or greater than the comparison-use posture deviation amount (Zn - Zs ≧ Zcomp) and the posture has deteriorated, the system control unit 202 makes the light notification unit 103 dimmer. Further, as the amount of posture deviation increases, the system control unit 202 increases the amplitude of the vibration control unit 201, and lengthens the period of the vibration control unit 201 and the lighting time of the light notification unit 103.

[0103] According to the embodiment described above, the notification robot 101 can notify whether the user's posture is approaching the reference posture.

[0104] Note that S304 to S328 are repeated at a fixed period to acquire posture information and notify the user. After notifying of posture deterioration in S314 and S319, the period of S304 to S328 may be shortened. As a result, since the notification frequency also increases until the posture is improved, it becomes easier for the user to improve the posture.

[0105] Note that although it has been described that the user is notified by the operation of the notification robot 101 in S314, S319, and S325, the user may be notified to an app in the user's smartphone or PC corresponding using the communication I / F unit 208. Further, information regarding the user's posture may be recorded in the app, and graphed or made numerical to make it easier for the user to grasp the tendency and habit of the user's posture. For example, if it is found that the number of notifications of the backward leaning posture increases towards Friday when Monday to Friday are working days, the user can consciously correct the posture.

[0106] Although the method of notification has been described as being performed by the movement or light of the notification robot 101 itself, in the case of a smartphone, the display unit may blink or a push notification for warning may be performed.

[0107] In addition, although the notification robot 101 in the present embodiment detects its posture based on the distance measurement sensor 102, posture detection may be performed using a visible light imaging sensor that does not measure distance. In that case, the distance information is estimated from the captured image obtained by the imaging sensor.

[0108] Note that each of the various controls described as being performed by the system control unit 101 may be performed by one piece of hardware, or the entire apparatus may be controlled by a plurality of pieces of hardware sharing the processing.

[0109] In addition, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope not departing from the gist of this invention are also included in the present invention. Furthermore, each of the above-described embodiments merely shows one embodiment of the present invention, and it is also possible to appropriately combine the embodiments.

[0110] In addition, in the above-described embodiment, the case where the present invention is applied to the notification robot 101 has been described as an example, but this is not limited to this example, and the present invention is applicable to any electronic device capable of acquiring information regarding the positional relationship and posture between the user and the device. That is, the present invention is applicable to a mobile phone terminal, a portable image viewer, a printer device equipped with a finder, a digital photo frame, a music player, a game machine, an electronic book reader, and the like.

[0111] (Other Embodiments) The present invention can also be realized by executing the following processes. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or device reads and executes program codes. In this case, the program and the recording medium storing the program constitute the present invention.

Explanation of Signs

[0112] 101 Notification robot 102 Distance measuring sensor 103 Light notification unit 104 Power button 106 Desk 201 Lens 202 System control unit 203 Non-volatile memory 205 System timer 206 Power control unit 207 Power supply unit

Claims

1. Subject image acquisition means for acquiring an image of a subject; Posture information acquisition means for acquiring posture information of the subject from the subject image at a predetermined period; Reference posture holding means for holding reference posture information regarding the posture targeted by the subject; Posture deviation amount calculation means for calculating the deviation amount between the posture information acquired by the posture information acquisition means and the reference posture information; Posture deviation change determination means for determining the temporal change of the posture deviation amount; Notification means for giving a predetermined notification to the user; characterized by comprising; when the posture deviation amount calculated by the posture deviation amount calculation means is greater than a predetermined amount, the posture deviation change determination means determines whether the current posture is closer to the reference posture than the previous posture from the temporal change of the posture deviation amount, and the previous notification means switches the notification according to the determination result. An electronic device characterized by this.

2. In the electronic device according to Claim 1, further, the posture information acquired by the posture information acquisition means is organ position information of the subject. An electronic device characterized by this.

3. In the electronic device according to Claim 1, further, the posture information acquired by the posture information acquisition means is generated from a distance map. An electronic device characterized by this.

4. In the electronic device according to any one of Claims 1 to 3, further, when the posture deviation amount calculated by the posture deviation amount calculation means is greater than a predetermined amount, the posture information acquisition means changes the period for acquiring posture information. An electronic device characterized by this.

5. In the electronic device according to any one of Claims 1 to 4, further, the notification means gives a notification when the electronic device shakes. An electronic device characterized by this.

6. In the electronic device according to any one of Claims 1 to 5, further, the notification means switches the notification according to the posture deviation amount. An electronic device characterized by this.

7. A subject image acquisition step of acquiring an image of a subject; A posture information acquisition step of acquiring posture information of the subject from the subject image at a predetermined period; A reference posture holding step of holding reference posture information regarding the posture targeted by the subject; A posture deviation amount calculation step of calculating the deviation amount between the posture information acquired by the posture information acquisition step and the reference posture information; A posture deviation change determination step of determining the temporal change of the posture deviation amount; A notification step of giving a predetermined notification to the user; characterized by comprising; When the amount of posture deviation calculated in the posture deviation amount calculation step is greater than a predetermined amount, the posture deviation change determination step determines whether the current posture is approaching the reference posture from the previous posture based on the temporal change of the amount of posture deviation, and the previous notification step has a step of switching the notification according to the determination result. A method for controlling an electronic device, characterized in that.

8. A program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device according to any one of claims 1 to 6.

Images