Electronic equipment and method of controlling the same

Abstract

To specify a person of notification target relating to posture information even if there are a plurality of persons whose posture information has been obtained.SOLUTION: Electronic equipment according to the present invention has: posture information acquiring means capable of acquiring posture information of persons as a plurality of subjects; subject region acquiring means capable of acquiring, with respect to the plurality of persons, subject regions where subjects whose posture information can be acquired occupy; notification target determination means for determining, as a notification target, a subject occupying the largest subject region; posture determination means capable of acquiring the posture of the notification target as a posture determination result; and notification means for notifying the posture based on the posture determination result.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an electronic device and a control method thereof, and particularly to a technique for notifying regarding a user's posture. [Background Art]

[0002] There is a technique for determining whether a user's posture deviates from an ideal posture. Patent Document 1 discloses that by normalizing user posture information according to the positional relationship with the user, it is possible to determine whether the user's posture is the target posture regardless of the posture relationship with the user, and to notify. Further, Patent Document 2 discloses that when there are a plurality of photographed persons, a limit is placed on the number of persons for whom a subject can be detected. [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2021-135337 [Patent Document 2] Japanese Patent Application Laid-Open No. 2019-133436 [Summary of the Invention] [Problems to be Solved by the Invention]

[0004] The method of Patent Document 1 does not mention posture notification for a plurality of persons for whom posture information has been acquired. Further, in the method of Patent Document 2, although a maximum limit can be placed on the number of notified persons for the photographed persons, the persons to be notified of the posture cannot be specified.

[0005] An object of the present invention is to provide an electronic device that can specify a person to be notified of a posture by using a subject area and a distance for a person who can acquire posture information. [Means for Solving the Problems]

[0006] In order to achieve the above object, the electronic device of the present invention includes a posture information acquisition means capable of acquiring posture information of a plurality of subjects, i.e., people, a subject area acquisition means capable of acquiring a subject area occupied by a subject for which the posture information can be acquired for a plurality of people, a notification target determination means for determining a subject occupying the area with the largest subject area as a notification target, a posture determination means capable of acquiring the posture of the notification target as a posture determination result, and a notification means for performing a notification of the posture based on the posture determination result.

Effect of the Invention

[0007] According to the present invention, even when there are a plurality of people for whom posture information has been acquired, the person who is the notification target of the posture can be specified.

Brief Description of the Drawings

[0008]

Fig. 1a

Fig. 1b

Fig. 1c

Fig. 1d

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6a

Fig. 6b

Fig. 6c

Fig. 7a

Fig. 7b

Fig. 8

Fig. 9a

Fig. 9b

Fig. 10

Fig. 11

Mode for Carrying Out the Invention

[0009] Hereinafter, a preferred Embodiment 1 of the present invention will be described with reference to the drawings.

[0010] Figs. 1a, 1b, and 1c are external views of a notification robot 101 as an example of an imaging control device of the present invention.

[0011] Fig. 1a is a front perspective view, Fig. 1b is a left side view, and Fig. 1c is a rear perspective view.

[0012] In Fig. 1a, the exterior 105 of the notification robot 101 is composed of a resin molded product having a substantially spherical shape at the lower part.

[0013] The distance measurement sensor 102 is provided at the upper front part of the notification robot 101 and measures the distance from the notification robot 101 to any part of the user.

[0014] In Fig. 1b, the distance measurement sensor 102 is configured to have an elevation angle α so that it can photograph the upper body of the user when the notification robot 101 is placed on a desk and used. Here, α is preferably 20° to 50°.

[0015] The light notification unit 103 is a semi-transparent part of the main body 101 and transmits the light of the internal light source so that it can be visually recognized by the user.

[0016] In Fig. 1c, the power button 104 is provided on the back of the main body 101, and by pressing it, the user can switch the power of the notification robot 100 on and off.

[0017] Fig. 1d is a top view for explaining the installation position during the use of the notification robot 101.

[0018] The size of the notification robot 101 is approximately 6 cm in diameter and approximately 8 cm in height, and it can be installed at a free position on the desk 106. For example, if there is a monitor of a PC (personal computer) on the desk 106, it can be placed in front of the user and below the monitor (position P), or when using a notebook PC or when it cannot be placed in the front, it can be installed diagonally forward (position Q or position R).

[0019] Fig. 2 is a block diagram showing a configuration example of the notification robot 101 in the first embodiment.

[0020] Lens 201 is a lens group composed of a plurality of lenses, but here it is simply shown as a single lens for simplicity. The light collected by lens 201 is guided to the distance measurement sensor 102. The distance measurement sensor 102 is an element that converts an optical image into an electrical signal, and transfers the converted electrical signal (data) to the system control unit 202. Among the transferred data, distance information from the distance measurement sensor 102 to the user 211 is included. Also, the timing at which the distance measurement sensor 102 converts the optical image into an electrical signal, etc. is controlled by the system control unit 202.

[0021] The non-volatile memory 203 is an electrically erasable and recordable memory, and for example, a Flash-ROM or the like is used. In the non-volatile memory 203, constants, programs, etc. for the operation of the system control unit 202 are stored. The program mentioned here is a program for executing various flowcharts described later in Embodiment 1.

[0022] The system control unit 202 is a control unit composed of at least one processor or circuit, and controls the entire notification robot 101. By executing the program recorded in the aforementioned non-volatile memory 203, each process of Embodiment 1 described later is realized. For the system memory 204, for example, RAM is used, and constants, variables, programs read from the non-volatile memory 203, etc. for the operation of the system control unit 202 are expanded. The system control unit 202 performs person detection based on the data obtained from the distance measurement sensor 102, and extracts distance information to feature points of a person such as the face, right shoulder, left shoulder, chest, etc. Also, the system control unit 202 performs calculations using the extracted distance information to determine the posture of the user 211. At this time, the extracted distance information and the determined posture are recorded in the non-volatile memory 203 as the posture determination result.

[0023] The system timer 205 is a timing unit that measures the time used for various controls and the time of the built-in clock.

[0024] The power button 104 is an operation member that switches the power supply of the notification robot 101 on and off. The power control unit 206 is composed of a DCDC converter, a switch circuit that switches energized blocks, and the like. The power control unit 206 controls the DCDC converter based on an instruction from the system control unit 202, and supplies the necessary voltage to the appropriate location within the notification robot 101 for the necessary period. The power supply unit 207 consists of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, and the like.

[0025] The communication I / F unit 208 is an interface that connects to an external device via a wireless or wired cable and performs data transmission and reception.

[0026] The optical notification unit 103 is composed of a light-emitting element such as an LED controlled by the system control unit 202. The system control unit 202 controls the optical notification unit 103 based on the determined posture determination result and notifies the user 211 of the posture determination result. Alternatively, the system control unit 202 controls the optical notification unit 103 to notify the user 211 that the reference posture of the user 211 is being acquired (during calibration).

[0027] The vibration unit 209 is composed of an actuator or the like, is a mechanism that physically vibrates the notification robot 101, and is controlled by the vibration control unit 210.

[0028] The vibration control unit 210 is composed of a motor driver or the like and is controlled by the system control unit 202. The vibration control unit 210 enables the notification robot 101 to vibrate at an arbitrary amplitude and an arbitrary period by controlling the presence or absence, timing, and energization amount of power supplied to the vibration unit 209.

[0029] The above is the description of the block diagram in the embodiment of the present invention, but it is not limited to this.

[0030] Next, the notification process of the notification robot 101 in the first embodiment will be described with reference to the flowcharts of FIGS. 3 and 4. This process is realized by expanding the program recorded in the non-volatile memory 203 into the system memory 204 and executing it by the system control unit 202. Note that this process starts when the power button 104 of the notification robot 101 is pressed to turn on the power and the distance measurement sensor 102 becomes operable, and the distance measurement sensor starts shooting.

[0031] In S301, the system control unit 202 starts shooting by the distance measurement sensor 102.

[0032] In S302, the system control unit 202 determines whether the reference posture data is recorded in the non-volatile memory 203. If the reference posture data is recorded, the process proceeds to S304; otherwise, the process proceeds to S303.

[0033] In S303, the system control unit 202 acquires the reference posture data. The acquisition of the reference posture data will be described later with reference to the flowchart of the reference data acquisition process in FIG. 4. Here, the reference posture data is the posture taken by the user after notifying the user to take a good posture, and in the process described later, it is determined whether the current user's posture is a good posture based on the reference posture data. By acquiring the reference posture data, it is possible to determine whether a user who originally has the habit of shrugging his shoulders is further shrugging his shoulders from his original habit. If the reference posture data is used to determine the goodness or badness of the posture without acquiring the reference posture data from the target user, the possibility of frequent notifications described later can be reduced when the original user has a habit or in a posture where the user's work is easy.

[0034] In S304, the system control unit 202 acquires a captured image using the distance measurement sensor 102.

[0035] In S305, the system control unit 202 detects a person from the captured image acquired in the immediately preceding S304.

[0036] In S306, the system control unit 202 acquires the distances of the head, both shoulders, and chest of the subject detected in S305. The distances of the head, both shoulders, and chest of the subject will be described with reference to FIGS. 6a to 6c. FIGS. 6a to 6c are top views of the notification robot 101 and the subject 600. Note that the actual position of the notification robot 101 in FIG. 6a is 101_01, and the position when it is virtually made to face the front is 101_02. The distance of the head is the distance 614 from the notification robot 101_01 to the head 603 of the subject. The distances of the shoulders are the distance 610 from the notification robot 101_01 to the left shoulder 605 and the distance 611 from the notification robot 101_01 to the right shoulder 606. The distance of the chest is the distance 615 from the notification robot 101_01 to the chest 604.

[0037] In S307, it is a sub-flow of the main subject determination process for determining the person to be notified of the posture from a typical subject area in the first embodiment 1, and will be described later after the description of the basic flow.

[0038] In S327, the system control unit 202 determines whether or not the main subject has been specified in S307. If it is determined that the main subject has been specified, the process proceeds to S308; otherwise, the processes from S304 to S327 are repeated.

[0039] In S308, the system control unit 202 normalizes the distances of the face, both shoulders, and chest of the subject acquired in S306 to the posture when the notification robot 101 is in front of the user's face. It is converted into data when the notification robot 101 is placed at a specific position in the direction perpendicular to the line connecting both shoulders 605 and 606. The specific position is 101_02 in FIG. 6a. The distance of the head is the distance 616 from the virtual notification robot 101_01 position to the head 603 of the subject. The distance of the head is converted into the distance 616 from the virtual notification robot 101_01 position to the head 603 of the subject.

[0040] Here, a specific conversion method for normalization will be described with reference to FIG. 6b. It is necessary to perform correction regarding distance and angle, and correction of the rotation of the subject. First, the correction regarding distance will be described.

[0041] In FIG. 6b, the virtual subject 630 is an image obtained by imaging the subject 600 with the virtual notification robot 101_02 in FIG. 6a, but is an image without correcting the left shoulder 605 and the right shoulder 606. The virtual notification robot 101_03 in FIG. 6b is a device that virtually superimposes the notification robot 101 in FIG. 6a and the virtual notification robot 101_02 in FIG. 6a.

[0042] In FIG. 6b, the distance 614 between the head 603 of the subject 600 and the distance 616 between the head 633 of the virtual subject 630 form an angle 631. Taking the direction in which the head 633 of the virtual subject 630 is orthogonal to the imaging plane of the virtual notification robot 101_03 as the z-axis and the direction orthogonal to the z-axis as the x-axis. Assuming the position of the virtual notification robot 101_03 as the origin, and the z-coordinate of the left shoulder 606 as coordinate 606_z and the x-coordinate as coordinate 606_x, the z-coordinate 636_z and the x-coordinate 636_x of the left shoulder 636 of the virtual subject 630 are (A) coordinate 636_z = (distance 616 / distance 614) × coordinate 606_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 606_x × sin(angle 631), (B) coordinate 636_x = (distance 616 / distance 614) × coordinate 606_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 606_x × cos(angle 631), and can be obtained.

[0043] Similarly, in FIG. 6b, assuming the position of the virtual notification robot 101_03 as the origin, and the z-coordinate of the right shoulder 605 as coordinate 605_z and the x-coordinate as coordinate 605_x, the z-coordinate 635_z and the x-coordinate 635_x of the left shoulder 635 of the virtual subject 630 are (C) coordinate 635_z = (distance 616 / distance 614) × coordinate 605_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 605_x × sin(angle 631), (D) coordinate 635_x = (distance 616 / distance 614) × coordinate 605_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 605_x × cos(angle 631), It can be obtained by

[0044] Next, the correction regarding the rotation of the subject will be described with reference to FIG. 6c.

[0045] The virtual subject 650 is the virtual subject after performing the rotation correction with respect to the virtual subject 630.

[0046] The line segment connecting the left shoulder 655 and the right shoulder 656 of the virtual subject 650 and the line segment connecting the left shoulder 635 and the right shoulder 636 of the virtual subject 630 have an angle 651. Assuming that the position of the virtual notification robot 101_03 is the origin, the z-coordinate 656_z and the x-coordinate 656_x of the left shoulder 656 of the virtual subject 650 are (E) Coordinate 656_z = Coordinate 636_z × cos(angle 651) - (Coordinate 636_x - Distance 616) × sin(angle 651), (F) Coordinate 656_x = Coordinate 636_z × sin(angle 651) + (Coordinate 636_x - Distance 616) × cos(angle 651) + Distance 616, It can be obtained by

[0047] Similarly, assuming that the position of the virtual notification robot 101_03 in FIG. 6c is the origin, the z-coordinate 655_z and the x-coordinate 656_x of the left shoulder 655 of the virtual subject 650 are (G) Coordinate 655_z = Coordinate 636_z × cos(angle 651) - (Coordinate 635_x - Distance 616) × sin(angle 651), (H) Coordinate 655_x = Coordinate 636_z × sin(angle 651) + (Coordinate 635_x - Distance 616) × cos(angle 651) + Distance 616, It can be obtained by

[0048] The order of the correction of the distance and angle and the correction of the rotation described above does not matter. Also, the above example is a two-dimensional correction when viewed from above, and the correction is performed using a two-dimensional rotation matrix and distance. When correction is required for the height of the subject, correction using a three-dimensional rotation matrix and distance with the correction in the height direction added to the above formula is required.

[0049] In S309, based on the values normalized in S308, the system control unit 202 calculates the difference value Zh n between the head and the chest and the difference value Zc n between the midpoint of the line connecting both shoulders and the chest, and records them in the system memory 204.

[0050] As shown in FIG. 7a, it is the difference value Zh s between the head 711 and the chest 712 in the reference posture, and an example of the posture at the time of notification processing is the difference value Zh n between the head 713 and the chest 714. The coordinate axis 701 is positive in the direction away from the notification robot 101. When Zh n is smaller than Zh s (negative value), that is, when the head comes forward more than the chest, it is determined that the posture is a forward lean posture. When Zh n is larger than Zh s, it is determined that the posture is a backward lean (slouching) posture. As shown in FIG. 7b, it is the difference value Zc s between the midpoint 715 of the line connecting both shoulders and the chest 716 in the reference posture, and an example of the posture at the time of notification processing is the difference value Zc n between the midpoint 717 of the line connecting both shoulders and the chest 718. When Zc n is smaller than Zc s (negative value), it is determined that the posture is a hunched shoulder posture.

[0051] In S310, the system control unit 202 compares the difference value Zh s between the head and the chest in the reference posture obtained in S303, the difference value Zc s between the midpoint of the line connecting both shoulders and the chest, and Zh n and Zc n obtained in S309, respectively. That is, it compares Zh s with Zh n and Zc s with Zc n.

[0052] In S311, the system control unit 202 determines whether either (I) Zh n - Zh s < x1 or (II) Zh n - Zh s > x2 holds, or neither holds. That is, in (I), it can be determined whether the posture is a forward lean posture. Here, x1 is a value such as -6 cm, -8 cm, or -10 cm. In (II), it can be determined whether the posture is a backward lean (slouching) posture. Here, x2 is a value such as 2 cm, 4 cm, or 6 cm.

[0053] If it is determined that either (I) or (II) holds, proceed to S312; otherwise, proceed to S313. Note that x1 and x2 may be arbitrarily set by the user, or may be automatically changed according to the input user profile (age, gender, physique). For example, if the shoulder width is detected and the shoulder width is large, change x1 and x2 to large values. Alternatively, if the gender is female, change x1 and x2 to small values.

[0054] In S312, the system control unit 202 starts counting the elapsed time T1 of the forward / backward tilt posture. T1 indicates the time during which the user's posture is in a state of leaning forward from the reference posture or in a slouching state. If the counting of T1 has already started, the counting continues in S312.

[0055] In S313, the system control unit 202 determines whether the elapsed time T1 of the forward / backward tilt posture has exceeded 5 minutes. If it is determined that T1 has exceeded 5 minutes, proceed to S314; otherwise, proceed to S316.

[0056] In S314, the system control unit 202 notifies the user that the posture is in a forward / backward tilt state. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period. Note that the notification method may be changed depending on whether the forward-leaning state is continuing or the slouching state is continuing.

[0057] In S315, the system control unit 202 resets the forward / backward tilt posture time T1 to 0. That is, if the user was temporarily in a forward-leaning or slouching state but returned to a posture close to the reference posture before 5 minutes elapsed, T1 is reset.

[0058] In S316, the system control unit 202 determines whether (III) Zcn - Zcs < x3. That is, in (III), it is possible to determine whether it is a hunched shoulder. Note that x3 is a value such as -1.0 cm, -1.5 cm, -2.0 cm. x3 may be arbitrarily set by the user, or may be automatically changed according to the input user profile (age, gender, build). For example, when the shoulder width is detected and the shoulder width is large, x3 is changed to a large value. Alternatively, when the gender is female, x3 is changed to a small value.

[0059] Thus, since Zhs, Zhs, Zcn, and Zcs all use the z - coordinate of the chest, it is necessary to detect the position of the user's chest from the notification robot 101. Therefore, the angle 651 in Fig. 6c, that is, the angle formed by the z - axis plane of the notification robot 101 and the left and right shoulders of the user who is the subject, is desirably in the range of approximately -80° to +80°. If it is within that range, the notification robot 101 can be installed at a free position with respect to the user.

[0060] If it is determined that (III) holds, the process proceeds to S319; otherwise, it proceeds to S321.

[0061] In S317, the system control unit 202 starts counting the time of the hunched - shoulder posture count T2. T2 indicates the time during which the user's posture is in a hunched state continuously compared to the reference posture. If the counting of T2 has already started, the counting continues in S317.

[0062] In S318, the system control unit 202 determines whether the count T2 of the hunched - shoulder posture has elapsed for 5 minutes or more. If it is determined that T2 has elapsed for 5 minutes or more, the process proceeds to S319; otherwise, it proceeds to S321.

[0063] In S319, the system control unit 202 notifies the user that the posture is in a hunched - shoulder state. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0064] In S320, the system control unit 202 resets the hunchback posture time T2 to 0. That is, if the user was in a hunchback state temporarily but returned to a posture close to the reference posture before 5 minutes elapsed, T2 is reset.

[0065] In S321, the system control unit 202 determines whether the user has stood up. If it is determined that the user has stood up, the process proceeds to S322; otherwise, the process proceeds to S323.

[0066] In S322, the system control unit 202 resets the seating time T3, which will be described later.

[0067] In S323, the system control unit 202 starts or continues to measure the seating time T3. The seating time T3 is the time for detecting that the user is continuously sitting. Note that the measurement of T1, T2, and T3 is performed by the system timer 205.

[0068] In S324, the system control unit 202 determines whether the seating time T3 has reached 30 minutes or more. If it is determined that T3 has reached 30 minutes or more, the process proceeds to S325; otherwise, the process proceeds to S326.

[0069] In S325, the system control unit 202 notifies the user that they have been sitting too long. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0070] Here, various notification methods of the notification robot 101 will be described with reference to FIG. 5.

[0071] In FIG. 5, the states of the notification robot 101 are set as A (correct position), B (left swing), and C (right swing).

[0072] In notification pattern 1, the state transitions as A → B → C → A → stop for 1 second → B → C → A → stop. By swaying left and right, even when the notification robot 101 is placed on the desk, the user can easily confirm the notification.

[0073] In notification pattern 2, the state transitions as A → B → C → B → C → B → C → A → stop.

[0074] In notification pattern 3, while in state A, the light notification unit 103 blinks 2 times at 2 Hz.

[0075] In notification pattern 4, while in state A, the light notification unit 103 lights up once for 2 seconds.

[0076] In notification pattern 5, while in state A, the light notification unit 103 blinks 3 times at 1 Hz.

[0077] In notification pattern 6, while in state A, the light notification unit 103 lights up brightly once for 1 second.

[0078] Notification pattern 1 is used for the poor posture notification S314 and the hunched shoulder notification S319 in FIG. 3. Notification pattern 2 is used for the sitting too long notification S325.

[0079] Notification pattern 3 is used as the notification when the power button 104 is pressed and the shooting of the notification robot 101 is started (S301).

[0080] Notification pattern 4 is used as the notification when a person is detected (S305). This is also used in the same way when a person is detected again after the user stands up in the sitting too long detection S325 and the standing detection (S321).

[0081] In S326, the system control unit 202 determines whether the power of the notification robot 101 has been turned off. If it is determined that the power has been turned off, the process of FIG. 3 is terminated; otherwise, the process returns to S304. Note that in S326, it may be determined as Yes when the power of the PC (personal computer) used by the user is turned off, or it may be determined as Yes according to the arrival of a predetermined time.

[0082] Next, with reference to FIG. 4, a flowchart for acquiring reference posture data will be described. This process starts when proceeding to S303 in FIG. 3.

[0083] In S401, the system control unit 202 controls the optical notification unit 103 to notify the user to assume the reference posture. At this time, the notification is made using notification pattern 5.

[0084] In S402, it waits for a signal indicating that the user is ready. It is determined that the user is ready when the power button 104 is long-pressed. The user's OK signal may also be a gesture such as the user raising both hands as the OK signal.

[0085] In S403, the system control unit 202 acquires the distances of the head, both shoulders, and chest as the reference posture of the detected subject. At this time, the successful acquisition of the reference posture is notified using notification pattern 6.

[0086] In S404, the system control unit 202 normalizes the distances of the reference face, both shoulders, and chest of the subject acquired in S403 to the posture when the notification robot 101 is in front of the user's face directly.

[0087] In S405, based on the values normalized in S404, the system control unit 202 calculates the difference value Zhs between the line connecting both shoulders and the head, and Zcs between the line connecting both shoulders and the chest, and records them in the volatile memory 203.

[0088] Next, in the main subject determination process of S307, when multiple people are being photographed, the method for determining the person to be notified of the posture using the subject area will be described with reference to the flowchart of FIG. 8.

[0089] In S801, the system control unit 202 acquires the posture information of the people in the captured image. In the first embodiment, the posture that can be calculated from the face and both shoulders in the captured image is acquired as the posture information. In this embodiment, the coordinates corresponding to the face and both shoulders detected in S305 are acquired as the posture information.

[0090] In S802, the system control unit 202 determines whether a person for whom posture information can be acquired is being photographed in the captured image acquired in S305. If it is determined that a person for whom posture information can be acquired is being photographed, the process proceeds to S803; otherwise, the process ends as undeterminable.

[0091] Here, the people for whom posture information can be acquired and the people for whom posture information cannot be acquired will be described with reference to FIGS. 9a and 9b.

[0092] In the first embodiment, as described above, a person for whom posture information can be acquired is a person in the captured image who has a face and both shoulders and for whom the posture can be detected. If both shoulders are not included, the person is considered a person for whom the posture cannot be detected.

[0093] FIG. 9a shows an example in which, while the notification robot 101 is operating, the main subject 901 who is the user, other subjects 902, 903, a subject 904 for whom posture information cannot be acquired, the left shoulder 905 and the right shoulder 906 of the main subject 901, and the left shoulders 907, 909 and the right shoulders 908, 910 of the subjects 902, 903 are being photographed.

[0094] FIG. 9b shows the captured image of the notification robot 101, and shows the correspondence with FIGS. 9a 901 to 910 and the differences 911 to 913 in the x coordinates of both shoulders of each person.

[0095] In S803, the system control unit 202 acquires the differences 911 to 913 of the x - coordinates described above. In this example, the magnitude relationship of the differences of the x - coordinates is 911 > 912 > 913.

[0096] In S804, the system control unit 202 determines the person who appears the largest as the subject area in terms of shoulder width. In the examples of FIGS. 9a and 9b, the main subject 901 with the largest difference in the x - coordinates of both shoulders is determined as the person for whom posture information is to be notified. When only the posture information of one person can be acquired, that person is determined as the main subject.

[0097] According to the first embodiment described above, even when a person in whom the posture information of a plurality of people has been acquired is photographed, the person to be notified of the posture can be specified by determining the person with the largest subject area.

[0098] Note that in the first embodiment, the person to be notified of the posture is determined only by the difference value of the x - coordinate, but it is not limited to this, and the size may be obtained from the x - coordinate and the y - coordinate.

[0099] Note that in the first embodiment, the person to be notified of the posture is determined by both shoulders, but it is not limited to this, and the size may be obtained by using the coordinate difference between the position of the face and the shoulder, the position of the face and the chest, etc. as the subject area and then comparing them.

[0100] Note that in the first embodiment, the description is made on the premise that the subject is facing forward, but the difference value of the x - coordinate may be normalized according to the orientation of each subject, and the largest person may be set as the notification target.

[0101] Next, a second embodiment of the present invention will be described.

[0102] In the second embodiment, the notification robot 101 will explain the method for determining the person to be notified of the posture using the distance information when a plurality of people are being photographed. Note that in the second embodiment, the method for identifying the person to be notified of the posture when posture information of a plurality of people can be obtained is different from that of the first example. That is, FIG. 3 is common but there are differences in the process of determining the main subject in S307, and the differences from the first embodiment will be described using the flowchart of FIG. 10.

[0103] In FIG. 10, in the main subject determination process of S307, the process flow changes for S803 and S804 in FIG. 8 in the first embodiment are set as S1001 and S1002. Since the steps of S801 and S802 are the same as those in the first embodiment, only the differences from FIG. 8 will be described.

[0104] In S1001, the system control unit 202 obtains the average distance between the two shoulders of the person from whom the posture information can be obtained.

[0105] Here, the method for obtaining the distance of the person from whom the posture information can be obtained will be described with reference to FIG. 11.

[0106] FIG. 11 shows 901 to 910 of FIGS. 9a and 9b described above, the distance 1101 of the main subject 901, and the distances 1102 and 1103 of the subjects 902 and 903. The method for obtaining each of the distances 1101 to 1103 is obtained by the average value of the distances between the two shoulders. In the second embodiment, it is assumed that the distance 1101 > 1102 > 1103.

[0107] In S1002, the system control unit 202 determines the person with the average distance between the two shoulders obtained in S1001 being the closest distance from the notification robot 101. Here, the person of the main subject 901 with the closest average distance between the two shoulders is set as the person to be notified of the posture. Note that when posture information of only one person can be obtained, that person is determined as the main subject.

[0108] According to Embodiment 2 described above, even when a person in which pose information of a plurality of persons has been acquired is photographed, the person to be notified of the pose can be specified by determining the person with the closest distance.

[0109] Note that the present invention is not limited to this, and the distance between either the left or right shoulder and the distance of parts other than the shoulder necessary for acquiring the pose information may be used.

[0110] Note that when a plurality of persons from whom pose information can be acquired are photographed, the system control unit 202 may not perform a notification without determining a notification target.

[0111] Note that during a period in which a plurality of persons from whom pose information can be acquired are photographed, the system control unit 202 may make the pose information unidentifiable, and for example, the notification robot 101 may perform a different type of notification from the pose notification such as vibration.

[0112] Note that regarding the determination of the notification target when a plurality of persons from whom pose information can be acquired are photographed, for example, in S303, personal face information may be recorded in the nonvolatile memory 203, and in S804 and S1002, the system control unit 102 may identify an individual from the face of the photographed image and determine the notification target. For example, if there is only one person who can be identified, the identified person may be set as the notification target.

[0113] Note that although the operation of the notification robot 101 has been described to notify the user in S314, S319, and S325, a notification may be sent to an application in the user's smartphone or PC corresponding using the communication I / F unit 208. Further, information regarding the user's pose may be recorded in the application, and by graphing or numericalizing it, it may be easier for the user to grasp the tendency and habit of the user's pose. For example, if it is found that the number of notifications of the backward leaning pose increases towards Friday when Monday to Friday are working days, the user can consciously correct the pose.

[0114] Although the method of notification has been described as using the movement or light of the notification robot 101 itself, in the case of a smartphone, it may also blink the display or send a push notification for warning.

[0115] Also, the notification robot 101 in the present embodiment detects its posture based on the distance measurement sensor 102, but may detect the posture with a visible light imaging sensor that does not measure the distance. In that case, the distance information is estimated from the captured image obtained by the imaging sensor.

[0116] Note that each of the various controls described as being performed by the system control unit 101 may be performed by one piece of hardware, or the entire apparatus may be controlled by a plurality of pieces of hardware sharing the processing.

[0117] Moreover, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope not departing from the gist of the present invention are also included in the present invention. Further, each of the above-described embodiments merely shows one embodiment of the present invention, and it is also possible to appropriately combine the embodiments.

[0118] Also, in the above-described embodiment, the case where the present invention is applied to the notification robot 101 has been described as an example, but this is not limited to this example, and it is applicable to any electronic device capable of acquiring information regarding the positional relationship and posture between the user and the device. That is, the present invention is applicable to a mobile phone terminal, a portable image viewer, a printer device equipped with a finder, a digital photo frame, a music player, a game machine, an electronic book reader, and the like.

[0119] (Other Embodiments) The present invention can also be realized by executing the following processes. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or device reads and executes program codes. In this case, the program and the recording medium storing the program constitute the present invention.

Explanation of Signs

[0120] 101 Notification robot 102 Distance measuring sensor 103 Optical notification unit 104 Power button 106 Desk 201 Lens 202 System control unit 203 Non-volatile memory 204 System memory 205 System timer

Claims

1. Posture information acquisition means capable of acquiring the posture information of a plurality of subjects, which are people; Subject area acquisition means capable of acquiring, for a plurality of people, the subject area occupied by the subject for which the posture information can be acquired; Notification target determination means for determining, as the notification target, the subject occupying the largest area among the subject areas; Posture determination means capable of acquiring, as the posture determination result, the posture of the notification target; An electronic device characterized by having notification means for performing a notification of the posture based on the posture determination result.

2. Posture information acquisition means capable of acquiring the posture information of a plurality of subjects, which are people; Subject distance acquisition means capable of acquiring, for a plurality of people, the distance between the subject for which the posture information can be acquired and the electronic device and the subject; Notification target determination means for determining, as the notification target, the subject having the shortest distance to the electronic device; Posture determination means capable of acquiring, as the posture determination result, the posture of the notification target; An electronic device characterized by having notification means for performing a notification of the posture based on the posture determination result.

3. The notification target determination means: Is provided with personal authentication means for identifying the faces of a plurality of subjects, which are people, and Determines the notification target based on the output of the personal authentication means when the posture information acquisition means can acquire the posture information of a plurality of subjects. The electronic device according to claim 1 or claim 2, characterized in that.

4. The predetermined notification is performed when the electronic device shakes. The electronic device according to any one of claims 1 and 2, characterized in that.

5. The subject distance acquisition means is capable of acquiring subject information acquired by a distance measurement sensor, and The distance measurement sensor is at a first position that is the front of the subject or at a second position shifted from the first position. The control means normalizes the posture information of the subject based on the positional relationship acquired by the acquisition means, and When the normalized posture information of the subject satisfies the predetermined conditions, the predetermined notification is not performed, and when the normalized posture information of the subject is not in the predetermined posture, the control is performed so as not to perform the predetermined notification. The electronic device according to claim 2, characterized in that.

Images