Electronic apparatus and control method of the same

Abstract

To provide an electronic apparatus which enables a user to make an effective recommendation to people close to himself or herself and increases his or her buying intention.SOLUTION: An electronic apparatus comprises: acquisition means for acquiring the positional relation between a subject and the electronic apparatus; control means for normalizing posture information of the subject based on the acquired positional relation and performing predetermined report when the normalized posture information of the subject does not meet a predetermined condition; person detection means for detecting a person from a subject image acquired by the acquisition means; individual identification means for identifying the individual of the acquired person; posture estimation means for estimating the posture of the identified person, the persons being a user individual and a person other than the user individual; first type reporting means for reporting a result of the posture estimation generated by the posture estimation means to the user individual; connection means with a coordination device linked with the user individual; and second type reporting means for reporting to the coordination device the posture information on the person other than the user individual generated by the posture estimation means.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an electronic device and a control method thereof, and particularly to a technique for notifying regarding a user's posture.

Background Art

[0002] There is a technique for determining whether a user's posture deviates from an ideal posture. Patent Document 1 discloses that by normalizing user posture information according to the positional relationship with the user, it is possible to determine whether the user's posture is the target posture regardless of the posture relationship with the user, and to notify.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the method of Patent Document 1, for those other than the user, there has been no established means that is persuasive and effective in giving recommendations regarding posture correction.

[0005] An object of the present invention is to provide an electronic device that can also give an effective recommendation to those other than the user, allowing them to experience the usability, being persuasive to people close to the user, and improving the willingness to purchase.

Means for Solving the Problems

[0006] To achieve the above object, the electronic device of the present invention includes an acquisition unit that acquires the positional relationship between the electronic device and the subject, normalizes the posture information of the subject based on the positional relationship acquired by the acquisition unit, and does not issue a predetermined notification when the normalized posture information of the subject satisfies a predetermined condition, and controls so as not to issue the predetermined notification when the normalized posture information of the subject is not in the predetermined posture, a person detection unit that detects a person from the subject image acquired by the acquisition unit, a personal identification unit that identifies the individual of the person acquired by the person detection unit, a posture estimation unit for the person identified by the personal identification unit, the person being an individual user and a person other than the individual user, a first type of notification unit that notifies the individual user of the posture information generated by the posture estimation unit, a connection unit to a cooperation device associated with the individual user, and a second type of notification unit that notifies the cooperation device of the posture information of the person other than the individual user generated by the posture estimation unit.

Effect of the Invention

[0007] According to the present invention, in addition to the user, it is possible to let others experience the feeling of use, which is persuasive to people close to the user and can effectively recommend products to improve the purchasing desire.

Brief Description of the Drawings

[0008]

Figure 1a

Figure 1b

Figure 1c

Figure 1d

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6a

Figure 6b

Figure 6c

Figure 7a

Figure 7b

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0010] FIG. 1a, FIG. 1b, and FIG. 1c are external views of a notification robot 101 as an example of the imaging control device of the present invention.

[0011] FIG. 1a is a front perspective view, FIG. 1b is a left side view, and FIG. 1c is a rear perspective view.

[0012] In FIG. 1a, the exterior 105 of the notification robot 101 is composed of a resin - molded product with a substantially spherical shape at the bottom.

[0013] The distance measurement sensor 102 is provided at the upper front part of the notification robot 101, and measures the distance from the notification robot 101 to any part of the user.

[0014] In Fig. 1b, the distance measurement sensor 102 is configured to have an elevation angle α so that it can photograph the upper body of the user when the notification robot 101 is placed on a desk for use. Here, α is preferably 20° to 50°.

[0015] The light notification unit 103 is a semi-transmissive part of the main body 101, and transmits the light of the internal light source so that it can be visually recognized by the user.

[0016] In Fig. 1c, the power button 104 is provided on the back of the main body 101, and by pressing it, the user can switch the power of the notification robot 100 on / off.

[0017] Fig. 1d is a top view for explaining the installation position during the use of the notification robot 101.

[0018] The size of the notification robot 101 is approximately 6 cm in diameter and approximately 8 cm in height, and it can be installed at a free position on the desk 106. For example, when there is a monitor of a PC (personal computer) on the desk 106, it can be placed in front of the user and below the monitor (position P), or when using a notebook PC or when it cannot be placed directly in front, it can be installed diagonally forward (position Q or position R).

[0019] Fig. 2 is a block diagram showing a configuration example of the notification robot 101 in this embodiment.

[0020] The lens 201 is a lens group composed of a plurality of lenses, but here it is simply shown as a single lens for simplicity. The light collected by the lens 201 is guided to the distance measuring sensor 102. The distance measuring sensor 102 is an element that converts an optical image into an electrical signal, and transfers the converted electrical signal (data) to the system control unit 202. Among the data transferred, distance information from the distance measuring sensor 102 to the user 211 is included. Also, the timing at which the distance measuring sensor 102 converts the optical image into an electrical signal, etc. is controlled by the system control unit 202.

[0021] The non-volatile memory 203 is an electrically erasable and recordable memory, and for example, a Flash-ROM or the like is used. In the non-volatile memory 203, constants, programs, etc. for the operation of the system control unit 202 are stored. The program mentioned here refers to the program for executing various flowcharts described later in this embodiment.

[0022] The system control unit 202 is a control unit composed of at least one processor or circuit, and controls the entire notification robot 101. By executing the program recorded in the aforementioned non-volatile memory 203, each process of this embodiment described later is realized. In the system memory 204, for example, RAM is used, and constants, variables, programs read from the non-volatile memory 203, etc. for the operation of the system control unit 202 are expanded. The system control unit 202 performs person detection based on the data obtained from the distance measuring sensor 102, and extracts distance information to feature points of a person such as the face, right shoulder, left shoulder, chest, etc. Also, the system control unit 202 performs calculations using the extracted distance information to determine the posture information of the user 211. At this time, the extracted distance information and the determined posture information are recorded in the non-volatile memory 203.

[0023] The system timer 205 is a timing unit that measures the time used for various controls and the time of the built-in clock.

[0024] The power button 104 is an operating member for switching the power of the notification robot 101 on and off. The power control unit 206 is composed of a DCDC converter, a switch circuit for switching the energized blocks, and the like. The power control unit 206 controls the DCDC converter based on the instructions of the system control unit 202, and supplies the necessary voltage to the appropriate place in the notification robot 101 for the necessary period. The power supply unit 207 consists of a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, and the like.

[0025] The communication I / F unit 208 is an interface that connects to an external device via a wireless or wired cable and performs data transmission and reception. For example, it connects to the user's smartphone or the like.

[0026] The light notification unit 103 is composed of a light-emitting element such as an LED controlled by the system control unit 202. The system control unit 202 controls the light notification unit 103 based on the determined posture information and notifies the user 211 of the posture determination result. Alternatively, the system control unit 202 controls the light notification unit 103 to notify the user 211 that the reference posture of the user 211 is being acquired (during calibration).

[0027] The vibration unit 209 is composed of an actuator or the like, is a mechanism for physically vibrating the notification robot 101, and is controlled by the vibration control unit 210.

[0028] The vibration control unit 210 is composed of a motor driver or the like and is controlled by the system control unit 202. The vibration control unit 210 enables the notification robot 101 to vibrate at an arbitrary amplitude and an arbitrary period by controlling the presence or absence, timing, and amount of energization to the vibration unit 209.

[0029] The above is the description of the block diagram in the embodiment of the present invention, but it is not limited to this.

[0030] Next, the notification process in this embodiment will be described using the flowcharts of FIGS. 3, 4, 8, and 9. This process is realized by expanding the program recorded in the non-volatile memory 203 into the system memory 204 and executing it by the system control unit 202. Note that in this process, when the power button 104 of the notification robot 101 is pressed to turn on the power and the distance measurement sensor 102 becomes operable, the distance measurement sensor starts shooting.

[0031] In S301, the system control unit 202 starts shooting by the distance measurement sensor 102.

[0032] In S302, the system control unit 202 determines whether the reference posture data is recorded in the non-volatile memory 203. If the reference posture data is recorded, the process proceeds to S304; otherwise, the process proceeds to S303.

[0033] In S303, the system control unit 202 acquires the reference posture data. The acquisition of the reference posture data will be described later using the flowchart of the reference data acquisition process in FIG. 4. Here, the reference posture data is the posture taken by the user after notifying the user to take a good posture, and in the processes described later, it is used to determine whether the current posture of the user is a good posture based on the reference posture data. By acquiring the reference posture data, it is possible to determine whether a user who originally has the habit of shrugging his shoulders has further shrugged his shoulders from his original habit. When using the reference posture data as the basis for determining the quality of the posture without acquiring the reference posture data from the target user, the possibility of frequent notifications described later can be reduced when the original user has a habit or in a posture where the user can work comfortably.

[0034] In S304, the system control unit 202 acquires a captured image using the distance measurement sensor 102.

[0035] In S305, the system control unit 202 detects a person from the captured image obtained in the immediately preceding S304. If multiple persons are detected, the information of the person closest or largest in the image is used in S307 described below.

[0036] In S306, the system control unit 202 determines whether a person is detected in the captured image obtained in S305. If it is determined that a person is detected, the process proceeds to S307; otherwise, the processes from S304 to S306 are repeated.

[0037] In S307, the system control unit 202 identifies the face (head) of the subject detected in S306 and verifies it with the user. If it is determined to be the user, the process proceeds to S308; otherwise, the process proceeds to S309.

[0038] Here, the posture detection and notification process of S308 will be described using the sub - flowchart of FIG. 8.

[0039] The system control unit 202 obtains the distances between the head, both shoulders, and chest of the subject detected in S801. The distances between the head, both shoulders, and chest of the subject will be described with reference to FIGS. 6a - 6c. FIGS. 6a - 6c are top views of the notification robot 101 and the subject 600. Let the actual position of the notification robot 101 in FIG. 6a be 101_01 and the position when virtually facing the front be 101_02. The distance of the head is the distance 614 from the notification robot 101_01 to the head 603 of the subject. The shoulder distances are the distance 610 from the notification robot 101_01 to the left shoulder 605 and the distance 611 from the notification robot 101_01 to the right shoulder 606. The chest distance is the distance 615 from the notification robot 101_01 to the chest 604.

[0040] In S308, the system control unit 202 normalizes the distances of the face, both shoulders, and chest of the subject acquired in S801 to the posture when the notification robot 101 is in front of the user's face directly. It converts the data when the notification robot 101 is placed at a specific position in the direction perpendicular to the line connecting both shoulders 605 and 606. Let the specific position be 101_02 in Fig. 6a. The distance of the head is the distance 616 from the position of the virtual notification robot 101_01 to the head 603 of the subject. The distance of the head is converted to the distance 616 from the position of the virtual notification robot 101_01 to the head 603 of the subject.

[0041] Here, with reference to Fig. 6b, a specific conversion method for normalization will be described. It is necessary to correct the distance and angle, and correct the rotation of the subject. First, the correction regarding the distance will be described.

[0042] The virtual subject 630 in Fig. 6b is the image of the subject 600 captured by the virtual notification robot 101_02 in Fig. 6a, but it is an image without correcting the left shoulder 605 and the right shoulder 606. The virtual notification robot 101_03 in Fig. 6b is a device obtained by virtually overlapping the notification robot 101 in Fig. 6a and the virtual notification robot 101_02 in Fig. 6a.

[0043] In Fig. 6b, the distance 614 between the head 603 of the subject 600 and the distance 616 between the head 633 of the subject 630 form an angle 631. Taking the direction in which the head 633 of the subject 630 is orthogonal to the imaging plane of the virtual notification robot 101_03 as the z-axis, and the direction orthogonal to the z-axis as the x-axis. With the position of the virtual notification robot 101_03 as the origin, if the z-coordinate of the left shoulder 606 is coordinate 606_z and the x-coordinate is coordinate 606_x, then the z-coordinate 636_z and the x-coordinate 636_x of the left shoulder 636 of the virtual subject 630 are (A) Coordinate 636_z = (distance 616 / distance 614) × coordinate 606_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 606_x × sin(angle 631), (B) Coordinate 636_x = (distance 616 / distance 614) × coordinate 606_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 606_x × cos(angle 631), It can be obtained by

[0044] Similarly, when the position of the virtual notification robot 101_03 in Fig. 6b is taken as the origin, and the z coordinate of the right shoulder 605 is coordinate 605_z and the x coordinate is coordinate 605_x, the z coordinate 635_z and the x coordinate 635_x of the left shoulder 635 of the virtual subject 630 are (C) Coordinate 635_z = (distance 616 / distance 614) × coordinate 605_z × cos(angle 631) - (distance 616 / distance 614) × coordinate 605_x × sin(angle 631), (D) Coordinate 635_x = (distance 616 / distance 614) × coordinate 605_z × sin(angle 631) + (distance 616 / distance 614) × coordinate 605_x × cos(angle 631), It can be obtained by

[0045] Next, the correction regarding the rotation of the subject will be described with reference to Fig. 6c.

[0046] The virtual subject 650 is the virtual subject after performing rotation correction on the virtual subject 630.

[0047] The line segment connecting the left shoulder 655 and the right shoulder 656 of the virtual subject 650 and the line segment connecting the left shoulder 635 and the right shoulder 636 of the virtual subject 630 have an angle 651. When the position of the virtual notification robot 101_03 is taken as the origin, the z coordinate 656_z and the x coordinate 656_x of the left shoulder 656 of the virtual subject 650 are (E) Coordinate 656_z = coordinate 636_z × cos(angle 651) - (coordinate 636_x - distance 616) × sin(angle 651), (F) Coordinate 656_x = coordinate 636_z × sin(angle 651) + (coordinate 636_x - distance 616) × cos(angle 651) + distance 616, It can be obtained by

[0048] Similarly, when the position of the virtual notification robot 101_03 in Fig. 6c is taken as the origin, the z coordinate 655_z and the x coordinate 656_x of the left shoulder 655 of the virtual subject 650 are (G) Coordinate 655_z = Coordinate 636_z × cos(Angle 651) - (Coordinate 635_x - Distance 616) × sin(Angle 651), (H) Coordinate 655_x = Coordinate 636_z × sin(Angle 651) + (Coordinate 635_x - Distance 616) × cos(Angle 651) + Distance 616, can be obtained.

[0049] The order of the correction of the distance and angle and the correction of rotation described above does not matter. Also, the above example is a two-dimensional correction when viewed from the top surface, and the correction is performed using a two-dimensional rotation matrix and distance. When correction is required for the height of the subject, correction using a three-dimensional rotation matrix and distance with the addition of height-direction correction to the above formula is required.

[0050] In S803, based on the values normalized in S802, the system control unit 202 calculates the difference value Zh between the head and the chest and the difference value Zc between the midpoint of the line connecting both shoulders and the chest, and records them in the system memory 204.

[0051] As shown in FIG. 7a, it is the difference value Zhs between the head 711 and the chest 712 in the reference posture, and an example of the posture at the time of notification processing is the difference value Zh between the head 713 and the chest 714. The coordinate axis 701 is positive in the direction away from the notification robot 101. When Zh is smaller than Zhs (negative value), that is, when the head comes forward more than the chest, it is determined that the posture is a forward inclination posture. Also, when Zh is larger than Zhs, it is determined that the posture is a backward inclination (slumping) posture. As shown in FIG. 7b, it is the difference value Zcs between the midpoint 715 of the line connecting both shoulders and the chest 716 in the reference posture, and an example at the time of notification processing is the difference value Zc between the midpoint 717 of the line connecting both shoulders and the chest 718. When Zc is smaller than Zcs (negative value), it is determined that the posture is a hunched shoulder posture.

[0052] In S804, the system control unit 202 compares the difference value Zhs between the head and the chest in the reference posture obtained in S303, the difference value Zcs between the midpoint of the line connecting both shoulders and the chest, and Zh and Zc obtained in S803, respectively. That is, it compares Zhs with Zh and Zcs with Zc.

[0053] In S805, the system control unit 202 determines whether either (I) Zh n-Zhs <x1 or (II) Zh n-Zhs > x2 holds, or neither holds. That is, in (I), it is possible to determine whether it is a forward leaning posture. Note that x1 is a value such as -6 cm, -8 cm, or -10 cm. In (II), it is possible to determine whether it is a backward leaning posture (slouching). Note that x2 is a value such as 2 cm, 4 cm, or 6 cm.

[0054] If it is determined that either (I) or (II) holds, the process proceeds to S806; otherwise, it proceeds to S809. Note that x1 and x2 may be arbitrarily set by the user, or may be automatically changed according to the input user profile (age, gender, physique). For example, if the shoulder width is detected and the shoulder width is large, x1 and x2 are changed to larger values. Alternatively, if the gender is female, x1 and x2 are changed to smaller values.

[0055] In S806, the system control unit 202 starts counting the elapsed time of the forward / backward tilt posture count T1. T1 indicates the time during which the user's posture is in a state of being bent forward from the reference posture or in a state of slouching. If the counting of T1 has already started, the counting continues in S806.

[0056] In S807, the system control unit 202 determines whether the elapsed time of the forward / backward tilt posture count T1 has exceeded 5 minutes. If it is determined that T1 has exceeded 5 minutes, the process proceeds to S808; otherwise, it proceeds to S810.

[0057] In S808, the system control unit 202 notifies the user that the posture is in a forward / backward tilt state. The notification method is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period. Note that the notification method may be changed depending on whether the state of bending forward continues or the state of slouching continues.

[0058] In S809, the system control unit 202 resets the forward / backward tilt posture time T1 to 0. That is, if the user was temporarily in a slouching or slumping state but returned to a posture close to the reference posture before 5 minutes elapsed, T1 is reset.

[0059] In S810, the system control unit 202 determines whether (III) Zcn - Zcs < x3. That is, in (III), it is possible to determine whether the user has hunched shoulders. Note that x3 is a value such as -1.0 cm, -1.5 cm, -2.0 cm. x3 may be arbitrarily set by the user, or may be automatically changed according to the input user profile (age, gender, physique). For example, if the shoulder width is detected and the shoulder width is large, x3 is changed to a large value. Alternatively, if the gender is female, x3 is changed to a small value.

[0060] Thus, since Zhn, Zhs, Zcn, and Zcs all use the z - coordinate of the chest, it is necessary to detect the position of the user's chest from the notification robot 101. Therefore, the angle 651 in Fig. 6c, that is, the angle formed by the z - axis plane of the notification robot 101 and the left and right shoulders of the user who is the subject, is desirably in the range of approximately -80° to +80°. If it is within that range, the notification robot 101 can be installed at a free position with respect to the user.

[0061] If it is determined that (III) holds, proceed to S813; otherwise, proceed to S815.

[0062] In S811, the system control unit 202 starts counting the hunch - shoulder posture count T2. T2 indicates the time during which the user's posture continues to be in a hunched state relative to the reference posture. If the counting of T2 has already started, in S811, the counting continues.

[0063] In S812, the system control unit 202 determines whether or not the count T2 of the hunchback posture has elapsed for 5 minutes or more. If it is determined that T2 has elapsed for 5 minutes or more, the process proceeds to S813; otherwise, the process proceeds to S815.

[0064] In S813, the system control unit 202 notifies the user that the posture is in the hunchback state. The method of notification is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0065] In S814, the system control unit 202 resets the hunchback posture time T2 to 0. That is, when the user was temporarily in the hunchback state but returned to a posture close to the reference posture before 5 minutes elapsed, T2 is reset.

[0066] In S815, the system control unit 202 determines whether or not the user has stood up. If it is determined that the user has stood up, the process proceeds to S819; otherwise, the process proceeds to S816.

[0067] In S819, the system control unit 202 resets the seating time T3, which will be described later.

[0068] In S816, the system control unit 202 starts or continues to measure the seating time T3. The seating time T3 is a time for detecting that the user is continuously sitting. Note that the measurement of T1, T2, and T3 is performed by the system timer 205.

[0069] In S817, the system control unit 202 determines whether or not the seating time T3 has become 30 minutes or more. If it is determined that T3 has become 30 minutes or more, the process proceeds to S818; otherwise, the process exits the sub - flow of posture detection and notification processing.

[0070] In S818, the system control unit 202 notifies the user that they are sitting too much. The method of notification is that the system control unit 202 controls the vibration control unit 210 to vibrate the notification robot 101 with an arbitrary amplitude and an arbitrary period.

[0071] Here, various notification methods of the notification robot 101 will be described with reference to FIG. 5.

[0072] In FIG. 5, the states of the notification robot 101 are set as A (correct position), B (left swing), and C (right swing).

[0073] In notification pattern 1, the state transitions as A→B→C→A→stop for 1 second→B→C→A→stop. By swinging left and right, even when the notification robot 101 is placed on the desk, the user can easily confirm the notification.

[0074] In notification pattern 2, the state transitions as A→B→C→B→C→B→C→A→stop.

[0075] In notification pattern 3, while in state A, the light notification unit 103 blinks twice at 2 Hz.

[0076] In notification pattern 4, while in state A, the light notification unit 103 lights up once for 2 seconds.

[0077] In notification pattern 5, while in state A, the light notification unit 103 blinks three times at 1 Hz.

[0078] In notification pattern 6, while in state A, the light notification unit 103 lights up brightly once for 1 second.

[0079] Notification pattern 1 is used for the poor posture notification S808 and the hunched shoulder notification S813 in FIG. 8. Notification pattern 2 is used for the sitting too much notification S818.

[0080] Notification pattern 3 is used as a notification when the power button 104 is pressed and the shooting of the notification robot 101 is started (S301).

[0081] Notification pattern 4 is used as the notification when a person is detected (S305). This is also used when a person is detected again after it has been detected in the over-sitting detection S818 that the user has stood up (S815).

[0082] In S309, it is determined whether a person other than the user has been detected. If it is determined that a person other than the user has been detected, the process proceeds to S310; otherwise, the process proceeds to S316.

[0083] In S310, the system control unit 202 collates with the registrant of the face (head) image other than the user. If it matches the registrant, the process proceeds to S314; if it does not match, the process proceeds to S311.

[0084] In S311, the system control unit 202 determines whether a face can be detected. If it is determined that a face has been detected, the process proceeds to S312 and the face is registered. If it is determined that a face cannot be detected, the process proceeds to S313 and the head image is temporarily registered.

[0085] In S313, the system control unit 202 determines whether posture detection is possible.

[0086] S313 will be described with reference to FIG. 9.

[0087] In S314, for the subject within the shooting angle of the notification robot 101, it is determined whether posture detection is possible.

[0088] If it is determined that information for posture detection (face (head), distance between both shoulders, chest) cannot be obtained for a person whose face is moving like the passerby 903, a person in a blind spot 904, a person facing backward 902, etc., the process proceeds to S309. If it is determined that information for posture detection (face (head), distance between both shoulders, chest) can be obtained for persons 901, 905, etc., the process proceeds to S314.

[0089] In S314, the same posture detection and notification as in S308 are performed. When there is insufficient information for posture detection, only the detection of any of posture deterioration, hunched shoulders, or sitting too much is detected and notification is carried out.

[0090] In S314, using the communication I / F unit 208, the posture detection results of users other than the corresponding user are notified to the applications in the user's smartphone or PC.

[0091] In S315, the system control unit 202 determines whether the power of the notification robot 101 has been turned off. If it is determined that the power has been turned off, the process of FIG. 3 is terminated; otherwise, the process returns to S304. Note that in S315, it may be determined as Yes when the power of the PC (personal computer) used by the user is turned off, or it may be determined as Yes according to the arrival of a predetermined time.

[0092] Next, with reference to FIG. 4, a flowchart for obtaining reference posture data will be described. Note that this process starts when proceeding to S303 in FIG. 3.

[0093] In S401, the system control unit 202 controls the optical notification unit 103 to notify the user to assume a reference posture. At this time, notification pattern 5 is used for notification.

[0094] In S402, it waits for a signal indicating that the user is ready. It is determined that the user is ready when the power button 104 is long-pressed. The user's signal of OK may be a gesture such as the user raising both hands as a signal of OK.

[0095] In S403, the system control unit 202 acquires the distances of the head, both shoulders, and chest as the reference posture of the detected subject. At this time, the success of acquiring the reference posture is notified using notification pattern 6.

[0096] In S404, the system control unit 202 normalizes the distances of the reference face, both shoulders, and chest of the subject acquired in S403 to the posture when the notification robot 101 is in front of the user's face directly.

[0097] In S405, based on the values normalized in S404, the system control unit 202 calculates the difference value Zhs between the line connecting both shoulders and the head, and Zcs between the line connecting both shoulders and the chest, and records them in the volatile memory 203.

[0098] According to the embodiments described above, by notifying the posture detection results of persons other than the user to the applications on the user's smartphone or PC, other people than the user can also experience the usability regarding posture correction based on this, which is persuasive to people close to the user and can effectively recommend to improve the purchase intention.

[0099] Furthermore, by recording information regarding the user's posture in the application, graphing it, and presenting it numerically, it may be easier for the user to grasp the tendencies and habits of the user's posture. For example, if it is found that the number of notifications of the backward leaning posture increases towards Friday when Monday to Friday are working days, the user can consciously correct the posture.

[0100] Furthermore, by including discount coupons and the like that can be used at the time of purchase of the notification robot 101 in the information regarding the postures of persons other than the user in the application, it may also aim to further improve the purchase intention.

[0101] Note that the discount coupon is controlled to be issued only once per person to whom the purchase is recommended, and if it has already been recommended at the time of face authentication, it may be determined not to perform posture detection.

[0102] Note that the control may be made possible to turn off so that the user does not perform posture detection of persons other than the user.

[0103] Note that the posture detection of the user and the posture detection of persons other than the user may prioritize the posture detection of the user and reduce the frequency of the posture detection of persons other than the user.

[0104] In addition, when detecting that the user is having a conversation, the notification timing may be changed so that the posture detection result is notified at the timing when the conversation ends.

[0105] Although it has been described that face detection is also performed for persons other than the user and posture detection is performed after identifying the individual, even when the face of a person facing away cannot be seen, by detecting the human body, it is also possible to perform only the detection of excessive sitting.

[0106] Although it has been described that the notification method is to notify by the movement and light of the notification robot 101 itself, in the case of a smartphone, the display unit may be blinked or a push notification for warning may be performed.

[0107] In addition, the notification robot 101 in the present embodiment detects the posture based on the distance measurement sensor 102, but the posture may be detected by a visible light imaging sensor that does not measure the distance. In that case, the distance information is estimated from the captured image obtained by the imaging sensor.

[0108] The various controls described above as being performed by the system control unit 101 may be performed by one piece of hardware, or the entire apparatus may be controlled by a plurality of pieces of hardware sharing the processing.

[0109] In addition, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope not departing from the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely shows one embodiment of the present invention, and it is also possible to appropriately combine the embodiments.

[0110] In addition, in the above-described embodiment, the case where the present invention is applied to the notification robot 101 has been described as an example, but this is not limited to this example, and it can be applied to any electronic device capable of acquiring information regarding the positional relationship and posture between the user and the device. That is, the present invention can be applied to a mobile phone terminal, a portable image viewer, a printer device equipped with a finder, a digital photo frame, a music player, a game machine, an electronic book reader, and the like.

[0111] (Other embodiments) The present invention can also be realized by executing the following processes. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or device via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or device reads and executes the program code. In this case, the program and the recording medium storing the program constitute the present invention.

Explanation of reference numerals

[0112] 101 Notification robot 102 Distance measurement sensor 103 Light notification unit 104 Power button 105 Exterior 106 Desk 201 Lens 202 System control unit 203 Non-volatile memory 204 System memory

Claims

1. an acquisition means for acquiring a positional relationship between the electronic device and the object to be imaged; a control means for normalizing the posture information of the object to be imaged based on the positional relationship acquired by the acquisition means, and performing control so as to issue a predetermined notification when the normalized posture information of the object to be imaged does not satisfy a predetermined condition; a person detection means for detecting a person from the object image acquired by the acquisition means; an individual identification means for identifying an individual of the person acquired by the person detection means; a posture estimation means for estimating the posture of the person identified by the individual identification means; the person is a user individual and a person other than the user individual; a first type of notification means for notifying the user individual of the posture estimation result generated by the posture estimation means; a connection means for connecting to a linked device associated with the user individual; a second type of notification means for notifying the linked device of the posture information of a person other than the user individual generated by the posture estimation means, an electronic device characterized by comprising the same.

2. The electronic device according to claim 1, wherein the second type of notification includes information recommending the purchase of the electronic device according to claim 1.

3. The electronic device according to claim 1, wherein the second type of notification includes a purchase coupon for another person other than the user individual.

4. The electronic device according to claim 3, wherein the purchase coupon is issued only once per target person recommended for purchase.

5. The electronic device according to claim 1, wherein the second type of notification is performed at a timing different from that of the first notification.

6. The electronic device according to claim 1, wherein the first type of notification and the second type of notification prioritize the first type of notification.

7. The electronic device according to claim 1, wherein the posture estimation result includes a determination result of sitting too much, and the notification of sitting too much notifies the first type of notification and the second notification at the same time.

8. The electronic device according to claim 1, wherein the predetermined condition of the posture information in the control means is any one of a forward inclination posture, a backward inclination posture, a hunched shoulder posture, and a sitting state, and the state has elapsed for a predetermined time.

9. The electronic device according to claim 1, wherein the individual identification means includes a determination as to whether a conversation is being held, and does not notify during the conversation, but notifies at the timing when the conversation ends.

10. An acquisition step of acquiring the positional relationship between the electronic device and the object; A control step of normalizing the pose information of the object based on the positional relationship acquired by the acquisition means, and performing control to issue a predetermined notification when the normalized pose information of the object does not satisfy a predetermined condition; A person detection step of detecting a person from the object image acquired by the acquisition means; An individual identification step of identifying the individual of the person acquired by the person detection means; A pose estimation step of estimating the pose of the person identified by the individual identification means; The person is an individual user and a person other than the individual user; A first type of notification step of notifying the individual user of the pose estimation result generated by the pose estimation means; A connection step with a linked device associated with the individual user; A second type of notification step of notifying the linked device of the pose information of a person other than the individual user generated by the pose estimation means. A control method for an electronic device, characterized by comprising:

11. A program for causing a computer to function as each means of the electronic device according to any one of Claims 1 to 9.

12. A computer-readable storage medium storing a program for causing a computer to function as each means of the electronic device according to any one of Claims 1 to 9.

Images