Method and device for generating, on basis of sound intensity and rhythm pattern, exercise program corresponding to music
The method and device synchronize exercise programs with music intensity and rhythm by analyzing sound patterns to adjust resistance and feedback in wearable devices, improving the workout experience through dynamic alignment with music tempo and intensity.
Patent Information
- Application Number
- PCT/KR2024/005509
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-19
- Filing Date
- 2024-04-24
- Publication Date
- 2024-12-26
AI Technical Summary
Existing exercise programs lack synchronization with music intensity and rhythm, leading to suboptimal workout experiences, as they do not dynamically adjust resistance and feedback based on real-time music analysis.
A method and device that analyze music sound intensity and rhythm patterns to generate guide data for exercise programs, adjusting resistance and haptic feedback in wearable devices to match the music, allowing for real-time synchronization of exercise movements with the music tempo and intensity.
Enhances the workout experience by ensuring that exercise intensity and feedback are aligned with the music, providing a more engaging and effective exercise routine through dynamic adjustment based on real-time music analysis.
Smart Images

Figure KR2024005509_26122024_PF_FP_ABST
Abstract
Description
Method and device for generating a movement program corresponding to music based on the loudness and rhythm pattern of the sound
[0001] The disclosure below relates to a method for generating a movement program corresponding to music based on the loudness and rhythmic patterns of the sound.
[0002] Wearable devices, which users can wear on their bodies, are being developed in various forms, depending on their purpose: they can be worn on a single body part, worn across the entire body, or even as replacements for existing body parts. In particular, wearable devices are often developed to assist elderly users in performing specific actions. For example, wearable devices may be developed for the rehabilitation of specific body parts or to replace certain body functions when they are no longer functioning.
[0003] Wearable devices can utilize actuators to assist the user's body movements in the same direction as the user's body movements, thereby assisting the user's body movements, or can resist the user's body movements in the opposite direction to the user's body movements, thereby training the user's body. Such wearable devices can have actuator movements pre-programmed and stored, and can operate the actuators according to the stored movements.
[0004] An electronic device according to one embodiment may include a communication module connected to a wearable device and receiving sensor data including movement information of the wearable device, a processor electrically connected to the communication module, and a memory electrically connected to the processor. The processor may analyze target music to extract intensity and rhythm pattern of the target music. The processor may generate guide data for an exercise program corresponding to the target music based on the extracted intensity and rhythm pattern. The processor may determine exercise content of the exercise program based on the generated guide data and movement information received from the wearable device.
[0005] The processor may generate intensity guide data regarding exercise intensity to be applied to the wearable device when executing the exercise program based on the extracted sound intensity. The processor may generate tempo guide data regarding haptic feedback to be generated in the wearable device when executing the exercise program based on the extracted rhythm tempo.
[0006] The processor can extract frequency information for a plurality of notes that constitute the target music. Based on the extracted frequency information, the processor can identify a chorus interval of the target music. The processor can correct the generated intensity guide data by correcting the exercise intensity for notes included in the identified chorus interval.
[0007] The processor can identify starting points of beats within the target music based on the generated tempo guide data, and determine exercise content of the exercise program so that haptic feedback is generated on the wearable device at some of the identified starting points when the exercise program is executed.
[0008] The processor can play the target music after generating the guide data, and receive movement information of the wearable device from the wearable device in real time according to the playback of the target music.
[0009] The processor can divide the entire playback time of the target music into a plurality of movement sections, and determine a movement mode corresponding to each of the divided movement sections as one of the first mode and the second mode.
[0010] The processor may determine an exercise mode corresponding to the target exercise section based on determining whether there is a candidate motion that matches the movement information of the wearable device received while the target exercise section is being played among a plurality of candidate motions stored in the electronic device.
[0011] The processor may determine, when there is a candidate motion matching the motion information of the wearable device received while the target motion section is being played, the motion mode corresponding to the target motion section as a first mode for the matching candidate motion, and, when there is no candidate motion matching the motion information of the wearable device received while the target motion section is being played, the motion mode corresponding to the target motion section as a second mode representing a free motion section.
[0012] The processor may determine the exercise content of the exercise program so that, when the exercise mode corresponding to the target exercise section is determined as the first mode, resistance is generated at a location according to the exercise motion recorded corresponding to the target exercise section in the wearable device when the target exercise section is played according to the execution of the exercise program.
[0013] The processor determines the exercise content from the start time of the target music of the exercise program to the current time based on the movement information received from the wearable device, and when receiving a request for automatic generation of the exercise program, determines the exercise content from the current time of the target music of the exercise program to the end time based on the determined exercise content.
[0014] The processor may determine whether to insert the received speech information into the exercise content of the exercise program based on whether the received speech information is speech related to an exercise motion or exercise sequence in the target exercise section when the speech information of the wearable device is received while the target exercise section is in progress.
[0015] The above processor may determine the exercise content of the exercise program so that, when inserting the received speech information into the exercise content of the exercise program, the wearable device outputs the received speech information through TTS (Text to Speech) when playing the target exercise section according to the execution of the exercise program.
[0016] A method performed by a processor of an electronic device according to one embodiment may include a step of analyzing target music to extract a sound intensity and a rhythm pattern from the target music. The method may include a step of generating guide data for an exercise program corresponding to the target music based on the extracted sound intensity and the extracted rhythm pattern. The method may include a step of determining an exercise content of the exercise program based on the generated guide data and movement information received from a wearable device connected to the electronic device.
[0017] The step of generating the above guide data may include a step of generating intensity guide data regarding exercise intensity to be applied to the wearable device when executing the exercise program based on the extracted sound intensity, and a step of generating tempo guide data regarding haptic feedback to be generated in the wearable device when executing the exercise program based on the extracted rhythm tempo.
[0018] The step of determining the exercise content of the above exercise program may include a step of playing the target music after generating the guide data, and a step of receiving movement information of the wearable device in real time from the wearable device according to the playing of the target music.
[0019] The step of determining the exercise content of the exercise program may include the step of dividing the entire playback time of the target music into a plurality of exercise sections, and the step of determining an exercise mode corresponding to the target exercise section based on determining whether there is a candidate motion that matches the movement information of the wearable device received while the target exercise section is played among candidate motions stored in the electronic device.
[0020] The step of determining the exercise content of the exercise program may include a step of determining the exercise content from the start time of the target music of the exercise program to the current time based on movement information received from the wearable device, and a step of determining the exercise content from the current time to the end time of the target music of the exercise program based on the determined exercise content when a request for automatic generation of the exercise program is received.
[0021] FIG. 1 is a drawing for explaining an overview of a wearable device worn on a user's body according to one embodiment.
[0022] FIG. 2 is a diagram illustrating a management system including a wearable device and an electronic device according to one embodiment.
[0023] FIG. 3 illustrates a rear schematic diagram of a wearable device according to one embodiment.
[0024] FIG. 4 illustrates a left side view of a wearable device according to one embodiment.
[0025] FIGS. 5A and 5B are diagrams illustrating the configuration of a control system of a wearable device according to one embodiment.
[0026] FIG. 6 is a diagram illustrating interaction between a wearable device and an electronic device according to one embodiment.
[0027] FIG. 7 is a diagram illustrating a configuration of an electronic device according to one embodiment.
[0028] FIG. 8 is a flowchart schematically illustrating a process by which an electronic device according to one embodiment generates an exercise program corresponding to target music.
[0029] FIG. 9 is a diagram illustrating a process in which an electronic device according to one embodiment analyzes target music to generate guide data for an exercise program.
[0030] FIG. 10 is a diagram illustrating a process in which an electronic device according to one embodiment generates intensity guide data of an exercise program based on the loudness of target music.
[0031] FIG. 11 is a diagram illustrating a process by which an electronic device determines haptic feedback of an exercise program based on a rhythm pattern of target music according to one embodiment.
[0032] FIG. 12 is a diagram illustrating a process in which an electronic device determines exercise content of an exercise program by receiving exercise records from a wearable device according to one embodiment.
[0033] FIG. 13 is a diagram illustrating a process in which an electronic device according to one embodiment automatically determines exercise content from the current point in time to the end point of the target music of an exercise program.
[0034] FIG. 14 is a diagram illustrating a process in which an electronic device according to one embodiment changes a recorded exercise motion corresponding to an exercise intensity or exercise section of an exercise program.
[0035] FIG. 15 is a diagram illustrating a process in which an electronic device according to one embodiment generates and executes an exercise program corresponding to target music.
[0036] FIG. 16 is a diagram illustrating a process in which an electronic device according to one embodiment outputs an avatar to a display while determining exercise content of a target exercise section in an exercise program.
[0037] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified and implemented in various forms. Therefore, the actual implementation is not limited to the specific embodiments disclosed, and the scope of this specification includes modifications, equivalents, or alternatives within the technical concepts described in the embodiments.
[0038] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "has" should be understood to indicate the presence of a described feature, number, step, operation, component, part, or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.
[0039] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.
[0040] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.
[0041]
[0042] FIG. 1 is a drawing for explaining an overview of a wearable device worn on a user's body according to one embodiment.
[0043] Referring to FIG. 1, in one embodiment, a wearable device (100) may be a device worn on a user's (110) body to assist the user's (110) walking, exercise, and / or work. In one embodiment, the wearable device (100) may also be used to measure the user's (110) physical ability (e.g., walking ability, exercise ability, exercise posture). In the embodiments, the term 'wearable device' may be replaced with 'wearable robot', 'walking assistance device', or 'exercise assistance device'. The user (110) may be a human or an animal, but is not limited thereto. A wearable device (100) may be worn on a user's (110) body (e.g., lower body (legs, ankles, knees, etc.), upper body (torso, arms, wrists, etc.), or waist) to apply external forces, such as assistance force and / or resistance force, to the body movements of the user (110). Assistance force refers to a force applied in the same direction as the body movement direction of the user (110), and represents a force that assists the body movements of the user (110). Resistance force refers to a force applied in the opposite direction to the body movement direction of the user (110), and represents a force that hinders the body movements of the user (110). The term 'resistance force' may also be referred to as 'exercise load'.
[0044] In one embodiment, the wearable device (100) may operate in a walking assistance mode to assist the walking of a user (110). In the walking assistance mode, the wearable device (100) may assist the walking of the user (110) by applying an assistive force generated from the driving module (120) of the wearable device (100) to the body of the user (110). The wearable device (100) may assist the force required for the walking of the user (110), thereby enabling the user (110) to walk independently or to walk for a long time, thereby expanding the walking ability of the user (110). The wearable device (100) may also help improve the walking of a pedestrian with abnormal walking habits or walking posture.
[0045] In one embodiment, the wearable device (100) may operate in an exercise assistance mode to enhance the exercise effect of the user (110). In the exercise assistance mode, the wearable device (100) may impede the body movement of the user (110) or provide resistance to the body movement of the user (110) by applying a resistance force generated from the driving module (120) to the body of the user (110). If the wearable device (100) is a hip-type wearable device worn on the waist (or pelvis) and legs (e.g., thighs) of the user (110), the wearable device (100) may provide an exercise load to the leg movement of the user (110) while being worn on the legs, thereby further enhancing the exercise effect on the legs of the user (110). In one embodiment, the wearable device (100) may also apply an assistive force to the body of the user (110) to assist the exercise of the user (110). For example, when a disabled person or an elderly person wears a wearable device (100) to exercise, the wearable device (100) may provide assistive force to assist body movements during the exercise. In one embodiment, the wearable device (100) may provide a combination of assistive force and resistance force by exercise section or time section, such as providing assistive force in some exercise sections and resistance force in other exercise sections.
[0046] In one embodiment, the wearable device (100) may operate in a physical ability measurement mode for measuring the physical ability of a user (110). The wearable device (100) may measure the movement information of the user by using sensors (e.g., an angle sensor (125), an inertial measurement unit (IMU) (135)) provided in the wearable device (100) while the user is walking or exercising, and may evaluate the physical ability of the user based on the measured movement information. For example, the gait index or exercise ability index (e.g., muscle strength, endurance, balance, exercise movement) of the user (110) may be estimated through the movement information of the user (100) measured by the wearable device (100). The physical ability measurement mode may include an exercise movement measurement mode for measuring the exercise movement of the user.
[0047] In various embodiments of the present disclosure, for convenience of explanation, a hip-type wearable device (100) as illustrated in FIG. 1 is used as an example, but the present disclosure is not limited thereto. As described above, the wearable device (100) may be worn on other body parts (e.g., upper arms, lower arms, hands, calves, feet) other than the waist and legs (particularly, thighs), and the shape and configuration of the wearable device (100) may vary depending on the body part on which it is worn.
[0048] According to one embodiment, the wearable device (100) includes a support frame (e.g., a waist support frame (20)) for supporting the body of the user (110) when the wearable device (100) is worn on the body of the user (110), a drive module (120) for generating a torque applied to the legs of the user (110) (e.g., a drive module (35, 45) of FIG. 3), a leg drive frame for transmitting the torque generated by the drive module (120) to the legs of the user (110) (e.g., a leg drive frame (50, 55) of FIG. 3), a sensor module including one or more sensors for obtaining sensor data including movement information about the body movement of the user (110) (e.g., a sensor module (520) of FIG. 5A), and a control module (130) for controlling the wearable device (100) (e.g., a control module (510) of FIGS. 5A and 5B). May include.
[0049] The sensor module may include an angle sensor (125) and an inertial sensor (135). The angle sensor (125) may measure a rotation angle of a leg drive frame of the wearable device (100) corresponding to a hip joint angle value of a user (110). The rotation angle of the leg drive frame measured by the angle sensor (125) may be estimated to be a hip joint angle value (or leg angle value) of the user (110). The angle sensor (125) may include, for example, an encoder and / or a hall sensor. In one embodiment, the angle sensor (125) may be disposed near a location where a motor included in the drive module (120) is connected to the leg drive frame. The inertial sensor (135) may include an acceleration sensor and / or an angular velocity sensor, and may measure a change in acceleration and / or angular velocity according to a movement of the user (110). The inertial sensor (135) can measure, for example, a movement value of a waist support frame or base body (e.g., base body (80) of FIG. 3) of a wearable device (100). The movement value of the waist support frame or base body measured by the inertial measurement device (135) can be estimated as an upper body movement value of the user (110).
[0050] In one embodiment, the control module (130) and the inertial sensor (135) may be placed within a base body (e.g., base body (80) of FIG. 3) of the wearable device (100). The base body may be positioned at the lumbar region (waist region) of the user (110) while the user (110) is wearing the wearable device (100). The base body may be formed or attached to the outside of the waist support frame of the wearable device (100).
[0051]
[0052] FIG. 2 is a diagram illustrating a management system including a wearable device and an electronic device according to one embodiment.
[0053] Referring to FIG. 2, the management system (200) may include a wearable device (100), an electronic device (210), another wearable device (220), and a server (230). In one embodiment, the management system (200) may omit at least one of these devices (e.g., another wearable device (220) or the server (230)), or may add one or more other devices (e.g., a dedicated controller device for the wearable device (100)).
[0054] In one embodiment, the wearable device (100) may be worn on the user's body in a walking assistance mode to assist the user's movements. For example, the wearable device (100) may be worn on the user's leg to generate an assistive force to assist the user's leg movements, thereby assisting the user's walking.
[0055] In one embodiment, the wearable device (100) may generate and apply to the user's body a resistance force to hinder the user's body movement or an assistive force to assist the user's body movement in order to enhance the user's exercise effect in the exercise assistance mode. In the exercise assistance mode, the user may select an exercise program (e.g., squats, split lunges, dumbbell squats, lunges and knee ups, stretching, etc.) to be exercised using the wearable device (100) through the electronic device (210) and / or an exercise intensity to be applied to the wearable device (100). The wearable device (100) may control the drive module of the wearable device (100) according to the exercise program selected by the user, and may acquire sensor data including information on the user's movement through the sensor module. The wearable device (100) may adjust the strength of the resistance force or the assistive force to be applied to the user according to the exercise intensity selected by the user. For example, the wearable device (100) can control the drive module to generate a resistance corresponding to the exercise intensity selected by the user.
[0056] In one embodiment, the wearable device (100) may be used to measure a user's physical ability in conjunction with an electronic device (210). The wearable device (100) may operate in a physical ability measurement mode, which is a mode for measuring the user's physical ability under the control of the electronic device (210), and may transmit sensor data acquired by the user's movements in the physical ability measurement mode to the electronic device (210). The electronic device (210) may analyze the sensor data received from the wearable device (100) to evaluate the user's physical ability.
[0057] The electronic device (210) can communicate with the wearable device (100), remotely control the wearable device (100), or provide the user with status information about the status of the wearable device (100) (e.g., booting status, charging status, sensing status, error status). The electronic device (210) can receive sensor data acquired by a sensor module of the wearable device (100) from the wearable device (100), and estimate the user's physical ability or exercise results based on the received sensor data. The electronic device (210) can provide the user's physical ability or exercise results to the user through a graphical user interface.
[0058] In one embodiment, a user may execute a program (e.g., an application) on an electronic device (210) to control a wearable device (100), and the user may adjust the operation or setting values (e.g., the torque intensity output from a driving module (e.g., the driving module (35, 45) of FIG. 3), the volume of audio output from an audio output module (e.g., the audio output module (550) of FIGS. 5A and 5B), the brightness of a lighting unit (e.g., the lighting unit (85) of FIG. 3)) of the wearable device (100) through the program. The program executed on the electronic device (210) may provide a graphical user interface (GUI) for interaction with the user. The electronic device (210) may be a device of various forms. For example, the electronic device (210) may include, but is not limited to, a portable communication device (e.g., a smartphone), a computer device, an access point, a portable multimedia device, or a home appliance device (e.g., a television, an audio device, a projector device).
[0059] According to one embodiment, the electronic device (210) may be connected to the server (230) using short-range wireless communication or cellular communication. The server (230) may receive user profile information of a user using the wearable device (100) from the electronic device (210) and store and manage the received user profile information. The user profile information may include, for example, information on at least one of name, age, gender, height, weight, or body mass index (BMI). The server (230) may receive exercise history information on exercise performed by the user from the electronic device (210) and store and manage the received exercise history information. The server (230) may provide various exercise programs or physical ability measurement programs that may be provided to the user to the electronic device (210).
[0060] According to one embodiment, the wearable device (100) and / or the electronic device (210) can be connected to another wearable device (220). The other wearable device (220) can be, for example, wireless earphones (222), a smartwatch (224), or smartglasses (226), but is not limited to the aforementioned devices. In one embodiment, the smartwatch (224) can measure a bio-signal including heart rate information of the user, and transmit the measured bio-signal to the electronic device (210) and / or the wearable device (100). The electronic device (210) can estimate heart rate information of the user (e.g., current heart rate, maximum heart rate, average heart rate) based on the bio-signal received from the smartwatch (224), and can provide the estimated heart rate information to the user.
[0061] In one embodiment, the user's exercise result information, physical ability information, and / or exercise motion evaluation information determined by the electronic device (210) may be transmitted to another wearable device (220) and provided to the user through the other wearable device (220). Status information of the wearable device (100) may also be transmitted to another wearable device (220) and provided to the user through the other wearable device (220). In one embodiment, the wearable device (100), the electronic device (210), and the other wearable device (220) may be connected to each other through wireless communication (e.g., Bluetooth communication, Wi-Fi communication).
[0062] In one embodiment, the wearable device (100) may provide (or output) feedback (e.g., visual feedback, auditory feedback, tactile feedback) corresponding to the state of the wearable device (100) according to a control signal received from the electronic device (210). For example, the wearable device (100) may provide visual feedback through a lighting unit (e.g., lighting unit (85) of FIG. 3) and may provide auditory feedback through an audio output module (e.g., audio output module (550) of FIGS. 5A and 5B).
[0063]
[0064] Fig. 3 illustrates a rear schematic diagram of a wearable device according to one embodiment. Fig. 4 illustrates a left side view of a wearable device according to one embodiment.
[0065] Referring to FIGS. 3 and 4, a wearable device (100) according to one embodiment may include a base body (80), a waist support frame (20), a driving module (35, 45), a leg driving frame (50, 55), a thigh fastening part (1, 2), and a waist fastening part (60). The base body (80) may include a lighting unit (85). In one embodiment, at least one of these components (e.g., the lighting unit (85)) may be omitted from the wearable device (100), or one or more other components may be added.
[0066] The base body (80) can be positioned on the user's lower back while the user wears the wearable device (100). The base body (80) can be mounted on the user's lower back to provide a cushioning feeling to the user's lower back and support the user's lower back. The base body (80) can be hung over the user's buttocks (hip area) to prevent the wearable device (100) from falling downward due to gravity while the user wears the wearable device (100). The base body (80) can distribute a portion of the weight of the wearable device (100) to the user's lower back while the user wears the wearable device (100). The base body (80) can be connected to the lower back support frame (20). The base body (80) can be provided with lower back support frame connection elements (not shown) that can be connected to the lower back support frame (20) at both ends.
[0067] In one embodiment, a lighting unit (85) may be disposed on the outer surface of the base body (80). The lighting unit (85) may include a light source (e.g., a light emitting diode (LED)). The lighting unit (85) may emit light under the control of a processor (not shown) (e.g., the processor (512) of FIGS. 5A and 5B). According to an embodiment, the processor may control the lighting unit (85) so that visual feedback corresponding to the status of the wearable device (100) may be provided (or output) through the lighting unit (85).
[0068] The lumbar support frame (20) can support the user's body (e.g., waist) when the wearable device (100) is worn on the user's body. The lumbar support frame (20) can extend from both ends of the base body (80). The user's lumbar region can be accommodated on the inside of the lumbar support frame (20). The lumbar support frame (20) can include at least one rigid body beam. Each beam can have a curved shape having a preset curvature so as to surround the user's lumbar region. A lumbar fastening part (60) can be connected to an end of the lumbar support frame (20). A driving module (35, 45) can be connected to the lumbar support frame (20).
[0069] In one embodiment, a processor, a memory, an inertial measurement unit (e.g., an inertial measurement unit (135) of FIG. 1, an inertial measurement unit (522) of FIG. 5B), a communication module (e.g., a communication module (516) of FIGS. 5A and 5B), an audio output module (e.g., an audio output module (550) of FIGS. 5A and 5B), and a battery (not shown) may be disposed inside the base body (80). The base body (80) may protect the components disposed inside. The processor may generate a control signal for controlling the operation of the wearable device (100). The processor may control the actuators of the drive modules (35, 45). The processor and the memory may be included in a control circuit. The control circuit may further include a power supply circuit for supplying power from the battery to each component of the wearable device (100).
[0070] In one embodiment, the wearable device (100) may include a sensor module (not shown) (e.g., sensor module (520) of FIG. 5A) that obtains sensor data from one or more sensors. The sensor module may obtain sensor data including movement information of the user and / or movement information of components of the wearable device (100). The sensor module may include, but is not limited to, an inertial measurement device (e.g., inertial measurement device (135) of FIG. 1 and inertial measurement device (522) of FIG. 5B) for measuring a movement value of the user's upper body or a movement value of the lumbar support frame (20) and an angle sensor (e.g., angle sensor (125) of FIG. 1 and a first angle sensor (524) and a second angle sensor (524-1) of FIG. 5B) for measuring a movement value of the user's hip joint or a movement value of the leg drive frame (50, 55). For example, the sensor module may further include at least one of a position sensor, a temperature sensor, a biosignal sensor, or a proximity sensor.
[0071] The waist fastening member (60) can be connected to the waist support frame (20) and can secure the waist support frame (20) to the user's waist. The waist fastening member (60) can include, for example, a pair of belts.
[0072] The drive module (35, 45) can generate an external force (or torque) applied to the user's body based on a control signal generated by the processor. For example, the drive module (35, 45) can generate an assistive force or a resistance force applied to the user's leg. In one embodiment, the drive module (35, 45) can include a first drive module (45) positioned corresponding to the user's right hip joint position and a second drive module (35) positioned corresponding to the user's left hip joint position. The first drive module (45) can include a first actuator and a first joint member, and the second drive module (35) can include a second actuator and a second joint member. The first actuator can provide power transmitted to the first joint member, and the second actuator can provide power transmitted to the second joint member. The first actuator and the second actuator may each include a motor (e.g., motors 534 and 534-1 of FIG. 5B) that receive power from a battery and generate power (or torque). When powered and driven, the motor may generate a force to assist the user's body movement (assistive force) or a force to impede the body movement (resistive force). In one embodiment, the control module may control the magnitude and direction of the force generated by the motor by adjusting the voltage and / or current supplied to the motor.
[0073] In one embodiment, the first joint member and the second joint member can receive power from the first actuator and the second actuator, respectively, and apply an external force to the user's body based on the received power. The first joint member and the second joint member can be disposed at positions corresponding to the user's joints, respectively. One side of the first joint member can be connected to the first actuator, and the other side can be connected to the first leg drive frame (55). The first joint member can be rotated by the power received from the first actuator. An encoder or a hall sensor that can act as an angle sensor for measuring a rotation angle of the first joint member or the first leg drive frame (55) (corresponding to the user's joint angle) can be disposed on one side of the first joint member. One side of the second joint member can be connected to the second actuator, and the other side can be connected to the second leg drive frame (50). The second joint member (333) can be rotated by power transmitted from the second actuator. An encoder or hall sensor that can function as an angle sensor for measuring the rotation angle of the second joint member or the second leg drive frame (50) can also be placed on one side of the second joint member.
[0074] In one embodiment, the first actuator may be disposed laterally of the first joint member, and the second actuator may be disposed laterally of the second joint member. The rotational axis of the first actuator and the rotational axis of the first joint member may be disposed to be spaced apart from each other, and the rotational axis of the second actuator and the rotational axis of the second joint member may also be disposed to be spaced apart from each other. However, the present invention is not limited thereto, and the actuator and the joint member may share a rotational axis. In one embodiment, each actuator may be disposed to be spaced apart from the joint member. In this case, the drive module (35, 45) may further include a power transmission module (not shown) that transmits power from the actuator to the joint member. The power transmission module may be a rotating body such as a gear, or a longitudinal member such as a wire, a cable, a string, a spring, a belt, or a chain. However, the scope of the embodiment is not limited by the positional relationship between the actuator and joint member and the power transmission structure described above.
[0075] In one embodiment, the leg drive frame (50, 55) can transmit the torque generated by the drive module (35, 45) to the user's body (e.g., leg) when the wearable device (100) is worn on the user's leg. The transmitted torque can act as an external force applied to the movement of the user's leg. One end of the leg drive frame (50, 55) is connected to a joint member and can rotate, and the other end of the leg drive frame (50, 55) is connected to a thigh fastening part (1, 2), so that the leg drive frame (50, 55) can support the user's thigh while transmitting the torque generated by the drive module (35, 45) to the user's thigh. For example, the leg drive frame (50, 55) can push or pull the user's thigh. The leg drive frame (50, 55) can extend along the longitudinal direction of the user's thigh. The leg drive frame (50, 55) can be bent to wrap at least a portion of the circumference of the user's thigh. The leg drive frame (50, 55) may include a first leg drive frame (55) for transmitting torque to the user's right leg and a second leg drive frame (50) for transmitting torque to the user's left leg.
[0076] The thigh fastening parts (1, 2) are connected to the leg drive frame (50, 55) and can secure the wearable device (100) to the user's leg (particularly, the thigh). For example, the thigh fastening parts (1, 2) may include a first thigh fastening part (2) for securing the wearable device (100) to the user's right thigh and a second thigh fastening part (1) for securing the wearable device (100) to the user's left thigh.
[0077] In one embodiment, the first thigh fastening unit (2) may include a first cover, a first fastening frame, and a first strap, and the second thigh fastening unit (1) may include a second cover, a second fastening frame, and a second strap. The first cover and the second cover may apply torque generated from the driving module (35, 45) to the user's thigh. The first cover and the second cover may be disposed on one side of the user's thigh and may push or pull the user's thigh. The first cover and the second cover may be disposed on the front side of the user's thigh, for example. The first cover and the second cover may be disposed along the circumferential direction of the user's thigh. The first cover and the second cover may extend in both directions with the other end of the leg driving frame (50, 55) as the center, and may include a curved surface corresponding to the user's thigh. One end of the first cover and the second cover may be connected to the fastening frame, and the other end may be connected to the strap.
[0078] The first fastening frame and the second fastening frame may be arranged to, for example, surround at least a portion of the user's thigh, thereby preventing the user's thigh from being detached from the wearable device (100). The first fastening frame may have a fastening structure connecting the first cover and the first strap, and the second fastening frame may have a fastening structure connecting the second cover and the second strap.
[0079] The first strap may encircle the user's right thigh, the remaining portion not covered by the first cover and the first fastening frame, and the second strap may encircle the user's left thigh, the remaining portion not covered by the second cover and the second fastening frame. The first strap and the second strap may comprise, for example, an elastic material (e.g., a band).
[0080]
[0081] FIGS. 5A and 5B are diagrams illustrating the configuration of a control system of a wearable device according to one embodiment.
[0082] Referring to FIG. 5A, a wearable device (e.g., wearable device (100)) may be controlled by a control system (500). The control system (500) may include a control module (510), a communication module (516), a sensor module (520), a drive module (530), an input module (540), and an audio output module (550). The drive module (530) may include a motor (534) capable of generating power (e.g., torque) and a motor driver circuit (532) for driving the motor (534). Although the embodiment of FIG. 5A illustrates a drive module (530) including one motor driver circuit (532) and one motor (534), this is merely an example. Referring to FIG. 5B, in the control system (500-1), as in the embodiment illustrated in FIG. 5B, there may be a plurality of motor driver circuits (532, 532-1) and motors (534, 534-1) (e.g., two or more). The drive module (530) including the motor driver circuit (532) and the motor (534) may correspond to the first drive module (45) of FIG. 3, and the drive module (530-1) including the motor driver circuit (532-1) and the motor (534-1) may correspond to the second drive module (35) of FIG. 3. The description of each of the motor driver circuit (532) and the motor (534) described below may also be applied to the motor driver circuit (532-1) and the motor (534-1) illustrated in FIG. 5B.
[0083] Returning to FIG. 5A, the sensor module (520) may include at least one sensor. The sensor module (520) may include sensor data including movement information of the user or movement information of the wearable device. The sensor module (520) may transmit the acquired sensor data to the control module (510). The sensor module (520) may include an inertial sensor (522) and an angular sensor (e.g., a first angular sensor (524) and a second angular sensor (524-1)) as illustrated in FIG. 5B. The inertial sensor (522) may measure movement values of the user's upper body. For example, the inertial sensor (522) may sense accelerations of the X-axis, Y-axis, and Z-axis and angular velocities of the X-axis, Y-axis, and Z-axis according to the movement of the user. In addition, the inertial sensor (522) may obtain movement values (e.g., acceleration values and angular velocity values) of the waist support frame of the wearable device. The angle sensor can measure the hip joint angle value according to the movement of the user's leg. Sensor data that can be measured by the angle sensor can include, for example, the hip joint angle value of the right leg, the hip joint angle value of the left leg, and information on the movement direction of the leg. The first angle sensor (524) of FIG. 5B can obtain the hip joint angle value of the user's right leg, and the second angle sensor (524-1) can obtain the hip joint angle value of the user's left leg. Each of the first angle sensor (524) and the second angle sensor (524-1) can include, for example, an encoder and / or a Hall sensor. In addition, the first and second angle sensors (520, 520-1) can obtain the movement value of the leg drive frame of the wearable device. For example, the first angle sensor (524) can obtain the movement value of the first leg drive frame (55), and the second angle sensor (524-1) can obtain the movement value of the second leg drive frame (50).
[0084] In one embodiment, the sensor module (520) may further include a position sensor for obtaining a position value of the wearable device, a proximity sensor for detecting the proximity of an object, a biosignal sensor for detecting a user's biosignal, and a temperature sensor for measuring the ambient temperature.
[0085] The input module (540) can receive commands or data to be used in a component of the wearable device (e.g., a processor (512)) from an external source (e.g., a user) of the wearable device. The input module (540) can include, for example, a key (e.g., a button) or a touch screen.
[0086] The audio output module (550) can output audio signals to the outside of the wearable device. The audio output module (550) can include a speaker that plays a guide audio signal (e.g., a driving start sound, an operation error notification sound), music content, or a guide voice.
[0087] In one embodiment, the control system (500) may further include a battery (not shown) for supplying power to each component of the wearable device. The wearable device may convert the power of the battery to an operating voltage of each component of the wearable device and supply the converted power to each component.
[0088] The drive module (530) can generate an external force applied to the user's leg under the control of the control module (510). The drive module (530) is located at a location corresponding to the user's hip joint position and can generate a torque applied to the user's leg based on a control signal generated by the control module (510). The control module (510) can transmit the control signal to the motor driver circuit (532), and the motor driver circuit (532) can control the operation of the motor (534) by generating a current signal (or voltage signal) corresponding to the control signal and supplying it to the motor (534). Depending on the control signal, the current signal may not be supplied to the motor (534). When the motor (534) is driven by supplying a current signal to the motor (534), the motor (534) can generate a force that assists the user's leg movement or a torque that hinders the leg movement.
[0089] The control module (510) controls the overall operation of the wearable device and can generate control signals for controlling each component (e.g., the driving module (530)). The control module (510) may include a processor (512) and a memory (514).
[0090] The processor (512) may, for example, execute software to control at least one other component (e.g., a hardware or software component) of a wearable device connected to the processor (512), and may perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (512) may store instructions or data received from another component (e.g., a communication module (516)) in the memory (514), process the instructions or data stored in the memory (514), and store the resulting data after the processing in the memory (514). According to one embodiment, the processor (512) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently of or together with the main processor. The auxiliary processor may be implemented separately from the main processor or as a part thereof.
[0091] The memory (514) can store various data used by at least one component (e.g., processor (512)) of the control module (510). The data can include, for example, input data or output data for software, sensor data, and commands related thereto. The memory (514) can include volatile memory or non-volatile memory (e.g., RAM, DRAM, SRAM).
[0092] The communication module (516) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the control module (510) and other components of the wearable device or an external electronic device (e.g., the electronic device (210) or the second wearable device (220) of FIG. 2), and the performance of communication through the established communication channel. The communication module (516) may, for example, transmit sensor data acquired by the sensor module (520) to an external electronic device (e.g., the electronic device (210) of FIG. 2) and receive a control signal from the electronic device. In one embodiment, the communication module (516) may include one or more communication processors that operate independently from the processor (512) and support direct (e.g., wired) communication or wireless communication. In one embodiment, the communication module (516) may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) and / or a wired communication module. Any of these communication modules may communicate with other components of the wearable device and / or external electronic devices via a short-range communication network such as, for example, Bluetooth, WiFi (wireless fidelity), ANT, or IrDA (infrared data association), or a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN).
[0093]
[0094] FIG. 6 is a diagram illustrating interaction between a wearable device and an electronic device according to one embodiment.
[0095] Referring to FIG. 6, the wearable device (100) can communicate with the electronic device (210). For example, the electronic device (210) can be a user terminal of a user using the wearable device (100) or a dedicated controller device for the wearable device (100). According to one embodiment, the wearable device (100) and the electronic device (210) can be connected to each other through short-range wireless communication (e.g., Bluetooth communication, Wi-Fi communication).
[0096] In one embodiment, the electronic device (210) may execute an application for checking the status of the wearable device (100) or controlling or operating the wearable device (100). By executing the application, a screen of a user interface (UI) for controlling the operation of the wearable device (100) or determining the operation mode of the wearable device (100) may be displayed on the display (212) of the electronic device (210). The UI may be, for example, a graphical user interface (GUI).
[0097] In one embodiment, a user may input a command to control the operation of the wearable device (100) (e.g., a command to execute a walking assistance mode, an exercise assistance mode, or a physical ability measurement mode) or change the settings of the wearable device (100) through a GUI screen on a display (212) of the electronic device (210). The electronic device (210) may generate a control command (or a control signal) corresponding to the motion control command or setting change command input by the user, and transmit the generated control command to the wearable device (100). The wearable device (100) may operate according to the received control command, and transmit a control result according to the control command and / or sensor data measured by a sensor module of the wearable device (100) to the electronic device (210). The electronic device (210) may analyze the control result and / or sensor data to provide the user with result information (e.g., walking ability information, exercise ability information, exercise movement evaluation information) through the GUI screen.
[0098]
[0099] FIG. 7 is a diagram illustrating a configuration of an electronic device according to one embodiment.
[0100] Referring to FIG. 7, the electronic device (210) may include a processor (710), a memory (720), a communication module (730), a display module (740), an audio output module (750), and an input module (760). In one embodiment, the electronic device (210) may omit at least one of these components (e.g., an audio output module (750)), or may have one or more other components added (e.g., a sensor module, a haptic module, a battery).
[0101] The processor (710) may control at least one other component (e.g., hardware or software component) of the electronic device (210) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (710) may store commands or data received from other components (e.g., communication module (730)) in the memory (720), process the commands or data stored in the memory (720), and store result data in the memory (720).
[0102] According to one embodiment, the processor (710) may include a main processor (e.g., a central processing unit or an application processor) or a secondary processor (e.g., a graphics processing unit, a neural network processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or in conjunction therewith.
[0103] The memory (720) can store various data used by at least one component (e.g., the processor (710) or the communication module (730)) of the electronic device (210). The data can include, for example, input data or output data for a program (e.g., an application) and commands related thereto. The memory (720) can include at least one instruction executable by the processor (710). The memory (720) can include volatile memory or non-volatile memory.
[0104] The communication module (730) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (210) and another electronic device (e.g., wearable device (100), another wearable device (220), server (230)), and the performance of communication through the established communication channel. The communication module (730) may include a communication circuit for performing a communication function. The communication module (730) may operate independently from the processor (710) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (290) may include a wireless communication module (e.g., a Bluetooth communication module, a cellular communication module, a Wi-Fi communication module, or a GNSS communication module) or a wired communication module (e.g., a LAN communication module or a power line communication module) that performs wireless communication. The communication module (730) may, for example, transmit a control command to the wearable device (100) and receive at least one of sensor data including body movement information of a user wearing the wearable device (100), status data of the wearable device (100), or control result data corresponding to the control command from the wearable device (100).
[0105] The display module (740) can visually provide information to an external device (e.g., a user) of the electronic device (210). The display module (740) can include, for example, an LCD or OLED display, a holographic device, or a projector device. The display module (740) can further include a control circuit for controlling display operation. In one embodiment, the display module (740) can further include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch. The display module (740) can output a user interface screen for controlling the wearable device (100) or providing various information (e.g., exercise evaluation information, setting information of the wearable device (100).
[0106] The audio output module (750) can output an audio signal to the outside of the electronic device (210). The audio output module (750) can include a speaker that plays a guide audio signal (e.g., a driving start sound, an operation error notification sound), music content, or a guide voice based on the status of the wearable device (100). For example, if it is determined that the wearable device (100) is not normally worn on the user's body, the audio output module (750) can output a guide voice to inform the user of the normal wearing of the wearable device (100) or to induce the user to wear the wearable device (100) normally.
[0107] The input module (760) can receive commands or data to be used in a component of the electronic device (210) (e.g., the processor (710)) from an external source (e.g., a user) of the electronic device (210). The input module (760) can include an input component circuit and can receive user input. The input module (760) can include, for example, a key (e.g., a button) and / or a touch recognition circuit for recognizing a touch on a screen.
[0108]
[0109] FIG. 8 is a flowchart schematically illustrating a process by which an electronic device according to one embodiment generates an exercise program corresponding to target music.
[0110] In operation (810), the electronic device can analyze the target music to extract sound intensity and rhythm pattern in the target music.
[0111] In one embodiment, the electronic device can generate a movement program corresponding to a target music. The movement program corresponding to the target music allows the user to perform various movement movements in sync with the target music by wearing the wearable device.
[0112] In one embodiment, an electronic device can receive multiple candidate music tracks from an external server and display the received multiple candidate music tracks on a display. The music program creator can select target music tracks from among the multiple candidate music tracks displayed on the display to generate an exercise program. The electronic device can receive the user's selection of the target music tracks.
[0113] In one embodiment, an electronic device can analyze target music to extract properties of the target music. More specifically, the electronic device can analyze the target music to extract loudness and rhythmic patterns from the target music. Loudness can represent sound amplitude, and the unit can be expressed in decibels (dB). By extracting the loudness of the target music, the electronic device can identify the loudness of each of the multiple notes that make up the target music. The rhythmic pattern is formed according to the beat, and can represent the emphasis and pattern of the music by combining the relative length, strong and weak, and interval of notes and rests. By extracting the rhythmic pattern from the target music, the electronic device can identify the length, strong and weak, and interval of notes and rests in the target music.
[0114] In operation (820), the electronic device may generate guide data for an exercise program corresponding to the target music based on the extracted sound intensity and the extracted rhythm pattern. The guide data may represent data that helps a creator more easily create an exercise program. The guide data may include intensity guide data regarding an exercise intensity to be applied to a wearable device when executing the exercise program and tempo guide data regarding haptic feedback to be generated in the wearable device when executing the exercise program. In other words, the electronic device may generate intensity guide data and tempo guide data as guide data for the exercise program.
[0115] In operation (830), the electronic device may determine the exercise content of the exercise program based on the generated guide data and movement information received from the wearable device. The exercise content of the exercise program may include the exercise intensity applied to the wearable device when the target music is played, the haptic feedback to be generated in the wearable device, and the exercise motions and number of exercises to be performed for each exercise section.
[0116] In one embodiment, a creator of an exercise program may wear a wearable device to determine the exercise content of the exercise program. The electronic device may play target music after generating guide data, and the creator may perform various exercise movements through the wearable device based on the playback of the target music. When the creator performs the exercise movements through the wearable device, the wearable device may transmit sensor data containing movement information of the wearable device to the electronic device in real time. The electronic device may determine the movement information of the wearable device through the received sensor data and analyze the movement information of the wearable device to determine the exercise content of the exercise program.
[0117]
[0118] FIG. 9 is a diagram illustrating a process in which an electronic device according to one embodiment analyzes target music to generate guide data for an exercise program.
[0119] In one embodiment, the electronic device (901) can visualize the creation status of an exercise program and output it to a display. FIG. 9 is a diagram illustrating a display screen showing the results of analysis of target music before the electronic device (901) receives exercise records from a wearable device (e.g., the wearable device (100) of FIG. 1).
[0120] In one embodiment, the electronic device (901) may analyze target music to generate guide data for an exercise program corresponding to the target music. The guide data may include intensity guide data (911) regarding exercise intensity to be applied to the wearable device when executing the exercise program, and tempo guide data (912) regarding haptic feedback to be generated on the wearable device when executing the exercise program.
[0121] In one embodiment, the electronic device (901) can extract the sound intensity (910) of the target music. The electronic device (901) can generate intensity guide data (911) based on the extracted sound intensity (910). The intensity guide data (911) can indicate information about the exercise intensity to be applied to the wearable device when the target music is played during execution of the exercise program. Setting an exercise intensity of '3' to one note constituting the target music in the intensity guide data (911) can indicate that the exercise intensity to be applied to the wearable device at the time when the corresponding note is played when the target music is played during execution of the exercise program is set to an intensity of '3'. Hereinafter, for the convenience of explanation, the exercise intensity that can be applied to the wearable device is defined as being between an intensity of '0' and an intensity of '5'. The wearable device can control a driving module (e.g., a driving module (35, 45) of FIG. 3) of the wearable device so as to generate a resistance force corresponding to the exercise intensity.
[0122] FIG. 10 is a diagram illustrating a process in which an electronic device according to one embodiment generates intensity guide data for an exercise program based on the loudness of sound in target music.
[0123] In one embodiment, an electronic device (e.g., the electronic device 901 of FIG. 9) may generate intensity guide data (1011) for an exercise program based on the loudness (1010) of a sound in the target music. The intensity guide data (1011) may include information regarding an exercise intensity corresponding to each of a plurality of notes constituting the target music. The electronic device may set the exercise intensity of a sound to be higher as the loudness of the sound ... The electronic device may set the exercise intensity for a note having a second volume (e.g., 90 dB) lower than the first volume within the target music to a second exercise intensity (e.g., an exercise intensity of '3') lower than the first exercise intensity.
[0124] In one embodiment, the electronic device can individually set a maximum exercise intensity (e.g., an exercise intensity of '4') and a minimum exercise intensity (e.g., an exercise intensity of '1') when generating intensity guide data (1011). The electronic device can correspond the loudest sound intensity among the sound intensities of the plurality of notes constituting the target music to the set maximum exercise intensity, and can correspond the quietest sound intensity among the sound intensities of the plurality of notes constituting the target music to the set minimum exercise intensity. The electronic device can set the exercise intensity of a note between the maximum exercise intensity and the minimum exercise intensity so as to be proportional to the sound intensity of the corresponding note.
[0125]
[0126] In one embodiment, the electronic device can correct intensity guide data (1011) generated based on the loudness of notes in the target music. Various methods by which the electronic device corrects the intensity guide data (1011) are described below.
[0127] In one embodiment, the electronic device may correct intensity guide data (1011) based on frequency information (1051) in the target music. First, the electronic device may extract frequency information (1051) for a plurality of notes constituting the target music. The electronic device may identify chorus sections (1061, 1062) of the target music based on the frequency information (1051) extracted from the target music. The electronic device may correct intensity guide data (1011) by correcting the exercise intensity for a plurality of notes included in the identified chorus sections (1061, 1062). In one embodiment, the electronic device may individually increase the exercise intensity of the plurality of notes included in the identified chorus sections (1061, 1062) by a preset ratio. The preset ratio may be, but is not limited to, 20%. For example, if the exercise intensity for one note included in the refrain section in the intensity guide data (1011) is set to '4', the electronic device may correct the exercise intensity for the note by increasing it by 20% to '4.8'. In another embodiment, the electronic device may correct the intensity guide data (1011) so that the exercise intensities of a plurality of notes included in the identified refrain sections (1061, 1062) collectively become the maximum exercise intensity set (e.g., an exercise intensity of '4'). In another embodiment, the electronic device may correct the intensity guide data (1011) by individually increasing the exercise intensities of a plurality of notes included in the identified refrain sections (1061, 1062) by a preset exercise intensity (e.g., an exercise intensity of '1').
[0128] Hereinafter, a process by which an electronic device identifies chorus sections (1061, 1062) of a target music based on frequency information (1051) extracted from the target music will be described. The electronic device can extract a melody of the target music based on the frequency information (1051) of the target music. The melody can represent the temporal progression of pitch. As the pitch frequency increases, the pitch increases, and as the pitch frequency decreases, the pitch decreases. The electronic device can determine whether two or more time sections having similar melodies exist within the entire playback time of the target music. Here, the time section can be a section in which one or more measures are continuously connected within the target music. The electronic device can calculate the similarity of the temporal progression of pitch between the time sections, and if the calculated similarity is greater than or equal to a threshold similarity, the electronic device can determine that the melodies of the two time sections are similar. If two or more time sections with similar melodies exist within a target music, the electronic device can calculate the loudness of each of the two or more time sections. The loudness of the time section can represent the average of the loudness of each of the multiple notes included in the time section. If the loudness of each of the two or more time sections all exceeds a threshold loudness, the electronic device can determine that the two or more time sections are chorus sections within the target music. For example, the threshold loudness can represent, but is not limited to, the average of the loudness of notes whose loudness falls within the top 20% among the multiple notes constituting the target music.
[0129] In one embodiment, the electronic device may divide the entire playback time of the target music into multiple time intervals and correct the intensity guide data (1011) so that the initial exercise intensity remains the same in each of the divided time intervals. If the exercise intensity applied to the wearable device changes continuously over time, a user wearing the wearable device and exercising according to an exercise program may be confused about how much force to apply to the wearable device. Therefore, the electronic device may correct the intensity guide data (1011) so that the exercise intensity applied to the wearable device remains the same within the same time interval when playing the target music according to the exercise program. For example, the electronic device may create multiple time intervals by dividing the entire playback time of the target music into preset minimum time units. The preset minimum time may be, but is not limited to, 3 seconds. The electronic device may calculate an average exercise intensity of the set exercise intensities for multiple notes included in one time interval. The electronic device may calculate the average exercise intensity as an arithmetic mean of the set exercise intensities for multiple notes. The electronic device can collectively correct the motion intensities of each of a plurality of sounds included in a single time interval with an average motion intensity calculated corresponding to the corresponding time interval. In the above example, the method of generating a plurality of time intervals by dividing the entire playback time of the target music into preset minimum time units has been described, but the method of generating a plurality of time intervals is not limited thereto, and the plurality of time intervals can be generated in various ways. For example, the electronic device can set the start time of a single time interval to the start time of the first bar in the target music, and set the end time of the time interval to the end time of the second bar in the target music. The first bar and the second bar may be the same bar or different bars.The electronic device may divide the total playback time of the target music into multiple time intervals such that each time interval includes a preset minimum time (e.g., 3 seconds).
[0130]
[0131] Referring back to FIG. 9, the electronic device (901) may generate tempo guide data (912) regarding haptic feedback to be generated in the wearable device when executing an exercise program based on the rhythm tempo of the target music. The tempo guide data (912) may represent data regarding the tempo and beat of the target music. For example, the display of a line (913) at a point in time within the target music in the tempo guide data (912) may indicate that a beat begins at that point in time. In other words, the tempo guide data (912) may display a line at each of the starting points of a beat within the target music. The electronic device (901) may determine the generation point of haptic feedback to be generated in the wearable device when playing the target music in the exercise program based on the tempo guide data (912).
[0132] FIG. 11 is a diagram illustrating a process by which an electronic device determines haptic feedback of an exercise program based on a rhythm pattern in target music according to one embodiment.
[0133] In one embodiment, an electronic device (e.g., the electronic device (901) of FIG. 9) may generate tempo guide data (1112) for an exercise program based on a rhythm pattern (1101) of a target music. The electronic device may determine the timing of generating haptic feedbacks to be generated in a wearable device when the exercise program is executed based on the generated tempo guide data (1112). Haptic feedback may refer to a technology that generates vibrations in the wearable device to convey a response to the user. By determining the timing of generating haptic feedbacks in the exercise program, the electronic device may enable a user who performs an exercise according to the exercise program to smoothly perform the exercise in time with the beat of the haptic feedbacks generated in the wearable device.
[0134] Referring to FIG. 11, the electronic device can identify the starting points of beats within the target music based on the generated tempo guide data. The electronic device can determine the exercise content of the exercise program so that haptic feedbacks (1121, 1122, 1123, 1124) are generated on the wearable device at some of the starting points of the identified beats when the target music is played according to the execution of the exercise program. Although the present specification describes an example in which the haptic feedback of the exercise program is generated through the wearable device, the present invention is not limited thereto, and the haptic feedback may be generated through another device (e.g., a smartwatch, etc.) connected to the electronic device.
[0135] Referring back to FIG. 9, when the electronic device generates an exercise program corresponding to the target music, the electronic device may receive input regarding the target exercise intensity and the target calorie consumption from the creator of the exercise program. The electronic device may display an interface object (931) corresponding to the target exercise intensity and an interface object (932) corresponding to the target calorie consumption on the display. In addition, after the analysis of the target music is completed and guide data is generated, the electronic device may display an interface object (943) for receiving an exercise record from a wearable device on the display. The exercise record may include movement information of the wearable device and speech information of the creator. The creator of the exercise program for the target music may wear the wearable device and perform various exercise movements through the wearable device. When the electronic device receives a touch input of the interface object (943), the target music may be played on the electronic device and the wearable device, and an exercise record including movement information and speech information of the wearable device may be received in real time from the wearable device according to the playback of the target music. The electronic device can determine the exercise content of the exercise program corresponding to the target music based on the exercise record received from the wearable device.
[0136]
[0137] FIG. 12 is a diagram illustrating a process in which an electronic device determines exercise content of an exercise program by receiving exercise records from a wearable device according to one embodiment.
[0138] In one embodiment, the electronic device (1201) may play the target music to determine the exercise content of the exercise program after completing analysis of the target music and generating guide data. The electronic device (1201) may receive movement information of the wearable device (e.g., the wearable device (100) of FIG. 1) in real time from the wearable device according to the playback of the target music.
[0139] In one embodiment, the electronic device (1201) may divide the entire playback time of the target music into a plurality of exercise sections (1271, 1272, 1273). The electronic device (1201) may determine a corresponding exercise mode for each of the divided plurality of exercise sections (1271, 1272, 1273). The electronic device (1201) may determine the exercise mode corresponding to one exercise section as one of the first mode and the second mode.
[0140] Hereinafter, a method for an electronic device (1201) to determine a movement mode of a target movement section will be described. The first mode may be a mode related to one or more candidate movements among candidate movements pre-stored in the electronic device (1201). In other words, the fact that the movement mode of the target movement section is determined to be the first mode may indicate that one or more candidate movements stored in the electronic device (1201) are recorded corresponding to the target movement section. The second mode may be a mode indicating that the movement mode is not related to the candidate movement movements pre-stored in the electronic device (1201). In other words, the fact that the movement mode of the target movement section is determined to be the second mode may indicate that the target movement section is a free movement section in which a corresponding movement movement is not recorded.
[0141] The fact that the electronic device (1201) stores candidate motions may indicate that the wearable device stores movement information of the wearable device required to perform each of the candidate motions. For example, the fact that the electronic device (1201) stores a lunge motion as a candidate motion may indicate that the electronic device (1201) stores information on changes in the rotation angle of the leg drive frame of the wearable device that appear when the wearable device performs the lunge motion, as well as information on the acceleration and / or angular velocity of the wearable device.
[0142] In one embodiment, the electronic device (1201) may determine an exercise mode corresponding to a target exercise section based on movement information received from a wearable device while the target exercise section is being played. The electronic device (1201) may determine an exercise mode corresponding to the target exercise section based on determining whether a candidate movement that matches the movement information of the wearable device received while the target exercise section is being played exists among a plurality of candidate movements (e.g., 'lunge movement', 'squat movement', etc.) stored in the electronic device (1201).
[0143] If there is at least one candidate motion that matches the motion information of a wearable device received while the target motion section is being played among a plurality of candidate motions stored in the electronic device (1201), the electronic device (1201) may determine the motion mode corresponding to the target motion section as a first mode for the matching candidate motion. On the other hand, if there is no candidate motion that matches the motion information of a wearable device received while the target motion section is being played among a plurality of candidate motions stored in the electronic device (1201), the electronic device (1201) may determine the motion mode corresponding to the target motion section as a second mode representing a free motion section.
[0144] Hereinafter, a process by which the electronic device (1201) determines whether the movement information of the wearable device matches a candidate motion is described. If, while the target exercise section is being played, the electronic device (1201) detects a threshold number of times or more of movement information similar to the movement information for the wearable device to perform one candidate motion, the electronic device (1201) may determine that the movement information of the wearable device received while the target exercise section is being played matches one candidate motion. Here, the threshold number of times may be three times, but is not limited thereto. On the other hand, if, while the target exercise section is being played, the electronic device (1201) does not detect movement information similar to a plurality of movement information for the wearable device to perform a plurality of candidate motions, the electronic device (1201) may determine that there is no candidate motion matching the movement information of the wearable device received while the target exercise section is being played. Furthermore, when movement information similar to movement information for the wearable device to perform one candidate movement is detected more than once or less than a threshold number of times among the movement information of the wearable device received while the target movement section is being played, the electronic device (1201) may output an interface on the display that inquires the manufacturer whether to determine the movement movement to be recorded in response to the target movement section as one candidate movement.
[0145] When the electronic device (1201) determines the exercise mode of the target exercise section as the first mode, the exercise motion that the user should perform during playback of the target exercise section according to the execution of the exercise program may be recorded corresponding to the target exercise section. When the electronic device (1201) determines the exercise mode of the target exercise section as the first mode, the electronic device (1201) may control the wearable device to repeatedly perform the exercise motion recorded corresponding to the target exercise section during playback of the target exercise section according to the execution of the exercise program until the target exercise section ends. At this time, a rest time during which the exercise motion is not performed may be set within the target exercise section. When the electronic device (1201) determines the exercise mode of the target exercise section as the second mode, the exercise motion corresponding to the target exercise section may not be recorded separately. Referring to FIG. 12, the exercise modes of the first exercise section (1271) and the second exercise section (1272) within the exercise program may be determined as the first mode, and the exercise mode of the third exercise section (1273) may be determined as the second mode. A lunge motion may be recorded corresponding to the first exercise section (1271), and a knee-up motion may be recorded corresponding to the second exercise section (1272). The third exercise section (1273) is a free motion section, and the exercise motion corresponding to the third exercise section (1273) may not be recorded separately.
[0146] In one embodiment, when the exercise mode corresponding to the target exercise section is determined as the first mode, the electronic device (1201) may determine the exercise content in the target exercise section of the exercise program so that resistance is generated at a position according to the exercise motion recorded in response to the target exercise section in the wearable device. For example, when the exercise motion recorded in response to the target exercise section is a knee-up motion, the electronic device (1201) may determine the exercise content in the target exercise section of the exercise program so that an action of applying an exercise load to the left leg movement and then applying an exercise load to the right leg movement in the wearable device is repeated until the target exercise section ends. At this time, the electronic device (1201) may control the driving module of the wearable device so that a resistance force of a magnitude corresponding to the exercise intensity set for the target exercise section in the intensity guide data (1211) is generated during the reproduction of the target section according to the execution of the exercise program.
[0147] In one embodiment, when the exercise mode corresponding to the target exercise section is determined as the second mode, the electronic device (1201) may not separately designate a location where resistance is generated in the wearable device. When the exercise mode corresponding to the target exercise section is determined as the second mode, the electronic device (1201) may generate resistance overall at each location where resistance can be applied in the wearable device. In this case, the electronic device (1201) may control the driving module of the wearable device so that a resistance of a magnitude corresponding to the exercise intensity set for the target exercise section in the intensity guide data (1211) is generated during reproduction of the target section according to the execution of the exercise program.
[0148] In one embodiment, the electronic device (1201) may determine that the next exercise segment begins when the previous exercise segment ends. The electronic device (1201) may determine the end point of the exercise segment (hereinafter, referred to as the "end point") in various ways. For example, the electronic device (1201) may determine the end point of the exercise segment as the point in time when a preset amount of time has elapsed from the start point of the exercise segment (hereinafter, referred to as the "start point"). As another example, the electronic device (1201) may determine the end point of the exercise segment as the point in time when the exercise movement recorded in response to the exercise segment is repeated a preset number of times (e.g., 30 times) within the exercise segment. As another example, if the electronic device (1201) receives an instruction to end the exercise segment from the producer while the exercise segment is being played, the electronic device (1201) may determine the point in time at which the instruction is received as the end point of the exercise segment. At this time, the electronic device (1201) can perform correction for the end point of the exercise section based on the tempo guide data (1212). For example, since the electronic device (1201) can identify the start points of beats within the target music based on the tempo guide data (1212), the end point of the exercise section can be changed to the start point of the beat that is closest to the end point of the existing exercise section among the start points of beats.
[0149] In one embodiment, the electronic device (1201) can extract the exercise tempo of the wearable device through the movement information of the wearable device. The electronic device (1201) can correct the tempo guide data (1212) based on the extracted exercise tempo of the wearable device. In one embodiment, the electronic device (1201) can correct the tempo guide data (1212) for each exercise section. The electronic device (1201) can obtain the interval at which the exercise movement performed by the wearable device is repeated within the exercise section (1272) through the movement information of the wearable device received while the exercise section (1272) is being played. If the interval between the extracted interval and the start time of the beats displayed in the exercise section (1272) of the tempo guide data (1212) differs by more than a threshold time, the electronic device (1201) can correct the tempo guide data (1212) within the exercise section (1272). For example, the electronic device (1201) may correct the tempo guide data (1212) so that the interval between the start points of beats displayed within the exercise section (1272) in the tempo guide data (1212) exceeds the interval at which the exercise motion is repeated, obtained through the movement information of the wearable device.
[0150] In one embodiment, the electronic device (1201) may display a line (1280) indicating the current point in time at which the target music is played on the display. In addition, the electronic device (1201) may calculate the average exercise intensity from the start time of the target music to the current point in time and the total calories consumed from the start time of the target music to the current point in time during the playback of the target music (e.g., 87 kcal). The electronic device (1201) may display an interface object (1233) corresponding to the size of the average exercise intensity and an interface object (1234) corresponding to the amount of the total calories consumed on the display. For example, the electronic device (1201) may calculate the average exercise intensity as the arithmetic mean of the exercise intensities of each note from the start time of the target music to the current point in time. For example, the electronic device (1201) may calculate the total calories consumed from the start time of the target music to the current point in time based on the producer's heart rate information and the average exercise intensity up to the current point in time. The electronic device can receive the creator's heart rate information through the smartwatch worn by the creator.
[0151] In one embodiment, the electronic device (1201) may display an interface object (1281) on the display for stopping playback of the target music. When receiving a selection for the interface object (1281), the electronic device (1201) may stop receiving movement information from the wearable device and temporarily stop determining the exercise content of the exercise program. In addition, the electronic device (1201) may display an interface object (1282) on the display for stopping determining the exercise content of the exercise program. Furthermore, the electronic device (1201) may display an interface object (1283) on the display for automatically generating the exercise program. The automatic generation of the exercise program is described in detail in FIG. 13.
[0152]
[0153] FIG. 13 is a diagram illustrating a process in which an electronic device according to one embodiment automatically determines exercise content from the current point in time to the end point of the target music of an exercise program.
[0154] In one embodiment, the electronic device (1301) can automatically determine the exercise content of the exercise program. In other words, the electronic device (1301) can automatically determine the exercise content of the exercise program without receiving movement information of the wearable device from the wearable device. For example, when the electronic device (1301) receives an input for selecting an interface object for automatic generation of the exercise program (e.g., the interface object (1283) of FIG. 12), the electronic device (1301) can automatically determine the exercise content from the current point (1312) to the end point of the target music of the exercise program.
[0155] In one embodiment, the electronic device (1301) can determine the exercise content from the start time (1311) to the current time (1312) of the target music of the exercise program based on the movement information received from the wearable device. When the electronic device (1301) receives a request for automatic generation of the exercise program of the exercise program, the electronic device (1301) can determine the exercise content from the current time (1312) to the end time (not shown) of the target music of the exercise program based on the exercise content determined from the start time (1311) to the current time (1312) of the target music of the exercise program.
[0156] In one embodiment, the electronic device (1301) can extract the melody of the target music based on the frequency information of the target music (e.g., the frequency information (1051) of FIG. 10). The electronic device (1301) can automatically generate an exercise program so that exercise sections with similar melodies within the target music have the same exercise motion and the same number of exercises. Referring to FIG. 13, when the melodies of an exercise section (1371) in which the exercise motion and the number of exercises have been determined are similar to an exercise section (1372) in which the exercise motion and the number of exercises have not been determined, the electronic device (1301) can set the exercise motion and the number of exercises of the exercise section (1372) to be the same as those of the exercise section (1371). Furthermore, the electronic device (1301) can display on the display a minimum time point (1313) at which movement information of the wearable device must be received from the wearable device during playback of the target music in order to automatically generate the exercise program. The electronic device (1301) can activate an interface object (e.g., interface object (1283) of FIG. 12) corresponding to a request for automatic generation of the exercise program only when the exercise content of the exercise program is determined from the start time (1311) of the target music to the minimum time (1313), and can automatically generate the remaining exercise content of the exercise program.
[0157] In one embodiment, the electronic device (1301) can insert the producer's speech information into the exercise content of an exercise program corresponding to the target music. The electronic device can receive speech information recorded from the producer's speech in real time from a wearable device according to the playback of the target music. The electronic device can analyze the producer's speech information and determine whether to insert the corresponding speech information into the exercise content of the exercise program. In one embodiment, when the electronic device (1301) receives speech information from the wearable device while the target exercise section is in progress, the electronic device (1301) can determine whether to insert the received speech information into the exercise content of the exercise program based on whether the received speech information is an exercise movement corresponding to the target exercise section or an utterance related to an exercise sequence in the target exercise section.
[0158] In one embodiment, if the electronic device (1301) determines that the speech information (1381) of the wearable device received while the exercise section (1373) is being played (e.g., “This movement is a lunge movement”) is a speech related to an exercise movement recorded in response to the exercise section (1373) or an exercise sequence in the exercise section (1373), the electronic device (1301) may insert the speech information (1381) into the exercise content of the exercise program. The electronic device (1301) may store the time at which the speech information (1381) is received within the exercise section (1373). The electronic device (1301) may control the wearable device to output the speech information (1381) as a voice through TTS (Text to Speech) at the same time as the time at which the speech information (1381) is received during the playback of the target exercise section according to the execution of the exercise program.
[0159] In one embodiment, when the electronic device (1301) determines that the speech information (1382) (e.g., “Hello”) of the wearable device received while the exercise segment (1373) is being played is an utterance unrelated to the exercise motion recorded corresponding to the exercise segment (1373) or the exercise sequence in the exercise segment (1373), the electronic device (1301) may display an interface object on the display that inquires whether to insert the speech information (1382) into the exercise content of the exercise program. When the electronic device (1301) receives a request from the manufacturer to insert the speech information (1382) into the exercise content of the exercise program, the electronic device (1301) may store the point in time at which the speech information (1382) was received within the exercise segment (1373). The electronic device (1301) can control the wearable device so that the electronic device (1301) or the wearable device outputs the speech information (1382) as a voice through TTS at the same time as the speech information (1382) is received during the playback of the target exercise section according to the execution of the exercise program.
[0160]
[0161] FIG. 14 is a diagram illustrating a process in which an electronic device according to one embodiment changes a recorded exercise motion corresponding to an exercise intensity or exercise section of an exercise program.
[0162] In one embodiment, an electronic device (e.g., electronic device (901) of FIG. 9) may receive input from a creator during the creation of an exercise program to change an exercise intensity or exercise motion of the exercise program. FIG. 14 primarily describes an example in which the electronic device receives input from a creator through a smartwatch (1410) connected to the electronic device. In one embodiment, the electronic device may change at least one of an exercise intensity in a target exercise section of the exercise program and an exercise motion recorded corresponding to the target exercise section based on input from the creator received while the target exercise section is being played.
[0163] For example, while the electronic device is playing the target exercise section to determine the exercise content in the target exercise section of the exercise program, the exercise intensity (1421) in the target exercise section according to the intensity guide data (e.g., the intensity guide data (911) of FIG. 9) may be output in real time on the display of the smartwatch (1410). The smartwatch (1410) may receive an input for changing the exercise intensity at the current point in time from the manufacturer while the target exercise section is playing on the electronic device. For example, the smartwatch (1410) may determine a touch input for interface objects (1431, 1432) received from the manufacturer as an input for changing the exercise intensity. The smartwatch (1410) may transmit the input for changing the exercise intensity to the electronic device, and the electronic device may change the exercise intensity (1421) in the target exercise section.
[0164] For another example, while the electronic device is playing back the target exercise section to determine the exercise content in the target exercise section of the exercise program, the exercise motion (1422) recorded corresponding to the target exercise section can be output in real time on the display of the smartwatch (1410). The smartwatch (1410) can receive an input from the manufacturer to change the exercise motion recorded corresponding to the target exercise section while the target exercise section is played back on the electronic device. For example, the smartwatch (1410) can determine a swipe input (1441, 1442) received from the manufacturer as an input to change the exercise motion recorded corresponding to the target exercise section. The smartwatch (1410) can transmit the input to change the exercise motion to the electronic device, and the electronic device can change the exercise motion (1422) recorded corresponding to the target exercise section.
[0165]
[0166] FIG. 15 is a diagram illustrating a process in which an electronic device according to one embodiment generates and executes an exercise program corresponding to target music.
[0167] In one embodiment, the electronic device (1501) can generate an exercise program corresponding to target music (1502) without recording information (movement information and speech information) received from a wearable device. The electronic device (1501) can analyze the target music (1502) to extract the loudness (1510) and rhythm tempo (not shown) of the target music (1502). The electronic device (1501) can generate intensity guide data (1511) based on the extracted loudness (1510) and can generate haptic guide data based on the extracted rhythm tempo. The electronic device (1501) can determine the exercise content of the exercise program using the generated guide data (intensity guide data (1511) and haptic guide data).
[0168] In one embodiment, when the electronic device (1501) executes an exercise program corresponding to a target music, the electronic device (1501) may apply the exercise intensity at each point in time extracted from the intensity guide data (1511) to the wearable device while playing the target music. Furthermore, when the electronic device (1501) executes an exercise program corresponding to the target music, the electronic device (1501) may control the wearable device to generate haptic feedback based on the haptic guide data in the wearable device while playing the target music. The electronic device (1501) may initially set the exercise action (1521) in the exercise program as a basic exercise action (e.g., 'boost walking') and may change the exercise action (1521) in real time based on a user's input while executing the exercise program. The electronic device (1501) may apply the exercise intensity (1522) at each point in time extracted from the intensity guide data (1511) to the wearable device while playing the target music according to the execution of the exercise program. However, the electronic device (1501) can change the exercise intensity (1522) at each point in time in real time based on the user's input while playing the target music according to the execution of the exercise program.
[0169]
[0170] FIG. 16 is a diagram illustrating a process in which an electronic device according to one embodiment outputs an avatar to a display while determining exercise content of a target exercise section in an exercise program.
[0171] In one embodiment, the electronic device (1601) may output an avatar (1610) to the display during the process of generating an exercise program corresponding to target music. The electronic device (1601) may change the shape and voice of the avatar (1610) according to the input of the creator. While determining the exercise content of the target exercise section in the exercise program, the electronic device (1601) may output an avatar (1610) according to exercise movements recorded corresponding to the target exercise section and speech information inserted into the target exercise section on the display. For example, while the target exercise section is played to determine the exercise content of the target exercise section, the electronic device (1601) may cause an image effect of the avatar (1610) performing the exercise movements recorded corresponding to the target exercise section to be generated on the display. In addition, while the target exercise section is played to determine the exercise content of the target exercise section, the electronic device (1601) may cause a voice effect of the avatar (1610) outputting a voice corresponding to the speech information inserted into the target exercise section to be generated on the display.
[0172] Furthermore, the electronic device (1601) can generate an image effect in which an avatar (1610) performs a movement movement recorded corresponding to the target movement section while the target movement section is played according to the execution of a movement program corresponding to the target music, and a voice effect in which a voice corresponding to speech information inserted into the target movement section is output as the voice of the avatar (1610).
[0173]
[0174] The various embodiments of the present disclosure and the terminology used therein are not intended to limit the technical features described in the present disclosure to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In the present disclosure, each of the phrases "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.
[0175] The term "module" used in various embodiments of the present disclosure may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0176] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or may command the processing device, independently or collectively. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, or computer storage medium or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed across networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium. Various embodiments of the present disclosure may be implemented as software comprising one or more instructions stored on a storage medium (e.g., memory 514) that can be read by a machine. For example, a processor of the device may recall at least one of the one or more instructions stored from the storage medium and execute it. This enables the device to operate to perform at least one function according to at least one command called above. The one or more commands may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.
[0177] According to one embodiment, the method according to various embodiments disclosed in the present disclosure may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.
[0178] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
Claims
1. In electronic devices, A communication module connected to a wearable device and receiving sensor data including movement information of the wearable device; a processor electrically connected to the above communication module; and Memory electrically connected to the above processor Including, The above processor, Analyze the target music to extract the intensity and rhythm pattern of the target music, Based on the extracted sound intensity and the extracted rhythm pattern, guide data for an exercise program corresponding to the target music is generated. Based on the generated guide data and the movement information received from the wearable device, the exercise content of the exercise program is determined. Electronic devices.
2. In paragraph 1, The above processor, Based on the extracted negative intensity, intensity guide data on exercise intensity to be applied to the wearable device when executing the exercise program is generated, Based on the extracted rhythm tempo, generating tempo guide data regarding haptic feedback to be generated in the wearable device when executing the exercise program. Electronic devices.
3. In any one of paragraphs 1 and 2, The above processor, Extract frequency information for multiple notes that constitute the above target music, Based on the extracted frequency information, the chorus interval of the target music is identified, Correcting the generated intensity guide data by correcting the exercise intensity for the notes included in the identified refrain section. Electronic devices.
4. In any one of paragraphs 1 to 3, The above processor, Identifying start points of beats in the target music based on the generated tempo guide data, and determining exercise content of the exercise program so that haptic feedback is generated in the wearable device at some of the identified start points when the exercise program is executed. Electronic devices.
5. In any one of paragraphs 1 to 4, The above processor, After generating the above guide data, play the above target music, Receiving movement information of the wearable device in real time from the wearable device according to the playback of the target music. Electronic devices.
6. In any one of paragraphs 1 to 5, The above processor, Dividing the entire playback time of the above target music into a plurality of movement sections, and determining the movement mode corresponding to each of the divided plurality of movement sections as one of the first mode and the second mode. Electronic devices.
7. In any one of paragraphs 1 to 6, The above processor, Based on determining whether there is a candidate motion that matches the movement information of the wearable device received while the target exercise section is being played among a plurality of candidate motions stored in the electronic device, determining an exercise mode corresponding to the target exercise section. Electronic devices.
8. In any one of paragraphs 1 to 7, The above processor, If there is a candidate motion that matches the movement information of the wearable device received while the target motion section is being played, the motion mode corresponding to the target motion section is determined as the first mode for the matching candidate motion, If there is no candidate motion matching the motion information of the wearable device received while the target motion section is being played, the motion mode corresponding to the target motion section is determined as a second mode representing a free motion section. Electronic devices.
9. In any one of paragraphs 1 to 8, The above processor, When the exercise mode corresponding to the above target exercise section is determined as the first mode, the exercise content of the exercise program is determined so that resistance is generated at a position according to the exercise movement recorded corresponding to the target exercise section in the wearable device when the target exercise section is reproduced according to the execution of the exercise program. Electronic devices.
10. In any one of paragraphs 1 to 9, The above processor, Based on the movement information received from the wearable device, the exercise content from the start time of the target music of the exercise program to the present time is determined, When receiving a request for automatic generation of the above exercise program, the exercise content from the current point in time to the end point of the target music of the exercise program is determined based on the determined exercise content. Electronic devices.
11. In any one of paragraphs 1 to 10, The above processor, When receiving speech information of the wearable device while the target exercise section is in progress, determining whether the received speech information is speech related to a movement movement recorded corresponding to the target exercise section or an exercise sequence in the target exercise section, and determining whether to insert the received speech information into the exercise content of the exercise program. Electronic devices.
12. In any one of paragraphs 1 to 11, The above processor, When the received speech information is inserted into the exercise content of the exercise program, the exercise content of the exercise program is determined so that the wearable device outputs the received speech information through TTS (Text to Speech) when the target exercise section is played according to the execution of the exercise program. Electronic devices.
13. In any one of paragraphs 1 to 12, The above processor, Based on the input of the producer received while the target exercise section is being played, changing at least one of the exercise intensity in the target exercise section of the exercise program and the exercise movement recorded corresponding to the target exercise section. Electronic devices.
14. In any one of paragraphs 1 to 13, The above processor, While determining the exercise content of the target exercise section in the above exercise program, an avatar is displayed according to the exercise movement recorded corresponding to the target exercise section and the speech information inserted in the target exercise section. Electronic devices.
15. A method performed by a processor of an electronic device, A step of analyzing target music and extracting the loudness and rhythm pattern of the target music; A step of generating guide data for an exercise program corresponding to the target music based on the extracted sound intensity and the extracted rhythm pattern; and A step of determining exercise content of the exercise program based on the generated guide data and movement information received from a wearable device connected to the electronic device. A method including:
Citation Information
Patent Citations
Exercise capacity enhancement device based on human body limb exercise and instrument exercise simulation sound, and operating method thereof
CN110841244A
Electric stimulation guiding device
JP2006288622A
Music pacemaker for physical training
JP2008538295A
Information processing device, information processing system, program, and information processing method
JP7047295B2
Customized health care system using electric stimulation device and method thereof
KR102350320B1
