Smart mirror-based health-training coaching system and operating method thereof

Abstract

The present invention relates to a smart mirror-based health-training coaching system and an operating method thereof. The present invention is advantageous in that a user can exercise in a user's dedicated space at a required time according to a selected type of training by using a smart mirror and following trajectories of respective skeletal structures during training by a professional health trainer, and the accuracy of the training can be analyzed and training in a specific interval can be selected and repeatedly performed.

Description

Smart Mirror-Based Health Training Coaching System and Method of Operation

[0001] The present invention relates to a smart mirror-based health training coaching system and a method of operating the same. More specifically, the invention relates to a smart mirror-based health training coaching system and a method of operating the same, wherein the system films the movements of professional trainers in various fields training for a specific purpose from the front, side, and back, respectively, extracts the linked movements of each joint (hereinafter referred to as "skeleton"), records and stores the data in a smart mirror, and when user access is confirmed, uses a stereo camera to recognize each skeletal part of the user's body and analyzes physical characteristics based on the length of each skeleton, etc., matches the stored physical characteristic data of the trainer to be coached with the user's physical characteristic data, outputs the results so that the user can follow along alone, records and analyzes the user's training situation to analyze accuracy, exercise time, energy consumption, etc. in each part, and instructs the user to repetitively train specific parts.

[0002] With the general rise in interest in health, people are utilizing weight training, fitness, and sports dance as forms of exercise to maintain and improve their health. When using clubs or centers for weight training, users typically pay a fee in advance to sign up for membership and begin exercising using the equipment. While trainers manage the usage of various equipment and posture correction, it is common for individuals to manage their own daily exercise volume. Furthermore, when receiving one-on-one training, one must select a specific time slot and arrive on time.

[0003] In other words, as interest in health increases, personal investment in a healthy life is growing, and as the field of recreational sports gradually develops, interest in systematic and effective exercise methods is also increasing. Furthermore, there is a trend where many people pay trainers to manage their exercise. To ensure effective exercise without injury while engaging in various types of exercise (weight training, yoga, Pilates, etc.), there may be personal trainers, exercise prescription specialists, rehabilitation exercise specialists, and coaches who provide individual guidance on exercise practice. These trainers apply training programs based on training models that include the frequency, intensity, duration, and form of exercise (FITT principles), depending on an evaluation of the trainee's exercise performance ability.

[0004] In addition, when leading a social life, it is often difficult to arrive at the relevant club or center in time for training, and there are issues regarding the need to face or interact with many other people.

[0005] Meanwhile, it is very common to look in a mirror while exercising to maintain accurate and correct posture. Mirrors are very familiar objects that humanity has used for a long time, primarily serving as a means to check one's appearance. People look at themselves or their workout routines in mirrors several times a day, but this only allows for real-time verification and cannot save the image. Recently, due to advancements in camera technology, it has become possible to take selfies not only to check one's workout routine but also to record and review them later; however, taking a selfie with a portable camera every time one exercises presents a very cumbersome problem.

[0006] Therefore, technology has become necessary that allows one to record their exercise routine using a mirror and receive coaching directly from a professional trainer whenever needed.

[0007] A prior art that partially resolves these needs and problems is disclosed in the ‘smart mirror device’ of Korean Patent Registration No. 10-1494301 (February 11, 2015).

[0008] FIG. 1 is a diagram showing the configuration of a smart mirror device according to an embodiment of the prior art. Hereinafter, the prior art will be described in detail with reference to the attached drawings. A camera module (100), a printing module (110), and a display (130) are provided on one side of a main body (10) that has a mirror (20) on its front surface.

[0009] The image captured by the camera module (100) of the mirror device, or the image edited after capture, is displayed on the display (130) or output as a printed product through the print module (110). That is, when a user uses the mirror (20), the camera module (100) captures the user, and the user checks their own image through the display (130).

[0010] These conventional technologies still have the problem of not being able to verify or guess whether the exercise trajectory caused by the user's movement is the same as or different from the exercise trajectory caused by the trainer's training.

[0011] An improved prior art that partially resolves these problems is disclosed in the ‘Fun Extreme Management System for Health Training’ under Korean Patent Registration No. 10-1856054 (May 2, 2018).

[0012] FIG. 2 is a configuration diagram of a fun extreme management system for health training according to another embodiment of the prior art.

[0013] Hereinafter, the improved prior art is described in detail with reference to the attached drawings. A member (including a user) visiting a center (including a club) recognizes that they have visited the center through a center terminal (30). The center terminal (30) guides the member on the movement path and types of exercise equipment for the fitness training that the member must perform. After receiving and wearing a member terminal (40), the member moves to the location of the exercise equipment in the correct order and performs the exercise. An exercise equipment terminal (20) mounted on the exercise equipment communicates with the member terminal (40) worn by the approaching member to verify the member and provides video and various information corresponding to the member's exercise schedule information. It operates the corresponding exercise equipment according to the member's operation and exercise schedule information, analyzes the member's exercise volume, displays it to the member via a monitor, and also transmits it to a management server (10). The management server (10) transmits exercise volume, diet information, etc., through the member's mobile device (50) held by the member, thereby increasing the efficiency of the member's fitness training.

[0014] The improved conventional technology has the advantage of enhancing training effectiveness by communicating with exercise equipment required by members or users, providing information and videos suitable for exercise schedules, and displaying analyzed exercise volume; however, it still fails to resolve the problem that users cannot train in a dedicated user space using types of exercises arbitrarily selected by them according to the exercise trajectory of a professional trainer.

[0015] Therefore, there is a need to develop technology that allows a professional fitness trainer to perform a type of exercise selected by the user in a user's private space at the time required by the user, analyze the accuracy of the exercise while following the trajectory of the skeleton being exercised, and enable the repetition of specific sections of exercise by selection.

[0016] [Prior Art Literature]

[0017] [Patent Literature]

[0018] (Patent Document 1) Republic of Korea Patent Registration No. 10-1494301 (Feb. 11, 2015) 'Smart Mirror Device'

[0019] (Patent Document 2) Republic of Korea Patent Registration No. 10-1856054 (May 2, 2018) 'Fun Extreme Management System for Health Training'

[0020] The present invention, devised to resolve the problems and necessities of the conventional technology described above, aims to provide a smart mirror-based health training coaching system and a method of operation thereof, which allows a professional health trainer to analyze the accuracy of training and perform a selected type of training along the trajectory of each skeleton at the required time using a smart mirror in a user's dedicated space, and to enable repetitive training of specific sections by selection.

[0021] The smart mirror-based health training coaching system of the present invention, devised to achieve the above-mentioned purpose, comprises: a smart mirror terminal device (1000) that detects a state in which a user approaches within a predetermined distance using an infrared signal, extracts and analyzes feature points of the body and face by analyzing the captured image signal, prioritizes and displays a health activity preferred by a specific user, searches for and outputs a training video for coaching by a trainer regarding the selected health activity, films the user's training process as a video, and performs a three-dimensional comparative analysis of the movement trajectories of each skeleton to record and output the results of the similarity analysis of the skeletal trajectories of the user and the trainer in conjunction with the corresponding time information; and a smart big data server (3000) that connects to the smart mirror terminal device (1000) via a smart mirror multi-communication network (2000), periodically searches for a training video for coaching by each trainer for each health activity from a public big data server, records and manages it in an allocated area, and provides it upon request. It may include a smart mobile mirror phone unit (4000) that connects to the smart mirror terminal device (1000) and the smart mirror multi-communication network (2000), receives and stores health training results practiced by the corresponding control signal, and outputs them by selection.

[0022] The smart mirror terminal device (1000) comprises: a smart mirror centralized control unit (1010) that monitors the overall operation of the smart mirror terminal device (1000) in real time by connecting to each functional unit equipped with a built-in operation program and parameter values ​​and outputs a corresponding control signal; a smart mirror GPS unit (1020) that receives and analyzes a GPS signal based on the corresponding control signal of the smart mirror centralized control unit (1010), confirms and records the coordinate information and time information of the current location, and outputs it; and an infrared sensor unit (1030) that activates an infrared sensor based on the corresponding control signal of the smart mirror centralized control unit (1010), detects whether a user is approaching and the distance, and outputs it. A stereo shooting color analysis unit (1040) that captures an image of a user's face and body shape approaching in visible light in accordance with the corresponding control signal of the smart mirror central control unit (1010), records 64 color-specific image signals obtained by color separating each pixel of the captured stereoscopic image signal into 64 parts, and outputs them in assigned areas; a feature point extraction analysis unit (1050) that receives the stereoscopic face and body shape image signal of the user captured in visible light in accordance with the corresponding control signal of the smart mirror central control unit (1010), records and stores it in assigned areas, extracts feature points, analyzes and recognizes the pattern of the stereoscopic face and the pattern of the skeleton using the extracted feature points, records and outputs them; and a pattern name recording unit (1060) that records and manages the user's name corresponding to the stereoscopic feature point analyzed in accordance with the corresponding control signal of the smart mirror central control unit (1010), and outputs it through stereoscopic feature point search. A smart mirror circuit board (1130) having the above-mentioned infrared sensor unit (1030) and stereo imaging color analysis unit (1040) fixedly installed on parts of the four side edges respectively, arranged to face the total reflection and translucent multimedia touch display unit (1080);It may include a total reflection reinforcing mirror part (1140) made of a material that is relatively resistant to mechanical impact, which is made of a material that is similar in size to the smart mirror circuit board part (1130) above, reflects light entering from the outside in one direction to the front part or transmits light entering from the rear part in one direction to the front part.

[0023] The smart mirror terminal device (1000) comprises: a training state management unit (1070) that inputs a user's three-dimensional feature point pattern, which is captured and analyzed in three dimensions by the corresponding control signal of the smart mirror central control unit (1010), analyzes the distance of each captured feature point to the location of each corresponding target feature point in units of the diameter value of each pixel in comparison with the trainer's coaching training pattern, records and manages the data in conjunction with the corresponding shooting time value, and outputs the data; a multimedia touch display unit (1080) that outputs and displays image signals of the user's training movement and the trainer's training movement captured by the corresponding control signal of the smart mirror central control unit (1010), inputs command signals by touch, and inputs and outputs audio signals; and a trainer video database unit (1090) that records, stores, and outputs three-dimensional training video information of a trainer selected by health field and corresponding skeletal pattern information in an allocated area in conjunction with the corresponding input time by the corresponding control signal of the smart mirror central control unit (1010). The smart mirror packet frame unit (1100) converts a data signal transmitted externally by a corresponding control signal of the smart mirror centralized control unit (1010) into a series of smart mirror packet frames for each unit size and analyzes each of the series of smart mirror packet frames received from the outside to extract a corresponding data signal consisting of a series of unit sizes; the TTS and STT unit (1110) converts text data applied by a corresponding control signal of the smart mirror centralized control unit (1010) into audio speech data and outputs it, and converts the applied audio speech data into text data and outputs it; and the multipath simultaneous communication unit (1120) which simultaneously provides and simultaneously activates communication function units of CDMA, Wi-Fi, Bluetooth, and TRS methods by a corresponding control signal of the smart mirror centralized control unit (1010), and simultaneously transmits and receives communication signals of the same content through communication paths by each communication function unit.

[0024]

[0025] The method for operating a smart mirror-based health training coaching system according to the present invention, devised to achieve the above-mentioned purpose, comprises: a preparation step of activating an infrared distance sensor to monitor user approach when it is determined that a control signal for operating the smart mirror-based health training coaching system is input by the smart mirror terminal device, a smart mirror multi-communication network, a smart big data server, a smart mobile mirror phone unit, and a public big data server; a user identification step of photographing the face and body shape with visible light, analyzing the captured image using a spectroscopic method, determining exercise volume and energy consumption, analyzing patterns through feature point extraction to identify the user, searching for the designated name of the identified user, and outputting the designated name as a voice signal to provide guidance; and a big data activation step of outputting the health field and scheduled health time designated by the user confirmed in the user identification step as a voice signal to provide guidance, providing the analyzed current health status as a voice signal, and activating a trainer video for coaching by a designated trainer that is searched from big data, recorded, and stored, to determine whether to train. If it is determined that training is taking place in the above big data activation step, a training video for coaching by a trainer is output, a video of the user training is filmed to extract feature points and analyze patterns, and the training patterns of the user and the trainer are compared and analyzed to analyze and display similarity, exercise volume, energy consumption, and physical condition, and guidance is provided via voice signal; if it is determined that the training results are being checked while moving, the registered smart mobile mirror phone unit is accessed to control the operation of the smart mirror phone routine, and the training operation step proceeds to termination; may be included.

[0026] The present invention, configured as described above, has the advantage of allowing a user to use a smart mirror in their private space, analyze the accuracy of the training while a professional fitness trainer trains a selected type of training along the trajectory of each skeleton at the required time, and repeat training of a specific section by selection.

[0027] FIG. 1 is a configuration diagram of a smart mirror device according to an embodiment of the prior art,

[0028] FIG. 2 is a configuration diagram of a fun extreme management system for health training according to another embodiment of the prior art,

[0029] FIG. 3 is a functional configuration diagram of a smart mirror-based health training coaching system according to an embodiment of the present invention,

[0030] FIG. 4 is an explanatory diagram of a state in which a smart mirror terminal device (1000) is installed and used according to an embodiment of the present invention.

[0031] FIG. 5 is a detailed functional configuration diagram of a circuit portion of a smart mirror terminal device according to an embodiment of the present invention,

[0032] FIG. 6 is a detailed configuration diagram of a mechanical part of a smart mirror terminal device according to an embodiment of the present invention,

[0033] FIG. 7 is a detailed functional configuration diagram of a smart mobile mirror phone unit according to an embodiment of the present invention,

[0034] FIG. 8 is a diagram illustrating the field configuration of a smart mirror packet frame according to an embodiment of the present invention,

[0035] FIG. 9 is a flowchart of a method for operating a smart mirror-based health training coaching system according to an embodiment of the invention,

[0036] and

[0037] FIG. 10 is a diagram illustrating a state in which feature points of a user training using a smart mirror terminal device are extracted according to an embodiment of the present invention.

[0038] The present invention is capable of various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. In describing the present invention, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions may obscure the essence of the present invention.

[0039] In the following explanation, health is described as including all exercises that utilize the body, such as fitness, sports dance, weight training, dance, choreography, yoga, Pilates, pole gymnastics, and calisthenics.

[0040]

[0041] FIG. 3 is a functional configuration diagram of a smart mirror-based health training coaching system according to an embodiment of the present invention, FIG. 4 is an explanatory diagram of a state in which a smart mirror terminal device (1000) is installed and used according to an embodiment of the present invention, FIG. 5 is a detailed functional configuration diagram of a circuit part of a smart mirror terminal device according to an embodiment of the present invention, FIG. 6 is a detailed configuration diagram of a mechanical part of a smart mirror terminal device according to an embodiment of the present invention, FIG. 7 is a detailed functional configuration diagram of a smart mobile mirror phone part according to an embodiment of the present invention, and FIG. 8 is a field configuration diagram of a smart mirror packet frame according to an embodiment of the present invention.

[0042] Hereinafter, with reference to all attached drawings, the smart mirror-based health training coaching system (900) is configured to include a smart mirror terminal device (1000), a smart mirror multi-communication network (2000), a smart big data server (3000), a smart mobile mirror phone unit (4000), and a public big data server (5000).

[0043] The smart mirror terminal device (1000) detects the state in which a user approaches within a predetermined distance using an infrared signal, extracts and analyzes feature points of the body and face by analyzing the captured image signal, prioritizes and displays the fitness type preferred by the specific user, searches for and outputs a training video for coaching by a trainer for the selected fitness type, films the user's training process as a video, and compares and analyzes the skeletal movement trajectories of the user and the trainer in three dimensions to determine how similar the user's training three-dimensional skeletal movement and the trainer's coaching training three-dimensional skeletal movement are, and records and outputs the similarity analysis result in conjunction with the corresponding time information.

[0044] The smart mirror terminal device (1000) is configured to include a smart mirror central control unit (1010), a smart mirror GPS unit (1020), an infrared sensor unit (1030), a stereo imaging color analysis unit (1040), a feature point extraction analysis unit (1050), a pattern naming record unit (1060), a training status management unit (1070), a multimedia touch display unit (1080), a trainer video database unit (1090), a smart mirror packet frame unit (1100), a TTS and STT unit (1110), a multipath simultaneous communication unit (1120), a smart mirror circuit board unit (1130), and a total reflection mirror unit (1140).

[0045] The smart mirror central control unit (1010) connects to each functional unit equipped with a built-in operation program and parameter values ​​for the overall operation of the smart mirror terminal device (1000), monitors it in real time, and outputs the corresponding control signal.

[0046] The smart mirror GPS unit (1020) receives and analyzes a GPS signal from a well-known GPS satellite by means of a corresponding control signal from the smart mirror central control unit (1010), confirms, records, and outputs coordinate information and time information based on longitude, latitude, and elevation at the current location.

[0047] The infrared sensor unit (1030) activates the infrared sensor by means of the corresponding control signal of the smart mirror central control unit (1010) and detects and outputs whether a user is approaching and the distance.

[0048] It is very obvious that the infrared sensor unit (1030) may be equipped with one first to fourth infrared sensor (1031, 1032, 1033, 1034) on each of the four side edge portions of the smart mirror terminal device (1000) to increase the reliability of detecting user approach, and may further add or subtract as needed.

[0049] The stereo shooting color analysis unit (1040) captures an image of the user's face and body shape approaching in visible light in accordance with the corresponding control signal of the smart mirror concentration control unit (1010) in three dimensions, and records and outputs 64 color-specific image signals, each separated into 64 pixels of the captured three-dimensional image signal, in the allocated area.

[0050] Stereo imaging utilizes stereo cameras, which employ a general technique where two cameras are installed at a fixed distance apart to capture images of the same object while maintaining a constant angle, allowing for three-dimensional viewing, analysis, or verification. This configuration mimics the structure of the human eye, with two identical cameras positioned in a straight line at regular intervals. It is well known that the greater the distance between identical cameras installed in a straight line, the higher the depth of the analysis becomes. This separation distance is generally selected from a range of 5 to 20 centimeters (cm), and a separation of 10 centimeters is relatively desirable for standardization purposes. It is only natural that the range of separation distance may be adjusted as needed, such as when the installation area is relatively small or when the depth of stereoscopic perception is enhanced without being limited by the installation area. Here, depth refers to the sense of depth and precision perceived in the video image.

[0051] Meanwhile, the stereo imaging color analysis unit (1040) can be equipped with one first to fourth stereo imaging color analysis unit (1041, 1042, 1043, 1044) on each of the four side edge portions of the smart mirror terminal device (1000) to further enhance the sense of depth and separate only the stereoscopic image of the left or right side while capturing the user's body shape in a three-dimensional manner from the front and capturing the user's side, for example, the left side and the right side, as needed, and it is very obvious that more can be added or removed as needed.

[0052] The feature point extraction analysis unit (1050) receives the user's three-dimensional face and body shape image signal captured by visible light by the corresponding control signal of the smart mirror concentration control unit (1010), records and stores it in an allocated area, extracts feature points, analyzes and recognizes the pattern of the three-dimensional face and the pattern of the skeleton using the extracted feature points, records and outputs them.

[0053] Various feature points including eyes, nose, mouth, and ears exist in the face, and in the body shape, feature points classified into shoulders, shoulder joints, upper arms, elbow joints, lower arms, wrist joints, hands, fingers, torso, hip joints, upper legs, knee joints, lower legs, ankle joints, feet, toes, etc., as well as the connection angles (three-dimensional angles) and distances (three-dimensional distances) between each feature point are used in feature point analysis and can be added or subtracted as necessary.

[0054] Increasing the number of recognized feature points allows for more precise analysis of the training state, but since there are significant limitations regarding data usage, memory capacity, and processing time, it is advisable to configure the settings appropriately based on the performance of the available hardware and software.

[0055] The pattern naming record unit (1060) records and manages the user's naming corresponding to the three-dimensional feature point analyzed by the corresponding control signal of the smart mirror central control unit (1010), and outputs it by searching for the three-dimensional feature point. The three-dimensional feature point can be divided into a three-dimensional feature point of the face and a three-dimensional feature point of the body shape, and either one of the selected feature points of the face or the body shape, or both feature points, can be used.

[0056] The training status management unit (1070) inputs the user's three-dimensional feature point pattern, which is captured and analyzed in three dimensions by the corresponding control signal of the smart mirror central control unit (1010), analyzes the distance of each captured feature point to the location of each corresponding target feature point in units of the diameter value of each pixel in comparison with the trainer's coaching training pattern, records and manages the data in conjunction with the corresponding shooting time value, and outputs it.

[0057] When comparing the feature points of the user and the trainer that are mutually identical or similar in location, if the locations are close or the same, the training similarity can be classified as high. Additionally, if the length connecting the feature points, or the angle formed by the line connecting the feature points with an adjacent line, is close or the same, the training similarity can also be classified as high.

[0058] The multimedia touch display unit (1080) visually displays the image signals of the user's training motion and the trainer's training motion, which are captured by the corresponding control signal of the smart mirror central control unit (1010), and inputs and outputs command signals by touch and audio signals.

[0059] The multimedia touch display unit (1080) is in a semi-transparent state and becomes partially completely transparent by the corresponding control signal, and the video-grade signal output and displayed by the multimedia touch display unit (1080) is output in one direction toward the front.

[0060] The trainer video database unit (1090) records, stores, and outputs the three-dimensional training video information of a trainer selected by health field and the corresponding skeletal pattern information in the allocated area in conjunction with the corresponding input time, based on the corresponding control signal of the smart mirror central control unit (1010).

[0061] The 3D training video information of a selected trainer by health field and the corresponding skeletal pattern information are recorded, stored, and managed in the smart big data server (3000). The smart mirror central control unit (1010) searches for and downloads the 3D training video information and the corresponding skeletal pattern information of a trainer preferred and designated by the user from the smart big data server (3000) at a time unit selected from either 12 hours or 24 hours, and records, stores, and manages them in the allocated area in conjunction with the corresponding time information. Meanwhile, the time for periodic access can be increased or decreased as needed.

[0062] The smart mirror packet frame unit (1100) converts the data signal transmitted externally by the corresponding control signal of the smart mirror central control unit (1010) into a series of smart mirror packet frames in each unit size, and analyzes each of the series of smart mirror packet frames received from the outside to extract the corresponding data signal consisting of a series of unit sizes.

[0063] The TTS and STT (text to speech / speech to text) (1110) convert text data applied by the corresponding control signal of the smart mirror central control unit (1010) into audio speech data and output it, and convert the applied audio speech data into text data and output it. Since these TTS and STT functions are generally well known, further specific explanation will not be provided.

[0064] The multipath simultaneous communication unit (1120) is equipped with and simultaneously activates communication function units of CDMA, Wi-Fi, Bluetooth, and TRS methods by means of the corresponding control signal of the smart mirror central control unit (1010), and simultaneously transmits and receives communication signals of the same content through the communication paths of each communication function unit.

[0065] The smart mirror circuit board (1130) has an external size similar to that of the multimedia touch display (1080) and is installed in a fixed position behind the multimedia touch display (1080).

[0066] The smart mirror circuit board section (1130) is a double-sided printed circuit board (PCB) that incorporates all circuit section configurations including a smart mirror centralized control section (1010), a smart mirror GPS section (1020), an infrared sensor section (1030), a stereo imaging color analysis section (1040), a feature point extraction analysis section (1050), a pattern naming record section (1060), a training status management section (1070), a trainer video database section (1090), a smart mirror packet frame section (1100), a TTS and STT section (1110), and a multipath simultaneous communication section (1120).

[0067] The smart mirror circuit board (1130) has an infrared sensor (1030) and a stereo imaging color analysis unit (1040) installed on parts of the four side edges, respectively, and is positioned to face the multimedia touch display (1080) in a semi-transparent state and total reflection state.

[0068] The multimedia touch display unit (1080) can control the corresponding portion of the four edge portions where the infrared sensor unit (1030) and the stereo imaging color analysis unit (1040) are installed to always be in a semi-transparent state or always in a transparent state by means of the corresponding control signal of the smart mirror central control unit (1010), and preferably controls it to always be in a transparent state.

[0069] The total reflection mirror part (1140) is made of a size similar to that of the multimedia touch display part (1080) or the smart mirror circuit board part (1130), is fixedly installed at the front position of the multimedia touch display part (1080), reflects light entering from the outside in one direction to the front or transmits light entering from the rear in one direction to the front, and is made of a material that is relatively resistant to mechanical impact.

[0070] The smart mirror central control unit (1010) selects and uses the communication signal of the communication function unit that has the best signal-to-noise ratio among the communication signals received by the four communication function units and has no transmission error, and if no transmission error occurs in two or more communication function units, it uses the communication signal received by any one of the communication function units that is randomly selected.

[0071] The smart mirror multi-communication network (2000) simultaneously provides a CDMA mobile communication system network, a Wi-Fi communication system network, a Bluetooth communication system network, and a TRS communication system network, and simultaneously activates and operates all of them. It simultaneously allocates all four communication system networks provided as communication paths with designated callees based on the request of a connected caller, and performs switching processing to ensure that signals of the same content are transmitted and received or sent while allowing simultaneous connection via the four communication paths. Since the communication technologies of the CDMA, Wi-Fi, Bluetooth, and TRS methods are already well known, further detailed explanations will be omitted. Meanwhile, it is very obvious that the smart mirror multi-communication network (2000) may further include a wired telephone communication network and a wired LAN communication network.

[0072] The smart big data server (3000) connects to the smart mirror terminal device (1000) and the smart mirror multi-communication network (2000), periodically accesses and searches the training videos for coaching by each trainer for each fitness category on the public big data server (5000), records and manages them in the allocated area, and provides them upon request.

[0073] It is very obvious that the time at which the smart big data server (3000) periodically connects to the public big data server (5000) can be set as a period of either 12 hours or 24 hours, and can be increased or decreased as needed.

[0074] The smart mobile mirror phone unit (4000) is connected to the smart mirror terminal device (1000) and the smart mirror multi-communication network (2000), receives the results of health training practiced while moving by means of the corresponding control signal, records, stores, and manages them in an allocated area, and converts and outputs them as voice signals by selection.

[0075] The smart mobile mirror phone unit (4000) periodically connects to the public big data server (5000) to collect big data and collect necessary information using machine learning methods. Since the technology for collecting big data using machine learning is diverse and well known, a detailed explanation will be omitted. Meanwhile, it is very obvious that the periodic connection time can be set to one of the selected time units between 12 hours and 24 hours, and can be increased or decreased as needed.

[0076] The smart mobile mirror phone unit (4000) is configured to include a smart mirror phone control unit (4010), a phone unit multi-path simultaneous communication unit (4020), a touch display unit (4030), an image capturing unit (4040), and an audio input / output unit (4050).

[0077] The smart mirror phone control unit (4010) is connected to each functional unit constituting the smart mobile mirror phone unit (4000) and monitors the operating status through various operating programs and operating parameters that are built-in and installed, and controls it by outputting the corresponding control signal.

[0078] The phone multipath simultaneous communication unit (4020) is intended to simultaneously connect to the four communication methods via the multipath simultaneous communication unit (1120) and the smart mirror multipath communication network (2000) and simultaneously transmit or receive communication signals of the same content. It is equipped with and simultaneously activates communication function units of CDMA, Wi-Fi, Bluetooth, and TRS methods, and simultaneously transmits and receives communication signals of the same content through communication paths by each communication function unit.

[0079] The smart mirror phone control unit (4010) selects and uses the communication signal of the communication function unit that has the best signal-to-noise ratio among the communication signals received by the four communication function units and has no transmission error, and if no transmission error occurs in two or more communication function units, it uses the communication signal received by any one of the communication function units that is randomly selected.

[0080] The touch display unit (4030) is operated in an activated state by the corresponding control signal of the smart mirror phone control unit (4010), and the touch input signal can be input using an electrostatic method or a pressure method, wherein the electrostatic method is relatively preferable.

[0081] The touch display unit (4030) visually outputs a video image to be displayed while inputting a command signal by touch, and can be made of either an LED or an LCD selected from, and is well known.

[0082] The image capturing unit (4040) is operated in an activated state by the corresponding control signal of the smart mirror phone control unit (4010) and captures an image of a designated subject as a still image or video using visible light. It is very obvious that the image capturing unit (4040) can capture an image of the subject using an infrared signal as needed.

[0083] The audio input / output unit (4050) is operated in an activated state by the corresponding control signal of the smart mirror phone control unit (4010) and inputs an external audio-grade signal or outputs an internal audio-grade signal to the outside.

[0084] Meanwhile, data transmitted and received or transmitted between the smart mirror terminal device (1000), the smart mirror multi-communication network (2000), the smart big data server (3000), and the smart mobile mirror phone unit (4000) is unitized into smart mirror packet frames (6000) and transmitted.

[0085] The smart mirror packet frame (6000) consists of a total of 185 bytes and is configured to include an overhead field, a first packet frame data field, a second packet frame data field, a third packet frame data field, a check data field, and an end field.

[0086] A byte refers to a group of 8 bits, which are the smallest unit that records the presence or absence of a single pulse signal in a digital signal. Although 8 bits or more may be defined as one byte if necessary, in the following description, 8 bits are defined as one byte.

[0087] The overhead (OVHD) field consists of 10 bytes and includes the starting position where the data constituting the frame begins to be recorded, the transmission destination, the transmission time, the creation coordinate information, the size of the entire data in bits, and frame sequence number information indicating the connection status, as the data may consist of multiple frames if the amount of data is large.

[0088] The first packet frame data field consists of 50 bytes with a 1-byte gap from the overhead field, and specified gap control data is recorded.

[0089] The second packet frame data field consists of 50 bytes with a 1-byte gap from the first packet frame data field, and the same data stored in the first packet frame data field is repeatedly recorded.

[0090] The third packet frame data field consists of 50 bytes with a 1-byte gap from the second packet frame data field and is managed as a blank space, allowing it to be used for undefined purposes or uses, thereby increasing the usability and scalability of the gap data block unit frame.

[0091] The check (CHK) data field consists of 10 bytes with a 1-byte gap from the third packet frame data field, and if it is determined that the same data is not recorded by comparing the data recorded in the first packet frame data field and the second packet frame data field, error detection and recovery are performed using the cyclic redundancy check method.

[0092] The END field consists of 10 bytes separated by a 1-byte gap from the check data field and includes information on error detection and recovery, retransmission request status, data creator information, frame end position, and frame serial number information.

[0093] The public big data server (5000) is described as being composed of two or more of the following: a portal site server, a media company server, a broadcasting company server, a financial server, an industrial server, an educational server, a weather station server, a government office server, etc.

[0094]

[0095] FIG. 9 is a flowchart of a method for operating a smart mirror-based health training coaching system according to an embodiment of the invention.

[0096] Referring to all attached drawings, the method of operating a smart mirror-based health training coaching system is described as follows: in the preparation stage, when it is determined that a control signal for operating the smart mirror-based health training coaching system is input by the smart mirror terminal device (S100), an infrared distance sensor is activated to monitor whether the user is approaching (S110).

[0097] The smart mirror terminal device is based on the user identification stage. When it is determined that a user is nearby during the preparation stage (S110), the user is identified by photographing the face and body shape with visible light, analyzing the captured image using a spectroscopic method, analyzing momentum and energy consumption, and analyzing patterns by extracting feature points (S120), and the user is identified by searching for the designated name of the identified user and outputting the designated name as a voice signal to provide guidance (S130).

[0098] The smart mirror terminal device is based on the big data activation stage, and provides guidance by outputting the health field and scheduled health time specified by the user confirmed in the user-specific stage as a voice signal (S140), provides guidance on the analyzed current health status as a voice signal, and determines whether to train by activating a trainer video for coaching by a designated trainer that is searched from big data, recorded and stored (S150).

[0099] When the smart mirror terminal device determines that training will be performed by activating a trainer video for coaching by a designated trainer during the big data activation stage (S150), it outputs the trainer's coaching training video, captures a video of the user training to extract feature points and analyze patterns (S160), analyzes and displays similarity, exercise volume, energy consumption, and physical condition by comparing the training patterns of the user and the trainer, and provides guidance via voice signal (S170). When it determines that the user is checking the training results while moving (S180), it connects to the registered smart mobile mirror phone unit to control the operation of the smart mirror phone routine and proceeds to termination.

[0100] The Smart Mirror Phone Routine is controlled by the Smart Mirror Phone Control Unit constituting the Smart Mirror terminal device. While the Smart Mirror terminal device is in motion, it receives a control signal to verify the user's training results via the Phone Unit Multipath Simultaneous Communication Unit, and controls the Audio Input / Output Unit to output the training results as a voice message. When it is confirmed that the user responds to a guidance message or requests a video of the training process via the touch display unit, the user's training video downloaded from the Smart Mirror terminal device is output and displayed.

[0101]

[0102] FIG. 10 is a diagram illustrating a state in which feature points of a user training using a smart mirror terminal device are extracted according to an embodiment of the present invention.

[0103] When a user trains in a selected field of health on the front of the smart mirror terminal, the smart mirror terminal extracts feature points of each part of the body and links them to the corresponding time information to record and manage them as a video.

[0104] The smart mirror terminal compares and analyzes the location of each feature point, connected length, angle, etc., of the user and the trainer, and determines the degree of similarity or the need for more training in terms of pixel diameter.

[0105]

[0106] Although the present invention has been described in detail with respect to specific embodiments described above, it is obvious to those skilled in the art that various modifications and variations are possible within the scope of the technical spirit of the invention, and such modifications and variations are naturally included in the appended claims.

[0107]

[0108]

[0109] [Explanation of Symbols]

[0110] 900: Smart Mirror-Based Health Training Coaching System

[0111] 1000 : Smart mirror terminal device 2000 : Smart mirror multi-communication network

[0112] 3000 : Smart Big Data Server 4000 : Smart Mobile Mirror Phone Unit

[0113] 5000 : Public Big Data Server

Claims

1. A smart mirror terminal device (1000) that detects the state in which a user approaches within a predetermined distance using an infrared signal, extracts and analyzes feature points of the body and face by analyzing the captured image signal, prioritizes and displays a fitness activity preferred by a specific user, searches for and outputs a training video for coaching by a trainer regarding the selected fitness activity, records the user's training process as a video, and compares and analyzes the movement trajectories of each skeleton in three dimensions to record and output the results of the similarity analysis between the user's and trainer's skeleton trajectories in conjunction with the corresponding time information; A smart big data server (3000) that connects to the smart mirror terminal device (1000) and the smart mirror multi-communication network (2000), periodically searches for training videos for coaching by each trainer for each fitness category from a public big data server, records and manages them in an allocated area, and provides them upon request; A smart mirror-based health training coaching system comprising: a smart mobile mirror phone unit (4000) that connects to the smart mirror terminal device (1000) and the smart mirror multi-communication network (2000), receives and stores health training results practiced by the corresponding control signal, and outputs them by selection.

2. In Paragraph 1, The smart mirror terminal device (1000) above is A smart mirror centralized control unit (1010) that connects to each functional unit equipped with a built-in operation program and parameter values ​​for the overall operation of the smart mirror terminal device (1000), monitors the operation in real time, and outputs corresponding control signals; A smart mirror GPS unit (1020) that receives and analyzes a GPS signal based on a corresponding control signal of the smart mirror central control unit (1010), confirms, records, and outputs the coordinate information and time information of the current location; An infrared sensor unit (1030) that activates an infrared sensor by means of a corresponding control signal of the smart mirror central control unit (1010) and detects and outputs whether a user is approaching and the distance; A stereo imaging color analysis unit (1040) that captures an image of a user's face and body shape approaching in visible light in accordance with the corresponding control signal of the smart mirror central control unit (1010), records 64 color-specific image signals obtained by color-separating each pixel of the captured stereoscopic image signal into 64 parts, and outputs each of them to an allocated area; A feature point extraction analysis unit (1050) that receives a three-dimensional face and body shape image signal of a user captured in visible light by the corresponding control signal of the smart mirror central control unit (1010), records and stores it in an allocated area, extracts feature points, and analyzes, recognizes, records, and outputs the pattern of the three-dimensional face and the pattern of the skeleton using the extracted feature points; A pattern naming record unit (1060) that records and manages user naming corresponding to three-dimensional feature points analyzed by the corresponding control signal of the smart mirror central control unit (1010) and outputs it by searching for three-dimensional feature points; A smart mirror circuit board (1130) having the above-mentioned infrared sensor unit (1030) and stereo imaging color analysis unit (1040) fixedly installed on parts of the four side edges respectively, arranged to face the total reflection and translucent multimedia touch display unit (1080); A smart mirror-based health training coaching system comprising: a total reflection reinforcing mirror part (1140) made of a material that is relatively resistant to mechanical shock, which is made of a material that is similar in size to the smart mirror circuit board part (1130) and reflects light entering from the outside in one direction to the front part or transmits light entering from the rear part in one direction to the front part.

3. In Paragraph 2, The smart mirror terminal device (1000) above is A training state management unit (1070) that inputs the user's three-dimensional feature point pattern, which is captured and analyzed in three dimensions by the corresponding control signal of the smart mirror central control unit (1010), analyzes the distance of each captured feature point to the location of each corresponding target feature point in units of the diameter value of each pixel in comparison with the trainer's coaching training pattern, records and manages the data in conjunction with the corresponding shooting time value, and outputs the result. A multimedia touch display unit (1080) that outputs and displays image signals of the user's training motion and the trainer's training motion captured by the corresponding control signal of the smart mirror central control unit (1010), inputs a command signal by touch, and inputs and outputs audio signals; A trainer video database unit (1090) that records, stores, and outputs three-dimensional training video information of a trainer selected by health field and skeletal pattern information in a designated area in conjunction with the corresponding input time, based on the corresponding control signal of the smart mirror central control unit (1010); A smart mirror packet frame unit (1100) that converts a data signal transmitted externally by the corresponding control signal of the smart mirror central control unit (1010) into a series of smart mirror packet frames in each unit size, and analyzes each of the series of smart mirror packet frames received from the outside to extract a corresponding data signal consisting of a series of unit sizes; A TTS and an STT unit (1110) that convert text data applied by the corresponding control signal of the smart mirror central control unit (1010) into audio speech data and output it, and convert the applied audio speech data into text data and output it; A smart mirror-based health training coaching system further comprising: a multipath simultaneous communication unit (1120) that simultaneously provides and simultaneously activates communication function units of CDMA, Wi-Fi, Bluetooth, and TRS methods by means of a corresponding control signal of the smart mirror central control unit (1010), and simultaneously transmits and receives communication signals of the same content through communication paths by each communication function unit.

4. A method for operating a smart mirror-based health training coaching system comprising a smart mirror terminal device, a smart mirror multi-communication network, a smart big data server, a smart mobile mirror phone unit, and an open big data server, wherein A preparation step of activating an infrared distance sensor to monitor user approach when it is determined that a control signal for operating a smart mirror-based health training coaching system is input by the smart mirror terminal device; A user identification step that, when it is determined that a user has approached in the above preparation step, photographs the face and body shape using visible light, analyzes the captured image using a spectroscopic method, determines momentum and energy consumption, analyzes patterns through feature point extraction to identify the user, searches for the designated name of the identified user, and outputs the corresponding designated name as a voice-grade signal to provide guidance; A big data activation step that outputs and provides guidance on the health field and scheduled health time specified by the user identified in the above user-specific step as voice signals, provides guidance on the analyzed current health status as voice signals, and determines whether to train by activating a trainer video for coaching from a designated trainer that is searched from big data, recorded and stored; A method for operating a smart mirror-based health training coaching system comprising: a training operation step in which, if it is determined that training is taking place during the big data activation step, a training video for coaching by a trainer is output, a video of the user training is captured to extract feature points and analyze patterns, a comparative analysis of the training patterns of the user and the trainer is performed to analyze and display similarity, exercise volume, energy consumption, and physical fitness status, and guidance is provided via voice signal, and if it is determined that the training results are being checked while moving, the registered smart mobile mirror phone unit is connected to control the operation of the smart mirror phone routine and proceed to termination.

Images