Generative model-based method and system for providing exercise guide

Abstract

The present invention relates to a generative model-based method and system for providing an exercise guide. A generative model-based method for providing an exercise guide according to the present invention may comprise the steps of: receiving sensing information from at least one sensor in response to an activation event of an application; generating a prompt using the sensing information and user-related information of a user account logged into the application while the application is in an activated state; inputting the prompt to an agent while the application is in the activated state; and obtaining, from the agent, a user exercise guide corresponding to the prompt and providing the obtained user exercise guide through the application.

Description

Method and System for Providing Exercise Guides Based on Generative Models

[0001] The present invention relates to a method and system for providing exercise guidance based on a generative model.

[0002] Recently, with the rapid advancement of artificial intelligence (AI) technology, generative models capable of natural conversation with humans (e.g., ChatGPT) have emerged. In particular, generative models that include a Large Language Model are demonstrating innovation in the AI ​​market by showcasing technological capabilities that allow them to communicate naturally, almost like humans, and provide fast and accurate information, unlike traditional chatbots that are manually built and provide only limited answers.

[0003] Furthermore, the utilization of artificial intelligence (AI) technology in the medical industry is rapidly expanding. In particular, there is growing interest in AI technology that uses generative models to provide personalized, non-face-to-face care to patients requiring consistent management and rehabilitation during treatment.

[0004] For example, generative models can be effectively utilized for the management and rehabilitation of musculoskeletal disorders. Musculoskeletal disorders refer to pain or injury occurring in the musculoskeletal system, including muscles, nerves, tendons, ligaments, bones, and surrounding tissues. As a principle, the treatment of musculoskeletal disorders should begin with less invasive procedures; non-pharmacological conservative treatments (e.g., exercise therapy and education, cognitive therapy, or relaxation therapy) should be implemented first, followed by pharmacological treatment and surgical treatment in sequence. Treatment guidelines strongly recommend non-pharmacological conservative treatment for musculoskeletal disorders, and active research on methods for implementing such treatments is being conducted, primarily in the United States and Europe. However, since continuous treatment and rehabilitation are crucial for non-pharmacological conservative treatment, the need for patients to visit the hospital frequently poses a significant burden.

[0005] To address these issues and facilitate the consistent management and rehabilitation of specific diseases, there is a need to provide non-face-to-face exercise guidance services using generative models.

[0006] The present invention is intended to provide a generative model-based exercise guide provision method and system capable of providing a customized exercise guide to a user.

[0007] More specifically, the present invention aims to provide a generative model-based exercise guide provision method and system capable of providing customized exercise guides based on an artificial intelligence agent capable of natural language processing and image analysis.

[0008] Furthermore, the present invention aims to provide a generative model-based exercise guide provision method and system capable of updating the exercise guide based on a prompt that reflects user feedback in real time.

[0009] To solve the problem described above, the present invention proposes a method that utilizes a generative model to interact with a user remotely and guide exercise for the user's indications. The method for providing exercise guidance based on a generative model according to the present invention may include the steps of: receiving sensing information from at least one sensor in response to an activation event of an application; generating a prompt using user-related information of a user account logged into the application and the sensing information while the application is active; inputting the prompt into an agent while the application is active; and obtaining a user exercise guide corresponding to the prompt from the agent and providing the obtained user exercise guide through the application.

[0010] Furthermore, the above-mentioned at least one sensor includes a camera, and in a user terminal where the application is activated, the camera is activated based on the activation of the application, and an image received through the camera is output in real time to the display unit of the user terminal, and a graphic object indicating that the agent has been activated can be displayed on the display unit.

[0011] Furthermore, the agent can obtain the exercise guide corresponding to the prompt by linking with a generative model.

[0012] Furthermore, the above user-related information includes at least one of user account information and prescription information, the user account information includes at least one of the user's ID, name, date of birth, and gender information, and the prescription information may include information on indications and information on a treatment plan prescribed by a medical institution for said indications.

[0013] Furthermore, the exercise guide may include information related to at least one of an exercise video, exercise plan, exercise schedule, exercise movement, exercise method, exercise difficulty, number of exercises, exercise time, timing of exercise, and exercise description related to the exercise that the user must perform in relation to the indication.

[0014] Furthermore, the at least one sensor includes a camera provided in a user terminal in which the application is activated, and in the step of generating the prompt, an image received through the camera is included in the prompt along with the user-related information, and the prompt may include a first request to the agent to analyze the user's actions included in the image.

[0015] Furthermore, the prompt may further include a second request to generate a comment regarding the indication included in the prescription information as a result of analyzing the user's actions.

[0016] Furthermore, the at least one sensor includes a microphone provided in the user terminal on which the application is activated, and in the step of generating the prompt, voice information corresponding to the voice received through the microphone is included in the prompt, and the prompt may further include a third request to generate the exercise guide by reflecting user requests based on the voice information.

[0017] Furthermore, in the step of generating the above prompt, the voice can be converted into text based on a STT (Speech-to-Text) algorithm, and the converted text can be included in the prompt as voice information.

[0018] Furthermore, in the step of generating the above prompt, text of a pre-set topic related to the generation of the exercise guide can be extracted from the converted text, and the text of the pre-set topic can be included in the above prompt.

[0019] Furthermore, the aforementioned pre-set topic is related to at least one of adjusting the difficulty of the exercise, selecting the type of exercise, and changing the type of exercise, and the agent can check the exercise plan information previously provided to the user account based on the third request, and generate the exercise guide based on the checked exercise plan so as to reflect the user request.

[0020] Furthermore, the agent is configured to be linked with a database in which exercise videos corresponding to each of a plurality of different exercise items are stored, and the agent can generate an exercise guide that extracts at least one exercise item related to the indication from the database based on the prompt and enables the user to perform the exercise according to the extracted exercise item.

[0021] Furthermore, the agent generates the exercise plan composed of the at least one exercise item, and the application can control the user terminal so that exercise videos corresponding to the at least one exercise item are played sequentially according to the exercise plan.

[0022] Furthermore, in the above database, exercise videos corresponding to each of the exercise items are stored linked to the identifier (ID) of the exercise item, and the extraction of an exercise item by the agent corresponds to extracting an identifier corresponding to at least one exercise item related to the indication, and the application receives the identifier corresponding to the extracted exercise item from the agent and can play the exercise video corresponding to the identifier corresponding to the extracted exercise item on a user terminal.

[0023] Furthermore, while the exercise video is being played on the user terminal, the camera and microphone of the user terminal are maintained in an active state, and while the exercise video is being played on the user terminal, user feedback information is obtained through at least one of the camera and microphone; a feedback prompt is generated to update the exercise guide using the user feedback information; the feedback prompt is input to the agent to obtain an updated exercise guide from the agent, and the updated exercise guide is provided to the user terminal.

[0024] Furthermore, while the exercise video is being played on the user terminal, an image received from an activated camera is displayed in real time in a part of the user terminal, and feedback information regarding the user's exercise movements included in the image may be displayed in at least a part of the image.

[0025] Meanwhile, the generative model-based exercise guide providing system according to the present invention includes a communication unit that receives sensing information from at least one sensor in response to an activation event of an application, and a control unit that generates a prompt using user-related information of a user account logged into the application and the sensing information when the application is activated. The control unit inputs the prompt to an agent that guides user exercise when the application is activated, obtains a user exercise guide corresponding to the prompt from the agent, and can provide the obtained user exercise guide through the application.

[0026] Meanwhile, it is executed by one or more processes in an electronic device and can be read by a computer.

[0027] A program stored on a recording medium, wherein the program may include instructions for performing the steps of: receiving sensing information from at least one sensor in response to an activation event of an application; generating a prompt using user-related information of a user account logged into the application and the sensing information while the application is active; inputting the prompt to an agent that guides user exercise while the application is active; and obtaining a user exercise guide corresponding to the prompt from the agent and providing the obtained user exercise guide through the application.

[0028] The method and system for providing exercise guidance based on a generative model according to the present invention can resolve physical limitations in rehabilitation treatment and improve user accessibility by providing exercise guidance remotely based on prescription information including information on the user's indications.

[0029] Furthermore, the method and system for providing exercise guidance based on a generative model according to the present invention can provide customized exercise guidance that takes into account the user's current performance ability by reflecting user feedback based on a generative model.

[0030] Furthermore, the generative model-based exercise guide provision method and system according to the present invention can provide a user-customized exercise guide using automatically collected information without the need for separate document creation or data input, thereby providing a seamless user experience and improving user convenience.

[0031] FIG. 1 is a conceptual diagram illustrating a generative model-based exercise guide providing system according to the present invention.

[0032] FIG. 2 is a conceptual diagram for explaining the operation of an agent according to the present invention in general.

[0033] FIG. 3 is a flowchart illustrating a method for providing exercise guidance based on a generative model according to the present invention.

[0034] FIGS. 4a and FIGS. 4b are conceptual diagrams for explaining a prompt generation method according to the present invention.

[0035] FIG. 4c is a conceptual diagram illustrating an exercise plan included in an exercise guide according to the present invention.

[0036] FIG. 5 is a conceptual diagram illustrating the operation process of an agent according to the present invention.

[0037] FIGS. 6a, FIGS. 6b, and FIGS. 7 are conceptual diagrams for explaining the process of providing an exercise guide by analyzing image data according to the present invention.

[0038] FIG. 8 is a conceptual diagram illustrating the process of performing user authentication according to the present invention.

[0039] FIGS. 9a and 9b are conceptual diagrams illustrating the process of updating an exercise guide based on user feedback according to the present invention.

[0040] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components are assigned the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the spirit and technical scope of the present invention.

[0041] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0042] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0043] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0044] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0045] The present invention relates to a method and system for providing exercise guidance based on a generative model. More specifically, the present invention relates to a method and system for providing user-customized exercise guidance using an agent capable of natural language understanding and image analysis.

[0046] An agent according to the present invention may refer to an intelligent system that autonomously executes specific tasks without human intervention, based on a generative model. Specifically, the agent may provide a user-customized exercise guide to a user terminal by utilizing a generative model that includes a “Large Language Model (LLM).” Here, the “Large Language Model (LLM)” may refer to an artificial intelligence model capable of understanding and generating natural language by learning a vast amount of data.

[0047] The “exercise guide” according to the present invention may refer to various information related to exercise, such as exercise motion videos, exercise methods, analysis services for exercise motions, and feedback based on analysis results, which an agent provides to a user terminal in relation to the user’s indications. Here, “indications” refer to symptoms or clinical situations requiring specific treatment or examination, which may be understood as the user’s disease or symptoms. Furthermore, indications may refer to symptoms regarding a specific part of the user that is physically damaged.

[0048] According to the present invention, a user (or patient) can execute an application of a user terminal (10) to perform exercises according to an exercise guide provided in relation to the user's indications, and can receive an analysis service for the user's exercise movements.

[0049] In this case, the present invention can provide a user-customized exercise guide based on an automatic process that collects various information using a camera, a microphone, and a plurality of different sensors, and processes the collected information based on an agent. For example, the present invention can provide a user-customized exercise guide based on the user's voice and video recording of the user's exercise movements after activating a separate application, without the need for a separate menu selection.

[0050] Furthermore, when a user according to the present invention activates an application of a user terminal (10), the user can perform exercises according to an exercise guide provided by an agent and receive an analysis service for the user's exercise movements.

[0051] For convenience of explanation, the present invention focuses on “indications related to musculoskeletal disorders,” but is not necessarily limited thereto. As an example, the exercise guide described in the present invention may be for the treatment of users requiring rehabilitation treatment for musculoskeletal disorders and users suffering from various diseases (e.g., cancer, diabetes, hypertension, etc.).

[0052] Furthermore, the present invention may provide exercise guides necessary for health promotion in daily life, rather than rehabilitation exercises for therapeutic purposes related to the user's indications. For example, the exercise according to the present invention is not limited to any specific purpose and may be an exercise performed for various purposes, such as rehabilitation exercises, fitness exercises, ball sports, or dance exercises, for therapeutic, health promotion, or cosmetic purposes.

[0053] Furthermore, the exercise guide according to the present invention may refer to a guide for various exercises, such as rehabilitation exercises, fitness exercises, ball sports, and dance exercises, for various purposes including therapeutic, health promotion, and cosmetic purposes, and it can be understood that it is not limited to a specific category of exercise. That is, there is no limitation on the type of exercise according to the present invention, nor is there a limitation on the location of the exercise, such as indoor or outdoor exercise.

[0054] In the foregoing, the provision of an exercise guide based on a generative model according to the present invention has been generally described, and this can be implemented by the exercise guide provision system described below. Below, the exercise guide provision system based on a generative model according to the present invention will be described in detail with reference to FIGS. 1 and FIGS. 2. FIGS. 1 is a conceptual diagram for explaining the exercise guide provision system based on a generative model according to the present invention, and FIGS. 2 is a conceptual diagram for generally explaining the operation of an agent according to the present invention.

[0055] As illustrated in FIG. 1, the generative model-based exercise guide providing system according to the present invention (hereinafter referred to as the “exercise guide providing system,” 100) may include at least one of a communication unit (110), a storage unit (120), a control unit (130), and an artificial intelligence agent (200). At this time, the exercise guide providing system (100) according to the present invention is not limited to the components described above and may further include components that perform the same or similar roles as the functions described in the specification. Meanwhile, the exercise guide providing system (100) according to the present invention may be implemented as an application or software. The exercise guide providing system (100) implemented as software in this manner may be downloaded via a program (e.g., Play Store) that allows the application to be downloaded on a user terminal (10), or implemented via an initial installation program on the user terminal (10). In this case, the communication unit (110), the storage unit (120), and the control unit (130) according to the present invention may be utilized as components of the user terminal (10). In the present invention, the user terminal (10) can be understood to mean an application (1) installed on the user terminal (10). Such an application (or software, 1) can be understood as a component of the exercise guide providing system (100) according to the present invention.

[0056] In the present invention, the user terminal (10) may also be named a 'mobile terminal' or 'electronic device', and the user terminal (10) described in this specification may include a mobile phone, a smartphone, a smart TV, a laptop computer, a digital broadcasting terminal, a PDA (personal digital assistants), a PMP (portable multimedia player), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a smartwatch, a smart glass, a head-mounted display), etc.

[0057] More specifically, the user terminal (10) according to the present invention is not limited to an electronic device in which an application is activated, but may refer to an electronic device connected to an electronic device in which an application is activated. As an example, based on the fact that the user terminal (10) according to the present invention is a smartphone, the user terminal (10) may refer to a smart TV connected to said smartphone.

[0058] Meanwhile, the exercise guide providing system (100) may exist inside a server (hereinafter referred to as the server) built to perform a specific purpose (e.g., providing an exercise guide), or it may exist as a separate device from the server. When the exercise guide providing system (100) exists inside the server, the exercise guide providing system (100) according to the present invention may provide a user-customized exercise guide through at least one component among a communication unit (110), a storage unit (120), a control unit (130), and an artificial intelligence agent (200) located inside the server, or through a module that performs a function similar to each of the above components. In this case, the application may provide an exercise guide on a user terminal (10) on which the application is installed through communication with the server. Furthermore, the exercise guide providing system (100) according to the present invention may provide an exercise guide providing method according to the present invention to a user terminal (10) by linking with a plurality of different external servers.

[0059] Meanwhile, the artificial intelligence agent according to the present invention (hereinafter referred to as “Agent,” 200) may include at least one of a prompt generation unit (201) and a generative model (202). In the specification of the present invention, the agent (200) is described as including a generative model (202), but this is merely one embodiment, and the agent (200) and the generative model (202) may refer to the same artificial intelligence model. Accordingly, for convenience of explanation, the use of the generative model (202) by the agent (200) will be described as the agent (200) and the generative model (202) operating in conjunction.

[0060] As another embodiment, the present invention describes an agent (200) including a prompt generation unit (201), but is not limited thereto, and the agent (200) and the prompt generation unit (201) may exist separately. In this case, the agent (200) can generate a prompt in conjunction with the prompt generation unit (201), and the agent (200) inputting the prompt into the generative model (202) can be understood as the prompt being input into the agent (200). That is, the input of the prompt according to the present invention into the agent (200) and the input into the generative model (200) can be understood to have the same meaning.

[0061] Furthermore, the agent (200) according to the present invention may operate in conjunction with at least one of a database and a plurality of external servers in relation to providing exercise guides. For example, the agent (200) may be linked with a voice recognition server to convert user voice data received from a microphone equipped in a user terminal into text and reflect it in a prompt. Furthermore, the agent (200) may be linked with a posture estimation server to analyze the user's exercise movements using video data received from a camera equipped in a user terminal.

[0062] The types of generative models (202) used in the present invention may vary. For example, the generative model (202) may include at least one of GPT (Generative Pre-trained Transformer), T5 (Text-to-Text Transfer Transformer), Vision-Language Models (e.g., CLIP, Flamingo), Multimodal Generative Models (GPT-4 Multimodal, PaLM-E), Generative Adversarial Networks (GANs), Reinforcement Learning-based generative models (e.g., DeepMind's AlphaStar-like Models), and Custom Fine-tuned Models.

[0063] Referring to FIG. 2, the user (U, or patient) can perform an exercise plan according to prescription information prescribed by a medical institution for the indications of the user (U) through an application or webpage provided by the exercise guide providing system (100) according to the present invention.

[0064] A user (or patient, U) may possess a user account registered in the exercise guide providing system (100) according to the present invention. For convenience of explanation, the account of a user who is a patient is referred to as a "user account (or patient account)." The "account" described above may be created through a page linked to the exercise guide providing system (100). Alternatively, the "account" may be created on at least one other server (e.g., a medical staff server) linked to the exercise guide providing system (100) according to the present invention. Accordingly, in this specification, without distinguishing the server where the account was issued, all accounts based on the exercise guide providing system (100) according to the present invention are referred to as "accounts already registered in the exercise guide providing system (100) according to the present invention."

[0065] Meanwhile, a doctor can issue a prescription related to rehabilitation treatment to a user (U) through a doctor terminal (20). At this time, the doctor (D) may possess a doctor account already registered in the exercise guide providing system (100) according to the present invention. In this specification, a user terminal logged in with a doctor account is referred to as the doctor terminal (20). As an example, the exercise guide providing system (100) according to the present invention may receive prescription information prescribed by the doctor (D) to the user (U) by linking with a medical staff server.

[0066] The exercise guide providing system (100) according to the present invention can collect sensing information including at least one of voice data and video data based on a microphone (32), a camera (33), and a sensor unit (34) provided in a user terminal (10). Here, the sensor unit (34) may include at least one sensor among an infrared sensor, a LiDAR sensor, an accelerometer, an illuminance sensor, a proximity sensor, a position sensor, a face recognition sensor, an iris scanner, a heart rate sensor, a touch sensor, and a pressure sensor.

[0067] Meanwhile, the exercise guide providing system (100) according to the present invention may receive prescription information (210) prescribed by a doctor (D) for the rehabilitation treatment of a user (U) from a doctor terminal (20). The exercise guide providing system (100) may input at least one of user-related information and sensing information of a user account logged into an application into the prompt generation unit (201) of an agent (200). At this time, the “user-related information” may include at least one of prescription information (210) and user account information (220), and the sensing information may include at least one of voice data (230) and video data (240).

[0068] As illustrated in FIG. 2, the agent (200) can generate a prompt and input it into a generative model (202) by using at least one of prescription information entered into the prompt generation unit (201), user account information (220) logged into the user terminal, voice data (230), and video data (240).

[0069] The generative model (202) according to the present invention can generate an answer (260) corresponding to the input prompt based on the input prompt and provide it to the service page (30) of the user terminal (10).

[0070] According to the present invention, the communication unit (110) of the exercise guide providing system (100) can communicate with at least one of a user terminal (10) and a doctor terminal (20). Specifically, the communication unit (110) can receive prescription information (210) including information on indications assigned to the user (U) account (hereinafter referred to as “indication information”) from the doctor terminal (20) so that the user (U) can perform exercise according to the exercise guide. Furthermore, the communication unit (110) can receive different types of information (or data) from the user terminal in response to an application activation event (2).

[0071] The communication unit (110) can receive user account information (220) corresponding to a user account logged into the user terminal (10) from the user terminal (10). Additionally, the communication unit (110) can receive sensing information including at least one of collected user voice data and video data by using a microphone (32), a camera (33), and a sensor unit (34) provided in the user terminal (10).

[0072] Meanwhile, the communication unit (110) can support various communication methods depending on the communication standard of the communicating device. For example, the communication unit (110) may be configured to perform communication using at least one of the following technologies: WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth (Bluetooth™ Frequency Identification), Infrared Communication (Infrared Data Association; IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus).

[0073] Next, the storage unit (120) may be configured to store various information related to the present invention. In the present invention, the storage unit (120) may be provided within the exercise guide providing system (100) itself, or alternatively, at least a portion of the storage unit (120) may refer to a database (DB). That is, the storage unit (120) is sufficient as a space where information necessary to provide an exercise guide according to the present invention is stored, and the storage unit (120) can be understood as having no physical space constraints. Accordingly, below, the storage unit (120) and the database will not be distinguished separately and will all be referred to as the storage unit (120).

[0074] Data and commands necessary for the operation of the exercise guide providing system (100) according to the present invention may be stored in the storage unit (120).

[0075] User-related information may be stored in the storage unit (120). User-related information may include various information related to providing a user-customized exercise guide, such as i) user ID, ii) name, iii) date of birth, iv) user indications, v) exercise history, vi) medical history, vii) treatment plan, etc., according to at least one of user account information and prescription information. Furthermore, user authentication information included in the user account information may be stored in the storage unit (120). Here, “user authentication information” may refer to information used in a user authentication process performed to log in to a user account on a user terminal. As an example, user authentication information may be various, such as i) ID, ii) password, iii) password pattern, iv) user fingerprint authentication information, v) facial authentication information, vi) voice authentication information, vii) iris authentication information, viii) vein authentication information, etc., set by the user.

[0076] In the storage unit (120), information (e.g., exercise name, number of exercises, timing of exercises, difficulty of exercises, etc.) associated with exercise movements corresponding to each of the multiple pain sites (e.g., shoulder, elbow, wrist & hand, hip & pelvis, knee, ankle & foot, neck, back, waist, abdomen) related to the indication may be stored. Additionally, at least one of an exercise video and an exercise description corresponding to each exercise movement may be stored in the storage unit (120).

[0077] Next, the control unit (130) may be configured to control the overall operation of the exercise guide providing system (100) related to the present invention. The control unit (130) may process signals, data, information, etc. that are input or output through the components described above, or provide or process appropriate information and functions to the user.

[0078] Meanwhile, the control unit (130) can control the operation for providing exercise guidance by linking with the agent (200). Here, the specific operation performed by the agent (200) can be understood as being performed by the control unit (130). In the present invention, the control unit (130) and the agent (200) exist separately, and the control unit (130) controls the agent (200); however, this is not limited thereto, and the agent (200) may be configured to include the control unit (130), or the agent (200) may perform the same function as the control unit (130). For convenience of explanation, the following description describes the control unit (130) and the agent (200) performing operations in conjunction.

[0079] The control unit (130) can control the output of a service page (30) for providing an exercise guide through a display unit (or touchscreen) provided on the user terminal (10), as illustrated in FIGS. 1 and 2. Such a service page (30) can be output on the user terminal (10) through an application or web page installed on the user terminal (10). The service page (30) is a page linked to the exercise guide providing system (100) according to the present invention and is configured to be controlled by the exercise guide providing system (100) according to the present invention.

[0080] Furthermore, if the above page (30) is provided in the form of an application, the above page (30) can be controlled by the CPU (Central Processing Unit) of the user terminal (10) on which the application is installed. In this case, the CPU of the user terminal (10) can provide a customized exercise guide to the user based on the information provided by the exercise guide providing system (100) according to the present invention.

[0081] Meanwhile, the control unit (130) can work in conjunction with the agent (200) to extract text related to the creation of an exercise guide from voice data received from the microphone (32) equipped in the user terminal, and analyze the user's exercise movements from video data received from the camera (33).

[0082] The control unit (130) according to the present invention can provide a comment (260) of a generative model (202) to a service page (30) of a user terminal (10) in conjunction with an agent (200).

[0083] The control unit (130) can change (or update) the exercise guide based on user feedback information received from the user terminal (10) in conjunction with the agent (200). The control unit (130) can provide the updated exercise guide to the service page (30) of the user terminal (10) in conjunction with the agent (200).

[0084] Meanwhile, the exercise guide providing system (100) may include one or more processors, and such processors may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., digital signal processors, tensor processing units (TPUs), graphics processing units (GPUs), neural network processing units (NPUs), application integrated circuits, application semiconductors (ASICs), etc.). One or more processors may be configured to execute instructions, computer-readable instructions, and / or other instructions described herein that are stored (or included) in the storage unit (120). The exercise guide providing system (100) may perform data processing described below in cooperation with memory and at least one processor. The processor may perform a series of operations and data processing using data and information stored in memory. Here, “memory” may be a component of the storage unit (120), and “processor” may be used interchangeably with the control unit (130).

[0085] In the above description, the exercise guide providing system (100) of the present invention has been described, and it can be implemented based on the generative model-based exercise guide providing method described below.

[0086] Hereinafter, with reference to FIG. 3 together with FIG. 4a to 4c, FIG. 5, FIG. 6a, FIG. 6b, and FIG. 7, a method for providing an exercise guide based on a generative model according to the present invention will be described in more detail. FIG. 3 is a flowchart for explaining a method for providing an exercise guide based on a generative model according to the present invention, and FIG. 4a and FIG. 4b are conceptual diagrams for explaining a prompt generation method according to the present invention. FIG. 4c is a conceptual diagram for explaining an exercise plan included in an exercise guide according to the present invention, and FIG. 5 is a conceptual diagram for explaining the operation process of an agent according to the present invention. FIG. 6a, FIG. 6b, and FIG. 7 are conceptual diagrams for explaining a process of providing an exercise guide by analyzing image data according to the present invention.

[0087] In the present invention, in response to an activation event of an application, a process of receiving sensing information from at least one sensor may be performed (S310, see FIG. 3).

[0088] In the present invention, an application activation event may occur based on a specific user input entered into a user terminal (10) to execute an application. Here, the “activation event” may refer to an event in which an application already installed on the user terminal (10) is executed to implement an exercise guide providing system (100). At this time, the “specific user input” may refer to an input provided to activate an application that provides content services. For example, the specific user input may include at least one user input among i) touch, ii) click, iii) voice command, iv) gesture recognition, and v) QR code scanning for application activation on the user terminal.

[0089] Furthermore, the control unit (130) can activate at least one sensor in response to an activation event in which an application is executed. For example, when an application is executed on a user terminal, the control unit (130) can activate a sensor unit including a microphone, a camera, and a plurality of different sensors equipped on the user terminal (10).

[0090] The control unit (130) can receive sensing information based on the fact that at least one of the microphone, camera, and sensor unit provided in the user terminal (10) is activated. For example, the control unit (130) can activate at least one of the camera and microphone based on the occurrence of an application activation event, and receive at least one of the user's video and the user's voice from the camera. At this time, the explanation assumes that a user account is logged in to the application in advance.

[0091] More specifically, in a user terminal where the application is activated, the camera can be activated based on the activation of the application. The display unit of the user terminal (10) can display the image received through the camera in real time.

[0092] Furthermore, the control unit (130) can activate the agent (200) based on an application activation event, and the display unit can display an agent graphic object (500) indicating that the agent (200) has been activated.

[0093] At this time, the “agent graphic object (500)” may be provided in various ways. For example, the agent graphic object may be provided in the form of an avatar (e.g., human form, character form, etc.) so that the avatar’s behavior changes according to the response of the generative model. As another example, the agent graphic object may be provided in the form of a visual indicator, such as a waveform (e.g., voice waveform) or a shape (e.g., circle), rather than in the form of an avatar. Here, “visual indicator” comprehensively refers to an element that visually displays the state or activity of the system and may include visual effects (or animations) that are actions on the service page.

[0094] The control unit (130) can collect user-related information based on an application activation event. Here, “user-related information” may include at least one of prescription information and user account information. Specifically, “user account information” may include at least one of the user’s ID, name, date of birth, and gender information, and “prescription information” may include at least one of information regarding indications and information regarding a treatment plan prescribed by a medical institution for said indications.

[0095] As previously explained, “indication” refers to a symptom or clinical situation requiring specific treatment or examination, which can be understood as the user’s disease or symptoms. The control unit (130) may collect prescription information including at least one of information regarding the user’s indication and information regarding a treatment plan from at least one of the doctor terminal (20) and the medical staff server.

[0096] Meanwhile, the control unit (130) can count the rehabilitation period matched to the exercise plan based on the activation event of the application. In the present invention, the day on which the counting of the rehabilitation period begins can be described as the rehabilitation exercise “start date” or “reference date.” For example, in the present invention, the reference date can be counted as Day 1, and the day after the reference date as Day 2. As another example, in the present invention, the 7 days including the reference date can be described as Week 1, and the 7 days following Week 1 as Week 2. Such counting of the rehabilitation period can be utilized to provide an exercise plan. For example, the control unit (130) can provide different exercise plans based on the counting of the rehabilitation period. As one example, the control unit (130) may provide an exercise plan related to recovery exercises based on the fact that the counted rehabilitation period is week 1, and as another example, the control unit (130) may provide an exercise plan related to strength exercises based on the fact that the counted rehabilitation period is week 4.

[0097] Meanwhile, as illustrated in FIG. 1, the control unit (130) may provide a service page (30) on the user terminal (10) based on an application activation event. At this time, an image captured through a camera (33) equipped on the user terminal (10) may be displayed on the service page (30), and a message (31) related to an exercise guide may be provided to the user. As an example, the control unit (130) may count the rehabilitation period based on the application activation event and utilize the rehabilitation period to provide an exercise guide. Furthermore, based on the counted rehabilitation period, the agent (200) may provide a message related to an exercise guide on the service page (30), such as “You have already been exercising for two weeks. That is amazing!”

[0098] As another example, the agent (200) can check information related to providing exercise guidance to the user by using the collected prescription information and user account information. For example, the agent (200) can check exercise history information stored in the storage unit (120). Here, the “exercise history information” may include at least one of information about exercise plans performed by the user in the past, user feedback information regarding said exercise plans, and past exercise summary information (e.g., exercise result sheet).

[0099] Furthermore, the agent (200) may provide a message (31) related to an exercise guide, such as “We are rooting for a healthy day, Young-hee Kim! How was your workout yesterday?” on the initial page (30) via at least one of sound and text in order to collect user feedback information about previously performed exercises.

[0100] Next, in the present invention, while the application is active, a process of generating a prompt using user-related information and sensing information of a user account logged into the application may be performed (S320, see FIG. 3).

[0101] The control unit (130) can check user-related information in response to an application activation event.

[0102] The control unit (130) can analyze verified user-related information to extract information to be included in the prompt. More specifically, the control unit (130) can analyze user account information to extract the user's age and gender so that different user account information is reflected in the prompt according to the user's age and gender.

[0103] As an example, the control unit (130) may reflect in the prompt that exercise items with an exercise difficulty level lower than or equal to a preset difficulty level are included in the exercise plan based on the fact that the user is female. Here, the “preset difficulty level” may be set by at least one of the user’s age, gender, and indication information. Furthermore, the control unit (130) may request the agent (200) to generate a prompt containing extracted user-related information.

[0104] Additionally, the control unit (130) can analyze the confirmed prescription information to extract information to be included in the prompt. For example, the control unit (130) can analyze at least one of the information regarding the user's indication and treatment plan to extract detailed information related to the indication, such as the user's current condition and past medical history. Furthermore, the control unit (130) can request the agent (200) to generate a prompt containing the extracted user-related information.

[0105] Furthermore, the control unit (130) may request the agent (200) to generate a prompt containing extracted prescription information.

[0106] Furthermore, the control unit (130) can activate at least one sensor based on an application activation event. The agent (200) can generate a prompt using the sensing information and user-related information received from the at least one sensor. At this time, in order to generate a prompt for providing exercise guidance, the control unit (130) can analyze the sensing information received from the at least one sensor and extract information to be included in the prompt.

[0107] For example, the control unit (130) can activate a camera equipped on a user terminal where the application is activated among at least one sensor. Furthermore, the control unit (130) can link with the agent (200) to include the video received through the camera in a prompt along with user-related information.

[0108] At this time, the control unit (130) can analyze the video received through the camera and extract the video necessary for providing an exercise guide. More specifically, the control unit (130) can extract a video containing the user's exercise movements from the video received through the camera and request the agent (200) to include it in the prompt.

[0109] That is, the control unit (130) does not extract images containing movements other than exercise movements from the images received from the camera, but extracts only images containing exercise movements, so that image data for analyzing the user's exercise movements is reflected in the prompt.

[0110] The control unit (130) can provide a user-customized exercise guide through an organic process that analyzes user-related information and sensing information without separate user input based on an application activation event, extracts necessary information, and reflects it in a prompt.

[0111] Accordingly, the agent (200) can generate a prompt based on extracted user-related information and sensing information. More specifically, the agent (200) can generate a prompt related to at least one of an analysis of user exercise movements and a comment on the results of the analysis of user exercise movements. For example, the prompt may include a first request to the agent to analyze the user's movements included in the video. Furthermore, the prompt may further include a second request to generate a comment on the indications included in the prescription information as a result of the analysis of the user's movements.

[0112] Meanwhile, the control unit (130) can activate a microphone equipped in a user terminal where the application is activated among at least one sensor. The control unit (130) may include voice information (or voice data) corresponding to the voice received through the microphone in a prompt in conjunction with the agent (200). At this time, the prompt may further include a third request to generate an exercise guide by reflecting user requests based on the voice information.

[0113] As illustrated in FIG. 4a, the agent (200) may input at least one of prescription information (210), user account information (220), voice analysis data (231), and video analysis data (241) into the prompt generation unit (201). Here, “voice analysis data (231)” may refer to information obtained by analyzing voice data (or voice, 230) collected through a microphone provided in the user terminal (10). At this time, the subject of voice analysis according to the present invention may be diverse. As an example, the agent (200) may analyze voice data (or voice, 230) in conjunction with a voice recognition server and generate voice analysis data (231).

[0114] As another example, an agent (200) can use a generative model (202) to analyze the voice data (or voice, 230) and generate voice analysis data (231). More specifically, the generative model according to the present invention may be understood as a generative model capable of voice recognition, including a voice recognition model. For example, the generative model (202) can analyze the voice data (230) based on a STT (Speech-to-Text) algorithm, and the agent (200) can use the generative model (202) to generate voice analysis data (231).

[0115] Meanwhile, “video analysis data (241)” may refer to information obtained by analyzing video data (240) including the user’s exercise movements. At this time, the subject of the video analysis according to the present invention may be diverse.

[0116] As an example, the agent (200) may be linked with a posture estimation server to analyze video data (240) including user exercise movements and generate video analysis data (241). As another example, the agent (200) may use a generative model (202) to analyze the video data (240) and generate video analysis data (241).

[0117] More specifically, the generative model (202) according to the present invention may include a vision-based generative model. Here, the “vision-based generative model” may refer to a generative model capable of understanding, transforming, and generating image data, and capable of analyzing image data. For example, the generative model (202) may generate image analysis data (241) by using at least one of a Generative Adversarial Network (GAN)-based model, a Video Generation Model, a Video Understanding Model, VideoGPT, a Transformer-based model (e.g., TimeSformer), Diffusion Models for Video, and Spatio-temporal Generative Models.

[0118] Meanwhile, the generative model (202) according to the present invention may include a motion generation model. Here, the “motion generation model” may refer to a model that learns the movement of a user and generates a vector corresponding to the user’s movement (or motion).

[0119] A motion generation model can analyze a user's movement in a video to extract time-series data, and using the extracted time-series data, generate vector data including at least one of a joint position, joint angle, velocity, and acceleration corresponding to the user's skeletal structure. Furthermore, based on the generated vector data, the motion generation model can generate motion data in which the user's movement over a specific period of time is vectorized, and can visually output the motion data in at least one of a two-dimensional and a three-dimensional space.

[0120] In this way, the generative model (202) according to the present invention can analyze the movement of a user included in video data by using at least one of a vision-based generative model and a motion-generated model. Furthermore, the generative model (202) can generate video analysis data (241) regarding the movement of the user.

[0121] Furthermore, the generated video analysis data (241) may include a movement analysis result (242) obtained by analyzing the user's movement from video data including the user's movement video.

[0122] In the following description, the agent (200) is described as analyzing voice data and video data by linking with a voice recognition server and a pose estimation server, but is not limited thereto, and the agent (200) may also analyze voice data and video data using a generative model (202).

[0123] At least one of the voice recognition server and the posture estimation server according to the present invention may be provided inside the exercise guide providing system (100) according to the present invention or may be an external server. Accordingly, the agent (200) of the present invention may analyze voice data received through a microphone provided in the user terminal (10) by linking with at least one of the voice recognition server and the posture estimation server, and may analyze the user's exercise movements included in the video received through the camera.

[0124] As illustrated in FIG. 4b, an agent (200) can input voice data (230) received from a user terminal (10) into a voice recognition server (40). The voice recognition server (40) may include at least one voice recognition model (41) based on a Speech-to-Text (STT) algorithm that converts voice data into text. As an example, the voice recognition server (40) may include at least one voice recognition model (41) among a Hidden Markov Model (HMM), a Hidden Markov Model-Gaussian Mixture Model (HMM-GMM), a Connectionist Temporal Classification (CTC) based model, a Beam Search based model, a Deep Neural Network (DNN) based model, a Recurrent Neural Network (RNN) based model, a Sequence-to-Sequence (Seq2Seq) model, and a Transformer based model.

[0125] The agent (200) can convert voice (or voice data) into text based on a STT (Speech-to-Text) algorithm and include the converted text as voice information (or voice analysis data) in a prompt. Specifically, the agent (200) can preprocess voice data (230) received through the microphone (32) by linking with the voice recognition server (40). For example, the voice recognition server (40) can remove noise from the input voice data (230) through filtering and perform normalization through scaling.

[0126] Furthermore, the voice recognition server (40) can extract features of the voice data (230) from the preprocessed voice data (230) and perform mapping between voice features and linguistic units (phonemes or words). The voice recognition server (40) can generate a word sequence corresponding to the voice features and convert the voice data into text. The agent (200) can work in conjunction with the voice recognition server (40) to extract text of a pre-set topic related to the creation of an exercise guide from the converted text, and include the text of the pre-set topic in the prompt. Here, “pre-set topic” may refer to a topic related to at least one of adjusting the difficulty of the exercise, selecting the type of exercise, and changing the type of exercise.

[0127] For example, the agent (200) can recognize a pre-set topic related to changing the type of exercise from voice data such as “The exercise movement is too difficult! Change it to a different exercise movement!” and include voice analysis data for the voice data in the prompt.

[0128] More specifically, the agent (200) can include the text of a pre-set topic extracted from text corresponding to voice data (230) in the prompt by using voice analysis data (231) and inputting it into the prompt generation unit (201).

[0129] Furthermore, the agent (200) can analyze video data based on receiving a first request requesting analysis of the user's movements. As previously described, the subject of video data analysis according to the present invention may be diverse. As an example, the agent (200) can generate video analysis data regarding the user's exercise movements included in the video data based on a generative model (202). For example, the agent (200) can generate video analysis data regarding the user's exercise movements using a vision-based generative model that analyzes the user's exercise posture using at least one of a Generative Adversarial Network (GAN) based model, a Video Generation Model, a Video Understanding Model, VideoGPT, a Transformer based model (e.g., TimeSformer), Diffusion Models for Video, and Spatio-temporal Generative Models.

[0130] As another example, the agent (200) may input image data (240) received from the user terminal (10) into the pose estimation server (50). The pose estimation server (50) may include at least one of a learning unit (51) and a pose estimation model (52). Here, the “pose estimation model (52)” is a pose estimation model learned using a learning data set containing position information for joint points, and can estimate the exercise pose of the user (U) from the exercise video to be analyzed.

[0131] The agent (200) can determine the accuracy of the user's exercise movements included in the video data (240) by linking with the posture estimation server (50), and generate video analysis data (241) based on the determination result. Furthermore, the agent (200) can input the video analysis data (241) corresponding to the user's exercise movements included in the video data (240) into the prompt generation unit (201). The posture estimation server and user motion analysis according to the present invention will be described in detail below together with the related FIGS. 6a and 6b.

[0132] Meanwhile, the agent (200) may input prescription information, including at least one of information on indications and a treatment plan, into the prompt generation unit (201). Here, the “indication information (211)” may include various information regarding the user’s indications, such as i) information on pain conditions, and ii) information on pain sites (e.g., shoulder, elbow, wrist & hand, hip & pelvis, knee, ankle & foot, neck, back, lower back, abdomen). Additionally, the “treatment plan” may include various information such as treatment purposes, treatment goals, rehabilitation exercise items, rehabilitation period, and precautions for the user’s indications.

[0133] The agent (200) can generate an exercise plan corresponding to the user's indications based on at least one of user-related information and sensing information, based on the input prompt. The “exercise plan” according to the present invention may include various information related to exercise movements prescribed to a user with specific symptoms in a specific body part, based on the indication information (211). In the exercise plan, at least one exercise item may be assigned to each of the multiple days or multiple weeks constituting the rehabilitation period.

[0134] More specifically, the agent (200) may generate an exercise plan based on a third request to generate an exercise guide by reflecting user requests based on exercise history information and voice information. Here, “exercise history information” may include at least one of information on exercise plans performed by the user in the past, user feedback information on said exercise plans, and past exercise summary information (e.g., exercise results sheet).

[0135] Meanwhile, in the exercise plan according to the present invention, exercise items corresponding to indication information (211) may be predetermined and exist. The exercise items constituting each exercise plan may be exercise items selected to be effective for treatment based on the indication targeted by each exercise plan, based on experts, expert groups, or artificial intelligence algorithms.

[0136] That is, the exercise plan can be generated by the agent (200) based on at least one of user-related information and sensing information, and may also be pre-set by medical staff and included in the treatment plan.

[0137] As illustrated in FIG. 4c, the exercise plan (213) may include at least one of the following: i) rehabilitation period information (461), ii) exercise item (e.g., “squat,” “standing arm rotation,” 462), iii) intensity (or difficulty) information of the exercise item (463), iv) number of repetitions of the rehabilitation exercise item (e.g., 5 repetitions, 564), v) total exercise time information for the rehabilitation exercise (465), vi) precaution information (e.g., “Please apply an ice pack after the exercise”), and vii) information about the exercise tool. Here, “exercise item” can be understood as an exercise movement or a type of exercise, and in the present invention, “exercise item,” “exercise movement,” and “type of exercise” may be used interchangeably. Additionally, “intensity of the exercise item” may correspond to the degree of load during exercise performance. The intensity of the exercise item may be pre-set based on the composition of the exercise item, the number of repetitions, the total exercise time, etc.

[0138] The agent (200) according to the present invention can generate a prompt (250) using at least one of prescription information (210), user account information (220), voice analysis data (231), and video analysis data (241) input into the prompt generation unit (201).

[0139] The prompt generation unit (201) can utilize natural language processing (NLP) technology to receive multiple different data inputs, extract prompt information, and generate a prompt (250) to be input into a generative model (202). Here, natural language processing (NLP) technology may refer to technology capable of understanding, interpreting, and generating human language using artificial intelligence technology (e.g., deep learning).

[0140] As illustrated in FIG. 4a, the prompt generation unit (201) can extract prompt information from at least one of the indication information (211) and the treatment plan (212) included in the prescription information (210). For example, the prompt generation unit (201) can extract first prompt information (420) including “[Treatment Site]_{Patella}”, “[Disease]_{Arthritis}”, and “[Rehabilitation Period]_{Day 1 of Week 2}” from the prescription information (210). Additionally, the prompt generation unit (201) can extract prompt information from user account information (220). For example, the prompt generation unit (201) can extract second prompt information (410) including “[User Account Information]_{Name_Kim Young-hee}, {Gender_Female}, {Age_29 years old}” from the user account information (220).

[0141] The agent (200) may input voice analysis data, which analyzes voice data, into the prompt generation unit (201) based on a third request to generate the exercise guide by reflecting user requests based on voice information (or voice data). Furthermore, the prompt generation unit (201) may extract prompt information from the voice analysis data. For example, the prompt generation unit (201) may extract third prompt information (430), such as "[Voice Analysis Result] {User Question_ “Is this movement correct?}", from voice analysis data (231) which analyzes the user’s voice data “Is this movement correct?” (232).

[0142] The agent (200) may input video analysis data containing the results of analyzing the user's exercise movements into the prompt generation unit (201) based on a second request to generate a comment on the indication as an analysis result of the user's exercise movements. Furthermore, the prompt generation unit (201) may extract prompt information regarding the results of the exercise movements from the video analysis data. For example, the prompt generation unit (201) may extract a fourth prompt information (440), such as "[Video Analysis Result] {Posture Problem_ “Arm is too bent”}, from video analysis data (231) containing the results of the user's exercise movements (e.g., “User’s arm is too bent”, 242).

[0143] In this way, the agent (200) can actively analyze user-related information and sensing information of the user account without a separate user command. Furthermore, the control unit (130) can request the agent (200) to extract only the necessary information from the user-related information and sensing information of the user account and reflect it in the prompt in order to provide a user-customized exercise guide.

[0144] That is, the control unit (130) can request the agent (200) to provide an exercise guide related to the user's indications according to the user's needs based on the extracted multiple prompt information. Furthermore, the agent (200) can use the prompt generation unit (201) to generate a prompt (450) to be input into a generative model (202) based on the extracted multiple prompt information (410 to 440).

[0145] Next, in the present invention, while the application is active, a process of inputting a prompt into an agent may be performed (S330, see FIG. 3).

[0146] The agent (200) according to the present invention can input a generated prompt into a generative model based on user-related information and sensing information. The generative model can generate an answer to be provided to the user based on the input prompt. As previously described, the input of the prompt (250) generated by the prompt generation unit (201) into the generative model (202) can also be understood as the prompt (250) being input into the agent (200).

[0147] As previously explained, the specification of the present invention describes the agent (200) as including a generative model (202), but this is merely one embodiment, and the agent (200) and the generative model (202) may refer to the same artificial intelligence model. Accordingly, the agent (200) utilizing the generative model (202) can be understood as the agent (200) and the generative model (202) operating in conjunction with each other.

[0148] Furthermore, the agent (200) can obtain an exercise guide corresponding to the prompt by linking with a generative model. As previously described, the generative model (202) according to the present invention may include a large language model (LLM), and the agent (220) may output a response to the input prompt based on the LLM.

[0149] More specifically, the agent (200) can generate a prompt containing analysis results of the user's voice information and movement actions by using at least one of user-related information and sensing information based on prompt engineering. Here, prompt engineering may refer to a technique for designing and optimizing input text (prompt) to effectively utilize a natural language processing (NLP) model.

[0150] The agent (200) can provide an exercise guide corresponding to an input prompt using a generative model. For example, the agent (200) can provide at least one of an exercise motion video, an exercise method, an analysis service for the exercise motion, and feedback based on the analysis results in relation to the user's indication.

[0151] As illustrated in FIG. 5, the control unit (130) may provide a service page (30) to the user terminal (10) based on an application activation event. An image corresponding to the image data collected from the user terminal (10) may be displayed on the service page (30). For example, an image including the appearance of the user received through a camera equipped in the user terminal (10) may be displayed on the service page (30).

[0152] Additionally, as previously described, the control unit (130) may provide an agent graphic object (500) to the service page (30) based on the activation of the agent (200). The agent graphic object (500) may be displayed with different appearances based on the provided exercise guide. For example, based on the user performing an exercise according to the exercise guide provided by the agent (200), the appearance of the first agent graphic object (500) may be changed to display an agent graphic object corresponding to the exercise, such as the second agent graphic (500a).

[0153] Furthermore, the agent (200) may generate an initial question (510) to receive user feedback based on the service page (30) being provided to the user terminal (10). The initial question (510) may include questions related to the user's current pain state, the intensity of past exercises, satisfaction with the exercise plan, etc.

[0154] The agent (200) can control the generative model (202) to generate an initial question (510) based on collected user-related information. For example, the generative model (202) can generate an initial question (510) such as “We are rooting for a healthy day, Young-hee Kim! How is your pain level today?” based on exercise history information. Here, “exercise history information” may include at least one of information about exercise plans performed by the user in the past, user feedback information about said exercise plans, and past exercise summary information (e.g., exercise results sheet).

[0155] The agent (200) can provide an initial question (510) to the user through the user terminal (10). As an example, the agent (200) can visually output a text message corresponding to the initial question (510) through the display of the user terminal (10). As another example, the agent (200) can audibly output a voice message corresponding to the initial question (260) through a speaker provided in the user terminal (10).

[0156] The control unit (130) can activate at least one of the microphone, camera, and sensor unit of the user terminal (10) to obtain user feedback information (520) regarding an initial question (260) provided to the user terminal (10). Here, “feedback information (520)” may refer to information related to the user’s response to the initial question (260). More specifically, the feedback information (520) may be understood as information obtained by analyzing the user’s response from sensing information including at least one of voice data and video data.

[0157] More specifically, the feedback information (520) may refer to voice analysis data obtained by the agent (200) based on a third request regarding user voice information (or voice data). As an example, the agent (200) may analyze the user's voice data “My pain has decreased a lot! Let’s start exercising today!” in conjunction with the voice recognition server (40). Voice data analysis according to the present invention is omitted as it has been described above.

[0158] The agent (200) can process voice analysis data obtained from the voice recognition server (40) as feedback information (520). Furthermore, the agent (200) can check the exercise plan information previously provided to the user account based on the third request, and generate an exercise guide based on the confirmed exercise plan so that the user request is reflected. Furthermore, the agent (200) can input a prompt (250) reflecting user feedback into the generative model (202), and generate a feedback response (530) corresponding to the feedback information (520) based on the prompt (250). The control unit (130) can provide the feedback response (530) through the user terminal (10) in conjunction with the agent (200). As an example, the control unit (130) can visually output a text message corresponding to the feedback response (530) through the display of the user terminal (10). As another example, the control unit (130) can audibly output a voice message corresponding to the feedback response (530) through a speaker provided in the user terminal (10).

[0159] Meanwhile, as previously described, the control unit (130) can count the rehabilitation period matched to the exercise plan based on the application's activation event. This rehabilitation period can be used to provide exercise guidance. For example, the agent (200) can provide exercise guidance related to the exercise items assigned to a specific day on which the exercise plan is provided, based on the reference date on which the counting of the rehabilitation period began.

[0160] More specifically, the agent (200) can generate a prompt containing information about the rehabilitation period based on the counted rehabilitation period. Furthermore, the agent (200) can provide an exercise guide using the prompt containing information about the counted rehabilitation period. For example, the agent (200) can provide a feedback response (530) to the user terminal (10) including “Today is the first day of the second week of rehabilitation exercises.”

[0161] Next, in the present invention, a user exercise guide corresponding to a prompt is obtained from an agent, and the obtained user exercise guide is provided through an application (S340, see FIG. 3).

[0162] As previously explained, the agent (200) according to the present invention can obtain an exercise guide corresponding to a prompt by linking with a generative model. The agent (200) may be configured to be linked with a database in which exercise videos corresponding to each of a plurality of different exercise items are stored. Here, the “exercise guide” may include information related to at least one of an exercise video, exercise plan, exercise schedule, exercise movement, exercise method, exercise difficulty, number of exercises, exercise time, time of exercise, and exercise description related to the exercise that the user must perform in relation to the user’s indication.

[0163] As previously explained, the exercise plan may be generated by the agent (200) based on at least one of user-related information and sensing information, or it may be pre-set by medical staff and included in a treatment plan. In this case, at least one exercise item may be assigned to each of the multiple days or multiple weeks constituting the rehabilitation period in the exercise plan, and at least one of an exercise video and an exercise description corresponding to the exercise item may be stored in the storage unit (or database, 120).

[0164] The agent (200) may, based on a prompt, extract at least one exercise item related to an indication from a database and generate an exercise guide that allows the user to perform the exercise according to the extracted exercise item. At this time, an exercise video corresponding to each exercise item may be stored in the database linked to an identifier (ID) of the exercise item. Here, the “identifier of the exercise item” may refer to at least one of a unique value and a code used to distinguish each exercise item within the database.

[0165] The agent (200) may extract an identifier corresponding to at least one exercise item related to the user's indication in order to extract at least one exercise item according to the exercise plan. The application receives the identifier corresponding to the extracted exercise item from the agent and may play an exercise video corresponding to the identifier corresponding to the extracted exercise item on the user terminal.

[0166] In another example, the agent (200) may command the application to play an exercise video corresponding to an exercise item in order to provide an exercise video matched to an exercise item. More specifically, the agent (200) may create an exercise plan consisting of at least one exercise item, and the application may control the user terminal so that an exercise video corresponding to at least one exercise item is played sequentially according to the exercise plan. At this time, the agent (200) may generate a control command to the application to sequentially play at least one exercise video corresponding to an exercise item included in the exercise plan. Furthermore, the application may generate an exercise guide including the at least one exercise video based on the control command. More specifically, the agent (200) may provide an exercise video for at least one exercise item constituting the exercise plan to the service page (30) so that the user exercises according to the generated exercise guide.

[0167] At this time, the agent (200) can receive the user's exercise progress status in real time using the activated camera. The agent (200) can analyze the user's posture (or movement) regarding the exercise movement being performed based on the user performing the exercise movement following the exercise video played on the user terminal (10). Specifically, the agent (200) can analyze the user's exercise movement included in the video received through the activated camera based on a first request including a request for analysis of the user's movement.

[0168] The agent (200) can perform an analysis of user exercise movements in video data collected from the user terminal (10) by linking with the pose estimation server (50). Alternatively, the agent (200) can perform an analysis of user exercise movements based on a generative model. More specifically, the generative model of the present invention may include a pre-trained pose estimation model, and the agent (200) can analyze video data based on the generative model.

[0169] In the following, video data including user exercise movements may also be referred to as exercise video data.

[0170] Referring to FIG. 6a, the posture estimation server (50) according to the present invention may refer to a cloud server that performs analysis of a user's exercise motion from exercise video data that captures the user's exercise motion.

[0171] The agent (200) may select a data transmission strategy to transmit exercise video data (600) capturing the user's exercise movements to the posture estimation server (50). As an example, the agent (200) may transmit all frames of the exercise video data to the posture estimation server (50) for accurate exercise movement analysis. At this time, the posture estimation server (50) may perform exercise movement analysis on all frames of the exercise video data without omission.

[0172] As another example, the agent (200) can selectively transmit frames of motion video data to the pose estimation server (50) by removing unnecessary data to improve data processing speed. At this time, the pose estimation server (50) can improve data processing speed by removing unnecessary data.

[0173] Meanwhile, the posture estimation server (50) can analyze the relative positional relationship between key points (P1, P2) corresponding to multiple joint points of the user (U) extracted from the exercise video data (600) through a posture estimation model learned using training data related to joint points. Here, “joint point” may refer to multiple joints of the user (U) (or a part of the user (U)’s body including joints). And, “key point” may refer to an area corresponding to each of the multiple joint points of the user (U) in the exercise video data (600). Accordingly, in the present invention, “joint point” and “key point” may be used interchangeably, and the same reference numeral “P2” may be assigned to each joint point and key point to explain them.

[0174] The control unit (130) can use a posture estimation model (52) to extract key points (P1, P2) corresponding to joint points from the user's exercise video data (600), and analyze the user's (U) exercise motion based on an analysis of the positional relationship between the extracted key points (P1, P2). In the present invention, a series of processes for analyzing the user's exercise motion from the exercise video using key points extracted through the artificial intelligence posture estimation model (52) can be named the “exercise motion analysis process.”

[0175] In the present invention, the physical space and subject where the exercise motion analysis process takes place are not separately distinguished, and it can be described as taking place in the exercise guide providing system (100). The exercise motion analysis process can be performed using key points extracted from the posture estimation model (52). As previously described, the generative model may include a previously learned posture estimation model (52), and the agent (200) can extract key points from the exercise video data (600) using the generative model.

[0176] More specifically, the agent (200) can analyze video data received from the camera on a frame-by-frame basis to extract key points corresponding to joint points corresponding to the user's movement. For example, the agent (200) can use various object detection algorithms. For example, the agent (200) can use an algorithm that ensembles multiple bounding boxes (Weighted Box Fusion, WBF). However, it is obvious that the agent (200) is not limited to the object detection algorithm described above and can use various object detection algorithms capable of detecting objects corresponding to the user (U) from video data. As another example, the pose estimation model (52) included in the pose estimation server (50) can identify or estimate the user's joint points from the movement video data (600) through learning on training data specialized for joint points, and extract key points corresponding thereto.

[0177] In the present invention, the training data for which the artificial intelligence pose estimation model (52) performs training may be stored in a pose estimation database (60), and such a pose estimation database (60) may also be referred to as a “training data DB.” Further details regarding the training data will be described later.

[0178] According to the present invention, the posture estimation server (50) may include at least one of a learning unit (51) and a posture estimation model (52). The posture estimation server (50) may be provided inside the exercise guide providing system (100) according to the present invention or may be an external server. That is, the posture estimation server (50) according to the present invention performs the function of learning for posture estimation in conjunction with the agent (200), and it can be understood that there are no physical spatial constraints. Detailed information regarding the posture estimation server (50) will be described later along with the learning data.

[0179] The posture estimation database (60) is a storage facility where a learning data set is stored, and may be provided within the exercise guide providing system (100) according to the present invention itself or may be an external storage facility (or external DB). It can be understood that the posture estimation database (60) according to the present invention is sufficient as long as it is a space where the learning data set is stored, and there are no restrictions on the physical space.

[0180] Meanwhile, the “exercise video data (600)” described in the present invention may include at least one of “exercise video data to be analyzed” and “exercise video data to be learned.” The “exercise video data to be analyzed” is exercise video data that is the subject of posture estimation analysis of the user (U), and the “exercise video data to be learned” can be understood as exercise video data (600) that is the subject of machine learning for a posture estimation model. Here, “posture estimation analysis” may mean extracting key points from the exercise video data.

[0181] The learning unit (51) may be configured to perform learning for a posture estimation model (52) based on the exercise video data to be learned. The learning unit (51) may train the posture estimation model (52) using the learning data. The learning unit (51) may detect a user (U) in the exercise video data to be learned and extract various learning data used for estimating exercise posture from the detected user (U). Such learning data may be used interchangeably with “information,” “data,” “data value,” or “data value.”

[0182] Meanwhile, the extraction of training data may be performed by means other than the training unit (51). The training unit (51) may use various object detection algorithms to detect the user (U) from the training target motion video data. For example, the training unit (51) may use an algorithm that ensembles multiple bounding boxes (Weighted Box Fusion, WBF). However, it is obvious that the training unit (51) is not limited to the object detection algorithm described above and may use various object detection algorithms capable of detecting an object corresponding to the user (U) from the training target motion video data.

[0183] Furthermore, the learning unit (51) can perform learning for the posture estimation model (52) based on the learning data set existing in the posture estimation database (60). As previously explained, the learning data set may include location information of joint points.

[0184] The posture estimation model (52) is a posture estimation model learned using a learning data set containing position information for joint points, and can estimate the exercise posture of the user (U) from the exercise video to be analyzed.

[0185] Meanwhile, the posture estimation model (52) can extract key points corresponding to the user's joint points from the motion video data (600) using the learning data set generated in the learning unit (51). The posture estimation model (52) can analyze the user's motion in the motion video data (600) using the extracted key points. For example, it can estimate and analyze information regarding at least one of i) the position of the joint point, ii) the range of motion of the joint point, iii) the movement path of the joint point, iv) the connection relationship between joint points, and v) the symmetry relationship of the joint point for the user (U).

[0186] Furthermore, the posture estimation model (52) can perform an analysis of at least one of the following: the range of motion of the joint, the speed of movement (or acceleration) of the joint, the user's body balance, body equilibrium, and body alignment state (e.g., leg axis alignment state, spine alignment state, etc.) included in the exercise video to be analyzed. In the present invention, the posture estimation model (52) may also be configured to include a learning unit (51). Furthermore, conversely, the learning unit (51) may include the posture estimation model (52), and in this case, the learning unit (51) can train the posture estimation model (52) to perform the posture estimation function. Accordingly, in the present invention, the function performed by the posture estimation model (52) may be described interchangeably as being performed by the learning unit (51).

[0187] The agent (200) can generate video analysis data that analyzes the user's exercise movements based on the exercise movement analysis process of the posture estimation server (50). More specifically, the agent (200) can control the camera equipped in the user terminal (10) to capture the user's (U) exercise video based on a first request to the agent to analyze the user's exercise movements included in the exercise video.

[0188] The agent (200) can receive an exercise video of a user (U) performing an exercise movement according to an exercise guide video through a camera provided in the user terminal (10).

[0189] As illustrated in (a) of FIG. 6b, the agent (200) can output a guidance message (e.g., “Please stand inside the screen”) on the user terminal (10) so that the user’s entire body is included within a specific area of ​​the exercise video (or the display of the user terminal) in order to detect the user (U) from the video data.

[0190] As explained above, the agent (200) can detect the user (U) from the image data using an object detection algorithm based on the entire user body within a specific area.

[0191] Furthermore, the agent (200) can receive exercise video data (600) of a user performing an exercise movement according to an exercise plan through a camera based on the detection of the user's entire body within a specific area.

[0192] In this case, the agent (200) can photograph the user performing the exercise while an exercise video corresponding to an exercise item assigned to the user is being played. Then, the control unit (130) can control the exercise video data captured by the camera of the user terminal (10) to be matched to an exercise plan (or each of the plurality of exercise items included in the exercise plan) and recorded in the storage unit (120) of the user terminal (10).

[0193] As illustrated in FIG. 7, the agent (200) may input a prompt (250) into a generative model (202) that includes a second request to generate a comment on an indication included in the prescription information as a result of analyzing the user's actions. Furthermore, the agent (200) may generate feedback information (720) on the user's exercise actions corresponding to the comment on the indication based on the input prompt (250).

[0194] At this time, the user page (30) may be provided with an exercise guide including at least one of information (710) about the exercise plan currently being performed, feedback information (720) about the user's exercise movements, exercise movement analysis information (731), an exercise video (750), and a graphic object (770) that supports the exercise guide. As an example, while an exercise video is being played on the user terminal (10), a video received from an activated camera is displayed in real time in a part of the user terminal, and feedback information (720) about the user's exercise movements included in the video may be displayed in at least a part of the video.

[0195] Furthermore, while the exercise video (750) is being played on the user terminal (10), the camera and microphone of the user terminal may be maintained in an active state. While the exercise video (750) is being played on the user terminal (10), the agent (200) may obtain user feedback information (740) through at least one of the camera and microphone.

[0196] The agent (200) can control the agent (200) to generate a prompt reflecting user feedback information (740) based on the acquisition of user feedback information (740) corresponding to feedback information (720) regarding user exercise movements. More specifically, the agent (200) can generate a feedback prompt to update the exercise guide using user feedback information (740) received from at least one sensor. For example, the agent (200) can generate a feedback prompt reflecting user voice feedback information (e.g., “Is this movement correct?”, 740) based on sensing from a microphone equipped in the user terminal (10). Furthermore, the agent (200) can input the feedback prompt to the agent (200) to obtain an updated exercise guide from the agent (200) and provide the updated exercise guide to the user terminal. The agent (200) can generate an updated exercise guide (760) corresponding to the feedback prompt reflecting user feedback information (720).

[0197] More specifically, the user can correct the exercise motion according to the provided user exercise analysis feedback information (720) and exercise video (750), and the agent (200) can perform an analysis of the corrected exercise motion. For example, the agent (200) can re-analyze the user's exercise motion based on the first exercise motion information (731) and the corrected second exercise motion information (732) by linking with the posture estimation server (50). The posture estimation server (50) can analyze the connection relationships of multiple joints (e.g., angles between each joint) based on joint points extracted from the user's exercise video. The posture estimation server (50) can generate an exercise motion analysis result based on the fact that the connection relationships of the first joints (731a, 731b) have been corrected to the connection relationships of the second joints (732a to 732c). The agent (200) can generate an updated exercise guide (760), such as “Wow, that motion is correct! Hold for 5 seconds!” based on the corrected exercise motion analysis results, and provide it to the service page (30) of the user terminal (10).

[0198] Meanwhile, the control unit (130) can change the visual appearance of the agent graphic object (770) so that the user intuitively recognizes the exercise guide. The agent graphic object (770) provided on the service page (30) may be displayed with different appearances based on the provided updated exercise guide (760). The agent (200) may change at least one of the appearance and animation effects of the agent graphic object (770) based on the provided updated exercise guide (760). For example, the agent (200) may change the appearance of the agent graphic object (770) to correspond to the updated exercise guide (760), such as “Wow, that’s the correct move! Hold for 5 seconds!” At this time, the appearance of the agent graphic object corresponding to the exercise guide may be stored in the storage unit (120).

[0199] Furthermore, the agent (200) may store the user's exercise results as exercise history information in the storage unit (120). For example, the agent (200) may store at least one of information about the exercise plan performed by the user (e.g., rehabilitation period, exercise items, total exercise time), user feedback information regarding the exercise plan (e.g., request for difficulty adjustment), and past exercise summary information (e.g., exercise result sheet) in the storage unit (120).

[0200] The agent (200) can use the user's exercise results to update exercise history information and transmit the exercise results to the medical staff server and the user terminal (10).

[0201] In the foregoing, the generative model-based exercise guide provision method according to the present invention has been described in detail. Below, with reference to FIGS. 8, 9a, and 9b, the user authentication and exercise plan modification process using the generative model-based exercise guide provision method will be described in detail. FIGS. 8 is a conceptual diagram illustrating the process of performing user authentication according to the present invention, and FIGS. 9a and 9b are conceptual diagrams illustrating the process of modifying an exercise plan using user feedback according to the present invention.

[0202] Meanwhile, the control unit (130) may perform user authentication for security purposes when an application is activated on a user terminal (10) that is not logged into an account. At this time, the user terminal (10) on which user authentication is performed may refer to a terminal used by an unspecified number of people in a public place, rather than a terminal (or electronic device) owned by the user.

[0203] As illustrated in FIG. 8, the control unit (130) can check whether a user account is logged in based on an application activation event. The control unit (130) can perform a user authentication process based on whether a specific user account is not logged in. At this time, each user authentication process can be performed on at least one of the exercise guide providing system (100) and an external server (200).

[0204] Meanwhile, “user authentication” can be performed in various ways. For example, user authentication may be performed by at least one process among knowledge-based authentication (e.g., password, pattern input, etc.), possession-based authentication (e.g., OTP, message authentication number, etc.), and biometric-based authentication (e.g., facial recognition, fingerprint recognition, voice recognition, etc.). Although biometric-based authentication is described as a representative method in this specification, user authentication according to the present invention is not limited to biometric-based authentication.

[0205] The control unit (130) can activate at least one of the microphone, camera, and sensor unit of the user terminal (10) based on the fact that a specific user account is not logged in. Furthermore, the control unit (130) can perform user authentication using at least one of the activated microphone, camera, and sensor unit. Here, the types of user authentication may vary, such as i) fingerprint recognition, ii) face recognition, iii) voice recognition, iv) iris recognition, v) vein recognition, vi) retina recognition, etc.

[0206] The control unit (130) can provide at least one of image data (810) including a user captured by a camera equipped in the user terminal (10) and a user authentication message (820) to the user page (30).

[0207] The control unit (130) can compare user authentication information stored in the storage unit (120) with sensing information sensed from the user terminal (10). Here, the “user authentication information” may be included in user account information and stored in the storage unit (120).

[0208] The control unit (130) can perform a user authentication process using at least one of the collected user's voice data (230) and video data (240) based on user authentication information included in the user account information (220).

[0209] The control unit (130) can log in to an authenticated user account based on the completion of the user authentication process. At this time, the control unit (130) can provide an authentication completion message (830) and an initial question (840) to the page (30) in conjunction with the agent (200). As previously described, the initial question (840) may include a greeting and questions related to the user's current pain status, the intensity of past exercises, and satisfaction with the exercise plan.

[0210] Furthermore, the agent (200) can update the exercise guide based on user feedback information input while providing the exercise guide. More specifically, while an exercise video is being played on the user terminal (10), the camera and microphone of the user terminal (10) are maintained in an active state, and while an exercise video is being played on the user terminal (10), user feedback information can be obtained through at least one of the camera and microphone.

[0211] More specifically, the agent (200) can update the exercise plan based on the user's request for an exercise plan update received through a microphone provided in the user terminal while the exercise video is being played. Here, “exercise plan update” may mean changing at least one of the exercise movements, the number of repetitions, and the total exercise time in the exercise plan (213). Below, an exercise plan update that changes the exercise movements is described as an example, but is not limited thereto, and at least one of the number of repetitions and the total exercise time may also be changed.

[0212] More specifically, the agent (200) can generate a feedback prompt to update the exercise guide using user feedback information. The agent (200) can generate the feedback prompt based on a third request requesting that the exercise guide be generated by reflecting user requests based on user voice information (or voice data).

[0213] More specifically, the agent (200) may extract text of a pre-set topic related to the generation of an exercise guide from voice analysis data and include the text of the pre-set topic in a prompt. Here, “pre-set topic” may mean at least one of adjusting the difficulty of the exercise, selecting the type of exercise, and changing the type of exercise.

[0214] For example, the agent (200) can generate a feedback prompt to change the “squat movement” to another exercise item based on receiving user voice information (or voice data) such as “I felt pain during the squat movement, please change to another exercise!” through the microphone.

[0215] Furthermore, the agent (200) can receive a feedback prompt and update the exercise guide to change the “squat” movement to a “standing leg bend” movement. The control unit (130) can obtain the updated exercise guide from the agent (200) and provide the updated exercise guide to the user terminal (10).

[0216] As another example, the agent (200) can collect user feedback information upon the end of the exercise plan. For example, the agent (200) can provide a feedback question (910) regarding the exercise plan performed to the user terminal based on the end of the exercise plan.

[0217] As illustrated in FIG. 9a, the agent (200) can provide a feedback question (910) to the user page (30). Furthermore, the agent (200) can collect and analyze user feedback information (920) by activating at least one of the microphone, camera, and sensor unit of the user terminal (10). For example, the agent (200) can analyze feedback information (e.g., “Movement 1, I felt pain while squatting!”, 920) from user voice data using the voice recognition model (41) of the voice recognition server (40).

[0218] Furthermore, the agent (200) can update the exercise plan (213) based on user feedback information (920). The control unit (130) can update the exercise guide (900) based on voice analysis data and according to the user feedback information (920).

[0219] In the present invention, updating the exercise guide can also be understood as updating the exercise plan. Hereinafter, the update of the exercise guide is described based on the update of the exercise plan, but is not limited thereto. The update of the exercise guide may refer to the updating of various information provided in relation to exercise, such as exercise motion videos, exercise methods, analysis services for exercise motions, and feedback based on analysis results, provided to the user terminal in relation to the user's indications.

[0220] As illustrated in FIG. 9b, the agent (200) can change the exercise movements according to the feedback information (920). The storage unit (120) may store a difficulty level corresponding to a specific exercise item (931, 932) that is matched. The agent (200) can change the exercise item based on the difficulty level matched to the exercise item. Here, the difficulty level corresponding to the specific exercise item may be pre-set and stored in the storage unit (120) based on an expert, a group of experts, or an artificial intelligence algorithm.

[0221] For example, the agent (200) can replace a specific exercise item with another exercise item having a lower difficulty level than the exercise intensity of the specific exercise item. For example, if the exercise intensity of the squat movement (931) is “ ”, the control unit (130) can change (or replace) the squat movement (931) with another exercise item (e.g., standing leg bending movement) having a difficulty level of “1”.

[0222] The agent (200) can update the exercise plan by changing at least one of the exercise items and difficulty levels assigned after the next day of the specific rehabilitation period (e.g., “Day 1 of Week 2”) based on user feedback information regarding exercise items assigned during the specific rehabilitation period (e.g., “Day 6 of Week 1”). The agent (200) can provide the updated exercise plan on the user terminal (10) from the day after the specific day.

[0223] The control unit (130) can input the updated exercise plan into the agent (200). The prompt generation unit (201) can generate a prompt (250) based on the updated exercise plan. Furthermore, the prompt generation unit (201) can input the generated prompt (250) into the generative model (202). The generative model (202) can generate an answer related to the updated exercise guide (900).

[0224] The agent (200) can provide an answer (940) related to the exercise plan update to the user terminal (10) and, based on user feedback, provide an updated exercise guide (900).

[0225] The method and system for providing exercise guidance based on a generative model according to the present invention can resolve physical limitations in rehabilitation treatment and improve user accessibility by providing exercise guidance remotely based on prescription information including information on the user's indications.

[0226] Furthermore, the method and system for providing exercise guidance based on a generative model according to the present invention can provide customized exercise guidance that takes into account the user's current performance ability by reflecting user feedback based on a generative model.

[0227] Furthermore, the generative model-based exercise guide provision method and system according to the present invention can improve user convenience by providing a user-customized exercise guide using automatically collected information without the need for separate document creation or data input.

[0228] Meanwhile, computer-readable media include all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SSD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0229] Furthermore, the computer-readable medium may be a server or cloud storage that includes a storage and is accessible to an electronic device via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage via wired or wireless communication.

[0230] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, namely a CPU (Central Processing Unit), and no special limitations are placed on its type.

[0231] Meanwhile, the above detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

Claims

1. Receiving sensing information from at least one sensor in response to an activation event of the application; A step of generating a prompt using user-related information of a user account logged into the application and the sensing information while the application is active; A step of inputting the prompt into an agent while the above application is active; and A generative model-based exercise guide provision method characterized by including the step of obtaining a user exercise guide corresponding to the prompt from the agent and providing the obtained user exercise guide through the application.

2. In Paragraph 1, The above-mentioned at least one sensor includes a camera, and In a user terminal where the above application is activated, the camera is activated based on the activation of the above application, and The display unit of the above user terminal displays the image received through the camera in real time, and A method for providing exercise guidance based on a generative model, characterized in that a graphic object indicating that the agent has been activated is displayed on the display unit.

3. In Paragraph 2, A method for providing exercise guidance based on a generative model, characterized in that the agent above obtains the exercise guide corresponding to the prompt by linking with the generative model.

4. In Paragraph 1, The above user-related information is, It includes at least one of user account information and prescription information, and The above user account information is, It includes at least one of the user's ID, name, date of birth, and gender information, and The above prescription information is, A method for providing an exercise guide based on a generative model, characterized by including information on indications and information on a treatment plan prescribed by a medical institution for said indications.

5. In Paragraph 4, The above exercise guide is, A generative model-based exercise guide provision method characterized by including at least one of information related to an exercise video, exercise plan, exercise schedule, exercise movement, exercise method, exercise difficulty, number of exercises, exercise time, timing of exercise, and exercise description related to the exercise that the user must perform in relation to the above indication.

6. In Paragraph 5, The above at least one sensor includes a camera provided on a user terminal in which the application is activated, and In the step of generating the above prompt, The video received through the camera is included in the prompt along with the user-related information, and The above prompt is, A method for providing exercise guidance based on a generative model, characterized by including a first request to the agent to analyze the user's actions included in the above video.

7. In Paragraph 6, The above prompt is, A method for providing an exercise guide based on a generative model, characterized by further including a second request to generate a comment on the indication included in the prescription information as a result of analyzing the action of the user.

8. In Paragraph 5, The above at least one sensor includes a microphone provided in a user terminal on which the application is activated, and In the step of generating the above prompt, Including voice information corresponding to the voice received through the above microphone in the above prompt, The above prompt is, A method for providing an exercise guide based on a generative model, characterized by further including a third request to generate the exercise guide by reflecting user requests based on the voice information.

9. In Paragraph 8, In the step of generating the above prompt, A method for providing exercise guidance based on a generative model, characterized by converting the speech into text based on a STT (Speech-to-Text) algorithm and including the converted text as speech information in the prompt.

10. In Paragraph 9, In the step of generating the above prompt, A generative model-based exercise guide provision method characterized by extracting text of a pre-set topic related to the generation of the exercise guide from the converted text and including the text of the pre-set topic in the prompt.

11. In Paragraph 10, The previously established topic above is, It relates to at least one of adjusting the difficulty of an exercise, selecting the type of exercise, and changing the type of exercise, and In the above agent, Based on the above third request, check the exercise plan information previously provided to the user account, and A method for providing an exercise guide based on a generative model, characterized by generating the exercise guide based on the confirmed exercise plan to reflect the user request.

12. In Paragraph 5, The above agent is configured to be linked with a database in which exercise videos corresponding to each of a plurality of different exercise items are stored, and The above agent is, Based on the above prompt, at least one exercise item related to the above indication is extracted from the above database, and A generative model-based exercise guide provision method characterized by generating an exercise guide that enables the user to perform exercises according to the extracted exercise items.

13. In Paragraph 12, The above agent is, Generate the exercise plan composed of at least one exercise item, and The above application is, A method for providing an exercise guide based on a generative model, characterized by controlling a user terminal to sequentially play exercise videos according to at least one exercise item according to the above exercise plan.

14. In Paragraph 12, In the above database, exercise videos corresponding to each of the above exercise items are stored linked to the identifier (ID) of the above exercise item, and The extraction of exercise items from the above agent is, Corresponds to extracting an identifier corresponding to at least one exercise item related to the above indication, and A method for providing a generative model-based exercise guide, characterized in that the application receives an identifier corresponding to the extracted exercise item from the agent and plays an exercise video corresponding to the identifier corresponding to the extracted exercise item on a user terminal.

15. In Paragraph 13, While the exercise video is being played on the user terminal, the camera and microphone of the user terminal are maintained in an active state, and A step of acquiring user feedback information through at least one of the camera and microphone while the exercise video is being played on the user terminal; A step of generating a feedback prompt to update the exercise guide using the above user feedback information; A generative model-based exercise guide provision method characterized by further including the step of inputting the above feedback prompt into the agent, obtaining an updated exercise guide from the agent, and providing the updated exercise guide to the user terminal.

16. In Paragraph 13, While the exercise video is being played on the user terminal, an image received from an activated camera is displayed in real time in one area of ​​the user terminal, and A generative model-based exercise guide provision method characterized by displaying feedback information regarding the user's exercise movements included in the video on at least a portion of the video.

17. A communication unit that receives sensing information from at least one sensor in response to an activation event of an application; and When the above application is active, the control unit generates a prompt using user-related information of a user account logged into the application and the sensing information. The above control unit is, With the above application activated, the above prompt is entered into an agent that guides the user's exercise, and A generative model-based exercise guide providing system characterized by obtaining a user exercise guide corresponding to the prompt from the agent and providing the obtained user exercise guide through the application.

18. Executed by one or more processes on an electronic device and can be read by a computer As a program stored on an existing recording medium, The above program is, A step of receiving sensing information from at least one sensor in response to an activation event of the application; A step of generating a prompt using user-related information of a user account logged into the application and the sensing information while the application is active; A step of inputting the prompt into an agent that guides user exercise while the above application is active; and A program stored on a computer-readable recording medium characterized by including instructions that perform the step of obtaining a user exercise guide corresponding to the prompt from the agent and providing the obtained user exercise guide through the application.

Images