Ai-based biofeedback content providing method and device

Abstract

A content providing device according to an embodiment may perform the operations of: acquiring medical history data including information about a user's body and disease; providing biofeedback content on the basis of a generative AI model by using a prompt in which environmental information for alleviating the disease is added to pre-stored content information; monitoring biometric information of the user while providing the content, and controlling the environmental information of the content to cause a change in the biometric information; and providing result data indicating a response of the user's biometric signal with respect to the content.

Description

Method and device for providing AI-based biofeedback content

[0001] The present invention relates to a method and device for providing AI-based biofeedback content. More specifically, the invention relates to a technology for providing mixed reality-based biofeedback content that reflects environmental information for alleviating a patient's disease into actual location information using generative AI, a technology for providing biofeedback content by adaptively controlling environmental information for alleviating a patient's disease according to changes in real-time biometric information, and / or a technology for providing biofeedback content using personalized result data that analyzes the directionality of a patient's biometric information according to the control of environmental information.

[0002]

[0003] Biofeedback technology collects and analyzes users' physiological data in real time to provide content tailored to their condition. It is actively being used to support mental health management, stress relief, rehabilitation, and personalized health management. Accordingly, biofeedback technology plays a crucial role in balancing the autonomic nervous system and promoting psychological well-being by monitoring users' physical responses and providing feedback.

[0004] Meanwhile, existing biofeedback technology has been used to collect physiological data and have experts assess the user's condition based on this data. This method has been primarily used in medical institutions or specialized treatment centers and requires specialized equipment and expert interpretation.

[0005] With the growing importance of personalized healthcare, the need for biofeedback devices that can be easily used at home is increasing. Accordingly, various commercial products are being developed. However, current biofeedback technology suffers from the following issues.

[0006] First, real-time data analysis and feedback are limited. Existing biofeedback systems require time to collect data, analyze it, and provide feedback. This makes it difficult to provide content tailored to the user's condition in real time.

[0007] Second, there's a lack of ability to comprehensively analyze diverse physiological data. Because HRV, EEG, and EMG data are often analyzed separately, it's difficult to synthesize HRV, EEG, and EMG values ​​to arrive at a comprehensive assessment.

[0008] Third, there's a lack of connection between the content users experience and their physiological data. For example, the lack of technology to adapt content to a user's medical condition or physiological signals makes it difficult to provide content that adapts to the user's real-time condition.

[0009] Therefore, this document proposes a new method of providing biofeedback content to improve the above problems.

[0010]

[0011] The purpose of the present invention is to solve the problems of existing biofeedback-based content providing devices and to propose a biofeedback system that provides more effective and personalized health management and psychological stability to users.

[0012] Specifically, the present invention seeks to solve the following problems.

[0013] First, we aim to address the challenges of real-time data analysis and feedback provision. Existing systems require time to collect, analyze, and provide feedback, making real-time responses difficult. The present invention addresses these challenges by implementing a system capable of analyzing data in real time and providing immediate feedback tailored to the user's condition. This allows for more effective biofeedback by immediately reflecting changes in the user's condition.

[0014] Furthermore, we aim to overcome the limitations of comprehensive condition assessment by introducing multimodal sensing technology capable of integrating and analyzing various physiological data. Existing technologies often analyze HRV, EEG, and EMG data separately, making comprehensive condition assessment difficult. The present invention proposes a system that comprehensively assesses user condition and provides more accurate feedback by comprehensively sensing biosignals and considering their directionality.

[0015] Furthermore, we aim to strengthen the connection between user experience and physiological data. Existing systems have limited ability to provide personalized experiences due to the lack of connection between the user's physiological state and content. The present invention dynamically adjusts content based on the user's physiological data to provide optimized content, maximizing personalized experiences. This allows users to experience customized content tailored to their individual conditions.

[0016] Therefore, the purpose of the present invention is to overcome the limitations of existing biofeedback systems by providing real-time data analysis and feedback, integrated analysis of multimodal sensing data, and strengthening the linkage between content and physiological data, and to provide a more innovative and efficient biofeedback-based content providing device.

[0017] Meanwhile, the technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below.

[0018]

[0019] A method performed by a content providing device operated by a processor according to one embodiment may include: acquiring medical history data including information on a user's body and disease; providing biofeedback content based on a generative AI model using a prompt that adds environmental information in a direction of alleviating the disease to previously stored content information; monitoring the user's bio-information while providing the content and controlling environmental information of the content so that the bio-information changes; and providing data as a result of the user's bio-signal reacting to the content.

[0020] Additionally, the environmental information may include a numerical value or range specifying one or more visual elements constituting the biofeedback content, and the numerical value or range may be set to have different numerical values ​​or ranges depending on the type of disease.

[0021] In addition, the above-mentioned pre-stored content information may include actual map information and location data visually implemented by rendering a specific location on the map information, and the operation of providing the content may include an operation of providing map information to the user to obtain a location of map information selected by the user; an operation of obtaining location data corresponding to the map information of the location; an operation of determining environmental information to correct the location data in a direction of alleviating the disease; and an operation of providing biofeedback content including location data corrected using generative AI based on a prompt that combines the environmental information with the obtained location data.

[0022] In addition, the place data may include first place data visually implemented by rendering an actual outdoor place in 2D or 3D, and the operation of providing the content may include an operation of determining first environmental information including a range of HSV (Hue, Saturation, Brightness Value) to be applied to a rendering model, a season of the place, weather, illuminance, a user's field of view, a gaze position, and a gaze movement speed as environmental information in the direction of alleviating the disease; and an operation of providing biofeedback content including first place data corrected using generative AI based on a first prompt that combines the first environmental information with the first place data.

[0023] In addition, the operation of determining the first environmental information may further include an operation of inputting a treatment image for a given disease into a classification model trained to extract a numerical value or range for one or more visual elements constituting the image when an image is input, and extracting the first environmental information including a numerical value or range for each visual element constituting the treatment image.

[0024] In addition, the above location data may include second location data visually implemented by rendering an actual building interior location in 2D or 3D, and the operation of providing the content may include an operation of determining second environmental information including an architectural style to be applied to a rendering model as environmental information in the direction of alleviating the disease; and an operation of providing biofeedback content including second location data corrected using generative AI based on a second prompt that combines the second environmental information with the second location data.

[0025] In addition, the operation of determining the second environmental information may further include an operation of inputting a treatment image for a given disease into a classification model trained to extract a numerical value or range for one or more visual elements constituting the image when an image is input, and extracting the second environmental information including a numerical value or range for each visual element constituting the treatment image.

[0026] In addition, the above-mentioned pre-stored content information may include narration data to be output in response to predetermined video information, and the operation of providing the content may include an operation of obtaining environmental information preset to correct the narration information as environmental information in the direction of alleviating the disease; and an operation of providing biofeedback content including narration data corrected using generative AI based on a prompt that combines the environmental information with the narration data.

[0027] Additionally, the environmental information may include the narrator's gender, age, voice, and speech rate.

[0028] A method performed by a content providing device operated by a processor according to one embodiment may include: acquiring medical history data including information on a user's body and disease; providing biofeedback content determined to be output based on the medical history data; monitoring the user's bioinformation while providing the content and controlling environmental information of the content so that the user's bioinformation changes in a preset direction according to the disease; and providing data as a result of the user's biosignal reacting to the content.

[0029] In addition, the bio-information includes HRV (Heart Rate Variability), EEG (Electroencephalography), and EMG (Electromyography), and the operation of controlling the environmental information of the content may include an operation of measuring the directionality of the bio-information by comparing the difference between the real-time measurement value of each bio-information with the cumulative average value of each bio-information.

[0030] In addition, the above-mentioned medical history data may include directional biometric information regarding a reference range or direction that the biometric information of the user should have according to the individual physical information of the user or the disease of the user, and the operation of controlling the environmental information of the content may include an operation of controlling the environmental information of the content so that the biometric information being monitored changes in the direction of the directional biometric information.

[0031] In addition, the above-mentioned troop data includes control information on the direction in which the user's biometric information changes as environmental information is controlled, and the operation of controlling the environmental information of the content may include an operation of controlling the environmental information of the content based on the control information so that the biometric information being monitored changes in the direction of the directional biometric information.

[0032] In addition, the control information may include information about a value or range that specifies one or more visual elements constituting biofeedback content, such that the user's HRV has a predetermined range, a value or range that specifies one or more visual elements constituting biofeedback content, such that the user's EEG has a predetermined directionality, or a value or range that specifies one or more visual elements constituting biofeedback content, such that the user's EMG has a predetermined directionality.

[0033] Additionally, information about the above values ​​or ranges may include the range of HSV (Hue, Saturation, Brightness Value) to be applied to the biofeedback content, the season of the location, weather, illuminance, the user's field of view, gaze position, and gaze movement speed.

[0034] Additionally, the action of controlling the environmental information of the content may include an action of providing recalibrated biofeedback content using generative AI based on a prompt that combines the environmental information based on the control information with the biofeedback content being provided to the user.

[0035] A method performed by a content providing device operated by a processor according to one embodiment may include: acquiring medical history data including information on a user's body and disease; providing biofeedback content determined to be output based on the medical history data; monitoring bioinformation of the user while providing the content and controlling environmental information of the content so that the bioinformation changes; and providing result data including information on a directionality of a biosignal of the user according to the environmental information based on a directionality to which the biosignal of the user reacts as a result of controlling the environmental information of the content.

[0036] Additionally, the environmental information may include information about a number or range that specifies one or more visual elements that constitute the biofeedback content.

[0037] In addition, the operation of controlling the environmental information of the content includes an operation of generating log data that records the direction in which the biometric information changes as the environmental information is controlled in a predetermined direction, and the log data can be mapped to biometric information that has changed by a predetermined value or more according to the controlled environmental information.

[0038] In addition, the operation of providing the result data may include an operation of filtering out cases in which the direction in which the biometric information changes is consistent more than a predetermined number of times by controlling the environmental information in a predetermined direction among the log data, and mapping the direction of the environmental information and the biometric information that have been consistently controlled more than a predetermined number of times and providing the result to the user.

[0039] In addition, the operation of providing the result data may include an operation of generating control information on the direction in which the user's biometric information changes according to controlling the environmental information based on the mapped environmental information and biometric information, and storing the control information in the user's medical history data.

[0040] Additionally, the operation of controlling the environmental information of the content may include an operation of controlling the environmental information of the content based on the control information so that the biometric information being monitored changes in the direction of the directional biometric information.

[0041] In addition, the control information may include information about a value or range that specifies one or more visual elements constituting biofeedback content, such that the user's HRV has a predetermined range, a value or range that specifies one or more visual elements constituting biofeedback content, such that the user's EEG has a predetermined directionality, or a value or range that specifies one or more visual elements constituting biofeedback content, such that the user's EMG has a predetermined directionality.

[0042] Additionally, information about the above values ​​or ranges may include the range of HSV (Hue, Saturation, Brightness Value) to be applied to the biofeedback content, the season of the location, weather, illuminance, the user's field of view, gaze position, and gaze movement speed.

[0043]

[0044] The present invention solves the problems of existing biofeedback-based content providing devices and implements a technology that provides users with improved personalized health management and psychological stability, thereby providing the following effects.

[0045] The present invention can provide real-time data analysis and feedback. By collecting and immediately analyzing physiological data such as HRV, EEG, and EMG in real time and providing real-time feedback to the user, the user can respond immediately to changes in their condition. This overcomes the delay issues of existing systems and maximizes the effectiveness of biofeedback by providing real-time, appropriate feedback tailored to the user's condition.

[0046] Furthermore, the present invention provides comprehensive status assessment by introducing multimodal sensing technology that integrates and analyzes various physiological data. Specifically, by comprehensively sensing biosignals and observing their directionality, customized content can be provided based on the user's condition.

[0047] Furthermore, the present invention can provide a personalized experience by strengthening the connection between user experience and physiological data. By dynamically adjusting content based on the user's physiological state to provide optimized content, users can enjoy a more immersive and personalized experience. This contributes to enhancing psychological stability and satisfaction, and can maximize the therapeutic effect of user treatment.

[0048] Therefore, the present invention overcomes the limitations of existing biofeedback systems by providing real-time data analysis and feedback, integrated multimodal sensing data analysis, and enhanced linkage between content and physiological data, thereby providing a more innovative and efficient biofeedback-based content provider. These benefits can provide users with personalized and effective health care and psychological stability, ultimately contributing to an improved quality of life.

[0049] Meanwhile, the effects of the present invention are not limited to those mentioned above, and other technical effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0050]

[0051] Figure 1 is a configuration diagram of a content providing device according to one embodiment.

[0052] Figure 2 is a flowchart illustrating steps of operations performed by a content providing device according to one embodiment.

[0053] FIG. 3 is an example diagram of an operation for providing biofeedback content with environmental information added to content information according to one embodiment.

[0054] FIG. 4 is an example diagram of an operation for extracting environmental information from a treatment image according to one embodiment.

[0055] FIG. 5 is an exemplary diagram of an operation for controlling environmental information of biofeedback content in response to a user's real-time biometric information according to one embodiment.

[0056] FIG. 6 is an example diagram of control information regarding the direction in which a user's biometric information changes according to controlling environmental information according to one embodiment.

[0057] FIG. 7 and FIG. 8 are examples of data showing the result of a user's biosignal response to biofeedback content according to one embodiment.

[0058]

[0059] The purpose, technical configuration, and resulting operational effects of the present invention will be more clearly understood through the following detailed description based on the drawings attached to the specification of the present invention. Reference will now be made to the accompanying drawings, which will further describe embodiments of the present invention.

[0060] The embodiments disclosed herein should not be construed or used to limit the scope of the present invention. Those skilled in the art will readily appreciate that the descriptions herein, including the embodiments, have a wide range of applications. Therefore, any embodiments described in the detailed description of the present invention are intended to serve as illustrative examples to better illustrate the present invention and are not intended to limit the scope of the present invention to the embodiments.

[0061] The functional blocks depicted in the drawings and described below are merely examples of possible implementations. Other implementations may utilize other functional blocks without departing from the spirit and scope of the detailed description. Furthermore, while one or more functional blocks of the present invention are depicted as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

[0062] Additionally, the expression “including certain components” is an “open” expression, simply indicating the presence of those components, and should not be construed as excluding additional components.

[0063] Furthermore, when it is said that a component is “connected” or “connected” to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

[0064] Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that various modifications, equivalents, and / or alternatives of the embodiments of the present invention are included.

[0065] The content providing device (100) according to the embodiment of this document is a device that provides biofeedback content to a user based on the user's biometric information. "Biofeedback content" may include content that collects the user's biometric information, analyzes it, and serves to improve the user's biometric information.

[0066] The content providing device (100) according to the embodiment of this document can analyze the user's bio-information and output bio-feedback content that improves the user's bio-information. To this end, the content providing device (100) can be linked with a module that measures the user's bio-signal, and can be implemented in the form of an HMD (Head Mounted Display) or a medical bio-feedback device equipped with software that is worn around the user's eyes and outputs content of visual and auditory stimulation to the user, or can be implemented as various computing devices that utilize the software.

[0067] A content providing device (100) according to an embodiment of this document may include a configuration as shown in the following Figure 1.

[0068] FIG. 1 is a configuration diagram of a content providing device (100) (hereinafter referred to as 'device (100)') according to one embodiment.

[0069] Referring to FIG. 1, a device (100) according to one embodiment may each include a memory (110), a processor (120), an input / output interface (130), and a communication interface (140).

[0070] The memory (110) can store data acquired from an external device or data generated by the device itself. The memory (110) can store commands that can cause the processor (120) to perform operations. For example, the memory (110) can store medical history data, environmental information, control information, map information, biofeedback content, classification models, etc. of the embodiments described below.

[0071] The processor (120) is a computing device that controls overall operations. The processor (120) can execute instructions stored in the memory (110). The operation of the device (100) according to the embodiment of this document can be understood as an operation performed by the processor (120).

[0072] The input / output interface (130) may include a hardware interface or software interface for inputting information or outputting content. For this purpose, the input / output interface (130) may include an input sensor module or an output display module.

[0073] The communication interface (140) enables information to be transmitted and received via a communication network. To this end, the communication interface (140) may include a wireless communication module or a wired communication module.

[0074] Meanwhile, the device (100) may further include a module for measuring the user's bio-information (e.g., HRV, EEG, EMG, etc.) or may be linked with an external device that measures the user's bio-information to obtain the user's bio-information in real time.

[0075] The device (100) may be implemented in various forms of devices capable of performing operations through a processor (120) and transmitting and receiving information through a network. For example, the device may be implemented in the form of a server, a computer device, a portable communication device, a smart phone, a portable multimedia device, a laptop, a tablet PC, an HMD device, smart glasses, a medical biofeedback device, etc., but is not limited to these examples.

[0076] FIG. 2 is a flowchart of operations performed by a device (100) according to one embodiment. The operations of the device (100) according to the embodiment of FIG. 2 can be understood as operations performed by a processor (120).

[0077] Each step disclosed in FIG. 2 is merely a preferred embodiment for achieving the purpose of the present invention, and some steps may be added or deleted as needed, and one step may be included and performed within another step. The order of each operation disclosed in FIG. 3 is merely an order arranged for ease of understanding, and this order is not limited to a chronological order, and the order may be changed and performed differently depending on the designer's choice.

[0078] Referring to FIG. 2, at step S1010, the device (100) may acquire the user's medical history. The medical history may include information about the user's physical condition and information about the user's medical conditions. For example, the device (100) may acquire the user's medical history by linking with an external database (e.g., EMR (Electronic Medical Record)) that stores the medical history data, or may directly input the medical history data from the user.

[0079] For example, medical history data may include biometric information, such as the user's individual physical information and the user's medical condition, which may indicate the standard range or directionality that the user's biometric information should have. For example, biometric information may include heart rate variability (HRV), electroencephalography (EEG), and electromyography (EMG), but the types of biometric information are not limited to these examples.

[0080] 'Targeted biometric information' refers to a numerical value or range of biometric information that a user should preferably have, based on the user's individual physical information (e.g., height, weight, obesity level, constitution, muscle mass, liver function test results, stress index, etc.) and the user's disease (e.g., depression, bulimia, anorexia, panic disorder, etc.). For example, the targeted biometric information can be set to a numerical value or range such as "For HRV: RMSSD 20-50ms, SDNN 100ms or more, LF / HF ratio 0.5-3.0, For EMG: Muscle potential 0.5-2.0 mV, Mean Frequency (MF) 70-100 Hz, For EEG: Electroencephalogram potential 20-60 μV, Mean Frequency (MF) 8-13 Hz."

[0081] For example, the troop data may include control information regarding the directionality of changes in the user's biometric information as environmental information is controlled. "Environmental information" and "control information" will be described in detail in the embodiments of steps S1020 to S1040 described below.

[0082] In step S1020, the device (100) may provide the user with biofeedback content that adds environmental information to previously stored content information based on the user's medical history data. For example, if the device (100) is implemented as an HMD device, the device (100) may output video or audio of the biofeedback content to the user through a display and speakers.

[0083] ‘Pre-saved content information’ may include certain image, video, and audio content.

[0084] "Environmental information" may include a numerical value or range that specifies one or more visual or auditory elements that constitute biofeedback content. The specified numerical value or range may vary depending on the type of disease. For example, the environmental information may include information on the range of HSV (Hue, Saturation, Brightness Value) to be applied to previously stored content information (e.g., rendering model), the season of the location, weather, illuminance, the user's viewing angle, gaze position, gaze movement speed, and architectural style (e.g., Gothic, Baroque, Modern, etc.).

[0085] FIG. 3 is an exemplary diagram illustrating an operation of providing biofeedback content with environmental information added to content information according to an embodiment of the present invention at step S1020. According to the example of FIG. 3, pre-stored content information may include actual map information and location data visually implemented by rendering a specific location on the map information.

[0086] Referring to FIG. 3, the device (100) can provide map information to the user, obtain the location of the map information selected by the user, and obtain location data corresponding to the map information of the location selected by the user.

[0087] For example, location data may include first location data visually realized by rendering an image of an actual outdoor location in 2D or 3D. For example, location data may include second location data visually realized by rendering an image of an actual indoor location in 2D or 3D. For example, the provision of actual map information and location data may be implemented through integration with the API of a map service (e.g., Google Maps, Naver Maps, etc.), and the 2D / 3D rendering of the image may be implemented using known rendering technologies.

[0088] The device (100) can determine environmental information to be used to correct location data in a direction that alleviates the user's condition. Here, the environmental information may include a numerical value or range that specifies one or more visual elements that constitute biofeedback content, and the specified numerical value or range may be set to different numerical values ​​or ranges depending on the type of disease. For example, the environmental information may include information on the range of HSV (Hue, Saturation, Brightness Value) to be applied to previously stored content information (e.g., rendering model), the season of the location, weather, illuminance, the user's field of view, gaze position, gaze movement speed, and architectural style (e.g., Gothic, Baroque, Modern, etc.).

[0089] Meanwhile, the environmental information reflected in the previously stored content information may vary depending on the user's physical or disease information. The environmental information may be applied to a previously stored numerical value or range setting, or the numerical value or range of environmental information extracted from a specific treatment image according to the embodiment of FIG. 4.

[0090] FIG. 4 illustrates an example of an operation for extracting environmental information from a therapeutic image according to an embodiment of the present invention at step S1020. The therapeutic image may include an image used to alleviate a patient's mental state or stress. Such a therapeutic image may include elements such as color, tone, HSV, and material (e.g., weather, season, etc.) that are helpful for alleviating the patient's mental state. The device (100) may extract environmental information contained in the therapeutic image and reflect the extracted environmental information from the therapeutic image in previously stored content.

[0091] Referring to FIG. 4, the device (100) may include a classification model (e.g., a neural network model based on a deep learning design) trained to extract environmental information (e.g., a numerical value or range for one or more visual elements) constituting an input image when an image is input. The device (100) may also pre-train the classification model to extract a numerical value or range of a visual element to be extracted from an image when an image is input.

[0092] The device (100) can extract environmental information from a treatment image based on this classification model.

[0093] In the case of FIG. 4, the picture on the left is a treatment image for alleviating depression, and the device (100) can input the treatment image for alleviating depression into a classification model to extract environmental information (e.g., numerical values ​​or ranges for visual elements) constituting the treatment image. For example, the environmental information of the treatment image extracted using the classification model according to the example of FIG. 4 can include numerical values ​​or ranges for visual elements such as "Hex code: #FFFF00, #0000FF, RGB: (230~255, 230~255, 0~50), HSV: (50~70, 90~100%, 90~100%), Season: Fall, Weather: Clear, Illuminance: 100,000 Lux."

[0094] Accordingly, referring back to FIG. 3, the device (100) can generate biofeedback content including corrected location data using generative AI based on an AI prompt that combines location data, which is a previously stored content, with the environmental information of FIG. 4. For example, the prompt may include a command that causes the generative AI to perform an operation to newly generate content in a form in which the environmental information of the therapeutic image is reflected in the previously stored content information.

[0095] Accordingly, the device (100) can input the AI ​​prompt into a predetermined generative AI model (e.g., AI RunwayML Gen-3, SORA, etc.) to generate biofeedback content in a form in which the environmental information is reflected in the location data.

[0096] Meanwhile, although Fig. 3 illustrates an example of pre-stored content information as video information, the pre-stored content information may further include narration data to be output in response to predetermined video information. In this case, in step S1020, the device (100) may obtain environmental information (e.g., the narrator's gender, age, voice, and speech rate) preset to correct narration information as environmental information for alleviating a specific disease, and may provide biofeedback content including corrected narration data using generative AI based on an AI prompt that combines the environmental information with the narration data.

[0097] At step S1030, the device (100) can monitor the user's biometric information and control the environmental information of the biofeedback content being output to the user based on the real-time biometric information.

[0098] According to one embodiment, in step S1030, the device (100) may provide an interface that can separately monitor a scene of biofeedback content being output to the user and the user's biometric information measured in real time accordingly. For example, the device (100) may measure the directionality of the biometric information by comparing the difference between the real-time measured value of each biometric information (e.g., HR, EEG, EMG) and the cumulative average value of each biometric information. For example, if the real-time HR is lower than the cumulative average value of the HR since the start of measuring the HR (Heart Rate), the device (100) may recognize that the user's HR is decreasing, and the device (100) may output a downward arrow image on the interface to provide information on the directionality of the HR. For example, if the real-time EEG is greater than the cumulative average of the EEG since the start of measuring the EEG potential, the device (100) can recognize that the user's EEG potential is increasing, and the device (100) can output an upward arrow image on the interface to provide information about the directionality of the EEG potential.

[0099] FIG. 5 is an exemplary diagram of an operation for controlling environmental information of biofeedback content according to a user's biometric information in step S1030 according to one embodiment.

[0100] Referring to FIG. 5, the device (100) can monitor the user's biometric information while providing content and control the environmental information of the content so that the user's biometric information changes in a preset direction according to the user's illness.

[0101] For example, the device (100) can control the environmental information of the content so that the biometric information of the user being monitored changes in the direction of the user's directional biometric information. Here, the directional biometric information refers to a numerical value or range of biometric information that the user should preferably have, depending on the user's individual physical information (e.g., height, weight, obesity, constitution, muscle mass, liver function test results, stress index, etc.) and the user's disease (e.g., depression, bulimia, anorexia, panic disorder, etc.). For example, the directional biometric information can be set to a numerical value or range such as "for HRV: RMSSD 20-50ms, SDNN 100ms or more, LF / HF ratio 0.5-3.0, for EMG: muscle potential 0.5-2.0 mV, mean frequency (MF) 70-100 Hz, for EEG: brain wave potential 20-60 μV, mean frequency (MF) 8-13 Hz."

[0102] At this time, if the device (100) wants to control environmental information so that the user's biometric information changes in the direction of the directional biometric information, information regarding the environmental information conditions under which the user's biometric information can change in the direction of the directional biometric information is required. This information is referred to as "control information" in the embodiments of this document. That is, the control information may include information regarding the environmental information conditions under which the user's biometric information can change in the direction of the directional biometric information.

[0103] FIG. 6 is an example diagram of control information regarding the direction in which a user's biometric information changes according to controlling the LA environment information according to one embodiment.

[0104] Referring to FIG. 6, the control information may include environmental information that needs to be controlled to change the user's biometric information in the direction of oriented biometric information. In the case of the control information of FIG. 6, control information indicating that H (Hue) in HSV must be increased to increase the RMSSD of the user's HRV, control information indicating that S (Saturation) in HSV must be decreased to reduce the muscle potential of the user's EMG, and control information indicating that V (Value) in HSV must be increased to reduce the average frequency of the user's EEG are stored.

[0105] In addition to these examples, the control information may include information about a value or range specifying one or more visual elements constituting the biofeedback content, such that the user's HRV has a predetermined range, a value or range specifying one or more visual elements constituting the biofeedback content, such that the user's EEG has a predetermined directionality, and a value or range specifying one or more visual elements constituting the biofeedback content, such that the user's EMG has a predetermined directionality. In this case, the value or range may include a range of HSV (Hue, Saturation, Brightness Value) to be applied to the biofeedback content, the season of the location, the weather, the illuminance, the user's field of view, the gaze position, and the gaze movement speed.

[0106] Meanwhile, control information may include elements that are generally common across individuals, while others may differ across individuals. For example, the control information that increasing "illumination" in the environmental information increases "HR" can be applied to all individuals. Conversely, the environmental information "rainy weather" in the background may increase EEG potential in some individuals, while decreasing it in others, depending on their characteristics.

[0107] According to the embodiment of this document, the control information for the directionality of biometric information according to the control of environmental information can be individually stored for each user based on the result data of step S1040, which will be described later. The specific operation for the individual storage (customization) of the control information for each user will be described later in the operation of S1040, and as described above in S1030, the purpose is to primarily explain the fact that the 'control information' is individually stored for each user, and environmental information can be controlled differently for each user, so the explanation will continue below.

[0108] Continuing with reference to FIG. 5, the device (100) can control environmental information according to control information by comparing the real-time measurement value of each biometric information to the cumulative average value of each biometric information, or by comparing the cumulative average value and real-time measurement value of each biometric information with biometric directional information, so that the user's biometric information (cumulative average value or real-time measurement value) has the directionality of the directional biometric information.

[0109] For example, if the real-time measurement value of HR (e.g., 73) is lower than the cumulative average value of HR (e.g., 80), the directionality of HR is decreasing. At this time, if the HR value (e.g., 90) according to the user's biometric information is higher than the cumulative average value of HR (e.g., 80), the directionality of HRV should have an increasing direction. In this case, the device (100) can control the environmental information in the direction of increasing the user's HRV (e.g., control to increase the brightness of content in real time) by referring to the user's individual control information (e.g., if the user increases the 'illuminance', the 'HR' increases).

[0110] To this end, the device (100) may provide biofeedback content that has been recalibrated using generative AI based on a prompt that combines environmental information based on control information with biofeedback content being provided to the user. For example, the prompt may include a command form that causes the generative AI to perform an operation to correct the current biofeedback content information into a content in which the environmental information according to the control information is reflected. Accordingly, the device (100) may input the AI ​​prompt into a predetermined generative AI model (e.g., AI RunwayML Gen-3, SORA, etc.) to provide biofeedback content that has been corrected in a form in which the environmental information according to the control information is reflected.

[0111] Meanwhile, in step S1030, the device (100) can record the direction in which the user's real-time biometric information changes as the environmental information is controlled in a predetermined direction. For example, if a control is performed to increase 'illuminance' among the environmental information and 'HR' increases in real time, the device (100) can generate log data that maps the controlled environmental information and the changed biometric information, such as [1st control: 'illuminance increase' -> 'HR increase']. At this time, the device (100) can map and store only the biometric information that has significantly changed (e.g., changed by more than a preset value) according to the controlled environmental information.

[0112] Figures 7 and 8 are examples of data showing the result of a user's biosignal response to biofeedback content at step S1040.

[0113] Referring to FIGS. 7 and 8, at step S1040, the device (100) can provide data on the result of the user's biosignal reaction to the biofeedback content.

[0114] For example, in step S1040, the device (100) can filter logs in which the direction in which the biometric information changes is consistent more than a predetermined number of times (e.g., more than 4 times) by controlling the environmental information in a predetermined direction among the log data described above, and provide the logs in the result data.

[0115] For example, if a specific user has log data such as [1st control: 'increase brightness' -> 'increase HR'], [2nd control: 'increase brightness' -> 'increase HR'], [3rd control: 'increase brightness' -> 'increase HR'], [4th control: 'increase brightness' -> 'increase HR'], then the user can be considered to show consistency in which the user's 'HR increases' every time the 'increase brightness' control is performed four or more times. In this case, since the pattern of 'increase brightness' -> 'increase HR' is meaningful control information to the user, the device (100) can provide the user with control information that maps the directionality of the environmental information and the biometric information that have been consistently controlled more than a predetermined number of times.

[0116] The control information generated in this way can be stored in the user's medical history data and can be used in step S1020 or step S1030 when the user reuses the device (100) and the embodiment of the present document is re-performed.

[0117] That is, depending on the operation of the S1040, control information can be individually stored and customized for each user, thereby enabling environmental information to be controlled differently for each user according to the user's characteristics.

[0118] According to the above-described embodiment, the present invention can provide the following effects by solving the problems of existing biofeedback-based content providing devices and implementing a technology that provides users with more improved personalized health management and psychological stability.

[0119] The present invention can provide real-time data analysis and feedback. By collecting and immediately analyzing physiological data such as HRV, EEG, and EMG in real time and providing real-time feedback to the user, the user can respond immediately to changes in their condition. This overcomes the delay issues of existing systems and maximizes the effectiveness of biofeedback by providing real-time, appropriate feedback tailored to the user's condition.

[0120] Furthermore, the present invention provides comprehensive status assessment by introducing multimodal sensing technology that integrates and analyzes various physiological data. Specifically, by comprehensively sensing biosignals and observing their directionality, customized content can be provided based on the user's condition.

[0121] Furthermore, the present invention can provide a personalized experience by strengthening the connection between user experience and physiological data. By dynamically adjusting content based on the user's physiological state to provide optimized content, users can enjoy a more immersive and personalized experience. This contributes to enhancing psychological stability and satisfaction, and can maximize the therapeutic effect of user treatment.

[0122] Therefore, the present invention overcomes the limitations of existing biofeedback systems by providing real-time data analysis and feedback, integrated multimodal sensing data analysis, and enhanced linkage between content and physiological data, thereby providing a more innovative and efficient biofeedback-based content provider. These benefits can provide users with personalized and effective health care and psychological stability, ultimately contributing to an improved quality of life.

[0123] The various embodiments and terms used in this document are not intended to limit the technical features described in this document to specific embodiments, but should be understood to encompass various modifications, equivalents, or alternatives of the embodiments. In connection with the description of the drawings, similar reference numerals may be used to refer to similar or related components. The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise.

[0124] In this document, the phrases "A or B", "at least one of A and B", "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can each include all possible combinations of the items listed together in that phrase. Terms such as "1", "2", or "first" or "second" may be used merely to distinguish the corresponding component from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a (e.g., a first) component is referred to as "coupled" or "connected" to another (e.g., a second) component, with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0125] The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0126] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., a memory) readable by a device (e.g., an electronic device). The storage medium may include random access memory (RAM), a memory buffer, a hard drive, a database, erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), read-only memory (ROM), and / or the like.

[0127] Additionally, the processor of the embodiments of the present document can call at least one command among one or more commands stored from a storage medium and execute it. This enables the device to operate to perform at least one function according to the at least one command called. These one or more commands may include code generated by a compiler or code executable by an interpreter. The processor may be a general-purpose processor, a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), and / or the like.

[0128] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0129] The methods according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated on a machine-readable storage medium, such as a manufacturer's server, an application store's server, or a server's memory.

[0130] According to various embodiments, each component (e.g., a module or a program) of the described components may include one or more entities. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In a method performed by a content providing device operated by a processor, An action to obtain medical history data containing information about the user's body and illness; An action of providing biofeedback content based on a generative AI model using a prompt that adds environmental information toward alleviating the disease to previously stored content information; An operation of monitoring the user's biometric information while providing the content and controlling the environmental information of the content so that the biometric information changes; and Including an action of providing data as a result of the user's bio-signal reaction to the above content. method.

2. In paragraph 1, The above environmental information is Contains a number or range that specifies one or more visual elements that constitute biofeedback content; The above figures or ranges are Set to have different values ​​or ranges depending on the type of disease, method.

3. In paragraph 1, The above-mentioned pre-saved content information Includes actual map information and visually implemented location data by rendering specific locations on the map information, The action of providing the above content is An action of providing map information to the user and obtaining the location of the map information selected by the user; An action of obtaining location data corresponding to map information of the above location; An action of determining environmental information to be used to correct the location data in a direction to alleviate the above disease; and An action including providing biofeedback content including corrected place data using generative AI based on a prompt combining the environmental information with the acquired place data. method.

4. In paragraph 3, The above location data is Includes first-place data that visually implements real-world outdoor locations by rendering them in 2D or 3D; The action of providing the above content is An operation for determining first environmental information including the range of HSV (Hue, Saturation, Brightness Value) to be applied to the rendering model, the season of the location, weather, illuminance, the user's angle of view, the position of the gaze, and the movement speed of the gaze as environmental information in the direction of alleviating the above-mentioned disease; and An operation including providing biofeedback content including first place data corrected using generative AI based on a first prompt that combines the first environment information with the first place data. method.

5. In paragraph 4, The operation of determining the above first environmental information is When an image is input, a classification model trained to extract a numerical value or range for one or more visual elements constituting the image further includes an operation of inputting a treatment image for a given disease and extracting the first environmental information including a numerical value or range for each visual element constituting the treatment image. method.

6. In paragraph 4, The above location data is Includes second location data that visually implements the actual interior location of a building by rendering it in 2D or 3D, The action of providing the above content is An operation of determining second environmental information including an architectural style to be applied to a rendering model as environmental information in the direction of alleviating the above disease; and An action including providing biofeedback content including second location data corrected using generative AI based on a second prompt that combines the second environment information with the second location data. method.

7. In paragraph 6, The operation of determining the above second environmental information is When an image is input, a classification model trained to extract a numerical value or range for one or more visual elements constituting the image further includes an operation of inputting a treatment image for a given disease and extracting the second environmental information including a numerical value or range for each visual element constituting the treatment image. method.

8. In paragraph 1, The above-mentioned pre-saved content information Contains narration data to be output in response to certain video information, The action of providing the above content is An operation of acquiring environmental information preset to correct the narration information as environmental information in the direction of alleviating the above disease; and An action including providing biofeedback content including corrected narration data using generative AI based on a prompt combining the environmental information with the narration data. method.

9. In paragraph 8, The above environmental information is Including the narrator's gender, age, voice, and speaking rate, method.

10. In a method performed by a content providing device operated by a processor, An action to obtain medical history data containing information about the user's body and illness; An action of providing biofeedback content determined to be output based on the above medical history data; An operation of monitoring the user's biometric information while providing the content and controlling the environmental information of the content so that the user's biometric information changes in a preset direction according to the disease; and Including an action of providing data as a result of the user's bio-signal reaction to the above content. method.

11. In paragraph 10, The above biometric information is Includes HRV (Heart Rate Variability), EEG (Electroencephalography), and EMG (Electromyography). The action to control the environmental information of the above content is Comprising an operation of measuring the directionality of the biometric information by comparing the difference between the real-time measurement value of each biometric information and the cumulative average value of each biometric information, method.

12. In paragraph 11, The above troop data Includes directional biometric information about the standard range or direction that the biometric information of the user should have according to the individual body information of the user or the disease of the user, The action to control the environmental information of the above content is Including an action of controlling the environmental information of the content so that the biometric information being monitored changes in the direction of the directional biometric information. method.

13. In paragraph 12, The above troop data Contains control information about the direction in which the user's biometric information changes as environmental information is controlled, The action to control the environmental information of the above content is An operation of controlling the environmental information of the content based on the control information so that the biometric information being monitored changes in the direction of the directional biometric information, method.

14. In paragraph 13, The above control information is A number or range specifying one or more visual elements constituting biofeedback content, wherein the number or range is such that the user's HRV has a predetermined range, a number or range specifying one or more visual elements constituting biofeedback content, wherein the number or range is such that the user's EEG has a predetermined direction, and a number or range specifying one or more visual elements constituting biofeedback content, wherein the number or range is such that the user's EMG has a predetermined direction, method.

15. In paragraph 14, Information about the above figures or ranges Including the range of HSV (Hue, Saturation, Value) to be applied to biofeedback content, the season of the location, weather, illuminance, user's field of view, gaze position and gaze movement speed. method.

16. In paragraph 15, The action to control the environmental information of the above content is An action including providing recalibrated biofeedback content using generative AI based on a prompt that combines environmental information based on the control information with the biofeedback content being provided to the user. method.

17. A method for performing a content providing device operated by a processor, An action to obtain medical history data containing information about the user's body and illness; An action of providing biofeedback content determined to be output based on the above medical history data; An operation of monitoring the user's biometric information while providing the content and controlling the environmental information of the content so that the biometric information changes; and An operation including providing result data including information on the directionality of the user's bio-signal according to the environmental information based on the directionality to which the user's bio-signal reacted by controlling the environmental information of the content, method.

18. In paragraph 17, The above environmental information is Contains information about a number or range that specifies one or more visual elements that constitute biofeedback content; method.

19. In paragraph 18, The action to control the environmental information of the above content is It includes an operation of generating log data that records the direction in which the biometric information changes as the above environmental information is controlled in a predetermined direction, The above log data is Biometric information that has changed beyond a preset value according to the controlled environmental information is mapped to each other. method.

20. In paragraph 19, The action that provides the above result data is An operation of filtering out cases in which the direction in which the biometric information changes is consistent more than a predetermined number of times by controlling the environmental information in a predetermined direction among the log data, and mapping the direction of the environmental information and biometric information that have been consistently controlled more than a predetermined number of times and providing the result to the user. method.

21. In paragraph 20, The action that provides the above result data is An operation including generating control information on the direction in which the user's biometric information changes according to controlling the environmental information based on the mapped environmental information and biometric information and storing the control information in the user's medical history data. method.

22. In paragraph 21, The action to control the environmental information of the above content is An operation of controlling the environmental information of the content based on the control information so that the biometric information being monitored changes in the direction of the directional biometric information, method.

23. In paragraph 22, The above control information is A number or range specifying one or more visual elements constituting biofeedback content, wherein the number or range is such that the user's HRV has a predetermined range, a number or range specifying one or more visual elements constituting biofeedback content, wherein the number or range is such that the user's EEG has a predetermined direction, and a number or range specifying one or more visual elements constituting biofeedback content, wherein the number or range is such that the user's EMG has a predetermined direction, method.

Images