System for providing customized health management service using ai-based image analysis

Abstract

A system for providing a customized health management service using AI-based image analysis is provided, the system comprising: an exercise facility terminal that recognizes entry and exit of a user; a user terminal that records the types and amounts of exercises performed by the user and provides results of analyzing exercise postures according to the types of exercises of the user; and a health management service provision server including a reception unit that receives entry and exit events of the user from the exercise facility terminal, an input unit that receives uploaded images input from at least one camera associated with the exercise facility terminal, a recording unit that searches for and identifies the user in the images, records the types and amounts of exercises performed by the user, and provides same to the user terminal, and a correction unit that transmits, to the user terminal, results of analyzing the exercise postures.

Description

A personalized healthcare service system using AI-based image analysis.

[0001] The present invention relates to a system for providing customized health management services using AI-based image analysis, and provides a system that automatically records the type and amount of exercise performed by a user and analyzes exercise posture to provide feedback by using AI-based image analysis.

[0002] Modern people spend more time indoors than outdoors, leading to chronic lack of exercise. This has led to a growing number of people engaging in personal training, which allows them to develop muscles indoors without equipment and involves simple movements. Maintaining proper posture during exercise is crucial, as it maximizes the effectiveness of exercise in a short period of time. While some people utilize coaching from fitness trainers to maintain proper posture, the high cost of training poses a barrier. Aside from those who receive such training, most newcomers to fitness are often unfamiliar with exercise routines and posture, making it difficult for beginners to overcome the barrier to entry. Recently, personal exercise devices such as smart bands have become popular. However, these devices only provide information on simple movements and calorie consumption for specific exercises, and lack feedback services for maintaining proper posture.

[0003] At this time, a method of performing exercise management by informing of exercise posture and exercise amount or performing motion recognition has been researched and developed. In relation to this, prior art Korean Patent Registration No. 10-0892665 (announced on April 15, 2009) and Korean Patent Publication No. 2023-0061876 (published on May 9, 2023) each disclose a configuration for detecting exercise information of an exerciser from an acceleration sensor installed in a health weight machine, analyzing the exercise state by comparing it with a reference acceleration, and outputting the analyzed result as an image and voice, and a configuration for reading the user's exercise state, analyzing exercise data, and transmitting exercise performance feedback to a user terminal.

[0004] However, in the former case, it does not correct the user's exercise posture, but only compares exercise speed using an acceleration sensor installed on the exercise equipment. In the latter case, although it is described as performing motion recognition, there is no configuration for how the exercise status is interpreted or how the privacy issues that arise here are resolved. Recently, various programs and applications that analyze and correct exercise posture using cameras have been released, but privacy issues arise when detecting and identifying users, and if this is not handled appropriately, the possibility of an individual's video being transmitted to others cannot be ruled out. Therefore, research and development of a system that can correct customized exercise posture while also considering privacy issues is required.

[0005] One embodiment of the present invention provides a system for providing customized health management services using AI-based image analysis, which tracks, detects, and identifies a user based on a unique identification code mapped to the user when the user enters a fitness facility, automatically records the type and amount of exercise when the user of the user terminal exercises, analyzes the exercise posture, and provides feedback to the user terminal in real time or after the fact, recommends various fitness products or health supplements based on the type and amount of exercise, and provides customized health management services, and attracts new customers and retains existing customers by performing consistent management based on a community and employing influencers for marketing. However, the technical problems to be achieved by the present embodiment are not limited to the technical problems described above, and other technical problems may exist.

[0006] However, the technical problems to be solved by the embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0007] As a technical means for achieving the above-described technical task, one embodiment of the present invention includes a health management service providing server including an exercise facility terminal that recognizes a user's entry and exit, a user terminal that records the type and amount of exercise performed by the user and provides a result of analyzing the exercise posture according to the type of exercise of the user, a receiving unit that receives a user's entry and exit event from the exercise facility terminal, an input unit that uploads an image input from at least one camera linked to the exercise facility terminal, a recording unit that searches and identifies a user in the image, records the type and amount of exercise performed by the user, and provides the result to the user terminal, and a correction unit that transmits a result of analyzing the exercise posture to the user terminal.

[0008] The above-described problem-solving methods are merely exemplary and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, additional embodiments may be included in the drawings and detailed description of the invention.

[0009] According to any one of the problem solving means of the present invention described above, when a user enters a sports facility, the user is tracked, detected, and identified based on a unique identification code mapped to the user, and when the user of the user terminal exercises, the type and amount of exercise are automatically recorded while the exercise posture is analyzed and feedback is provided to the user terminal in real time or after the fact, and various fitness equipment or health supplements are recommended based on the type and amount of exercise, thereby providing a customized health management service, and by performing continuous management based on a community and hiring influencers to conduct marketing, new customers can be attracted and existing customers can be retained at the same time.

[0010] However, the effects that can be obtained from the present invention are not limited to the effects described above, and other effects may exist.

[0011] FIG. 1 is a drawing for explaining a system for providing customized healthcare services using AI-based image analysis according to one embodiment of the present invention.

[0012] Figure 2 is a block diagram illustrating a health care service provision server included in the system of Figure 1.

[0013] FIG. 3 and FIG. 4 are drawings for explaining an embodiment of a customized healthcare service implemented using AI-based image analysis according to one embodiment of the present invention.

[0014] FIG. 5 is a flowchart illustrating a method for providing customized healthcare services using AI-based image analysis according to one embodiment of the present invention.

[0015] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement them. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar reference numerals have been used throughout the specification to indicate similar elements.

[0016] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element in between. Furthermore, when a part is said to "include" a component, this should be understood to mean that, unless specifically stated to the contrary, it may include other components rather than excluding them, and does not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0017] The terms "about," "substantially," and the like used throughout the specification are used in a sense of degree or in a sense close to the numerical value when manufacturing and material tolerances inherent to the meanings mentioned are presented, and are used to prevent unscrupulous infringers from unfairly exploiting disclosures that mention precise or absolute numerical values ​​to aid understanding of the present invention. The terms "step of doing" or "step of" used throughout the specification of the present invention do not mean "step for doing."

[0018] In this specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be realized by using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, the "unit" is not limited to software or hardware, and the "unit" may be configured to be on an addressable storage medium or may be configured to reproduce one or more processors. Accordingly, as an example, the "unit" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and "units" may be combined into a smaller number of components and "units," or further separated into additional components and "units." Additionally, components and '~parts' may be implemented to regenerate one or more CPUs within a device or secure multimedia card.

[0019] Some of the operations or functions described herein as being performed by a terminal, apparatus, or device may instead be performed by a server connected to the terminal, apparatus, or device. Similarly, some of the operations or functions described herein as being performed by a server may also be performed by a terminal, apparatus, or device connected to the server.

[0020] In this specification, some of the operations or functions described as terminal and mapping or matching may be interpreted to mean mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data.

[0021] The present invention will be described in detail with reference to the attached drawings below.

[0022] FIG. 1 is a diagram illustrating a customized healthcare service provision system using AI-based image analysis according to one embodiment of the present invention. Referring to FIG. 1, the customized healthcare service provision system (1) using AI-based image analysis may include at least one user terminal (100), a healthcare service provision server (300), at least one exercise facility terminal (400), and at least one camera (500). However, the customized healthcare service provision system (1) using AI-based image analysis of FIG. 1 is merely one embodiment of the present invention, and thus the present invention is not limited thereto through FIG. 1.

[0023] At this time, each component of FIG. 1 is generally connected via a network (Network, 200). For example, as illustrated in FIG. 1, at least one user terminal (100) can be connected to a health management service providing server (300) via a network (200). In addition, the health management service providing server (300) can be connected to at least one user terminal (100) and at least one exercise facility terminal (400) via the network (200). In addition, at least one exercise facility terminal (400) can be connected to the health management service providing server (300) via the network (200). In addition, at least one camera (500) can be connected to the health management service providing server (300) via the network (200).

[0024] Here, a network means a connection structure that enables information exchange between each node, such as multiple terminals and servers, and examples of such networks include a local area network (LAN), a wide area network (WAN), the Internet (WWW), a wired and wireless data communication network, a telephone network, and a wired and wireless television communication network. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), 5G NR (New Radio), 6G (6th Generation of Cellular Networks), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth networks, NFC (Near-Field Communication) networks, satellite broadcasting networks, analog broadcasting networks, and DMB (Digital Multimedia Broadcasting) networks.

[0025] In the following, the term "at least one" is defined as a term including both singular and plural, and it will be clear that even if the term "at least one" does not exist, each component can exist in the singular or plural and can mean either the singular or plural. Furthermore, whether each component is provided in the singular or plural may vary depending on the embodiment.

[0026] At least one user terminal (100) may be a user terminal that outputs the type and amount of exercise and receives feedback on exercise posture using a web page, app page, program, or application related to a customized health management service using AI-based image analysis.

[0027] Here, at least one user terminal (100) may be implemented as a computer capable of accessing a remote server or terminal via a network. Here, the computer may include, for example, a notebook, desktop, or laptop equipped with a navigation system or web browser. In this case, at least one user terminal (100) may be implemented as a terminal capable of accessing a remote server or terminal via a network. At least one user terminal (100) may include, for example, a wireless communication device that ensures portability and mobility, and may include all types of handheld-based wireless communication devices such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, smartpads, tablet PCs, etc.

[0028] The health management service providing server (300) may be a server that provides a customized health management service web page, app page, program, or application using AI-based image analysis. In addition, the health management service providing server (300) may be a server that registers an exercise facility terminal (400) and a camera (500), and registers and stores a unique identification code that identifies the user of the user terminal (100). In addition, the health management service providing server (300) may be a server that, when a user is recognized by the exercise facility terminal (400), identifies the type of exercise performed by the user through the unique identification code, checks the amount of exercise using the camera (500), analyzes the exercise posture, and transmits the result to the user terminal (100). Here, the health management service providing server (300) may not be a server connected to the camera (500) via the network (200), but may be installed on any device or terminal as a program, in which case it may be directly connected to the camera (500).

[0029] Here, the healthcare service provision server (300) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a notebook computer, desktop computer, or laptop computer equipped with a navigation system or web browser.

[0030] At least one exercise facility terminal (400) may be a terminal of an exercise facility that utilizes a web page, app page, program, or application related to a personalized health management service utilizing AI-based image analysis. The exercise facility terminal (400) may be a terminal that recognizes a user's entry and exit and stores and manages the user's unique identification code.

[0031] Here, at least one exercise facility terminal (400) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a notebook, desktop, or laptop equipped with a navigation system or web browser. In this case, at least one exercise facility terminal (400) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one exercise facility terminal (400) may include, for example, all kinds of handheld-based wireless communication devices such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, smartpads, tablet PCs, etc., as wireless communication devices that ensure portability and mobility.

[0032] At least one camera (500) may be a device that transmits images captured using or without using a web page, app page, program, or application related to a customized healthcare service utilizing AI-based image analysis to a healthcare service providing server (300). In this case, if the camera (500) is equipped with an intelligent CCTV, the camera (500) may be a device that not only captures images but also identifies and detects objects within the images.

[0033] Here, at least one camera (500) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a notebook, desktop, or laptop equipped with a navigation system or web browser. In this case, at least one camera (500) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one camera (500) may include, for example, all kinds of handheld-based wireless communication devices such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminals, smartphones, smartpads, tablet PCs, etc., as wireless communication devices that ensure portability and mobility.

[0034] FIG. 2 is a block diagram for explaining a health care service provision server included in the system of FIG. 1, and FIGS. 3 and 4 are diagrams for explaining an embodiment in which a customized health care service using AI-based image analysis according to one embodiment of the present invention is implemented.

[0035] Referring to FIG. 2, the health care service provision server (300) may include a receiving unit (310), an input unit (320), a recording unit (330), a correction unit (340), an instrument identification unit (350), an analysis unit (360), a customization unit (370), a personal information protection unit (380), and a profit model unit (390).

[0036] When a health management service providing server (300) according to one embodiment of the present invention or another server (not shown) operating in conjunction with it transmits a customized health management service application, program, app page, web page, etc. using AI-based image analysis to at least one user terminal (100), at least one exercise facility terminal (400), and at least one camera (500), the at least one user terminal (100), the at least one exercise facility terminal (400), and the at least one camera (500) may install or open a customized health management service application, program, app page, web page, etc. using AI-based image analysis. In addition, the service program may be driven on at least one user terminal (100), at least one exercise facility terminal (400), and at least one camera (500) using a script executed in a web browser. Here, a web browser is a program that allows the use of web (WWW: World Wide Web) services, and refers to a program that receives and displays hypertext written in HTML (Hyper Text Mark-up Language), and includes, for example, Chrome, Microsoft Edge, Safari, Firefox, Whale, and UC Browser. In addition, an application refers to an application program on a terminal, and includes, for example, an app running on a mobile terminal (smartphone).

[0037] Referring to FIG. 2, the receiver (310) can receive a user entry / exit event from the exercise facility terminal (400). The exercise facility terminal (400) can recognize the user's entry / exit. The user's entry / exit can be checked by utilizing a recently available exercise facility attendance check application, or by various other methods, such as entering the last four digits of the user's phone number or recognizing a QR code.

[0038] The input unit (320) can upload an image input from at least one camera (500) linked to an exercise facility terminal (400). At this time, after identifying the user in the image input from the camera (500), the user's face can be made into a fake face using a GAN (Generative Adversarial Network) algorithm or a fake face can be made by synthesizing the eyes, nose, and mouth of a celebrity so that the user's face is not directly included in the image. At this time, if there is a wearable device (not shown) linked to the user terminal (100), such as a smartwatch or fitness band, the input unit (320) can further collect various biometric data and exercise data therefrom. When collecting biometric data, for example, biometric data such as heart rate, blood pressure, and oxygen saturation can be collected to adjust the exercise intensity and provide the user with an optimal exercise environment.

[0039] The recording unit (330) can search and identify the user in the video, record the type and amount of exercise performed by the user, and provide the information to the user terminal (100). At this time, the amount of exercise can include the time or number of times the exercise was performed, and if the exercise equipment is a weight lifting equipment, it can include the weight or number of times lifted, etc. The user terminal (100) can record the type and amount of exercise performed by the user. It can record the number of repetitions with what weight the exercise equipment or exercise posture was used. At this time, the weight or number of repetitions can be read in the number in the video captured by the camera (500), or the weight can be detected using a weight sensor detected by each exercise equipment. For the number of repetitions, the number of rounds the exercise equipment has made can be counted, or in the case of a treadmill, the number written on the treadmill can be read using OCR or received directly through data communication. To this end, each exercise equipment can be linked to an IoT device, and for example, data can be transmitted from each exercise equipment through a dongle. Of course, it is self-evident that it can be processed using only video by reading the numbers on the treadmill without using communication such as a dongle.

[0040] The correction unit (340) can transmit the results of analyzing the user's exercise posture to the user terminal (100). The user terminal (100) can provide the results of analyzing the user's exercise posture according to the type of exercise performed by the user. The correction unit (340) can provide basic data for generating a personalized exercise program using the results of analyzing the user's exercise posture. In one embodiment of the present invention, a personalized exercise program can also be provided. An artificial intelligence algorithm can automatically generate and adjust a personalized exercise program based on user data. The exercise plan can be modified in real time according to the user's exercise progress, goals, and physical fitness level. In addition, the user's exercise data can be analyzed to provide advice for injury prevention, health status prediction, and long-term health management. Predictive analysis and health monitoring functions for this purpose can be further included.

[0041] The correction unit (340) can utilize AR technology to allow users to check their exercise posture in real time and receive visual guidance on areas for improvement. In addition to receiving corrections, it can also be implemented to allow users to receive fitness training through AR or VR. In one embodiment of the present invention, VR or AR technology can be used to provide an environment where users can exercise with a virtual trainer, thereby providing a more immersive exercise experience for users.

[0042] The equipment identification unit (350) can record the type, frequency, and weight of exercise equipment used by the user as the type and amount of exercise. The equipment identification unit (350) can perform an AI-based prediction and customized notification system, i.e., predictive maintenance. Based on the usage data of the exercise equipment, it can predict failures or problems and automatically adjust the maintenance schedule. It can also analyze the user's exercise patterns to provide customized notifications and recommendations. For example, it can send a notification encouraging exercise when the user has not exercised for a certain period of time.

[0043] The analysis unit (360) may create a database of data on basic exercise postures corresponding to at least one type of exercise, and may perform analysis based on the basic exercise posture when analyzing the user's exercise posture. While the platform according to one embodiment of the present invention may create a database of data on basic exercise postures, it may also utilize exercise postures such as those shown in FIGS. 3b to 3e, which are currently provided by the AI ​​hub.

[0044] The analysis unit (360) can detect and analyze the exercise posture based on an algorithm that detects and analyzes the user's skeleton and joints, for example, OpenPose. In one embodiment of the present invention, the exercise posture can be detected and analyzed using the above-described OpenPose-based deep learning models, DNN and CNN. Of course, OpenPose is only one embodiment and is not limited thereto, and it will be understood that various algorithms can be used. The Chat GPT, which is an LLM model, can be used to capture the user's movements in real time through a camera (500) and perform preprocessing, and then provide information to a trainer about how similar the posture is to the basic exercise posture and how to move which part to make it similar.

[0045] OpenPose, developed by the Perceptual Computing Lab at CMU (Camegie Mellon University), is designed to capture human motion. Based on the deep learning frameworks Caffe and OpenCV, it can capture faces, body parts, and fingers in real time, and can detect single and multiple models. The detected data is used to create a learning model using TensorFlow, a machine learning library developed by Google. The machine learning algorithms utilize deep neural networks (DNNs) and convolutional neural networks (CNNs). OpenPose estimates human body parts, marks each point, and outputs a skeleton-like output connecting those points. It is currently being applied to various fields (facial expressions, body movements and hand gestures, and body and hand movements). Based on the deep learning CNN algorithm, OpenPose uses images or videos as input to detect object position and orientation. The OpenPose network architecture utilizes VGGNet to output features with enhanced quality. VGGNet is a model within the CNN architecture, with 19 layers and an error rate of 7.3%. And it increases the depth of the network through small 3X3 filters, and it has the characteristic that the number of channels increases as the layer gets deeper. OpenPose uses only the first 10 layers of the VGG-19 network, and the output with emphasized features is re-inputted as the input of the stage. At this time, as the stage is repeated, the confidence map and affinity field values ​​are calculated. The confidence map is used to identify the location of a person's joints in the image, and the affinity field is used to identify the owner of the joint extracted from the image.After calculating the Confidence Map and Affinity Field, we combine them to extract each point and match it to find out who the point belongs to.

[0046] Deep Neural Networks (DNNs) are a type of machine learning algorithm that allows computers to form artificial neural networks similar to the human brain. DNNs address several issues with existing deep learning algorithms, such as difficulty finding optimal parameter values ​​during the learning process, overfitting, and slow learning times. Furthermore, they improve learning outcomes by increasing the number of hidden layers. DNNs typically have three or more hidden layers, located between the input and output layers. CNNs, one of the most widely used deep learning algorithms, automatically learn to adapt each element of a filter, expressed as a matrix, to the data processing requirements. Because DNNs typically use one-dimensional data, two-dimensional data (images) must be converted to one-dimensional data. This process results in the loss of spatial / topological information, and because the data is processed directly without abstraction, learning time and efficiency are reduced. CNNs were developed to address these issues. The convolution layer extracts features, and the pooling layer is a sub-sampling process to reduce the number of features due to the large number of pixels. Finally, the feed-forward layer uses the features from the convolution and pooling layers to classify the features. Because CNNs directly learn to find patterns and classify features, they require no manual intervention and have the advantage of producing high-quality recognition results.

[0047] The customization unit (370) can input and record the user's joint range of motion of the user terminal (100), and calibrate the basic exercise posture to correspond to the joint range of motion. Each individual inevitably has a different joint range of motion depending on age, gender, physique, weight, skeleton, body type, etc. Even if the physique or body type is the same, if flexibility is low, the joint range of motion is bound to be narrower than in other cases, and if the physique is large or heavy, the joint range of motion is bound to be narrower than in other cases. In addition, in the case of a patient with a lumbar disc, the movement of lifting the leg may stimulate the lumbar disc more or may be difficult, and in the case of a user with hunched shoulders, the shoulder range of motion is bound to be narrower than in other cases. In this way, since the range of motion that each individual can do may be different, the angle of the basic exercise posture can be adjusted to suit each individual. For example, if you have a bad back when doing squats and the angle between your upper body and legs is A, you can adjust the angle that was originally B to A.

[0048] The method for measuring the range of joint motion can be performed by automatically extracting the movement and angle of each human body part from the image captured by the user terminal (100), or by having the user move in front of a protractor and then extracting the degree of the protractor pointed by each human body part from the image using OCR to measure the range of joint motion, or by receiving data output from a device for measuring the range of joint motion or capturing the angle of the device and reading it using OCR. The following deep learning model can be used to identify the angle at which a human body part moves in the image. Of course, the method for measuring the range of joint motion as described above is not limited to the above-described method.

[0049] <mediapipe>

[0050] To extract skeletal coordinates, you can use MediaPipe, provided by Google. MediaPipe allows you to easily use various vision AI functions using video data in a pipeline format. It is a framework that provides various functions and models for human body recognition (Detect), such as Face Detection, Pose, Object Detection, and Motion Tracking. Among them, the Pose Estimation library identifies key joint points of the human body and connects them to estimate human motion. Leveraging these estimation results can play a key role in providing movement-related services. Using the MediaPipe framework to analyze posture enables MediaPipe-based action recognition for embedded environments.

[0051] <Human Pose Estimation>

[0052] For human pose estimation, features are first extracted from an image using a VGG network. Using the extracted F (feature) as input, part affinity fields (PAFs) are extracted through a convolution operation. The extracted information is organized into vectors, which are in the form of heatmaps. These PAFs are added to the original F to create a confidence map. The desired final joint coordinates can be obtained through confidence map matching, which become keypoints. The number of keypoint coordinates can vary depending on the trained model, so the location and quantity of keypoints to be tracked can be varied or increased or decreased to suit each measurement method. Joint angles can be obtained using keypoints, and GPUs can be used to track each center point and detect features.

[0053] Alternatively, you can use a PoseNet-based Teachable Machine. Using the Teachable Machine (TensorfLow PoseNet) library, you can train a model by providing positional information for 17 body parts through a camera alone, without any special sensor devices. Since PoseNet tracks only 17 human joints, it has the advantage of significantly reducing computational effort compared to full-size images. Using PoseNet, you can extract human joint data from input images, and by using the pose model learned through Teachable Machine, you can determine the user's range of motion.

[0054] The personal information protection unit (380) may identify users of a user terminal (100) based on at least one unique identification code when detecting and identifying them from images of at least one camera (500). At this time, users may be distinguished by equipping each piece of exercise equipment with a reader or scanner capable of scanning QR codes, NFC, or RFID tags. Alternatively, after recognizing the user's face, the face may be replaced with a unique identification code such as a QR code within the image, or the face may be synthesized with a fake face to detect, identify, and track the fake face. In this case, since the fake face or unique identification code remains within the image, the face of each user cannot be identified, and even if this image is leaked, there will be no privacy protection issues. However, since the fake face or unique identification code is linked to the real face or user information, the mapping table itself may be encrypted to prevent it from being linked to the user's personal information.

[0055] The Personal Information Protection Department (380) can perform data encryption and anonymization, strengthening personal information protection through the encryption and anonymization of user data. It can also manage user data permissions, providing users with the ability to directly manage how their data is used and shared.

[0056] The revenue model unit (390) can recommend products, including health supplements and fitness equipment, based on the type and amount of exercise performed by the user and provide them to the user terminal (100). At this time, in order to obtain data on nutritional supplements currently sold on the market in Korea, the nutritional supplement data can be extracted using the Health Functional Food Product Manufacturing Report (Raw Materials) OPEN-API from a public data utilization site. Products that are not provided in public data or that have the potential to cause serious allergies can be excluded to improve data quality. The basic algorithm of the recommendation system can be implemented based on a content-based recommendation system, and operates based on the selected nutritional supplement data. It can recommend useful nutritional supplements to the user by combining the characteristics of each existing nutritional supplement recommendation site. Additionally, a full view page and hashtag functions can be provided so that the user can conveniently search for nutritional supplements and check summarized information.

[0057] In order to recommend nutritional supplements, the user's condition can be first identified. For example, a questionnaire can be designed so that the user can select items in the following order: blood vessels / blood circulation, digestion / stomach / liver / intestines, fatigue, eyes, bones and joints, and immunity, regarding areas of discomfort or concern. The survey can be designed so that the user selects at least three items for each area. After selecting each questionnaire item, the user's nutritional needs are confirmed as a result, and the platform of the present invention recommends nutritional supplements appropriate for the user's needs, allowing the user to select the necessary nutritional supplements on their own.

[0058] Meanwhile, based on nutritional supplement data obtained from public data, i.e. OPEN-API, only the nutrient content of each supplement can be extracted, which can be set as a table used in a recommendation system. The recommendation system can identify deficient nutrients through a user-completed survey, assign a high importance to these nutrients, and recommend supplements with high nutrient content. Since each nutrient has different units and distributions, standardization can be applied during the preprocessing process. At this time, the standardization method used can be the MinmaxScaler provided by Sklearn. Here, the maximum nutrient content can be adjusted so that it does not exceed the daily intake standard.

[0059] The revenue model (390) can provide comprehensive wellness management by integrating comprehensive health data, including diet, sleep, and stress management, in addition to exercise data. Furthermore, medical MyData can be linked to facilitate regular health checkups and consultations. Furthermore, the revenue model (390) can incorporate social and community features. Through social feedback and challenges, users can share each other's exercise data, complete challenges, and receive social feedback. Furthermore, through online training sessions, professional trainers can provide real-time online training sessions, enabling interaction with users. This can help retain existing customers and attract new ones.

[0060] In one embodiment of the present invention, AI and machine learning technologies are advanced to create and adjust personalized exercise programs based on the user's progress and goals. Furthermore, integration with various wearable devices, including biometric data, allows for more accurate data collection and analysis. Furthermore, VR and AR technologies can be utilized to provide users with an immersive exercise experience and real-time posture correction, while community functions can be enhanced through social feedback and online training sessions. A comprehensive wellness management platform can be provided, integrating diet, sleep, and stress management beyond exercise, while privacy protection can be enhanced through the encryption and anonymization of user data.

[0061] Hereinafter, the operation process according to the configuration of the health care service provision server of FIG. 2 described above will be described in detail with reference to FIGS. 3 and 4 as examples. However, it will be apparent that the embodiment is merely one of various embodiments of the present invention and is not limited thereto.

[0062] Referring to Fig. 3a, (a) in each exercise facility, a camera (500) can be stored so as to be mapped to the exercise facility terminal (400), and a unique identification code for identifying users who visit the exercise facility can be mapped and stored. In addition, as in (b), after identifying a user through a video, the health management service provision server (300) checks the type and amount of exercise performed by the user, and as in (c), posture can be corrected, and a revenue model such as (d) can be used to retain current members and attract new members. Figs. 3b to 3e are datasets that can be used to identify exercise posture. A platform according to an embodiment of the present invention such as Fig. 4a can utilize the technology of Fig. 4d based on a strategy such as Fig. 4c to solve a problem such as Fig. 4b. In addition, an analysis method such as Fig. 4e can be used, privacy protection and data security such as Fig. 4f can be maintained, a revenue model such as Fig. 4g can be provided, and a reward system such as Fig. 4h can be provided.

[0063] Matters not described in the method for providing customized healthcare services using AI-based image analysis of FIGS. 2 to 4 are the same as or can be easily inferred from the contents described in the method for providing customized healthcare services using AI-based image analysis of FIG. 1 above, and therefore, description thereof will be omitted below.

[0064] FIG. 5 is a diagram illustrating a process of transmitting and receiving data between each component included in the personalized healthcare service provision system using AI-based image analysis of FIG. 1 according to one embodiment of the present invention. Hereinafter, an example of the process of transmitting and receiving data between each component will be described through FIG. 5, but the present invention is not limited to this embodiment, and it will be apparent to those skilled in the art that the process of transmitting and receiving data illustrated in FIG. 5 may be modified according to various embodiments described above.

[0065] Referring to FIG. 5, the health management service provision server receives a user entry / exit event from an exercise facility terminal (S5100) and uploads an image input from at least one camera linked to the exercise facility terminal (S5200).

[0066] In addition, the health management service providing server searches and identifies the user in the video, records the type and amount of exercise performed by the user, and provides the information to the user terminal (S5300), and transmits the results of analyzing the exercise posture to the user terminal (S5400).

[0067] The order of the above-described steps (S5100 to S5400) is merely an example and is not limited thereto. That is, the order of the above-described steps (S5100 to S5400) may be mutually changed, and some of the steps may be executed simultaneously or deleted.

[0068] Matters not described in the method for providing customized healthcare services using AI-based image analysis of FIG. 5 are the same as or can be easily inferred from the contents described in the method for providing customized healthcare services using AI-based image analysis through FIGS. 1 to 4, and thus, description thereof will be omitted below.

[0069] The method for providing a personalized healthcare service using AI-based image analysis according to one embodiment described through FIG. 5 may also be implemented in the form of a recording medium containing computer-executable instructions, such as an application or program module executed by a computer. The computer-readable medium may be any available medium that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include all computer storage media. The computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented by any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0070] The method for providing a customized healthcare service using AI-based image analysis according to an embodiment of the present invention described above can be executed by an application that is installed by default on the terminal (which may include a program included in a platform or operating system installed by default on the terminal), or by an application (i.e., a program) that the user directly installs on the master terminal through an application providing server such as an application store server, an application, or a web server related to the service. In this sense, the method for providing a customized healthcare service using AI-based image analysis according to an embodiment of the present invention described above can be implemented as an application (i.e., a program) that is installed by default on the terminal or directly installed by the user, and can be recorded on a computer-readable recording medium such as the terminal.

[0071] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0072] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.< / mediapipe>

Claims

1. A sports facility terminal that recognizes the user's entry and exit; A user terminal that records the type and amount of exercise performed by the user and provides the results of analyzing the user's exercise posture according to the type of exercise performed by the user; and A health management service providing server including a receiving unit that receives an entry / exit event of the user from the exercise facility terminal, an input unit that uploads an image input from at least one camera linked to the exercise facility terminal, a recording unit that searches and identifies the user in the image, records the type and amount of exercise performed by the user, and provides the recording to the user terminal, and a correction unit that transmits the results of analyzing the exercise posture to the user terminal; A system for providing customized healthcare services using AI-based image analysis, including .

2. In paragraph 1, The above health care service provision server is, When the user uses exercise equipment, an equipment identification unit records the type, number of times, and weight of the exercise equipment as the type and amount of exercise; A system for providing customized healthcare services using AI-based image analysis, characterized by further including:

3. In paragraph 1, The above health care service provision server is, An analysis unit that databases data on basic exercise postures corresponding to at least one type of exercise, and performs analysis based on the basic exercise posture when analyzing the user's exercise posture; A system for providing customized healthcare services using AI-based image analysis, characterized by further including:

4. In paragraph 3, The above analysis unit, A customized healthcare service provision system using AI-based image analysis, characterized in that the exercise posture is detected and analyzed based on an algorithm that detects and analyzes the user's skeleton and joints.

5. In paragraph 2, The above health care service provision server is, A custom unit that receives and records the user's joint motion range of the user terminal and calibrates the basic exercise posture to correspond to the joint motion range; A system for providing customized healthcare services using AI-based image analysis, characterized by further including:

6. In paragraph 1, The above health care service provision server is, A personal information protection unit that detects and identifies a user of the user terminal from an image of at least one camera, and identifies the user based on at least one unique identification code; A system for providing customized healthcare services using AI-based image analysis, characterized by further including:

7. In paragraph 1, The above health care service provision server is, A revenue model section that recommends and provides products, including health supplements and fitness equipment, to the user terminal based on the type and amount of exercise performed by the user; A system for providing customized healthcare services using AI-based image analysis, characterized by further including:

Images