Augmented reality game system for assisting users with health requirements

Abstract

An augmented reality gaming system for assisting a user having a health demand. The system comprises: a gateway for receiving an input associated with a preset event or an action performed by a user; the participation degree evaluation module is used for classifying and / or evaluating the preset event or the action executed by the user based on the action-health relationship; and a user interaction module for interacting with the user by providing recommendations associated with the health status of the user in response to a preset event or action performed by the user.

Description

Technical Field

[0001] This invention relates to an augmented reality (AR) gaming system for assisting users with health needs. In particular, but not exclusively, this invention relates to an AR gaming system with elderly care functions. Background Technology

[0002] Residents of nursing homes typically participate in a variety of activities to keep themselves busy and maintain their physical and mental well-being. Many nursing home residents enjoy spending time with peers, participating in group activities, and engaging in social events organized by the nursing home, such as volunteering or community service.

[0003] However, some elderly people may be introverted and prefer activities such as reading, watching TV, or playing games on tablets. Meanwhile, some residents may be more active and hope to maintain a healthy lifestyle through physical exercise. Summary of the Invention

[0004] According to a first aspect of the present invention, an AR gaming system for assisting users with health needs is provided. The system includes: a gateway for receiving input associated with a preset event or an action performed by a user; an engagement assessment module for classifying and / or assessing the preset event or the action performed by the user based on an action-health relationship; and a user interaction module for interacting with the user in response to the preset event or action performed by the user by providing recommendations associated with the user's health status.

[0005] According to a first aspect of the invention, the preset event includes a medication plan, and wherein the user interaction module is configured to provide the recommendation to the user, the recommendation including a reminder message that displays medication information to the user.

[0006] According to a first aspect of the invention, the action performed by the user includes physical exercise or recreational activities performed by the user.

[0007] According to a first aspect of the invention, the recommendation includes real-time guidance for improving the movement or posture of a user performing the physical exercise activity.

[0008] According to a first aspect of the invention, the system further includes: a user identification module for capturing the user's movement or posture and providing it as input to the gateway.

[0009] According to a first aspect of the invention, the user identification module is used to identify the user by performing at least one of pose estimation, hand and gesture detection, and face detection.

[0010] According to a first aspect of the invention, the engagement assessment module is used to analyze the user's movements or postures captured by the user identification module.

[0011] According to a first aspect of the invention, the engagement assessment module includes an AI-based assessment engine for analyzing the user's movements or postures and providing the user with real-time guidance to improve the user's movements or postures.

[0012] According to a first aspect of the invention, the system further includes: a social connection module for supporting multi-user participation, so that multiple users can simultaneously perform the physical exercise activity or the entertainment activity.

[0013] According to a first aspect of the invention, the system further includes: an interactive education module for providing the user with educational content having interactive features. Attached Figure Description

[0014] Embodiments of the present invention will be described below by way of example with reference to the accompanying drawings.

[0015] Figure 1 This is a schematic diagram of a computer server according to an embodiment of the present invention, which is implemented as an AR game system to assist users with health needs.

[0016] Figure 2 This is a block diagram of an AR gaming system for assisting users with health needs, according to an embodiment of the present invention.

[0017] Figure 3 To show in Figure 2 The screenshot illustrates the integrated platform implemented on the AR gaming system.

[0018] Figure 4A To show as Figure 3 A screenshot of the welcome page for the "CimdaCare" senior care app, which is part of the integrated platform.

[0019] Figure 4B To show Figure 4A A screenshot of the menu page of the "CimdaCare" app.

[0020] Figure 4C This is to show Figure 4A A screenshot of the "Workout" feature in the "CimdaCare" app.

[0021] Figure 4D To show Figure 4A A screenshot of the "Karaoke" feature in the "CimdaCare" app.

[0022] Figure 5A The image shows an example of an AR game system in operation with a "workout" function that is being launched, and the system analyzes the user's posture as the user performs physical exercises guided by the system and displayed on the system's screen.

[0023] Figure 5B To show Figure 5A Images of optional exemplary operations of an AR gaming system.

[0024] Figure 6A A screenshot is shown to illustrate a user interacting with the system using an application in the "Environment" category according to an embodiment of the present invention.

[0025] Figure 6B A screenshot is shown of a user interacting with the system using an application in the "Music" category according to an embodiment of the present invention.

[0026] Figure 6C A screenshot is shown illustrating a user interacting with the system using an application in the "Rehabilitation" category according to an embodiment of the present invention.

[0027] Figure 6D A screenshot is shown illustrating a user interacting with the system using an application in the "Multiplayer" category according to an embodiment of the present invention.

[0028] Figure 7 An image illustrating an example operation of an AR game system according to an embodiment of the present invention, wherein, Figure 6C Applications in the "Rehabilitation" category are launched by users who interact with the system. Detailed Implementation

[0029] refer to Figure 1 An embodiment of the present invention is illustrated. This embodiment provides an AR game system for assisting users with health needs. The system includes: a gateway for receiving input associated with a preset event or an action performed by a user; an engagement assessment module for classifying and / or assessing the preset event or the action performed by the user based on the action-health relationship; and a user interaction module for interacting with the user in response to the preset event or action performed by the user by providing recommendations associated with the user's health status.

[0030] In this exemplary embodiment, the interface and processor are implemented by a computer with a suitable user interface. The computer can be implemented by any computing architecture, including: a portable computer, tablet computer, personal computer (PC), smart device, Internet of Things (IoT) device, edge computing device, client / server architecture, "dumb" terminal / host architecture, cloud computing-based architecture, or any other suitable architecture. The computing device can be appropriately programmed to implement the present invention.

[0031] This system can interact with users, for example, by responding to user input or providing feedback after a user performs an action, or by requesting a response when certain conditions are met. The system is particularly helpful in providing users with health-related messages or suggestions to improve their health or physical condition. For instance, a user can launch a fitness app designed for seniors. In this app, in addition to displaying video content and instructions to help users exercise, the system can provide real-time feedback or suggestions to help users improve their posture or movements. Furthermore, the system can remind users to take breaks at specific time intervals.

[0032] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The term “and / or” as used herein includes any and all combinations of all associated listed items. As used herein, the singular forms “a,” “an,” “the,” and “described” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having the same meaning as they have in the context of the relevant technology and the invention, and will not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0034] like Figure 1The diagram illustrates a computer system or server labeled 100. This diagram represents an exemplary embodiment of a processor within the server that processes input, namely preset events or activities performed by a user, to determine how the system should interact with the user, for example by providing recommendations associated with the user's health status. In this embodiment, the system includes server 100. Server 100 includes appropriate components for receiving, storing, and executing appropriate computer instructions. These components may include: a processing unit 102, which includes a central processing unit (CPU), a math co-processing unit (Math Processor), a graphics processing unit (GPU), or a tensor processing unit (TPU) for tensor or multidimensional array computation or manipulation operations; read-only memory (ROM) 104; random-access memory (RAM) 106; input / output (I / O) devices (e.g., disk drive 108); input devices 110 (e.g., Ethernet port, USB port, etc.); a display 112, such as a liquid crystal display, a light-emitting display, or any other suitable display; and a communication link 114. Server 100 may include instructions that can be stored in ROM 104, RAM 106, or disk drive 108 and executed by processing unit 102. Multiple communication links 114 can be provided, which can be connected differently to one or more computing devices, such as servers, personal computers, terminals, wireless or handheld computing devices, IoT devices, smart devices, edge computing devices, and cloud devices. At least one of the multiple communication links can be connected to an external computing network via a telephone line or other types of communication link.

[0035] Server 100 may include storage devices such as disk drives 108, which may include solid-state drives, hard disk drives, optical drives, tape drives, or remote or cloud-based storage devices. Server 100 may use a single disk drive or multiple disk drives, or remote storage services 120. Server 100 may also have a suitable operating system 116 residing on the disk drives or in the ROM of server 100.

[0036] Computers or computing devices can also provide the necessary computing power to operate or interface with machine learning networks of neural networks to provide various functions and outputs. Neural networks can be implemented locally, or they can be accessed or partially accessed through servers or cloud-based services. Machine learning networks can also be untrained, partially trained, or fully trained, and / or can be retrained, modified, or updated over time.

[0037] refer to Figure 2 An embodiment of an AR gaming system for assisting users with health needs is illustrated, designated 200. In this embodiment, the AR gaming system 200 includes: a gateway 202 for receiving input that can be analyzed or evaluated by an engagement assessment module 204; and a user interaction module 206 for interacting with the user by providing interactive messages or content based on the assessment. For example, the system 200 may be mounted on a smartphone, (tablet) computer, television, or display panel 216 equipped with a suitable application executable to provide an interactive user interface and software functionality. On such exemplary devices, input devices such as touchscreens, camera modules, keyboards, and indicator devices can be used to capture user input initiated by the user or the AR gaming system 200. Furthermore, in some exemplary embodiments, events triggered or determined by program routines may also be provided as input.

[0038] Preferably, the actions performed by the user include physical activities such as exercise, or recreational activities such as karaoke or computer games. FunSpark is a dynamic and versatile platform that offers a variety of applications designed to appeal to diverse user groups, including those in rehabilitation, environmental, music, sports, and aged care fields. For example:

[0039] Applications categorized as environmental may promote practical actions for sustainable development, encourage users to actively participate in waste sorting, and raise their awareness of the recyclability of different items.

[0040] Rehabilitation apps may improve users’ math skills, and can also offer lateral scrolling apps that require users to control a character and reach a finish line by running and jumping, thereby improving hand-eye coordination.

[0041] Music-related apps might involve drumming and trombone playing, helping users explore multiple instruments, enhance their musical perception, and improve rhythmic accuracy.

[0042] Sports-related applications might include running, badminton, bowling, and table tennis, allowing users to participate in sports activities and collaborate with others in different locations.

[0043] Applications in the elderly care category may allow users to perform different movements and provide assessments and feedback through artificial intelligence to help users correct the correctness of their movements.

[0044] This invention will be referred to in the following sections. Figures 4A to 7 This will further explain examples of applications in each category.

[0045] Additionally or optionally, a medication reminder function may be included to provide users with information about their next dose, such as the time of administration and instructions for use. System 200 may also support karaoke functionality, allowing users to sing along to multiple songs, and is equipped with various tools to ensure a high-quality karaoke experience.

[0046] Preferably, by utilizing various advanced technologies such as pose estimation, hand and gesture detection, facial recognition, signal transmission, and WebRTC, the platform provides users with a rich and interactive experience.

[0047] See also Figure 3 A schematic diagram of an all-in-one software platform suitable for installation in an AR gaming system 200 is shown, providing interactive features described further below. In this example, the all-in-one platform, called FunSpark 300, includes multiple applications categorized as previously described, including elderly care, rehabilitation, exercise, music, and environment. Designed by the inventors, FunSpark 300 is an innovative platform aimed at meeting the needs of a broad range of users, with a particular focus on the fields of elderly care and rehabilitation. This integrated application combines entertainment, education, and health management functions to create an engaging and beneficial experience for users.

[0048] For example, apps focused on rehabilitation and exercise encourage users to fully stretch their bodies and maintain basic physical mobility. On the other hand, environmental apps educate users by incorporating sustainability, environmental protection, and ecological information into the app. Engaging with music apps helps develop a sense of rhythm and improve reaction time.

[0049] Preferably, a social connection module 208 can also be provided to support multi-user participation, facilitating simultaneous physical exercise or recreational activities for multiple users. For example, the social connection module 208 can be used in conjunction with applications that offer multiplayer game modes to increase entertainment value and enrich social interaction among multiple users, thus improving their mental health. Simultaneously, multiplayer games can be designed to train the cognitive and / or physical abilities of older users. Example multiplayer applications might include games and activities such as running, badminton, bowling, and table tennis to promote physical participation. Multiplayer games can be designed to encourage both physical participation and social interaction. The platform's multiplayer mode allows users to connect with others, fostering a sense of community and shared experiences.

[0050] In one exemplary embodiment, the FunSpark platform 300 may include an application called “CimdaCare” 400, which is one of the applications in the aged care category. One of FunSpark’s key strengths lies in its aged care support features. For example, the platform 300 can provide an application for exercise or movement, utilizing AI-driven assessment features to ensure the correctness and effectiveness of the movements. It also provides medication reminders and health management tools, making it a valuable resource for older adults to maintain their health and independence. Advantageously, CimdaCare 400 can help older adults or older users meet their physical and mental health needs. These unique features will be described further below.

[0051] Preferably, CimdaCare 400 may have the function of periodically checking or analyzing the user's medication plan 210, so that after the interaction evaluation module 204 determines that an active user should pause activity while using the application or system 200 and take medication according to the medication plan 210 specified by the doctor and / or caregiver, the user interaction module 206 may provide the user with a reminder message containing medication information as a recommendation.

[0052] For example, CimdaCare 400 can be equipped with a medication management system that uses automatic reminders and schedules to help older users maintain daily dosage management. This medication management system provides medication schedules 210 as input for analysis by the engagement assessment module 204. The user interaction module 206 can provide reminders displaying medication information to remind users of medication times and prevent missed doses. Alternatively or additionally, based on a preset action-health relationship 212, the system can suggest certain supplements that may improve the user's health and also remind the user to take these supplements.

[0053] In one example embodiment, see Figures 4A to 4D Users with health needs can interact with the system via the CimdaCare app (400) and system (200). Figure 4B On the menu page 402 shown, the user can select to perform exercise 404 and enter... Figure 4C The screen shown, or select Karaoke 406, will then take you to the next screen. Figure 4D The screen shown.

[0054] Preferably, the user can also... Figure 4BThe menu page 402 shows the option to view reminders 408 to see if there are any health-related suggestions, such as medication reminders. Alternatively, when a user is exercising or singing karaoke, an in-app pop-up notification may appear to remind the user if it's close to their next medication appointment. The app can also provide simple exercise plans tailored to different physical abilities as recommendations related to the user's health status. For example, if the engagement assessment module 204 determines that a user should try to do more exercise to work out other parts of their body, such suggestions will also be provided to the user.

[0055] The inventors believe that entertainment may be a key aspect of the FunSpark 300. See also... Figure 4B The platform includes a karaoke function, allowing users to sing along to various songs, providing a fun and interactive way to pass the time. This feature not only provides entertainment but also helps improve cognitive function and mood, especially for older users.

[0056] In addition, the CimdaCare 400 includes karaoke functionality, allowing users to sing along to a vast music library for an entertaining experience. It may come equipped with various controls to provide a high-quality karaoke experience, such as rewind and fast-forward, display lyrics for users to sing along, a spectrum bar for visual effects, and a slider for adjusting vocals, allowing users to sing along with softer vocals or mute them completely. Advantageously, karaoke can be seen as an alternative way for older users to enjoy exercise and maintain their health.

[0057] The FunSpark 300 also places special emphasis on rehabilitation functions. See also Figure 4C The platform offers interactive exercises designed to improve physical health and cognitive skills. These apps are particularly beneficial for users recovering from injuries or managing chronic conditions.

[0058] Preferably, system 200 may further include a user identification module 214 for capturing user actions or gestures and providing them as input to gateway 202. As mentioned above, the system may include a camera module for providing input for analysis. In this embodiment, the camera module can serve as user identification module 214, used to identify the user by performing at least one action estimation, gesture and pose detection, and facial recognition using computer vision.

[0059] In addition, the camera module can also capture images or videos of users or preset objects to generate AR content, which can then be displayed to users or other observers through a gaming system. For example, the AR gaming system can display a real-time generated avatar that moves based on user actions captured by the user recognition module 214, instead of directly displaying the user's video.

[0060] See Figure 5A and 5B Preferably, posture estimation can be applied to examine the user's posture and provide immediate feedback, for example, by capturing the user's posture and / or movements using the camera module built into the tablet 506. As shown, the real-time captured user image 504 is displayed on the display panel of the system 200. Advantageously, this allows users to safely receive real-time guidance for improvement while exercising. Advantageously, game content can be integrated into the display for the user by incorporating AR elements based on images / videos captured by the tablet's camera, sound captured by the microphone, or motion captured by sensors on the mobile device, thereby enriching the user experience.

[0061] In one exemplary operation, pose estimation can be performed by first capturing body postures or movements using a camera module. The images captured by the camera module can then be provided to the evaluation module 204 for further processing. For example, the evaluation module 204 may include an AI-based evaluation engine for analyzing the user's movements or postures and providing real-time guidance to improve them. The AI-based evaluation engine can be trained using an image or video database containing examples of good and bad postures or movements assessed by professional coaches or physical therapists. Optionally, the images or videos captured by the camera module can be preprocessed, such as extracting the skeleton representing the user's body posture, and then passed as input to the evaluation module 204 via gateway 202.

[0062] In some exemplary embodiments, pose estimation is improved by providing supplementary information related to the physical location of different parts of the user's body, such as depth information captured by a depth camera or time-of-flight sensor. Alternatively, pose or body motion can be tracked or estimated using other methods, such as inertial sensors or tags worn by the user, which can be captured by the system. Motion signals can then be provided to the system to estimate the user's pose or motion during various physical exercises.

[0063] In addition, the camera module can also be used for facial recognition. For example, when a user starts using the system, the system can automatically identify the user's identity, load the matching user profile, and provide corresponding reminders to the identified user.

[0064] Consistent with current global concerns, see [link / reference] Figure 6A FunSpark may integrate environmental education into its functionality. The platform could promote sustainability practices, such as waste sorting and recycling awareness, helping users become more environmentally conscious in their daily lives.

[0065] For music lovers, see Figure 6B FunSpark offers an app (604) for learning musical instruments and developing rhythm skills. This feature not only provides entertainment but also supports cognitive development and the improvement of fine motor skills.

[0066] See Figure 6C and Figure 7 Games 606 in the rehabilitation category can also be provided to help users exercise through the system. For example, virtual characters in the game can be controlled by movements or postures sensed by a camera module, allowing users to exercise their bodies while enjoying the game.

[0067] As mentioned earlier, the system also supports multi-user applications, promoting social interaction among different users. See also Figure 6D It can provide a table tennis game 608 that supports six players, with two players able to play simultaneously in battle mode.

[0068] Preferably, to support these diverse functionalities, FunSpark 300 may incorporate several key technological elements. Pose estimation technology can be used to track user movements during motion, ensuring correct posture and maximizing the effectiveness of physical activity. Hand and gesture detection enhances user interaction with the platform, while face detection supports a personalized user experience.

[0069] Furthermore, WebRTC facilitates real-time communication for multiplayer functionality by ensuring smooth communication between devices through efficient signal transmission. These technological elements work together to provide a responsive and immersive user experience.

[0070] FunSpark, an all-in-one platform, is a multi-functional platform that combines entertainment, health management, education, and social interaction. Its extensive functionality and advanced technological capabilities make it a valuable tool for users of all ages, with particular benefits for seniors and those undergoing rehabilitation. By providing such a comprehensive suite of applications, FunSpark aims to improve users' quality of life and promote physical and mental well-being in an engaging and enjoyable way.

[0071] Furthermore, the FunSpark platform can be implemented in a variety of environments, including schools, aged care facilities, residential areas, and government agencies. This multi-functional platform enables individuals to learn about environmental protection, participate in rehabilitation exercises, relax through music, improve their physical fitness through sports activities, and provide support for older adults, while enjoying the app's interactive and educational features.

[0072] The advantage of these embodiments lies in providing an integrated software platform for the elderly and education through AR gaming systems. It aims to offer users convenience, entertainment, and social connectivity through a variety of practical applications.

[0073] Compared to other existing senior care and education software, FunSpark stands out with its all-in-one approach. It integrates senior care, entertainment, and social functions into a single platform, enhancing usability and convenience. This fusion of different applications makes FunSpark more appealing than traditional apps that focus solely on medication or activities.

[0074] By comprehensively meeting the physical, mental, and social needs of its users, FunSpark has the potential to attract a broad audience beyond just seniors, including schools, aged care facilities, residential communities, and government agencies. Whether used at home or deployed in public facilities, users can benefit from this innovative software platform. FunSpark aims to make a profound impact by combining helpful care tools with simple gamification and creative entertainment, bringing more joy to the lives of its target audience, from young to old.

[0075] The AR gaming system integrated with FunSpark applications boasts innovation and versatility, attracting diverse user groups across fields including rehabilitation, environmental protection, music, sports, and elderly care. The FunSpark platform offers a range of interactive applications designed to meet the needs of different demographics and interests, and is expected to attract a large number of potential clients, including schools, elderly care facilities, residential communities, and government agencies.

[0076] These applications also emphasize practical actions to promote sustainable development, such as encouraging waste sorting and raising awareness of recycling, which aligns with the growing global trend of environmental consciousness. This aspect of the FunSpark platform contributes to its social impact and positions it favorably in the market, as sustainability has become an increasingly important factor for consumers.

[0077] Furthermore, FunSpark's extensive activities and features, including educational applications, physical activity, music exploration, and aged care support, enhance its market appeal and pave the way for widespread adoption. The app's interactivity and engagement make it ideal for individuals seeking entertainment, education, and wellness solutions, providing an integrated platform to meet diverse needs. Advantageously, the AR gaming system integrated with the FunSpark app offers a compelling solution for the digital gaming and wellness sectors. Its unique combination of entertainment, education, and social impact enables better aged care support.

[0078] While not strictly necessary, the embodiments described with reference to the accompanying drawings can be implemented as an application programming interface (API) or a set of libraries used by developers, or can be included in another software application, such as a terminal or personal computer operating system or a portable computing device operating system. Typically, since program modules include routines, programs, objects, components, and data files that help perform specific functions, those skilled in the art will understand that the functionality of a software application can be distributed among multiple routines, objects, or components to achieve the same functionality required herein.

[0079] It should also be understood that any suitable computing system architecture can be used where the methods and systems of the present invention are implemented wholly or partially by a computing system. This will include stand-alone computers, network computers, and dedicated hardware devices. When using the terms "computing system" and "computing device," these terms are intended to include any suitable configuration of computer hardware capable of implementing the described functions.

[0080] Those skilled in the art will understand that various changes and / or modifications can be made to the invention as illustrated in the specific embodiments without departing from the spirit or opposite scope of the invention as broadly described. Therefore, these embodiments are to be considered illustrative rather than restrictive in all respects.

[0081] Unless otherwise specified, any references to prior art contained herein should not be construed as an admission that the information is common general knowledge.

Claims

1. An augmented reality gaming system for assisting users with health needs, characterized in that, include: A gateway is used to receive input associated with preset events or actions performed by the user; The engagement assessment module is used to classify and / or assess the preset events or user actions based on the action-health relationship. as well as The user interaction module is used to respond to preset events or actions performed by the user and interact with the user by providing recommendations related to the user's health status.

2. The augmented reality game system according to claim 1, characterized in that, in, The preset events include medication plans; and the user interaction module is used to provide the recommendations to the user, the recommendations including reminder messages that display medication information to the user.

3. The augmented reality game system according to claim 1, characterized in that, in, The actions performed by the user include physical exercise or recreational activities performed by the user.

4. The augmented reality game system according to claim 3, characterized in that, in, The recommendations include real-time guidance to improve the movement or posture of users performing the physical exercise activities.

5. The augmented reality game system according to claim 4, characterized in that, Also includes: The user identification module is used to capture the user's movement or posture and provide it as input to the gateway.

6. The augmented reality game system according to claim 5, characterized in that, in, The user identification module is used to identify the user by performing at least one of pose estimation, hand and gesture detection, and face detection.

7. The augmented reality game system according to claim 5, characterized in that, in, The engagement assessment module is used to analyze the user's movements or postures captured by the user identification module.

8. The augmented reality game system according to claim 7, characterized in that, in, The engagement assessment module includes an AI-based assessment engine that analyzes the user's movements or postures and provides real-time guidance to the user to improve their movements or postures.

9. The augmented reality game system according to claim 3, characterized in that, Also includes: A social connection module is used to support multi-user participation, so that multiple users can perform the physical exercise activity or the entertainment activity simultaneously.

10. The augmented reality game system according to claim 1, characterized in that, Also includes: An interactive education module is used to provide the user with educational content featuring interactive characteristics.

Images