A Method for Constructing a Multimodal Resource Library for Smart Elderly Care

Abstract

This invention discloses a method for constructing a multimodal resource library for smart elderly care, including: acquiring a multimodal pre-trained large model; performing cross-domain transfer learning on the multimodal pre-trained large model to obtain a multimodal pre-trained large model after cross-domain transfer learning, wherein the multimodal pre-trained large model is obtained through training on a training set, which is a smart elderly care multimodal dataset; and constructing a smart elderly care multimodal nursing resource library based on the multimodal pre-trained large model after cross-domain transfer learning. This invention utilizes the generation and reasoning capabilities of the multimodal pre-trained large model to enrich and innovate the multimodal nursing resource library in the field of smart elderly care, providing more diversified and personalized nursing services and activities for the elderly; and utilizes the matching and decision-making capabilities of the multimodal pre-trained large model to automate and intelligently support multimodal nursing service matching and decision support in the field of smart elderly care.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method for constructing a multimodal resource library for smart elderly care. Background Technology

[0002] In recent years, a new model of elderly care—smart elderly care—has emerged. However, the field of smart elderly care still faces some problems and challenges, such as a lack of data resources, insufficient information analysis, inaccurate service matching, and inadequate decision support. These problems limit the effectiveness and efficiency of smart elderly care services and also affect the trust and satisfaction of the elderly in these services.

[0003] Therefore, there is an urgent need for a method to construct a multimodal resource library for smart elderly care, in order to address the shortcomings of existing technologies. Summary of the Invention

[0004] The purpose of this invention is to propose a method for constructing a multimodal resource library for smart elderly care. This method can effectively utilize the knowledge of other industry fields already mastered by pre-trained large models, especially the prominent knowledge in the fields of medical care, health and daily care, and transfer it to the field of smart elderly care, providing strong support for the construction of a multimodal resource library in this field.

[0005] To achieve the above objectives, this invention provides a method for constructing a multimodal resource library for smart elderly care, comprising the following steps:

[0006] Obtain a large pre-trained multimodal model;

[0007] Cross-domain transfer learning is performed on the multimodal pre-trained large model to obtain the multimodal pre-trained large model after cross-domain transfer learning. The multimodal pre-trained large model is obtained by training a training set, which is a smart health and wellness multimodal dataset.

[0008] Based on the multimodal pre-trained large model after cross-domain transfer learning, a smart elderly care multimodal nursing resource library is constructed.

[0009] According to the present invention, a method for constructing a multimodal resource library for smart elderly care is proposed, wherein the multimodal pre-trained large model is constructed using a GPT-4 network or a BERT network.

[0010] According to the present invention, a method for constructing a multimodal resource library for smart elderly care is proposed, wherein the various types of data related to elderly care include voice, text and image pairs of the elderly’s health status, living habits and care needs.

[0011] According to the present invention, a method for constructing a multimodal resource library for smart elderly care includes performing cross-domain transfer learning on the multimodal pre-trained large model to obtain the multimodal pre-trained large model after cross-domain transfer learning, comprising:

[0012] The source domain and target domain are obtained through the multimodal pre-trained large model, wherein the source domain is the industry domain that the multimodal pre-trained large model has mastered, and the target domain is the smart health and wellness field;

[0013] Obtain migration information and evaluation metrics;

[0014] Based on the transfer information, cross-domain transfer learning is performed on the multimodal pre-trained large model using data from the source domain and the target domain to obtain the multimodal pre-trained large model after cross-domain transfer learning.

[0015] According to the multimodal resource library construction method for smart elderly care proposed in this invention, after performing cross-domain transfer learning on the multimodal pre-trained large model and obtaining the multimodal pre-trained large model after cross-domain transfer learning, the method further includes: testing and evaluating the multimodal pre-trained large model after cross-domain transfer learning on the data in the target domain using the evaluation index, wherein the evaluation index includes accuracy, recall and F1 score.

[0016] According to the present invention, a method for constructing a multimodal resource library for smart elderly care is proposed, wherein the migration information includes migration tasks, migration strategies, and migration methods;

[0017] The migration tasks include text classification, image recognition, video analysis, and audio processing;

[0018] The transfer strategies include fine-tuning, multi-task learning, and meta-learning;

[0019] The transfer methods include adversarial learning, knowledge distillation, and attention mechanisms.

[0020] According to the present invention, a method for constructing a multimodal resource library for smart elderly care, based on the multimodal pre-trained large model after cross-domain transfer learning, constructs the smart elderly care multimodal nursing resource library, including:

[0021] Acquire the multimodal data to be processed and perform preprocessing;

[0022] Using the multimodal pre-trained large model after cross-domain transfer learning, a unified semantic feature representation is performed on the preprocessed multimodal data to obtain a new multimodal dataset;

[0023] The new multimodal data is stored and updated to construct the smart elderly care multimodal nursing resource library.

[0024] According to the present invention, a method for constructing a multimodal resource library for smart elderly care is proposed. After constructing the smart elderly care multimodal resource library, the method further includes: conducting a benefit evaluation on the smart elderly care multimodal resource library, wherein the benefit evaluation includes user satisfaction evaluation, service efficiency evaluation, and service quality evaluation.

[0025] According to the present invention, a method for constructing a multimodal resource library for smart elderly care is proposed, the method further includes using the smart elderly care multimodal resource library to realize a user interaction interface;

[0026] The user interface includes multimodal query, multimodal recommendation, and multimodal feedback;

[0027] The multimodal query is used to retrieve target multimodal data from the smart elderly care multimodal nursing resource library by allowing the user to input query conditions;

[0028] The multimodal recommendation is used to obtain multimodal data recommendation results from the smart elderly care multimodal nursing resource library based on user preferences and needs;

[0029] The multimodal feedback is used to provide feedback on the multimodal data in the smart elderly care multimodal nursing resource library, and to update or optimize the information of the multimodal data in the smart elderly care multimodal nursing resource library based on user feedback, and to improve and adjust cross-domain transfer learning and multimodal recommendation.

[0030] According to the present invention, a method for constructing a multimodal resource library for smart elderly care is proposed, wherein the multimodal recommendation includes content-based recommendation, collaborative filtering-based recommendation, and deep learning-based recommendation.

[0031] The present invention has the following beneficial effects:

[0032] This invention proposes a method for constructing a multimodal resource library for smart elderly care. This method utilizes cross-domain transfer learning technology with a multimodal pre-trained large model to effectively utilize existing massive multimodal data, improving data utilization and knowledge acquisition capabilities in the smart elderly care field. By leveraging the generation and reasoning capabilities of the multimodal pre-trained large model, it can enrich and innovate the multimodal nursing resource library in the smart elderly care field, providing more diversified and personalized nursing services and activities for the elderly. Furthermore, by utilizing the matching and decision-making capabilities of the multimodal pre-trained large model, it can automate and intelligently match multimodal nursing services and provide decision support in the smart elderly care field. Attached Figure Description

[0033] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0034] Figure 1 This is a flowchart illustrating a method for constructing a multimodal resource library for smart elderly care, as described in an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the technical framework proposed in the embodiments of the present invention;

[0036] Figure 3 This is a schematic diagram of cross-domain transfer learning proposed in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram illustrating the construction of a multimodal resource library according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of the user interface proposed in an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram illustrating the benefit assessment proposed in an embodiment of the present invention. Detailed Implementation

[0040] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0042] Smart elderly care refers to the use of information technology and smart devices to provide personalized, intelligent, and convenient elderly care services, enabling seniors to manage their health, receive daily care, and receive emotional support. Smart elderly care can effectively improve seniors' self-care abilities and quality of life, reduce the care burden on society and families, and promote the integration of seniors into society and their families.

[0043] like Figure 1 As shown in the figure, this embodiment provides a method for constructing a multimodal resource library for smart elderly care, including the following steps:

[0044] Obtain a large pre-trained multimodal model;

[0045] Specifically, the multimodal pre-trained large model serves as the basic framework. By pre-training on large-scale data in different domains, the model can learn and master the semantic and structural features of various types of data, thereby achieving a unified representation and processing capability for multimodal data. The model can process multiple types of data such as language, audio, and vision, and has the ability to generate and reason.

[0046] Cross-domain transfer learning is performed on the multimodal pre-trained large model to obtain the multimodal pre-trained large model after cross-domain transfer learning. The multimodal pre-trained large model is obtained by training the training set, which is the smart health and wellness multimodal dataset, that is, various types of data related to health and wellness care.

[0047] Specifically, cross-domain transfer learning is applied to multimodal pre-trained large models to adapt them to the data and tasks in the smart elderly care field, employing techniques such as parameter fine-tuning or parameter regularization. The smart elderly care field refers to the use of information technology and smart devices to provide personalized, efficient, and convenient health management and living services for the elderly. Parameter fine-tuning or regularization techniques involve adjusting or regularizing some or all parameters while maintaining the core structure of the pre-trained large model to adapt to the data distribution and task objectives of the target domain. Furthermore, the various types of data related to elderly care and nursing originate from datasets of a specific elderly care institution or other similar organizations.

[0048] Based on a multimodal pre-trained large model obtained through cross-domain transfer learning, a smart elderly care multimodal nursing resource library is constructed.

[0049] Specifically, by utilizing a multimodal pre-trained large model after cross-domain transfer learning, new voice, text, and image pairs are generated based on data and tasks in the field of smart elderly care, expanding the multimodal nursing data and knowledge in the field of smart elderly care. Then, the expanded and inferred multimodal nursing data and knowledge are integrated into an information platform to form a smart elderly care multimodal nursing resource library.

[0050] Among them, generating new speech, text and image pairs refers to using existing speech, text and image pairs as input, and leveraging the generative capabilities of a multimodal pre-trained large model to output new speech, text and image pairs as output, thereby enriching the multimodal nursing resource library.

[0051] Optionally, the multimodal pre-trained large model can be constructed using a GPT-4 network or a BERT network.

[0052] Specifically, it utilizes state-of-the-art multimodal pre-trained large models, such as GPT-4, BERT, or other similar models.

[0053] Optional, various types of data related to health and wellness care include voice, text, and image pairs of information on the health status, lifestyle habits, and care needs of older adults.

[0054] Specifically, the various types of data related to health and wellness care, namely multimodal datasets, contain a large number of voice, text, and image pairs, reflecting information such as the health status, lifestyle habits, and care needs of the elderly.

[0055] Optionally, cross-domain transfer learning can be performed on the multimodal pre-trained large model to obtain the multimodal pre-trained large model after cross-domain transfer learning, including:

[0056] By using a multimodal pre-trained large model, the source domain and target domain are obtained. The source domain is the industry domain that the multimodal pre-trained large model has mastered, and the target domain is the smart healthcare field.

[0057] Obtain migration information and evaluation metrics;

[0058] Based on the transfer information, cross-domain transfer learning is performed on the multimodal pre-trained large model using data from the source and target domains to obtain the multimodal pre-trained large model after cross-domain transfer learning.

[0059] Specifically, this invention utilizes cross-domain transfer learning technology to effectively transfer knowledge from other industry sectors already mastered by a pre-trained large model, particularly prominent knowledge in fields such as medicine, health, and daily care, to the smart elderly care field, thereby supplementing and enriching the data and knowledge system in this area. Specifically, this invention achieves cross-domain transfer learning through the following steps:

[0060] Step 1: Select appropriate source domain and target domain. The source domain refers to other industry domains that the pre-trained large model has mastered, such as medical, health and life care. Other industry domains include natural language processing, computer vision, multimodal and other fields. The target domain refers to the smart health care field.

[0061] Step 2: Select appropriate migration tasks and evaluation metrics;

[0062] Step 3: Select appropriate migration strategies and methods;

[0063] Step 4: Perform cross-domain transfer learning. Based on the selected transfer task, strategy and method, use data from the source domain and the target domain to perform cross-domain transfer learning on the basis of the pre-trained large model, so that the model can adapt to the data characteristics and task requirements of the target domain.

[0064] Optionally, cross-domain transfer learning is performed on the multimodal pre-trained large model. After obtaining the multimodal pre-trained large model after cross-domain transfer learning, the method further includes: testing and evaluating the multimodal pre-trained large model after cross-domain transfer learning on data in the target domain using evaluation metrics, including accuracy, recall and F1 score.

[0065] Specifically, after cross-domain transfer learning, the cross-domain transfer effect of the multimodal pre-trained large model is evaluated. Based on the selected evaluation metrics, the transferred model is tested and evaluated on data in the target domain to verify its effectiveness and advantages. The evaluation metrics refer to specific indicators used to measure the transfer effect, such as accuracy, recall, and F1 score.

[0066] Optionally, migration information includes migration tasks, migration strategies, and migration methods;

[0067] The transfer tasks include text classification, image recognition, video analysis, and audio processing;

[0068] Transfer strategies include fine-tuning, multi-task learning, and meta-learning;

[0069] Transfer methods include adversarial learning, knowledge distillation, and attention mechanisms.

[0070] Specifically, transfer tasks refer to specific tasks that need to be completed in the target domain, such as text classification, image recognition, video analysis, and audio processing; transfer strategies refer to specific strategies adopted during the transfer process, such as fine-tuning, multi-task learning, and meta-learning; and transfer methods refer to specific methods adopted during the transfer process, such as adversarial learning, knowledge distillation, and attention mechanisms.

[0071] Optionally, based on a multimodal pre-trained large model obtained through cross-domain transfer learning, a smart elderly care multimodal nursing resource library can be constructed, including:

[0072] Acquire the multimodal data to be processed and perform preprocessing;

[0073] By utilizing a large multimodal pre-trained model trained through cross-domain transfer learning, a unified semantic feature representation is performed on the preprocessed multimodal data to obtain a new multimodal dataset.

[0074] The new multimodal data is stored to build a smart elderly care multimodal nursing resource library.

[0075] Specifically, based on the results of cross-domain transfer learning, this invention constructs a structured multimodal resource library. This resource library contains various data types related to elderly care and nursing, such as text descriptions, images, videos, audio, and other possible data formats, to facilitate subsequent retrieval and utilization. Specifically, this invention constructs the multimodal resource library through the following steps:

[0076] Step 1: Data collection. A large amount of multimodal data related to health and wellness care was collected through various channels and methods, such as obtaining text, images, videos, and audio data from the Internet, social media, professional institutions, and individual users, or collecting physiological signals, behavioral data, and environmental data related to health and wellness care through self-designed and developed data collection devices and systems, such as smart bracelets, smart cameras, and smart microphones. Among them, multimodal data includes different types of data such as images, videos, voice, text, and tables.

[0077] Step 2: Data processing. Using the pre-trained large model after cross-domain transfer, the collected multimodal data is analyzed and processed to achieve unified semantic feature representation of the multimodal data, thereby providing consistent services for subsequent operations such as storage, analysis, retrieval and computation of multimodal data.

[0078] Specifically, different processing methods are applied to different types of data. For example, text data is processed by word segmentation, part-of-speech tagging, named entity recognition, and sentiment analysis; image data is processed by face detection, target recognition, and scene classification; video data is processed by frame extraction, action recognition, and event detection; audio data is processed by speech recognition, speech synthesis, and emotion recognition; and other types of data are processed accordingly.

[0079] Step 3: Data storage. The processed multimodal data and its related metadata (such as source, time, tags, etc.) are stored in a structured database. This database supports fast querying and retrieval of multimodal data, as well as operations such as adding, deleting, modifying, and querying multimodal data.

[0080] Step 4: Data Update. A complete data management and update mechanism has been established, including regularly or in real-time acquisition of new multimodal data related to health and wellness care from various channels and methods, processing and storing it; at the same time, existing multimodal data is regularly or in real-time cleaned, verified and optimized to ensure the timeliness, accuracy and completeness of the resource library.

[0081] Optionally, after constructing the smart elderly care multimodal nursing resource library, the following may also be included: conducting a benefit assessment of the smart elderly care multimodal nursing resource library, wherein the benefit assessment includes user satisfaction assessment, service efficiency assessment and service quality assessment.

[0082] Specifically, through practical application and benefit evaluation, this invention has verified its effectiveness and advantages, as well as its improvement in the efficiency and quality of elderly care services. Specifically, this invention achieves benefit evaluation through the following methods:

[0083] (1) User satisfaction assessment: By designing and releasing a series of questionnaires, we collect user satisfaction ratings on the content, quality, quantity, diversity, etc. of the multimodal data in the resource library, and conduct statistical analysis on the ratings to reflect the overall user satisfaction and preference for the resource library.

[0084] (2) Service efficiency evaluation: By recording and analyzing data such as the time, number of times, and results of users' queries and recommendations in the resource library, the service efficiency indicators of the resource library, such as response time, query accuracy, and recommendation recall rate, are calculated and compared with other similar resource libraries to reflect the service efficiency and advantages of the resource library.

[0085] (3) Service quality assessment: By collecting and analyzing the application effects and impacts of multimodal data obtained by users in the resource database on health care and nursing practice, such as the improvement or enhancement of users' health status, quality of life, and psychological state, the service quality indicators of the resource database, such as health index, happiness index, loyalty index, etc., are calculated and compared and evaluated with other similar resource databases to reflect the service quality and advantages of the resource database.

[0086] Optionally, the method for constructing a smart elderly care multimodal nursing resource library also includes using the smart elderly care multimodal nursing resource library to realize a user interaction interface;

[0087] The user interface includes multimodal query, multimodal recommendation, and multimodal feedback;

[0088] Multimodal query is used to retrieve target multimodal data from the smart elderly care multimodal nursing resource library by allowing users to input query conditions;

[0089] Multimodal recommendation is used to obtain recommendation results of multimodal data from the smart elderly care multimodal nursing resource library based on user preferences and needs;

[0090] Multimodal feedback is used to provide feedback on the multimodal data in the smart elderly care multimodal nursing resource library, and to update or optimize the information of multimodal data in the smart elderly care multimodal nursing resource library based on user feedback, and to improve and adjust cross-domain transfer learning and multimodal recommendation.

[0091] Specifically, this invention designs and implements a user-friendly interface that enables healthcare professionals and relevant stakeholders to easily access and utilize information in the resource database, while also providing users with personalized query and recommendation services. Specifically, this invention implements the user interface through the following functions:

[0092] (1) Multimodal query: It supports users to input query conditions in a variety of ways, such as inputting text keywords or sentences through the keyboard, clicking on images or video clips through the mouse, speaking voice commands or questions through the microphone, etc. Based on the query conditions input by the user, it retrieves matching or related multimodal data from the resource library and displays it to the user in an appropriate way, such as displaying text or image results in the form of a list or grid, or displaying video or audio results in the form of a player.

[0093] (2) Multimodal Recommendation: Supports users to obtain multimodal data recommendation results from the resource library according to their preferences and needs, and present them to users in an appropriate way, such as displaying text or image recommendation results in the form of a carousel or slideshow, or displaying video or audio recommendation results in the form of a playlist. Obtaining multimodal data recommendation results from the resource library means using the user's needs and preferences as input, utilizing the matching and decision-making capabilities of a multimodal pre-trained large model, selecting nursing services and activities that meet the conditions from the multimodal nursing resource library as output, and obtaining multimodal data recommendation results from the resource library.

[0094] (3) Multimodal feedback: It supports users to provide feedback on multimodal data in the resource library in various ways, such as entering text comments or evaluations through the keyboard, clicking on tags or buttons on images or videos with the mouse, and speaking opinions or suggestions through the microphone. Based on the feedback given by users, the content, tags, ratings and other information of multimodal data in the resource library are updated or optimized. At the same time, the feedback information of users is used to improve and adjust functions such as cross-domain transfer learning and multimodal recommendation.

[0095] Specifically, this invention achieves multimodal recommendation through the following methods:

[0096] (1) Content-based recommendation: Based on the multimodal data content that users have browsed or saved, the semantic features and themes are analyzed, such as keywords in texts, tags in images, and scenes in videos. Based on these features and themes, similar or related multimodal data is retrieved from the resource library and recommended to the user according to similarity or relevance. For example, if a user browses or saves a text about music therapy in health and wellness care, this invention will retrieve other texts, images, videos, or audio related to music therapy from the resource library and recommend them to the user. This method allows users to obtain multimodal data that matches their interests or needs, improving user satisfaction and loyalty.

[0097] (2) Recommendation based on collaborative filtering: Based on the similarity between users or between multimodal data, calculate the similar neighbors of users or multimodal data, and recommend multimodal data that users may be interested in based on the browsing or collection records of the neighbors.

[0098] (3) Deep learning-based recommendation: Using a pre-trained large model after cross-domain transfer, deep feature extraction and embedding are performed on users and multimodal data. Then, through neural network models, such as multilayer perceptron, recurrent neural network, convolutional neural network, etc., the user's rating or preference for multimodal data is predicted, and multimodal data that the user may be interested in is recommended according to the rating or preference.

[0099] This invention proposes a method for constructing a multimodal resource library for smart elderly care. This method utilizes pre-trained large models (such as GPT-4 or BERT) to analyze and process newly collected multimodal data (including text, images, videos, audio, and other data) related to health and elderly care. It achieves unified semantic feature representation of the multimodal data, thus providing consistent services for subsequent operations such as storage, analysis, retrieval, and computation of the multimodal data. Furthermore, addressing the current lack of high-quality multimodal data in the smart elderly care field, this method effectively transfers knowledge from other industry sectors learned and mastered by the pre-trained large models to the smart elderly care domain. These key technologies and methods provide strong support for the construction of a multimodal data resource library for smart health and elderly care.

[0100] Example

[0101] This invention implements a method for constructing a multimodal resource library for smart elderly care, specifically including the following steps:

[0102] Step 1, Technical Framework: such as Figure 2 As shown, pre-trained large models (such as GPT-4 or BERT) are used as the basic framework, and cross-domain transfer learning technology is adopted to supplement and enrich the data and knowledge system in the field of smart healthcare.

[0103] Specifically, this embodiment first utilizes the knowledge of other industry fields (such as medical care, health and life care) already mastered by the pre-trained large model to obtain the semantic features and knowledge representations of these data; then, these semantic features and knowledge representations are transferred to the smart elderly care field, and through fine-tuning and adaptive training, the pre-trained large model can adapt to the characteristics and needs of the smart elderly care field, and can effectively analyze and process newly collected multimodal data in health and elderly care.

[0104] Step 2, cross-domain transfer learning: such as Figure 3 As shown, through cross-domain transfer learning, knowledge from other industry sectors already mastered by pre-trained large models is effectively utilized, especially prominent knowledge in fields such as medical care, health, and daily care, and transferred to the field of smart elderly care, providing strong support for the construction of a multimodal resource library in this field.

[0105] Step 3: Construction of a multimodal resource library: such as Figure 4 As shown, based on the results of cross-domain transfer learning, a structured multimodal resource library is constructed. This resource library contains various data types related to health and wellness care, such as text descriptions, images, videos, audio, and other possible data formats, to facilitate subsequent retrieval and utilization.

[0106] Specifically, this embodiment first preprocesses the newly collected multimodal data on health and elderly care, including cleaning, labeling, and classification. Then, it uses a pre-trained large model to perform unified semantic feature representation on these data, thereby achieving consistent processing of multimodal data. Next, these semantic features are stored in a structured database, and corresponding indexes and labels are established according to different data types and attributes to facilitate subsequent operations such as storage, analysis, retrieval, and calculation of multimodal data.

[0107] Step 4, User Interface: such as Figure 5 As shown, a user-friendly interface was designed and implemented to enable healthcare professionals and relevant stakeholders to easily access and utilize the information in the resource library, while also providing users with personalized query and recommendation services.

[0108] Specifically, this embodiment first designs suitable interaction methods and functions based on different users and application scenarios, such as voice input, graphic output, and natural language question answering, and provides corresponding query conditions and filtering options according to user preferences and needs. Then, it uses a pre-trained large model to understand and analyze user input, retrieves relevant information from the resource library based on user input, and returns it to the user in an appropriate format. In addition, this embodiment also uses a pre-trained large model to analyze and mine user behavior, and provides personalized recommendation services based on user behavior, such as recommending relevant content, providing feedback and suggestions, etc.

[0109] Step 5: Data Management and Update Mechanism: Establish a complete data management and update mechanism, including data collection, processing, storage, and updating, to ensure the real-time, accuracy, and completeness of the resource repository, while also providing support for the continuous optimization and expansion of the resource repository;

[0110] Specifically, this embodiment first establishes a data collection platform to collect multimodal data related to health and wellness care through various channels and methods, such as web crawlers, social media, and user uploads, and then performs quality checks and screening on the collected data. Next, a pre-trained large model is used to process and represent the collected data, and the data is stored in a resource library. The data in the resource library is updated and maintained regularly to ensure the timeliness and accuracy of the resource library.

[0111] Step Six: Benefit Evaluation: (e.g.) Figure 6 As shown, through practical application and benefit evaluation, the effectiveness and advantages of this embodiment are verified, as well as its improvement in the efficiency and quality of health and nursing care services;

[0112] Specifically, this embodiment first selects suitable application scenarios and target users, such as the elderly, caregivers, and doctors, and designs appropriate application schemes and evaluation indicators, such as user satisfaction, service efficiency, and service quality, based on the application scenarios and target users. Then, user feedback and data are collected in actual applications, and a pre-trained large model is used to analyze and statistically process the user feedback and data. The effectiveness of the invention is evaluated based on the evaluation indicators. Finally, the resource library is optimized and expanded based on the evaluation results, such as adding new data types, improving data structures, and enhancing data quality. The invention is then summarized and improved, and future research directions and prospects are proposed.

[0113] This embodiment provides a powerful and flexible multimodal resource library for the field of smart elderly care, which greatly improves the efficiency and quality of elderly care services, meets the needs of different users and application scenarios, and contributes to the development of health management and nursing fields.

[0114] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing a multi-modal resource library for smart health and nursing care, characterized in that, Specifically comprising the following steps: Obtaining a multi-modal pre-training large model; Performing cross-domain transfer learning on the multi-modal pre-training large model to obtain a multi-modal pre-training large model after cross-domain transfer learning, wherein the multi-modal pre-training large model is obtained by training a training set, and the training set is a smart health care multi-modal data set; Performing cross-domain transfer learning on the multi-modal pre-training large model to obtain a multi-modal pre-training large model after cross-domain transfer learning, wherein the multi-modal pre-training large model is obtained by training a training set, and the training set is a smart health care multi-modal data set; Obtaining source domain and target domain through the multi-modal pre-training large model, wherein the source domain is an industry domain that the multi-modal pre-training large model has mastered, and the target domain is a smart health care domain; Obtaining transfer information and evaluation indicators; According to the transfer information, using the data of the source domain and the target domain to perform cross-domain transfer learning on the multi-modal pre-training large model, and obtaining the multi-modal pre-training large model after cross-domain transfer learning; Based on the multi-modal pre-training large model after cross-domain transfer learning, a smart health care multi-modal nursing resource library is constructed, comprising: Obtaining and preprocessing multi-modal data to be processed; Using the multi-modal pre-training large model after cross-domain transfer learning, uniformly representing the semantic features of the preprocessed multi-modal data to obtain a new multi-modal data set; Storing and updating the new multi-modal data to construct the smart health care multi-modal nursing resource library; After constructing the smart health care multi-modal nursing resource library, the smart health care multi-modal nursing resource library is also subjected to benefit evaluation, wherein the benefit evaluation includes user satisfaction evaluation, service efficiency evaluation and service quality evaluation; The method further comprises realizing a user interaction interface by using the smart health care multi-modal nursing resource library; The user interaction interface comprises multi-modal query, multi-modal recommendation and multi-modal feedback; The multi-modal query is used to retrieve target multi-modal data in the smart health care multi-modal nursing resource library by inputting query conditions by the user; The multi-modal recommendation is used to obtain multi-modal data recommendation results in the smart health care multi-modal nursing resource library according to user preferences and demands; The multi-modal feedback is used to give feedback on the multi-modal data in the smart health care multi-modal nursing resource library, and update or optimize the information of the multi-modal data in the smart health care multi-modal nursing resource library according to the user feedback information, and improve and adjust the cross-domain transfer learning and multi-modal recommendation.

2. The smart health and wellness oriented multi-modal resource library construction method of claim 1, wherein, The multi-modal pre-training large model is constructed by using GPT-4 network or BERT network.

3. The method of claim 1, wherein the method is characterized by, The various types of health care related data include voice, text and image pairs of the health status, living habits and nursing needs of the elderly.

4. The smart health and wellness oriented multi-modal resource library construction method of claim 1, wherein, After performing cross-domain transfer learning on the multi-modal pre-training large model to obtain a multi-modal pre-training large model after cross-domain transfer learning, the multi-modal pre-training large model after cross-domain transfer learning is also tested and evaluated on the data in the target domain by using the evaluation indicators, wherein the evaluation indicators include accuracy, recall rate and F1 value.

5. The smart health and wellness oriented multi-modal resource library construction method of claim 1, wherein, The migration information includes migration tasks, migration strategies, and migration methods. The migration tasks include text classification, image recognition, video analysis, and audio processing. The migration strategies include fine-tuning, multi-task learning, and meta-learning. The migration methods include adversarial learning, knowledge distillation, and attention mechanisms.

6. The smart health and wellness oriented multi-modal repository construction method of claim 1, wherein, The multi-modal recommendation includes content-based recommendation, collaborative filtering-based recommendation, and deep learning-based recommendation.

Images