Medical aesthetic consultation system and control method based on RAG and multi-modal large language model

Abstract

This invention discloses an intelligent medical aesthetic consultation system and its control method based on RAG and a multimodal large language model. Belonging to the category of multimodal large language model intelligent systems, this system includes multiple functional modules: a customer data acquisition module, an image analysis module, a knowledge base retrieval module, a solution generation module, and a business system interface module. Each module transmits, flows, and calculates data within the system to obtain the aesthetic consultation solution required by the customer, which is then presented on the client side. By integrating RAG knowledge retrieval, a multimodal large language model, and a medical aesthetics-specific knowledge base into the system, it can provide users with professional, personalized, and dynamically updated aesthetic consultation solutions in real time, solving the pain points of existing technologies such as heavy reliance on manual labor, high costs, and inconsistent user experience. This improves the efficiency and professionalism of the system's aesthetic consultation, reduces service costs, and optimizes the entire customer experience, introducing a new intelligent consultation model to the medical aesthetics industry.

Description

Technical Field

[0001] This invention relates to a multimodal large language model intelligent system, and more specifically, to a medical aesthetics consultation intelligent system and its control method based on RAG and a multimodal large language model. Background Technology

[0002] Currently, aesthetic consultations in the medical aesthetics field primarily rely on manual processes. The quality of aesthetic design plans heavily depends on the personal experience of doctors or designers, resulting in inconsistent outcomes and insufficient personalization. The high cost of aesthetic design raises the barrier to entry for customers, hindering the widespread adoption of medical aesthetic services. Furthermore, the traditional aesthetic consultation process lacks a tight connection with subsequent treatments, making it difficult to provide end-to-end intelligent services and leading to a fragmented customer experience. While some institutions are beginning to utilize artificial intelligence to assist in medical aesthetic consultations, these efforts are largely limited to single functions such as skin detection and 3D modeling, failing to form a complete closed-loop intelligent service system. For example, some solutions only provide skin quality assessments or post-operative effect simulations, failing to cover the entire process from consultation and plan development to pricing and post-operative follow-up. Additionally, with the rise of generalized languages, there have been attempts to use universal dialogue models for medical aesthetic consultations, but these have the following limitations: First, they lack a dedicated knowledge base; the generalized models do not integrate professional knowledge in the medical aesthetics field. Second, platform data is not integrated; existing models cannot automatically connect to hospital business platform data to obtain real-time information such as customer profiles and product inventory / prices, resulting in mechanical and unpersonalized consultation responses. Third, relying on a single LLM model is costly, and there are few alternatives when the model fails to cover certain knowledge gaps. Therefore, it is necessary to research and improve intelligent systems for the medical aesthetics industry. Summary of the Invention

[0003] One of the objectives of this invention is to address the aforementioned shortcomings by providing a medical aesthetics consultation system and control method based on RAG and a multimodal large language model. This aims to solve the technical problems of existing systems, such as a lack of domain-specific expertise, a lack of personalized consultation answers, and high costs due to the reliance on a single model.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: This invention provides an intelligent medical aesthetics consultation system based on RAG and a multimodal large language model. The system includes: a customer data acquisition module for acquiring customer data from the client and transmitting it to an image analysis module; the customer data includes current customer input data and historical information; the customer input data includes at least a facial photo, name, age, and desired outcome; an image analysis module for extracting and analyzing key facial parameters based on the customer data, generating an image analysis result including a 3D facial model and an effect image; a knowledge base retrieval module for performing knowledge retrieval based on keywords related to the desired outcome from the customer data; the knowledge retrieval involves searching for information related to the current desired outcome keywords in a medical aesthetics knowledge base and then embedding the information into the input information of the large language model; a solution generation module for generating an aesthetic consultation solution based on the image analysis results, customer needs, and the aforementioned information, and outputting it to the client; and a business system interface module for providing data from the backend business system to the customer data acquisition module via an API interface, serving as the current customer's historical information; the business interface module also receives the aesthetic consultation solution and transmits it to the backend business system.

[0005] As a preferred embodiment, a further technical solution is as follows: The system further includes: an aesthetic design recommendation module, used to output facial effect images under different schemes based on the client's three-dimensional facial model according to the aesthetic consultation plan, and to provide a score and improvement suggestions for the current client's face through an aesthetic evaluation algorithm, which are output to the client along with the aesthetic consultation plan. A quotation management module, used to calculate and generate quotation information based on the corresponding items in the aesthetic consultation plan; the quotation information is generated based on the item price list and consumable inventory in the back-end business system, and is output to the client along with the aesthetic consultation plan. A post-operative care module, used to provide continuous guidance and service information to clients according to the preset project cycle through a large language model, and to collect client feedback; the post-operative care module is also used to transmit the feedback to the back-end business system through the API interface of the business system interface module.

[0006] A further technical solution is that the aesthetic consultation plan includes aesthetic design suggestions, recommended specific medical aesthetic projects and products, implementation steps, and expected results.

[0007] A further technical solution is that the medical aesthetics knowledge base is updated in real time through a backend management system. The backend business system includes any one of a medical aesthetics business management system, a membership management system, an electronic medical record system, and a marketing system. The solution generation module is also used to answer or supplement client-side questions regarding the aesthetic consultation solution and the current facial assessment and improvement suggestions using a large language model.

[0008] Another aspect of the present invention provides a control method for the above-mentioned system, the method comprising the following steps: Step A: The customer data acquisition module obtains customer data from the client and transmits it to the image analysis module. The customer data includes the data input by the current customer and the historical information of the current customer. The data input by the customer includes at least a facial photo, name, age, and request. The historical information of the current customer is data from the backend business system provided by the business system interface module to the customer data acquisition module through the API interface.

[0009] Step B: The image analysis module extracts and analyzes key facial parameters based on customer data, generating image analysis results including a 3D model of the customer's face and a rendering of the appearance.

[0010] Step C: The knowledge base retrieval module performs knowledge retrieval based on the target keywords in the customer data. The knowledge retrieval involves searching for information related to the current target keywords in the medical aesthetics professional knowledge base, and then embedding the information into the input information of the large language model.

[0011] Step D: The solution generation module generates an aesthetic consultation solution and outputs it to the client based on the image analysis results, customer needs, and the aforementioned information using a large language model.

[0012] Step E: The business interface module receives the aesthetic consultation plan and transmits it to the back-end business system.

[0013] As a preferred embodiment, a further technical solution is that the method further includes: Step F: The aesthetic design recommendation module, based on the aesthetic consultation plan and the client's 3D facial model, outputs facial effect images under different plans, and provides a score and improvement suggestions for the current client's face through an aesthetic evaluation algorithm, which are then output to the client along with the aesthetic consultation plan.

[0014] Step G: The quotation management module calculates and generates quotation information based on the corresponding projects in the aesthetic consultation plan; the quotation information is generated based on the project price list and consumable inventory in the back-end business system, and is output to the client along with the aesthetic consultation plan.

[0015] Step H, the post-operative care module, uses a large language model to provide continuous guidance and service information to clients who have completed transactions, according to the project's preset cycle, and collects client feedback.

[0016] Step 1: The postoperative care module transmits the feedback to the backend business system through the API interface of the business system interface module.

[0017] Step J: The solution generation module uses a large language model to answer or supplement the client's questions regarding the aesthetic consultation solution and the current client's facial score and improvement suggestions.

[0018] Compared with existing technologies, one of the beneficial effects of this invention is that by integrating RAG knowledge retrieval, multimodal large language models, and a medical aesthetics-specific knowledge base into the system, it can provide users with professional, personalized, and dynamically updated aesthetic consultation solutions in real time. This solves the pain points of existing technologies, such as heavy reliance on manual labor, high costs, and inconsistent user experience. Consequently, it improves the efficiency and professionalism of the system's aesthetic consultation, reduces service costs, and optimizes the entire customer experience, introducing a brand-new intelligent consultation model to the medical aesthetics industry. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating a system architecture of one embodiment of the present invention.

[0020] Figure 2 This is a flowchart illustrating a method according to an embodiment of the present invention. Detailed Implementation

[0021] The invention will now be further described with reference to the accompanying drawings.

[0022] One embodiment of the present invention is an intelligent medical aesthetic consultation system based on RAG and a multimodal large language model. This system includes multiple functional modules: a customer data acquisition module, an image analysis module, a knowledge base retrieval module, a solution generation module, and a business system interface module. Each module, through data transmission, flow, and calculation within the system, generates the aesthetic consultation solution required by the customer and presents it on the client side. Specifically: The aforementioned customer data collection module is used to acquire customer data from the client and transmit it to the image analysis module. This customer data includes the data currently input by the customer and their historical information. The customer input data includes at least a facial photo, name, age, and request. Specifically, this module also interfaces with the hospital's existing business platform through a business system interface module, automatically acquiring customers' historical medical records and membership files, achieving data synchronization between the online mini-program and the backend "Ruimei Cloud" system. This ensures that the customer data obtained by the system is comprehensive and accurate, and is updated in real time as the business database is updated.

[0023] The aforementioned image analysis module is used to extract and analyze key facial parameters based on client data, generating image analysis results including a 3D model of the client's face and a rendering of the desired appearance. Specifically, this module uses computer vision and deep learning algorithms to identify and quantitatively evaluate facial features and skin condition. For example, it analyzes parameters such as facial proportions, facial contours, and skin imperfections, providing an objective basis for subsequent aesthetic design. Preferably, the image analysis module includes a 3D modeling unit that can generate a 3D model of the client's face and simulate post-operative changes, as well as a skin detection unit for detecting skin problems and aging. Through dynamic simulation of the 3D model, this module can intuitively demonstrate the potential effects of different cosmetic treatments. The analysis results will be output in structured data form (such as facial feature parameters and a list of skin problems) for the treatment plan generation module to use.

[0024] The aforementioned knowledge base retrieval module is used to perform knowledge retrieval based on the keywords of customer requests in the customer data. This retrieval involves searching for information related to the current request keywords in a medical aesthetics professional knowledge base, and then embedding this information into the input information of the large language model. The aforementioned knowledge base pre-collects a large amount of professional materials in the medical aesthetics field and internal hospital knowledge, such as introductions to various plastic surgery procedures, explanations of surgical principles, case studies, pre- and post-operative precautions, and product efficacy comparisons, stored in the database in vector form. When a customer raises a specific request or consultation question, the knowledge base retrieval module calculates semantic similarity based on the question content, retrieves the most relevant entries from the knowledge base, and embeds them as prompts into the input of the large language model (i.e., RAG prompt construction), assisting the large model in generating professional and data-supported responses. This module supports a real-time update mechanism; when new knowledge points, products, or solutions are added, entries can be quickly added or modified to the knowledge base through the backend management. For example, operations personnel can instantly "feed" the model with the latest aesthetic concepts, information on newly launched materials and products, or details of current promotional activities, ensuring that the model's responses reflect the latest knowledge dynamics.

[0025] The aforementioned solution generation module is used to generate aesthetic consultation solutions and output them to the client based on image analysis results, client needs, and the aforementioned information, using a large language model. Specifically, the solution generation module calls the multimodal large language model integrated within the system, integrating text and image information into the model to generate professional recommendations. The generated solution content includes: aesthetic design suggestions suitable for the client (such as facial feature adjustment plans, skin treatment plans, etc.), specific recommended medical aesthetic projects and products, implementation steps, and expected results. The model follows certain templates and format constraints when generating solutions to ensure that the output content is detailed and logical, including itemized explanations and justifications. Due to the integration of RAG retrieval information, key data and arguments in the solution are supported by sources. For example, for a client's request to improve their nose shape, the model will combine rhinoplasty cases and explanations of principles from the knowledge base to propose suitable rhinoplasty options (implant rhinoplasty or autologous cartilage rhinoplasty, etc.), and provide expected results and risk warnings for each option.

[0026] The aforementioned business system interface module is used to provide data from the backend business system to the customer data collection module via API, serving as the historical data of the current customer. This module also receives the aesthetic consultation plan and transmits it to the backend business system. Specifically, this module interfaces with the hospital's existing "Ruimei Cloud" medical aesthetic business management system, membership management system, and electronic medical record system. On one hand, it acquires customer appointment records, medical history files, and past consumption items as consultation references; on the other hand, it synchronizes the AI-generated plan and customer intentions back to the business system for doctors to further confirm and follow up. Furthermore, the business interface module can also call upon functions of the hospital's marketing system, such as generating customer-specific coupon codes and arranging follow-up visits. Through the integration of this module, the system of this invention forms a closed loop with the hospital's existing IT system: the AI ​​consultation plan can seamlessly connect with offline medical service processes, truly achieving an integrated intelligent service closed loop of online consultation and offline treatment.

[0027] Furthermore, in a preferred embodiment of the present invention, the system may further integrate an aesthetic design recommendation module, a pricing management module, and a postoperative care module, specifically: The aforementioned aesthetic design recommendation module, based on the aesthetic consultation plan and the client's 3D facial model, outputs facial effect images under different plans. It also provides a score and improvement suggestions for the client's current face using an aesthetic evaluation algorithm, and outputs these along with the aesthetic consultation plan to the client. As mentioned earlier, this module can simulate the effect of the recommended plan on the client's 3D facial model based on the suggestions output by the plan generation module, or provide comparison photos of similar cases to allow the client to intuitively understand the design concept. Simultaneously, the module uses aesthetic evaluation algorithms to provide a comprehensive score and suggestions for improving the client's image from an aesthetic perspective. For example, it assesses the room for improvement of the current face shape based on aesthetic standards such as the golden ratio and recommends specific design plan combinations (e.g., a "facial contour lifting + skin radiance management" combination). Furthermore, this module also combines current trends and the client's personal preferences to recommend personalized aesthetic styles. For example, if the knowledge base records that the client prefers a natural style, it prioritizes recommending less invasive and conservative plans; if the client seeks dramatic changes, it provides relatively radical design combinations. The output of the aesthetic design recommendation module complements the plan generation module, helping the client understand the plan from both professional rational and aesthetic perspectives.

[0028] The pricing management module calculates and generates pricing information based on the corresponding items in the aesthetic consultation plan. This pricing information is generated from the item price list and consumable inventory in the back-end business system and is output to the client along with the aesthetic consultation plan. Specifically, this module obtains information such as the hospital's internal item price list and consumable inventory through the business system interface, and calculates costs based on the customer's selected items and plan content. The pricing management module can provide detailed prices for each recommended sub-item, as well as possible discount options. For example, if the plan includes laser skin treatment and hyaluronic acid injection, the module will list the standard prices for both, and consider member discounts or package prices to generate a total price. Simultaneously, this module supports real-time updates, automatically applying the latest pricing strategies when the hospital adjusts prices or launches new packages to ensure accurate and reliable pricing. The pricing information will be presented to the customer along with the plan suggestions to help them make a decision. If needed, customers can also directly book services for the corresponding items through the system.

[0029] The post-operative care module utilizes a large language model to provide continuous guidance and service information to clients according to a pre-set project cycle, and collects client feedback. Simultaneously, this module transmits feedback to the back-end business system via the API interface of the business system interface module. Furthermore, leveraging the dialogue capabilities of the large language model, the module provides 24 / 7 intelligent Q&A support for common post-operative issues, such as wound care methods, swelling reduction techniques, and dietary precautions. The post-operative care module also automatically pushes a recovery period calendar based on the specific surgical procedure, sending reminders at key time points such as the 1st, 3rd, and 7th day after surgery, reminding clients of correct care measures or follow-up appointments. If clients experience anxiety or unease, the module can simulate psychological comfort dialogue, providing support and professional advice, acting as a "caring nurse." In addition, by integrating with the hospital's CRM system, the post-operative care module records client recovery progress and feedback, providing reference for doctors' follow-up examinations and subsequent services. This series of thoughtful post-operative services will significantly improve customer satisfaction, enhance their trust, and increase their willingness to repurchase.

[0030] As can be seen from this embodiment, the system of the present invention aims to create functions similar to an intelligent aesthetic consultant, achieving the following innovative effects: The multi-source information fusion system integrates customers' text and image data inputs, uses computer vision and multimodal large language models to analyze and understand customers' facial images and descriptive needs, and combines retrieval augmented generation (RAG) technology to extract professional knowledge from the medical aesthetics vertical knowledge base to form customized consultation results for customers.

[0031] The system integrates enhanced retrieval with a knowledge base. Through the RAG architecture, when generating responses or solutions, the system first retrieves relevant information (including aesthetic design principles, surgical cases, product / project descriptions, etc.) from the hospital's own medical aesthetics knowledge base. The search results are then incorporated into prompts within the larger model, ensuring the professionalism and accuracy of the generated content. The knowledge base uses a vector database to store knowledge, enabling efficient semantic retrieval and allowing for real-time synchronization and updates with business system data.

[0032] Multi-model collaboration: The system supports the integration of multiple mainstream large-scale language models, such as GPT, Claude, and Kimi. Through the model management and orchestration platform, the optimal model or combination of models is selected based on task requirements and model strengths. The complementary advantages of different models effectively compensate for the knowledge gaps of a single model, reducing usage costs and improving response reliability.

[0033] Real-time dynamic recommendations: This invention's system can dynamically adjust based on the latest industry knowledge and internal organizational data. For example, through the backend knowledge base management module, it instantly incorporates new aesthetic concepts, new product information, brand promotional points, the latest aesthetic scheme templates, and promotional activities, ensuring that the system's consultation and recommendation content is always synchronized with current trends and the organization's business strategies. When new technologies or products are launched, the system can learn in real time and incorporate them into consultation plans; when customers inquire, the system can also provide real-time updated quotes and solutions based on inventory and promotional information.

[0034] This invention provides end-to-end intelligent service, encompassing the entire customer service process and possessing comprehensive capabilities similar to a human consultant. Specifically, the system can perform functions such as pre-operative consultation and Q&A, personalized aesthetic design, plan and quotation generation, related product recommendations, closing the deal, and post-operative care guidance and emotional support. For example, after a customer uploads a front-facing photo and expresses their aesthetic requests, the system automatically provides aesthetic design suggestions and detailed plastic surgery / skincare plans, including simulated images of expected results and project quotations, for the customer's reference and decision-making. After the customer's treatment, the system continuously provides care services such as recovery period precautions and post-operative care Q&A, greatly improving the continuity and satisfaction of the customer experience.

[0035] This invention integrates RAG knowledge retrieval, a multimodal large language model, and a medical aesthetics-specific knowledge base. It provides users with professional, personalized, and dynamically updated aesthetic consultation solutions in real time, addressing the pain points of existing technologies such as heavy reliance on manual labor, high costs, and inconsistent user experience. This system improves the efficiency and professionalism of aesthetic consultations, reduces service costs, and optimizes the entire customer experience, introducing a brand-new intelligent consultation model to the medical aesthetics industry.

[0036] Another embodiment of the present invention is a control method based on the above-described system. In this embodiment, the method is performed according to the following steps: Step 1: Customer Data Acquisition. Customers initiate aesthetic consultation requests via user terminals (such as mobile applications or mini-programs), submitting personal information and consultation materials. This includes filling in basic information (name, age, main requests, etc.) and uploading images such as a front-facing facial photo. The system receives this data through the customer data collection module and retrieves the customer's historical data (such as past medical records, membership levels, etc.) from the backend through the business system interface module, integrating it to form a complete customer profile for subsequent steps.

[0037] Step 2: Image Analysis and Processing. The system sends the acquired customer facial photos to the image analysis module for processing. First, facial feature point localization and measurement are performed to extract key facial parameters; then, skin condition analysis is conducted to detect skin problems such as blemishes, wrinkles, and pores; finally, a 3D model of the customer's face is generated, and the appearance changes under different schemes are simulated. After the analysis is completed, the module outputs information including customer facial feature data, a skin problem report, and a comparison image after simulation.

[0038] Step 3: Knowledge Retrieval and Preparation. Before generating a consultation plan, the system calls the knowledge base retrieval module to perform a knowledge retrieval based on the customer's keywords. For example, if a customer inquires about "what are the options and effects of rhinoplasty," the module will search the knowledge base for relevant surgical methods, material types, case results, etc. The retrieved results (such as explanations of the principles of rhinoplasty, comparisons of the advantages and disadvantages of different procedures, and images of successful cases) will be organized and used as background information, which will be incorporated into the prompts generated by the subsequent large-scale model. At the same time, if the business system has any special notes related to the customer (such as an allergy to a certain drug), these will also be obtained through the interface and prompted to be considered by the model.

[0039] Step 4: Multimodal Dialogue Solution Generation. The system inputs the image analysis results (output from Step 2) and the organized knowledge retrieval results (output from Step 3), along with the customer's original question, into the multimodal large language model. The solution generation module guides the model to comprehensively analyze the customer's basic facial features and needs through customized prompts, and provides a personalized aesthetic consultation solution based on professional knowledge. The model's output includes: answers to the customer's needs, a list of recommended treatment / beautification projects, the expected effects and principles of each project, the corresponding implementation steps, and precautions. To improve the credibility of the results, the output cites evidence provided by the knowledge base (e.g., "Based on past cases at the hospital, the [Case Library] shows that this solution resulted in an average increase of approximately 2mm in nasal bridge height after surgery."). If the model determines that there are multiple options (such as different combinations of projects), it will also explain each option for the customer to weigh.

[0040] Step 5: Multimodal Presentation of Solution Results. While the solution generation module outputs the text solution, the aesthetic design recommendation module synthesizes a post-operative effect simulation image on the customer's 3D facial model based on the solution, and compares it with real before-and-after photos from similar cases. This visual aid information, along with the text solution, is displayed to the customer through the front-end interface. The customer can see the simulated changes after undergoing the recommended procedure, such as a preview of the effect of nasal bridge augmentation, thus gaining a more intuitive understanding of the expected results of the solution. Furthermore, the system interface provides interactive tools that allow customers to adjust certain solution parameters (such as rhinoplasty height, chin length, etc.) and instantly view the impact of different adjustments on the effect and price, enhancing their sense of participation and decision-making confidence.

[0041] Step 6: Price Calculation and Recommendation. Once the plan is finalized, the system enters the price management module to calculate the cost. This module retrieves the price entries corresponding to the items included in the plan from the backend, calculates the total price, and considers customer membership discounts or current promotional offers. Additionally, if the hospital offers installment payment plans, package deals, or other preferential policies, the system will reflect these in the price quote. The quote result is presented in list format, showing the price, subtotal, and total for each service, along with explanations of any discounts. The system will also intelligently recommend adjustments to the plan based on the customer's budget, such as suggesting, "If you wish to reduce the total budget, consider prioritizing item A and postponing item B," thus increasing the flexibility of the plan.

[0042] Step 7: Decision Support and Order Placement. The system presents the comprehensive solution (text description + simulation diagram) and quotation to the customer, and provides an online communication channel for them to ask questions. If the customer has any questions about a part of the solution, they can ask them in the chat interface, and the system will provide answers or further explanations in real time using a large language model. For example, if a customer asks, "Why are you recommending laser treatment instead of chemical peels?" the system will provide a professional explanation based on its knowledge base and the preceding context. After thorough communication, the customer can directly book the selected service in the system and generate an electronic order. The booking information is sent to the hospital's appointment management module through the business interface to arrange subsequent hospital services.

[0043] Step 8: Post-operative Follow-up Service. After the customer completes in-person treatment, the post-operative care module automatically initiates the corresponding service process. The system proactively sends post-operative precautions and rehabilitation guidance information to the customer's mobile phone and inquires about their feelings in the chat window. If the customer reports pain or anxiety, the system will provide reassurance and advice based on its knowledge base and psychological care dialogue templates. During the recovery period, the customer can also ask the system questions at any time, such as "Is it normal for the wound to be a little red?" The system will immediately answer based on its built-in knowledge or suggest a follow-up medical examination. Throughout the post-operative period, the system continuously collects and records the customer's recovery data and satisfaction feedback. This data can be used by the hospital to analyze service quality, improve treatment plans and strategies, and form a closed-loop optimization process.

[0044] In this embodiment, the intelligent system provided by this invention achieves intelligent management throughout the entire process from consultation to design, decision-making, treatment, and follow-up. The system not only improves consultation efficiency but also provides customers with a highly personalized and continuous service experience, addressing the pain points of strong reliance on specialized expertise and information fragmentation inherent in traditional models. Applications of multiple embodiments demonstrate that the system of this invention can significantly improve conversion rates and customer satisfaction, creating new service value and competitive advantages for medical aesthetic institutions.

[0045] In addition to the above, it should be noted that the terms "one embodiment," "another embodiment," and "embodiment" used in this specification refer to specific features, structures, or characteristics described in connection with that embodiment, which are included in at least one embodiment described in the general description of this application. The appearance of the same expression in multiple places in the specification does not necessarily refer to the same embodiment. Furthermore, when a specific feature, structure, or characteristic is described in connection with any embodiment, the intention is to suggest that implementing such a feature, structure, or characteristic in conjunction with other embodiments also falls within the scope of this invention.

[0046] Although the invention has been described herein with reference to several illustrative embodiments, it should be understood that many other modifications and implementations can be devised by those skilled in the art, which will fall within the scope and spirit of the principles disclosed herein. More specifically, various variations and modifications can be made to the components and / or layout of the subject matter arrangement within the scope of the disclosure, drawings, and claims. Besides variations and modifications to the components and / or layout, other uses will be apparent to those skilled in the art.

Claims

1. A medical and aesthetic consultation intelligent system based on RAG and a multi-modal large language model, characterized by The system includes: The customer data acquisition module is used to acquire customer data from the client and transmit it to the image analysis module. The customer data includes the data currently input by the customer and the historical information of the current customer. The data input by the customer includes at least a facial photo, name, age, and request. The image analysis module is used to extract and analyze key facial parameters based on customer data, and generate image analysis results including a 3D model of the customer's face and a rendering of the appearance. The knowledge base retrieval module is used to perform knowledge retrieval based on the keywords of customer requests in the customer data. The knowledge retrieval involves searching for information related to the current keyword of request in the medical aesthetics professional knowledge base, and then embedding the information into the input information of the large language model. The solution generation module is used to generate an aesthetic consultation solution and output it to the client based on the image analysis results, customer needs, and the aforementioned information, using a large language model. The business system interface module is used to provide data from the back-end business system to the customer data collection module via an API interface, as historical data of the current customer. The business interface module is also used to receive the aesthetic consultation plan and transmit it to the back-end business system.

2. The RAG and multi-modal large language model-based medical aesthetic consultation intelligent system according to claim 1, characterized in that The system also includes: The aesthetic design recommendation module is used to output facial effect images under different schemes based on the client's three-dimensional facial model according to the aesthetic consultation plan, and to give the current client's face score and improvement suggestions through the aesthetic evaluation algorithm, and output them to the client along with the aesthetic consultation plan.

3. The RAG and multi-modal large language model-based medical aesthetic consultation intelligent system according to claim 1 or 2, characterized in that The system also includes: The quotation management module is used to calculate and generate quotation information based on the corresponding projects in the aesthetic consultation plan. The quotation information is generated based on the project price list and consumable inventory in the back-end business system and is output to the client along with the aesthetic consultation plan.

4. The RAG and multi-modal large language model-based medical aesthetic consultation intelligent system according to claim 3, characterized in that The system also includes: The post-operative care module is used to provide continuous guidance and service information to clients who have completed transactions, according to the project's preset cycle, through a large language model, and to collect client feedback. The postoperative care module is also used to transmit the feedback to the backend business system through the API interface of the business system interface module.

5. The RAG and multi-modal large language model-based medical aesthetic consultation intelligent system according to claim 1 or 4, characterized in that: The aesthetic consultation plan includes aesthetic design suggestions, recommended specific medical aesthetic projects and products, implementation steps, and expected results.

6. The RAG and multi-modal large language model-based medical aesthetic consultation intelligent system according to claim 1 or 4, characterized in that: The medical aesthetics knowledge base is updated in real time through a back-end management system.

7. The RAG and multi-modal large language model-based medical aesthetic consultation intelligent system according to claim 1 or 4, characterized in that: The back-end business system includes any one of the following: medical aesthetics business management system, membership management system, electronic medical record system, and marketing system.

8. The RAG and multi-modal large language model-based medical aesthetic consultation intelligent system of claim 1 or 4, characterized in that: The solution generation module is also used to answer or supplement the client's questions regarding the aesthetic consultation solution and the current client's facial score and improvement suggestions through a large language model.

9. A control method of the system according to any one of claims 1 to 8, characterized by The method includes the following steps: The customer data acquisition module obtains customer data from the client and transmits it to the image analysis module. The customer data includes the data currently input by the customer and the historical information of the current customer. The data input by the customer includes at least a facial photo, name, age, and request. The historical information of the current customer is data from the backend business system provided by the business system interface module to the customer data acquisition module through the API interface. The image analysis module extracts and analyzes key facial parameters based on customer data, generating image analysis results including a 3D model of the customer's face and a rendering of the appearance. The knowledge base retrieval module performs knowledge retrieval based on the keywords of customer requests in the customer data. The knowledge retrieval involves searching for information related to the current keyword of request in the medical aesthetics professional knowledge base, and then embedding the information into the input information of the large language model. Based on the image analysis results, customer needs, and the aforementioned information, the solution generation module generates an aesthetic consultation solution using a large language model and outputs it to the client. The business interface module receives the aesthetic consultation plan and transmits it to the back-end business system.

10. The control method according to claim 1, characterized in that... The method further includes: Based on the aesthetic consultation plan, the aesthetic design recommendation module outputs facial effect images under different plans on the basis of the client's three-dimensional facial model. It also provides a score and improvement suggestions for the current client's face through the aesthetic evaluation algorithm, and outputs them to the client along with the aesthetic consultation plan. The quotation management module calculates and generates quotation information based on the corresponding items in the aesthetic consultation plan; the quotation information is generated based on the project price list and consumable inventory in the back-end business system, and is output to the client along with the aesthetic consultation plan; The postoperative care module uses a large language model to provide continuous guidance and service information to clients who have completed transactions, according to the project's preset cycle, and collects client feedback. The postoperative care module transmits the feedback to the backend business system through the API interface of the business system interface module; The solution generation module uses a large language model to answer or supplement the client's questions regarding the aesthetic consultation solution and the current client's facial score and improvement suggestions.

Images