AI intelligent assistant system supporting multi-modal input and personalized service

Abstract

The invention relates to the technical field of AI, in particular to an AI intelligent assistant system supporting multi-modal input and personalized service. Specifically, an AI intelligent assistant system supporting multi-modal input and personalized service uses a large language model to carry out semantic analysis and intention recognition. According to the personalized recommendation method, factors such as personal preference, budget and purposes of vehicle owners are combined, and multiple types of vehicle models meeting conditions are screened out from a vehicle database. The invention relates to a method for performing estimation calculation by using a machine learning model based on a random forest algorithm. And generating a maintenance suggestion by combining a vehicle maintenance suggestion model according to the driving record and the maintenance record of the vehicle owner. And recording personal information, preferences and behavioral habits of the user, and generating a user portrait. And recording information such as brand, model, mileage, maintenance record and the like of the vehicle, and generating a vehicle portrait. And dynamically adjusting the recommendation strategy and the service content according to the portraits of the user and the vehicle.

Description

Technical Field

[0001] This application relates to the field of AI technology, and particularly to an AI intelligent assistant system that supports multi-modal input and personalized services. Background Art

[0002] With the development of social economy, cars have gradually become an indispensable means of transportation in people's daily lives. However, with the continuous increase in the number of car owners, the demand for vehicle management and related services by car owners is increasingly showing a trend of diversification and personalization. Traditional vehicle management systems often have relatively single functions and can only meet some basic needs such as basic vehicle information recording and simple maintenance reminders, and it is difficult to adapt to the more complex and personalized service requirements of car owners in different scenarios.

[0003] In the field of intelligent assistants, although with the development of artificial intelligence technology in recent years, a series of intelligent assistant products applied to vehicle management and service scenarios have emerged in the market. However, these existing intelligent assistants have many limitations. On the one hand, they are relatively single in terms of data sources. For example, when making new car recommendations, they often only make recommendations based on limited new car data, lacking the construction and analysis of the car owner's personal profile, and unable to deeply understand the personalized preferences of car owners, resulting in inaccurate recommendation results; in used car valuation, they fail to fully combine market dynamics and the detailed conditions of the vehicle for comprehensive analysis, greatly reducing the accuracy of valuation results; in maintenance renewal services, they do not comprehensively track the usage and maintenance history of the vehicle and cannot provide timely and personalized maintenance suggestions; in the recommendation of driving knowledge, due to the lack of in-depth analysis of the car owner's driving habits and vehicle conditions, the recommended content lacks practicality. Summary of the Invention

[0004] In view of this, the embodiments of this application are committed to providing an AI intelligent assistant system that supports multi-modal input and personalized services.

[0005] This application provides an AI intelligent assistant system that supports multi-modal input and personalized services, including: a multi-modal input-output module, which is used to receive the interaction information input by the car owner through a variety of preset input methods; and present the response information corresponding to the interaction information to the car owner in the form of text, voice, image or web page; the information includes questions or requests; the variety of preset input methods includes: text, voice, image;

[0006] A human body sensing module, which is used to obtain the GPS location information of the car owner's mobile terminal;

[0007] A personalized enhancement module for obtaining the owner's intention; dynamically adjusting the recommendation strategy and service content based on the information provided by the human body sensing module, the owner's intention, the owner portrait, and the vehicle portrait data, where the owner portrait includes the owner's age, gender, occupation, brand, model of the vehicle already owned, and daily driving habits; the vehicle portrait includes the brand, model, mileage, and maintenance records;

[0008] An owner demand processing module for semantically understanding and sentiment analyzing the interaction information in combination with the owner portrait and vehicle portrait data, and generating corresponding response information based on the support of a preset service module; and determining the owner's intention based on the interaction information;

[0009] The preset service module includes: a new car recommendation module, a used car valuation module, a maintenance appointment module, and a vehicle usage knowledge module;

[0010] The new car recommendation module is used to receive the refined screening conditions from the large language model, adopt a hybrid recommendation method based on rules, collaborative filtering, and content-based filtering to screen out multiple vehicles that meet the user's needs from the vehicle database for the user to compare and view, display detailed vehicle model information in various forms, and the vehicle model details page supports the user to make an online test drive appointment;

[0011] The used car valuation module is used to calculate the valuation using a machine learning model based on the random forest algorithm according to the owner's city, vehicle brand, model, year, new car price, mileage, and maintenance records, and generate a detailed valuation report;

[0012] The maintenance appointment module is used to provide a list of service outlets for the owner to choose according to the text or voice description input by the owner, automatically send a reservation confirmation text message after the owner determines the service outlet and time, support the owner to upload pictures or videos for the service outlet to confirm the problem in advance, and can also actively remind the owner to perform maintenance according to the owner's driving record and maintenance record in combination with the vehicle maintenance advice model;

[0013] The vehicle usage knowledge module is used to provide a detailed vehicle user manual according to the owner's vehicle model information, and accelerate the full-text retrieval through a distributed search and analysis engine, and support the retrieval-enhanced generation method to obtain vehicle usage and maintenance knowledge;

[0014] A big data analysis module for processing the data of the owner and the vehicle using big data processing technology and machine learning technology, including data cleaning, integration, feature extraction, and update, to generate an owner portrait and a vehicle portrait.

[0015] In some embodiments, when the multi-modal input-output module outputs the response information, it uses the Volcano Engine text-to-speech API to convert the output text into audio and provides page display for the user in the form of an HTML page.

[0016] In some embodiments, the personalized enhancement module updates the vehicle owner profile and vehicle profile data.

[0017] In some embodiments, the maintenance appointment module is configured to automatically send a reminder text message for the appointment to the vehicle owner one day before the appointment expires.

[0018] In some embodiments, the big data analysis module uses Apache Spark to clean and integrate data from different sources.

[0019] In some embodiments, the multi-modal input / output module is further configured to provide an information input interface for the vehicle owner to fill in basic information when logging in for the first time.

[0020] In some embodiments, the multi-modal input / output module is further configured to use a real-time noise reduction library to perform noise reduction processing on the collected audio, improve the speech recognition accuracy when inputting speech, convert the speech into text through a speech recognition model, and perform intent recognition and keyword extraction on the text.

[0021] In some embodiments, the multi-modal input / output module is further configured to obtain vehicle photos taken by a camera and use a computer vision model to identify vehicle information and fault locations.

[0022] The AI intelligent assistant system provided by this application that supports multi-modal input and personalized services has a richer data source. For example, when making new car recommendations, it collects various information, constructs a vehicle owner profile and a vehicle profile, and combines multi-party data for new car recommendations. In terms of used car valuation, it can fully combine market dynamics and the detailed condition of the vehicle for comprehensive analysis to improve the accuracy of the valuation result; in the maintenance renewal service, it comprehensively tracks the usage and maintenance history of the vehicle and provides timely and personalized maintenance suggestions; in terms of driving knowledge recommendation, it deeply analyzes the driving habits of the vehicle owner and the condition of the vehicle, and the recommended content is more targeted. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By describing the embodiments of the present application in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present application will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. They are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 is a schematic diagram of an AI intelligent assistant system that supports multi-modal input and personalized services provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0026] Figure 1 It is a schematic diagram of an AI intelligent assistant system that supports multimodal input and personalized services provided by an embodiment of the present application. As Figure 1 shown, the method includes the following content.

[0027] The multimodal input-output module is used to receive the interaction information input by the vehicle owner through a variety of preset input methods; present the response information corresponding to the interaction information to the vehicle owner in the form of text, voice, image or web page; the information includes questions or requests; the variety of preset input methods include: text, voice, image;

[0028] The human body sensing module is used to obtain the GPS location information of the vehicle owner's mobile terminal;

[0029] The personalized enhancement module is used to obtain the vehicle owner's intention; based on the information provided by the human body sensing module, the vehicle owner's intention, the vehicle owner portrait and the vehicle portrait data, dynamically adjust the recommendation strategy and service content, where the vehicle owner portrait includes the vehicle owner's age, gender, occupation, brand, model of the vehicle already owned, and daily driving habits; the vehicle portrait includes the brand, model, mileage, and maintenance records;

[0030] The vehicle owner demand processing module is used to combine the vehicle owner portrait and vehicle portrait data, perform semantic understanding and sentiment analysis on the interaction information, and generate corresponding response information based on the support of the preset service module; and determine the vehicle owner's intention based on the interaction information;

[0031] The preset service module includes: a new car recommendation module, a used car valuation module, a maintenance appointment module, and a vehicle use knowledge module;

[0032] The new car recommendation module is used to receive the refined screening conditions of the large language model, and use a hybrid recommendation method based on rules, collaborative filtering and content-based to screen out multiple vehicles that meet the user's needs from the vehicle database for the user to compare and view, and display the detailed vehicle model information in various forms, and the vehicle model details page supports the user to make an online test drive appointment;

[0033] The used car valuation module is used to calculate the valuation according to the city where the vehicle owner is located, the brand, model, year, new car price, mileage and maintenance records of the vehicle, and generate a detailed valuation report using a machine learning model based on the random forest algorithm;

[0034] The maintenance reservation module is used to provide a list of service outlets for the vehicle owner to choose according to the text or voice description input by the vehicle owner. After the vehicle owner determines the service outlet and time, it automatically sends a reservation confirmation text message. At the same time, it supports the vehicle owner to upload pictures or videos so that the service outlet can confirm the problem in advance. It can also actively remind the vehicle owner to perform maintenance according to the vehicle owner's driving record and maintenance record in combination with the vehicle maintenance suggestion model;

[0035] The vehicle usage knowledge module is used to provide a detailed vehicle user manual according to the vehicle model information of the vehicle owner, and accelerate the full-text retrieval through a distributed search and analysis engine, and support retrieving enhanced generation methods to obtain vehicle usage and maintenance knowledge;

[0036] The big data analysis module is used to process the data of vehicle owners and vehicles using big data processing technology and machine learning technology, including data cleaning, integration, feature extraction and update, and generate vehicle owner portraits and vehicle portraits.

[0037] Specifically, the multi-modal input-output module is responsible for receiving and processing various input methods (text, voice, image, etc.) of the vehicle owner, and presenting the information generated by the system to the vehicle owner in various forms (text, voice, image, web page, etc.). This module improves the convenience and flexibility of user interaction through various input methods, and at the same time enhances the richness and comprehensibility of information through various output methods.

[0038] The vehicle owner enters a question or requirement through a text box, and the system uses a large language model for semantic analysis and intent recognition, extracting keywords and intents.

[0039] The vehicle owner inputs voice through a microphone. The system first uses a real-time noise reduction library (RNNoise) to perform noise reduction processing on the audio to improve the accuracy of speech recognition, and then converts the voice into text through a speech recognition model (Whisper), and performs subsequent intent recognition and keyword extraction.

[0040] The vehicle owner takes a photo of the vehicle through a camera. The system uses a computer vision model (YOLO) to identify vehicle information and fault locations. After obtaining the recognition result, it performs subsequent processing through a large language model.

[0041] The system performs multi-modal output for users according to different scenarios, including text, voice, image, web page, etc. It uses the Volcano Engine Text-to-Speech (TTS) API to convert the output text into audio; it returns a maintenance flow chart and other images for the user; it uses the HTML page method to provide a rich page display for the user, such as the vehicle details page for new car recommendations.

[0042] The system uses the Transport Layer Security (TLS) protocol to encrypt the communication between users and the server, ensuring that data is not stolen or tampered with during transmission.

[0043] According to laws, regulations, and business requirements, the system periodically deletes expired session records. For data that needs to be stored long-term, the system uses an archival storage solution to ensure data security and recoverability.

[0044] The system provides a user-friendly interface that supports access from multiple devices (such as smartphones, tablets, in-vehicle screens, etc.), making it convenient for users with different needs to use it friendly.

[0045] Human body sensing module:

[0046] The car owner can authorize the acquisition of the mobile GPS location to obtain the car owner's location information in real time. The system can dynamically adjust the recommendation strategy and service content based on the real-time location.

[0047] Personalization enhancement module:

[0048] Filter appropriate car owner portrait and vehicle portrait data based on the current car owner and the car owner's intention judged by the large language model, and dynamically adjust the recommendation strategy and service content.

[0049] After the large language model obtains the car owner's intention, it passes the car owner's intention to the personalization enhancement module.

[0050] The personalization enhancement module receives the car owner's intention and selects existing portrait data according to the car owner's intention and returns it to the large language model. For example, for new car recommendations, it returns the car owner portrait, which includes key information such as the car owner's age, gender, occupation, brand, model of the cars already owned, and daily driving habits; for used car valuation, it returns the basic vehicle portrait data, which includes key information such as the brand, model, mileage, and maintenance records.

[0051] When the large language model refines subsequent business parameters, it performs personalization enhancement on the context based on the obtained portrait data.

[0052] When the large language model refines business parameters, it will synchronously extract the car owner's basic information and vehicle basic information from the context information. If the car owner's basic information or vehicle basic information is extracted, it will use an asynchronous method to pass the car owner's basic information or vehicle basic information to the personalization enhancement module.

[0053] After receiving the car owner's basic information or vehicle basic information, the personalization enhancement module performs an update operation on the car owner's basic information portrait or vehicle basic information portrait.

[0054] Large language model car owner demand understanding module:

[0055] Use a large language model to perform semantic understanding and sentiment analysis on the owner's input, and generate the most suitable response for the owner by combining the context and personalized enhanced data.

[0056] First, use a large language model to judge the owner's intention based on the context, and then continue to use the large language model to refine subsequent business parameters from the context and personalized enhanced data according to the intention or directly return the user response.

[0057] New car recommendation module:

[0058] Receive the refined screening conditions from the large language model, and use a hybrid recommendation method to screen out multiple vehicles that meet the user's needs from the vehicle database for the user to compare and view.

[0059] The recommendation methods include the following: The first is rule-based recommendation, such as when the user specifies a budget, brand, model, etc.; the second is collaborative filtering recommendation, which recommends models that other similar users have purchased or liked by finding other users with similar interests and behaviors to the current user; the third is content-based recommendation, which matches based on the preference features in the user profile (such as whether they like electric, gasoline, SUV, sedan, etc.) and the attributes of each vehicle in the vehicle model library.

[0060] Display detailed vehicle model information (including configuration, price, performance), vehicle pictures, vehicle evaluations, etc. for the user in various forms such as text, voice, and image. The user interface is simple and intuitive, facilitating the user to view and compare.

[0061] At the same time, the vehicle model details page supports users to make an online test drive appointment. The system will recommend the relevant dealership closest to the user's location based on the vehicle model selected by the user. After the user selects the dealership, the system automatically sends a reservation confirmation message.

[0062] Used car valuation module:

[0063] The system calculates the valuation based on the owner's city, vehicle brand, model, year, new car price, mileage, repair records, etc., using a machine learning model based on the random forest algorithm to generate a detailed valuation report. It includes the market value of the vehicle and the basis for valuation, helping users understand the actual situation of the vehicle.

[0064] If the owner has bound the vehicle information when using the used car valuation function, the owner can obtain a more accurate valuation report without entering the relevant vehicle information.

[0065] Maintenance and appointment module:

[0066] This module only requires the car owner to input a text or voice description of the maintenance appointment, and the system will provide a list of the nearest service outlets for the car owner to choose based on the car owner's location (the car owner can also specify by the outlet name directly). After determining the service outlet and a suitable time, the system will automatically send a reservation confirmation text message, and the system will also automatically send a reminder text message for the appointment to the store one day before the appointment expires.

[0067] The maintenance appointment also supports the car owner to upload pictures or videos so that the service outlet can confirm the problem in advance and perform operations such as spare parts.

[0068] The system supports the car owner to view and manage existing maintenance appointment records via text or voice.

[0069] The system will also combine the car owner's driving records and maintenance records with the vehicle maintenance advice model to judge whether the car owner needs vehicle maintenance. If so, it will remind the car owner in the form of an in-system message or by actively responding when the car owner mentions maintenance, so as to prevent the car owner from failing to perform vehicle maintenance in a timely manner.

[0070] The system will also convert the results of the driving habit advice model into natural language using a large language model and remind the car owner to drive safely in the form of an in-system message.

[0071] Except for uploading pictures and videos, all operations in this module can be completed by voice input, greatly simplifying the reservation process.

[0072] Vehicle usage knowledge module:

[0073] According to the car owner's vehicle model information, provide a detailed vehicle user manual to guide the car owner in daily use and maintenance.

[0074] The system stores the vehicle user manual in a hierarchical manner in a document-based database (MongoDB) according to different chapter entries. According to the car owner's vehicle model information or the vehicle model information specified in the context, a distributed search and analysis engine (Elasticsearch) is used as the search engine to accelerate full-text retrieval and provide the car owner with a fast and accurate vehicle usage knowledge Q&A experience.

[0075] The system also supports obtaining knowledge about vehicle use and maintenance through the Retrieval-Augmented Generation (RAG) method.

[0076] Big data analysis module:

[0077] Use big data processing technologies (Hadoop, Spark) and machine learning technologies to process the data of car owners and vehicles. To ensure the accuracy and real-time nature of data analysis, we have adopted a complete set of data cleaning, integration, feature extraction and update mechanisms.

[0078] Clean and integrate data from different sources using Apache Spark. The key steps of data cleaning include:

[0079] Duplicate removal: Identify and remove duplicate data records to ensure data uniqueness.

[0080] Missing value filling: For missing data fields, adopt reasonable filling strategies (such as mean filling, interpolation, etc.) to complete them.

[0081] Data standardization: Standardize data from different sources to make the data formats unified, facilitating subsequent analysis and processing.

[0082] Extract features and conduct analysis on the cleaned and integrated data:

[0083] Car owner interest preference features: Use natural language processing technology (Bidirectional Encoder Representations from Transformers, BERT) to deeply analyze the text data (historical conversation records) of car owners, and extract valuable feature information from it, including but not limited to interests and needs.

[0084] Vehicle maintenance features: Use the Python language to analyze the maintenance manuals of each vehicle, and finally form vehicle maintenance features in cooperation with the vehicle maintenance suggestions provided by cooperative customers.

[0085] Vehicle driving behavior features: Use Apache Spark to analyze the vehicle driving records provided by the cooperative party to obtain features such as vehicle driving mileage, driving frequency, driving time distribution, mileage per trip, number of hard accelerations per trip, number of hard decelerations per trip, and average driving speed.

[0086] Vehicle maintenance record features: Use Apache Spark to analyze the vehicle maintenance records provided by the cooperative party to obtain features such as the date of the last maintenance, maintenance content, and maintenance frequency of the vehicle.

[0087] Used car transaction features: Use Apache Spark to analyze the used car transaction data provided by the cooperative party and publicly available online to obtain features such as used car brand, model, city, initial registration date, driving mileage, repair records, and search popularity.

[0088] Generate portraits and various machine learning models and update them regularly:

[0089] Car owner portrait: Generate a car owner portrait based on the basic information provided by the car owner and the extracted car owner interest preference features. Among them, the basic information part is updated when the car owner modifies it, and the car owner interest preferences are updated monthly regularly.

[0090] Vehicle Portrait: Generate a vehicle portrait based on the vehicle information bound by the vehicle owner, the vehicle model information publicly available on the Internet, as well as the vehicle driving behavior characteristics and vehicle maintenance record characteristics. The basic vehicle information remains unchanged, and the driving behavior characteristics and maintenance record characteristics (updated only when maintenance occurs) are updated regularly on a weekly basis.

[0091] Driving Habit Suggestion Model: Provide customized suggestions for vehicle owners in each category according to the vehicle driving behavior characteristics using the clustering algorithm (K-means), and update it regularly on a monthly basis.

[0092] Vehicle Maintenance Suggestion Model: Train the vehicle maintenance suggestion model according to the vehicle maintenance characteristics and vehicle driving behavior characteristics using the classification model (random forest), and update the model once every six months.

[0093] In some embodiments, predefined rules and logics can be used to process user requests instead of relying on complex machine learning models, which is applicable to application scenarios with clear and less-changing requirements. The advantages are simple logic and fast response speed. The disadvantages are that it is difficult to handle complex and diverse user requests, and as the requirements increase, the rules will become more and more complex, making it difficult to manage and maintain.

[0094] In summary, in the solution provided by this application, there are methods of using large language models for semantic analysis and intent recognition. Methods of converting speech to text by combining noise reduction algorithms and ASR technology and processing it through large language models. Methods of using computer vision technology to identify vehicle information and fault locations and generating repair suggestions through large language models. Methods of selecting appropriate output forms (text, speech, image) according to the content generated by large language models. Methods of using large language models for semantic understanding and screening out multiple eligible vehicle models from the vehicle database based on factors such as the vehicle owner's personal preferences, budget, and usage for personalized recommendation. Methods of using machine learning models based on random forest algorithms for valuation calculation. Methods of generating maintenance suggestions according to the vehicle owner's driving records and maintenance records in combination with the vehicle maintenance suggestion model. Methods of recording the user's personal information, preferences, and behavior habits to generate a user portrait. Methods of recording information such as the vehicle's brand, model, mileage, and repair records to generate a vehicle portrait. Methods of dynamically adjusting recommendation strategies and service contents according to the portraits of users and vehicles. With such settings, through multi-modal input and personalized services, the usage experience and satisfaction of vehicle owners are improved. Through big data analysis and machine learning technologies, intelligent management and services of vehicles are realized. Through online appointment and automatic reminder, the vehicle owner's maintenance process is simplified and service efficiency is improved. Through accurate new car recommendation and used car valuation, the market competitiveness of the system is enhanced.

[0095] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the form disclosed herein. Although numerous example aspects and embodiments have been discussed above, those skilled in the art will recognize some of their variations, modifications, alterations, additions, and subcombinations.

Claims

1. An AI intelligent assistant system supporting multimodal input and personalized services, characterized in that: include: A multi-modal input and output module, used to receive interactive information input by the car owner through multiple preset input methods; Presenting response information corresponding to the interactive information to the vehicle owner in the form of text, voice, image or web page; The information includes a question or request; The multiple preset input methods include: text, voice, and image; Human body sensing module, used to obtain the GPS location information of the owner's mobile terminal; A personalized enhancement module is used to obtain the intention of the owner; based on the information provided by the human body sensing module and the intention of the owner, the owner's portrait and the vehicle portrait data, dynamically adjust the recommendation strategy and service content, wherein the owner's portrait includes the owner's age, gender, occupation, brand and model of the car already owned and daily driving habits; the vehicle portrait includes the brand, model, mileage and maintenance record; The vehicle owner demand processing module is used to combine the vehicle owner portrait and vehicle portrait data, perform semantic understanding and sentiment analysis on the interaction information, and generate corresponding response information based on the support of the preset service module; and determine the vehicle owner's intention based on the interaction information; The preset service modules include: new car recommendation module, used car valuation module, maintenance reservation module, and car knowledge module; The new car recommendation module is used to receive the screening conditions refined by the large language model, and adopts a hybrid recommendation method based on rules, collaborative filtering and content to screen multiple cars that meet the user's needs from the vehicle database for the user to compare and view, and displays detailed model information in various forms. The model details page supports users to make online reservations for test drives; The used car valuation module is used to generate a detailed valuation report by using a machine learning model based on a random forest algorithm to perform valuation calculations based on the city where the car owner is located, the brand, model, year, new car price, mileage and maintenance record of the car; The maintenance reservation module is used to provide a list of service outlets for the car owner to choose from based on the text or voice description entered by the car owner, and automatically send a reservation confirmation SMS after the car owner confirms the service outlet and time. It also supports the car owner to upload pictures or videos so that the service outlet can confirm the problem in advance, and can also actively remind the car owner to perform maintenance based on the car owner's driving record and maintenance record combined with the vehicle maintenance suggestion model; The vehicle knowledge module is used to provide detailed vehicle manuals based on the vehicle model information of the owner, accelerate full-text retrieval through distributed search and analysis engines, and support retrieval enhancement generation to obtain vehicle use and maintenance knowledge; The big data analysis module is used to process the data of car owners and vehicles using big data processing technology and machine learning technology, including data cleaning, integration, feature extraction and updating, and generate owner portraits and vehicle portraits.

2. The AI ​​intelligent assistant system supporting multimodal input and personalized services according to claim 1, characterized in that: When outputting response information, the multimodal input and output module uses the Volcano Engine text-to-speech API to convert the output text into audio, and uses an HTML page to provide a page display to the user.

3. The AI ​​intelligent assistant system supporting multimodal input and personalized services according to claim 1, characterized in that: The personalized enhancement module updates the owner portrait and vehicle portrait data.

4. The AI ​​intelligent assistant system supporting multimodal input and personalized services according to claim 1, characterized in that: The maintenance reservation module is used to automatically send a reminder text message of appointment to the car owner one day before the appointment expires.

5. The AI ​​intelligent assistant system supporting multimodal input and personalized services according to claim 1, characterized in that: The big data analysis module uses Apache Spark to clean and integrate data from different sources.

6. The AI ​​intelligent assistant system supporting multimodal input and personalized services according to claim 1, characterized in that: The multimodal input and output module is also used to provide an information input interface for the car owner to fill in basic information when logging in for the first time.

7. The AI ​​intelligent assistant system supporting multimodal input and personalized services according to claim 1, characterized in that: The multimodal input and output module is also used to use a real-time noise reduction library to perform noise reduction on the collected audio, thereby improving the accuracy of speech recognition during voice input, and then converting the speech into text through a speech recognition model, and performing intent recognition and keyword extraction on the text.

8. The AI ​​intelligent assistant system supporting multimodal input and personalized services according to claim 1, characterized in that: The multimodal input and output module is also used to obtain vehicle photos taken by a camera and use a computer vision model to identify vehicle information and fault locations.

Images