Intelligent management method and system for message notification of intelligent vehicle

Abstract

The invention is suitable for the technical field of vehicle notification management, and provides an intelligent management method and system for message notification of an intelligent vehicle. The method comprises the steps of collecting user interaction data in a vehicle; performing data processing and multi-modal fusion on the user interaction data to generate multi-modal data; based on a preset multi-modal large model, identifying and analyzing the multi-modal data, and judging whether a notification needs to be sent or not in combination with historical notification response; when the notification needs to be sent, determining a notification mode and notification strength, and performing message notification to the user according to the notification mode and the notification strength; and judging whether the user has a notification response or not, and performing message grading and dynamic adjustment processing when the user has the notification response. According to the technical scheme, whether notification is needed or not and which mode is used for notification can be comprehensively judged, the notification strategy is dynamically adjusted according to the response and feedback of the user, and therefore unnecessary information interference is reduced, and meanwhile it is ensured that key information can be effectively conveyed.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle notification management, and in particular relates to a message notification intelligent management method and system for intelligent vehicles. Background Art

[0002] Smart vehicles need to process a large amount of information during driving and use, including vehicle status updates, navigation prompts, safety warnings, etc.

[0003] In the prior art, it is impossible to effectively distinguish the importance and urgency of information, which may cause the driver to miss key prompts or be distracted by unimportant information. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method and system for intelligent management of message notifications of intelligent vehicles, aiming to solve the technical problems existing in the prior art mentioned in the background technology.

[0005] The embodiment of the present invention is implemented as follows:

[0006] A method for intelligent management of message notifications of an intelligent vehicle, the method specifically comprising the following steps:

[0007] Collecting user interaction data in the vehicle;

[0008] Performing data processing and multimodal fusion on the user interaction data to generate multimodal data;

[0009] Based on a preset multimodal macro model, the multimodal data is identified and analyzed, and combined with historical notification responses, it is determined whether a notification needs to be sent;

[0010] When a notification needs to be sent, determine a notification method and a notification strength, and send a message notification to the user according to the notification method and the notification strength;

[0011] Determine whether the user has a notification response, and if so, perform message classification and dynamic adjustment processing.

[0012] As a further limitation of the technical solution of the embodiment of the present invention, the user interaction data includes vehicle sensor data, camera data and / or microphone data.

[0013] As a further limitation of the technical solution of the embodiment of the present invention, the step of performing data processing and multimodal fusion on the user interaction data to generate multimodal data specifically includes the following steps:

[0014] Performing data preprocessing on the user interaction data to generate preprocessed data;

[0015] Performing targeted processing of images, voices, texts and physiological signals on the pre-processed data to generate a plurality of targeted processed data;

[0016] Perform multimodal fusion on the plurality of processed data to generate multimodal data.

[0017] As a further limitation of the technical solution of the embodiment of the present invention, the data preprocessing of the user interaction data includes: data cleaning, standardization and enhancement.

[0018] As a further limitation of the technical solution of the embodiment of the present invention, the performing multimodal fusion on the plurality of the processed data to generate multimodal data specifically comprises the following steps:

[0019] Performing multimodal feature extraction on the plurality of processed data to obtain multiple modal feature representations;

[0020] Through the attention mechanism, the importance of multiple processing data is analyzed to extract important information of multiple tasks;

[0021] According to the multiple modal feature representations and the multiple task important information, multimodal fusion is performed to generate multimodal data.

[0022] As a further limitation of the technical solution of the embodiment of the present invention, the identification and analysis of the multimodal data based on the preset multimodal large model and the determination of whether to send a notification in combination with historical notification responses specifically include the following steps:

[0023] Based on the preset multimodal large model, context learning and instruction tuning are performed on multimodal data;

[0024] Combine preset historical notification responses to learn user preferences and habits;

[0025] Determine whether a notification needs to be sent.

[0026] As a further limitation of the technical solution of the embodiment of the present invention, the determining whether the user has a notification response and, if the user has a notification response, performing message classification and dynamic adjustment processing specifically includes the following steps:

[0027] Determine whether the user has a notification response;

[0028] When there is a notification response, dynamically adjust the message notification strategy;

[0029] Conducting notification priority analysis;

[0030] Different notification processing is performed for different notification priorities.

[0031] As a further limitation of the technical solution of the embodiment of the present invention, performing different notification processing for different notification priorities specifically includes the following steps:

[0032] For high-priority notifications, the preset fallback strategy is executed for direct notification;

[0033] For notifications of the same type or of low priority, summarize and notify them.

[0034] A message notification intelligent management system for intelligent vehicles, the system comprises an interactive data acquisition module, a multimodal data processing module, a data identification and analysis module, a message notification module and a notification response processing module, wherein:

[0035] An interactive data collection module, used to collect user interaction data in the vehicle;

[0036] A multimodal data processing module, used to perform data processing and multimodal fusion on the user interaction data to generate multimodal data;

[0037] A data identification and analysis module is used to identify and analyze the multimodal data based on a preset multimodal large model, and determine whether a notification needs to be sent in combination with historical notification responses;

[0038] A message notification module, used to determine a notification mode and a notification strength when a notification needs to be sent, and send a message notification to the user according to the notification mode and the notification strength;

[0039] The notification response processing module is used to determine whether the user has a notification response, and if so, to perform message classification and dynamic adjustment processing.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The embodiment of the present invention collects user interaction data in the vehicle; performs data processing and multimodal fusion on the user interaction data to generate multimodal data; based on a preset multimodal large model, identifies and analyzes the multimodal data, and judges whether it is necessary to send a notification in combination with historical notification responses; when it is necessary to send a notification, determines the notification method and notification strength, and sends a message notification to the user according to the notification method and notification strength; judges whether the user has a notification response, and performs message classification and dynamic adjustment processing when there is a notification response. It can comprehensively judge whether a notification is needed and in what manner, and dynamically adjust the notification strategy according to the user's response and feedback, thereby reducing unnecessary information interference and ensuring that key information can be effectively conveyed. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1A flow chart showing a method for intelligent management of message notifications of an intelligent vehicle provided by an embodiment of the present invention is shown;

[0043] Figure 2 The application architecture diagram of the intelligent management system for message notification of intelligent vehicles provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] It is understandable that smart vehicles need to process a lot of information during driving and use, including vehicle status updates, navigation prompts, safety warnings, etc. In the existing technology, it is impossible to effectively distinguish the importance and urgency of information, resulting in the driver missing key prompts or being distracted by unimportant information.

[0046] To solve the above problems, an embodiment of the present invention discloses an intelligent management method and system for message notifications of intelligent vehicles, which collects user interaction data in the vehicle; performs data processing and multimodal fusion on the user interaction data to generate multimodal data; identifies and analyzes the multimodal data based on a preset multimodal large model, and judges whether it is necessary to send a notification in combination with historical notification responses; when it is necessary to send a notification, determines the notification method and notification strength, and sends a message notification to the user according to the notification method and notification strength; judges whether the user has a notification response, and performs message classification and dynamic adjustment processing when there is a notification response. It can comprehensively judge whether a notification is needed and in what manner, and dynamically adjust the notification strategy according to the user's response and feedback, thereby reducing unnecessary information interference and ensuring that key information can be effectively conveyed.

[0047] Specifically, Figure 1 A flow chart of a method for intelligent management of message notifications of an intelligent vehicle provided by an embodiment of the present invention is shown.

[0048] In a preferred embodiment of the present invention, a method for intelligent management of message notifications of an intelligent vehicle is provided, and the method specifically comprises the following steps:

[0049] Step S101: Collect user interaction data in the vehicle.

[0050] In an embodiment of the present invention, user interaction data such as vehicle sensor data, camera data and / or microphone data in the vehicle are collected.

[0051] It can be understood that the corresponding data collection procedure is:

[0052] def collect_data():

[0053] vehicle_sensors=collect_sensor_data(vehicle_sensor_list)

[0054] camera_frame=collect_camera_data(main_camera)

[0055] audio_clip=collect_microphone_data(main_microphone)

[0056] user_interaction=collect_user_interaction_data(interaction_sensors)

[0057] return vehicle_sensors, camera_frame, audio_clip, user_interaction.

[0058] Step S102: performing data processing and multimodal fusion on the user interaction data to generate multimodal data.

[0059] In an embodiment of the present invention, preprocessing such as data cleaning, standardization and enhancement is performed on user interaction data to generate preprocessed data, and then targeted processing of images, voices, texts and physiological signals is performed on the preprocessed data to generate multiple targeted processed data, and then multimodal feature extraction is performed on the multiple targeted processed data to obtain multiple modal feature representations. Thereafter, importance analysis is performed on the multiple targeted processed data through an attention mechanism to extract multiple task important information, and then multimodal fusion is performed according to the multiple modal feature representations and the multiple task important information to generate multimodal data.

[0060] It can be understood that the corresponding data preprocessing procedure is:

[0061] def preprocess_vehicle_sensors(sensors):

[0062] #Filtering, normalization and other preprocessing operations

[0063] preprocessed_sensors=filter_and_normalize(sensors)

[0064] return preprocessed_sensors

[0065] def preprocess_camera_frame(frame):

[0066] #Image enhancement, size adjustment and other preprocessing operations

[0067] preprocessed_frame=image_enhancement_and_resize(frame)

[0068] return preprocessed_frame;

[0069] The procedures for processing are:

[0070] def process_image_data(image_data):

[0071] # Feature extraction and other image processing operations

[0072] features=feature_extraction(image_data)

[0073] return features def process_audio_data(audio_data):

[0074] #Speech recognition, feature extraction and other audio processing operations

[0075] features=speech_recognition_and_feature_extraction(audio_data)

[0076] return features;

[0077] The procedure for multimodal fusion is:

[0078] def fuse_multimodal_data(features_list):

[0079] #Fusing features from different modalities

[0080] fused_features=multimodal_fusion(features_list)

[0081] return fused_features.

[0082] Step S103: Based on the preset multimodal macro model, the multimodal data is identified and analyzed, and combined with historical notification responses, it is determined whether a notification needs to be sent.

[0083] In an embodiment of the present invention, context learning and instruction tuning are performed on multimodal data based on a preset multimodal large model, and user preferences and habits are learned in combination with preset historical notification responses, and then a comprehensive analysis is performed to determine whether a notification needs to be sent.

[0084] It can be understood that the corresponding context learning and instruction tuning procedures are:

[0085] def reasoning_and_decision(fused_features):

[0086] #Use pre-trained models for reasoning and decision making

[0087] decision=multimodal_model_inference(fused_features)

[0088] return decision;

[0089] The corresponding program for learning user preferences and habits and then conducting comprehensive analysis is: classDynamicNotificationAdjustment(nn.Module):

[0090] def __init__(self):

[0091] super(DynamicNotificationAdjustment,self).__init__()

[0092] self.user_preference_model=nn.Sequential(

[0093] nn.Linear(100,50), #Assume that the dimension of the user preference vector is 100

[0094] nn.ReLU(),

[0095] nn.Linear(50,1) )

[0097] def forward(self,user_preference_vector,notification_features):

[0098] #Dynamically adjust notification strategies based on user preferences and notification characteristics

[0099] adjusted_preference=self.user_preference_model(user_preference_vector)

[0100] notification_score=torch.matmul(adjusted_preference,notification_features)

[0101] return notification_score.

[0102] Step S104: when a notification needs to be sent, determine a notification method and a notification strength, and send a message notification to the user according to the notification method and the notification strength.

[0103] It can be understood that the corresponding procedures for determining the notification method and notification intensity are: def notification_decision(decision,user_preferences):

[0104] #Determine notification strategy based on user preferences and decision results

[0105] notification_strategy=determine_notification_strategy(decision,user_preferences)

[0106] return notification_strategy.

[0107] Step S105: determine whether the user has a notification response, and if so, perform message classification and dynamic adjustment processing.

[0108] In an embodiment of the present invention, by determining whether the user has a notification response, and if so, dynamically adjusting the message notification strategy, and performing a notification priority analysis, different notification processing is performed for different notification priorities. Specifically: for high-priority notifications, a preset fallback strategy is executed for direct notification; for similar or low-priority notifications, summary processing and notification are performed.

[0109] Furthermore, Figure 2 The application architecture diagram of the intelligent management system for message notification of intelligent vehicles provided by an embodiment of the present invention is shown.

[0110] Among them, in another preferred embodiment provided by the present invention, a message notification intelligent management system for intelligent vehicles includes:

[0111] The interaction data collection module 101 is used to collect user interaction data in the vehicle.

[0112] In the embodiment of the present invention, the interaction data collection module 101 collects user interaction data such as vehicle sensor data, camera data and / or microphone data in the vehicle.

[0113] The multimodal data processing module 102 is used to perform data processing and multimodal fusion on the user interaction data to generate multimodal data.

[0114] In an embodiment of the present invention, the multimodal data processing module 102 generates preprocessed data by performing data preprocessing such as data cleaning, standardization and enhancement on the user interaction data, and then performs targeted processing of images, voices, texts and physiological signals on the preprocessed data to generate multiple targeted processed data, and then performs multimodal feature extraction on the multiple targeted processed data to obtain multiple modal feature representations. Thereafter, through the attention mechanism, importance analysis is performed on the multiple targeted processed data to extract multiple task important information, and then multimodal fusion is performed according to the multiple modal feature representations and the multiple task important information to generate multimodal data.

[0115] The data identification and analysis module 103 is used to identify and analyze the multimodal data based on a preset multimodal macro model, and determine whether a notification needs to be sent in combination with historical notification responses.

[0116] In an embodiment of the present invention, the data identification and analysis module 103 performs context learning and instruction tuning on the multimodal data based on a preset multimodal large model, learns the user's preferences and habits in combination with preset historical notification responses, and then performs a comprehensive analysis to determine whether a notification needs to be sent.

[0117] The message notification module 104 is used to determine the notification mode and notification strength when a notification needs to be sent, and to send a message notification to the user according to the notification mode and the notification strength.

[0118] The notification response processing module 105 is used to determine whether the user has a notification response, and if so, to perform message classification and dynamic adjustment processing.

[0119] In an embodiment of the present invention, the notification response processing module 105 determines whether the user has a notification response, and if so, dynamically adjusts the message notification strategy, performs notification priority analysis, and performs different notification processing for different notification priorities. Specifically: for high-priority notifications, executes the preset fallback strategy for direct notification; for similar or low-priority notifications, performs summary processing and notification.

[0120] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0121] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0122] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for intelligent management of message notifications of intelligent vehicles, characterized in that: The method specifically comprises the following steps: Collecting user interaction data in the vehicle; Performing data processing and multimodal fusion on the user interaction data to generate multimodal data; Based on a preset multimodal macro model, the multimodal data is identified and analyzed, and combined with historical notification responses, it is determined whether a notification needs to be sent; When a notification needs to be sent, determine a notification method and a notification strength, and send a message notification to the user according to the notification method and the notification strength; Determine whether the user has a notification response, and if so, perform message classification and dynamic adjustment processing.

2. The intelligent vehicle message notification intelligent management method according to claim 1, characterized in that: The user interaction data includes vehicle sensor data, camera data and / or microphone data.

3. The intelligent vehicle message notification intelligent management method according to claim 1, characterized in that: The step of performing data processing and multimodal fusion on the user interaction data to generate multimodal data specifically includes the following steps: Performing data preprocessing on the user interaction data to generate preprocessed data; Performing targeted processing of images, voices, texts and physiological signals on the pre-processed data to generate a plurality of targeted processed data; Perform multimodal fusion on the plurality of processed data to generate multimodal data.

4. The intelligent vehicle message notification intelligent management method according to claim 2, characterized in that: The data preprocessing of the user interaction data includes: data cleaning, standardization and enhancement.

5. The intelligent vehicle message notification intelligent management method according to claim 2, characterized in that: The step of performing multimodal fusion on the plurality of processed data to generate multimodal data specifically comprises the following steps: Performing multimodal feature extraction on the plurality of processed data to obtain multiple modal feature representations; Through the attention mechanism, the importance of multiple processing data is analyzed to extract important information of multiple tasks; According to the multiple modal feature representations and the multiple task important information, multimodal fusion is performed to generate multimodal data.

6. The intelligent vehicle message notification intelligent management method according to claim 1, characterized in that: The method of identifying and analyzing the multimodal data based on the preset multimodal macro model and judging whether a notification needs to be sent in combination with historical notification responses specifically includes the following steps: Based on the preset multimodal large model, context learning and instruction tuning are performed on multimodal data; Combine preset historical notification responses to learn user preferences and habits; Determine whether a notification needs to be sent.

7. The intelligent vehicle message notification intelligent management method according to claim 1, characterized in that: The method of determining whether the user has a notification response and performing message classification and dynamic adjustment processing when the user has a notification response specifically includes the following steps: Determine whether the user has a notification response; When there is a notification response, dynamically adjust the message notification strategy; Conducting notification priority analysis; Different notification processing is performed for different notification priorities.

8. The intelligent vehicle message notification intelligent management method according to claim 7, characterized in that: The different notification processing for different notification priorities specifically includes the following steps: For high-priority notifications, the preset fallback strategy is executed for direct notification; For notifications of the same type or of low priority, summarize and notify them.

9. An intelligent vehicle message notification intelligent management system, characterized in that: The system includes an interactive data acquisition module, a multimodal data processing module, a data identification and analysis module, a message notification module and a notification response processing module, wherein: An interactive data collection module, used to collect user interaction data in the vehicle; A multimodal data processing module, used to perform data processing and multimodal fusion on the user interaction data to generate multimodal data; A data identification and analysis module is used to identify and analyze the multimodal data based on a preset multimodal large model, and determine whether a notification needs to be sent in combination with historical notification responses; A message notification module, used to determine a notification mode and a notification strength when a notification needs to be sent, and send a message notification to the user according to the notification mode and the notification strength; The notification response processing module is used to determine whether the user has a notification response, and if so, to perform message classification and dynamic adjustment processing.

Images