Human-computer interaction self-driving automobile passenger personalized information service system

Abstract

The invention relates to the technical field of intelligent transportation, and discloses a man-machine interaction autonomous vehicle passenger personalized information service system. A data capture module of the system divides data segments based on the occurrence time of an interaction event, and generates an interaction data set with aligned time sequences; a feature extraction module performs feature extraction on the interaction data set subjected to time sequence alignment to obtain a passenger behavior feature vector and a vehicle environment feature matrix; the graph neural network module inputs the feature vector and the feature matrix into a pre-trained graph neural network model, and outputs a service association graph; the random walk module applies a random walk algorithm on the service association map to generate a passenger preference path sequence; the dynamic adjustment module dynamically adjusts a service activation probability threshold according to the phase change of the real-time driving scene, and selects a service path of which the probability is higher than the threshold to generate a personalized service instruction; and the execution control module converts the personalized service instruction into an operation signal and controls the information output equipment to deliver the service.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, specifically to a human-computer interactive personalized information service system for autonomous vehicle occupants. Background Technology

[0002] Current autonomous vehicle information service systems primarily employ fixed menus and manually triggered interaction methods. Existing technologies rely heavily on single-modal data for understanding occupant behavior, failing to effectively integrate multi-source sensor information. Service recommendation mechanisms are static and fixed, unable to dynamically adjust based on real-time occupant interactions. Environmental perception capabilities are limited, and the correlation between vehicle status and occupant needs remains largely unexplored. Fixed service trigger thresholds are ill-suited to complex and ever-changing driving scenarios. Existing methods need to address key technical challenges such as multimodal data fusion, deep understanding of occupant behavior, and dynamic service recommendation.

[0003] Traditional in-vehicle information service systems have significant shortcomings in personalization and intelligence. Asynchronous multimodal data acquisition and temporal discrepancies between sensor data lead to inaccurate feature extraction. Simple behavioral feature extraction methods fail to capture temporal patterns and underlying intentions in occupant interactions. Linear service association modeling cannot represent the complex network relationships between occupants, services, and the environment. Recommendation algorithms are passively reactive, lacking proactive exploration and prediction of occupant preference paths. Rigid threshold setting mechanisms cannot dynamically adjust service delivery strategies based on safety requirements in driving scenarios. The service execution process lacks closed-loop feedback, making it impossible to optimize recommendation strategies based on service performance. Existing technologies necessitate the establishment of a comprehensive personalized service system encompassing multimodal perception and intelligent services. Summary of the Invention

[0004] The purpose of this invention is to provide a human-computer interaction-based personalized information service system for autonomous vehicle occupants, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a human-computer interaction-based personalized information service system for occupants of autonomous vehicles, the system comprising: The data capture module is used to capture multimodal sensor data streams in real time during the interaction between the occupant and the vehicle interface, and to divide the data into segments based on the occurrence time of the interaction event to generate a time-aligned interaction dataset. The feature extraction module is used to extract features from the time-aligned interactive dataset to obtain occupant behavior feature vectors and vehicle environment feature matrices. The graph neural network module is used to input the occupant behavior feature vector and the vehicle environment feature matrix into a pre-trained graph neural network model, update the node representation through multi-layer graph convolution operations, and output a service association graph. The random walk module is used to apply a random walk algorithm to the service association graph to generate a sequence of passenger preference paths and calculate the service activation probability based on the path weights. The dynamic adjustment module is used to dynamically adjust the threshold of service activation probability based on the phase changes in the real-time driving scenario, and select service paths with a probability higher than the threshold to generate personalized service instructions. The execution control module is used to convert the personalized service instructions into specific operation signals through the on-board execution unit, and control the information output device to deliver the service.

[0006] Preferably, the real-time capture of multimodal sensor data streams during the interaction between the occupant and the in-vehicle interface, and the division of data segments based on the occurrence time of the interaction event to generate a time-aligned interaction dataset, includes: A high-frame-rate visual sensor and microphone array are used to simultaneously collect occupant facial expression data, gesture trajectory data, and voice command data. Detect the start and end points of interactive events, and extract multimodal data within a fixed time window before and after the event center as the reference. The extracted data segments are timestamped to eliminate timing discrepancies between different sensors. Extract keyframes and audio features from each data segment and fuse them to generate a multidimensional feature vector; Arrange the multidimensional feature vectors of all data segments in chronological order to construct the time-aligned interactive dataset.

[0007] Preferably, the step of extracting features from the time-aligned interactive dataset to obtain occupant behavior feature vectors and vehicle environment feature matrices includes: Visual data segments are processed using a convolutional autoencoder, and spatial features are extracted after compressing the image dimensions to generate occupant attention distribution vectors. By processing audio data segments through a recurrent neural network, the temporal pattern of voice commands is learned to generate occupant intention encoding vectors. The system reads environmental sensor data from the vehicle bus system, including vehicle speed, steering angle, and obstacle distance, and calculates environmental complexity metrics. The occupant attention distribution vector and the occupant intention encoding vector are concatenated to form the occupant behavior feature vector; The environmental complexity index is combined with historical environmental data within the time window to form a vehicle environmental feature matrix.

[0008] Preferably, the step of inputting the occupant behavior feature vector and the vehicle environment feature matrix into a pre-trained graph neural network model, updating the node representations through multi-layer graph convolution operations, and outputting a service association graph includes: Construct a graph structure where nodes represent different service types and edges represent the strength of the association between services; Initialize node features as occupant behavior feature vectors and edge features as similarity scores of vehicle environment feature matrices. Perform graph convolutional layer operations to aggregate neighbor node information and update node embedding representations. Preserve original features through residual connections to prevent gradient vanishing. After multiple iterations, output a service association graph with stable node representations.

[0009] Preferably, the step of applying a random walk algorithm to the service association graph to generate a sequence of passenger preference paths and calculating the service activation probability based on the path weights includes: Starting from the root node of the service association graph, adjacent nodes are selected with transition probability to generate multiple random walk paths; The weight of each path is calculated. The weight is determined by the inner product of the edge features between nodes and the node features in the path. The path weights are normalized to obtain the activation probability of each path. The top k paths with the highest activation probabilities are selected as the passenger preference path sequence. The service activation probability is adjusted according to the cosine similarity between the current passenger behavior feature vector and the path nodes.

[0010] Preferably, the step of dynamically adjusting the threshold of service activation probability based on phase changes in the real-time driving scenario, and selecting service paths with probabilities higher than the threshold to generate personalized service instructions, includes: Obtain driving scenario phase identifiers from the vehicle control system, including straight-ahead, turning, and parking phases; Set a base threshold for each driving scenario phase and dynamically offset the threshold based on the environmental complexity index; Compare the service activation probability with the adjusted threshold, filter qualified service paths, map qualified service paths to specific service operations, and generate personalized service instructions that include service type and execution sequence.

[0011] Preferably, the step of converting the personalized service instructions into specific operation signals through the on-board execution unit and controlling the information output device to deliver the service includes: Parse the service type parameters in personalized service commands and match them with the vehicle device driver protocol; The service timing data is converted into a pulse width modulation signal to control the brightness changes of the display screen or the volume gradient of the speaker; Real-time monitoring of passenger feedback signals during service delivery, and dynamic adjustment of the intensity of operation signals based on the feedback signals to ensure smooth service delivery.

[0012] Preferably, the real-time monitoring of occupant feedback signals during service delivery further includes: collecting occupant micro-motion signals through a bio-radar sensor and extracting physiological rhythm patterns from the micro-motion signals through time-frequency analysis; performing pattern matching between the physiological rhythm patterns and benchmark patterns in a pre-stored comfort pattern library to calculate a pattern similarity score; generating an occupant comfort index based on the pattern similarity score; and adjusting the timing parameters of service instructions, including instruction execution delay and instruction interval period, based on the occupant comfort index.

[0013] Preferably, the step of adjusting the timing parameters of service instructions based on the occupant comfort index includes: increasing the instruction execution delay and extending the instruction interval when the occupant comfort index is lower than the dynamic comfort threshold; reducing the instruction execution delay and shortening the instruction interval when the occupant comfort index is higher than the dynamic comfort threshold; recording the adjusted timing parameters and the corresponding occupant comfort index in real time, and updating the baseline mode in the comfort mode library; recalculating the mode similarity score based on the updated baseline mode, and iteratively optimizing the timing parameter adjustment strategy.

[0014] Preferably, the method further includes a periodic model optimization phase: collecting passenger feedback signal sequences and adjusted time-series parameter sequences from historical service delivery records to construct a model retraining dataset; extracting the latest baseline pattern sequence from the comfort mode library and injecting it as node features into the graph neural network model; performing graph convolution operations to update the edge weights and node embeddings of the service association graph; verifying the path consistency of the updated service association graph through a random walk algorithm, and recalculating the service activation probability distribution.

[0015] Compared with the prior art, the beneficial effects of the present invention are: The data capture module captures multimodal sensor data streams in real time during the interaction between occupants and the in-vehicle interface. Data segments are generated based on the occurrence time of interaction events, resulting in a time-aligned interaction dataset. Multimodal sensor data includes various interaction information such as voice commands, gestures, facial expressions, and physiological signals. The occurrence time of interaction events is accurately identified through action detection and semantic analysis, ensuring accurate data segmentation. Time alignment processing employs interpolation methods and dynamic time warping algorithms to eliminate time offsets between different sensor data. The aligned dataset ensures strict synchronization of each modality's data on the time axis, providing a reliable foundation for subsequent analysis. The feature extraction module extracts features from the time-aligned interaction dataset, obtaining occupant behavior feature vectors and vehicle environment feature matrices. Behavioral feature extraction uses time series analysis methods to capture the dynamic changes and pattern characteristics of interaction behavior. Environmental feature extraction considers key factors affecting service delivery, such as vehicle speed, road condition complexity, and traffic density. The dimensions of the feature vectors and feature matrices are optimized to balance representational capability and computational efficiency. The extracted features retain key information from the original data while eliminating redundant noise interference.

[0016] The graph neural network module inputs occupant behavior feature vectors and vehicle environment feature matrices into a pre-trained graph neural network model. It updates node representations through multi-layer graph convolution operations, outputting a service association graph. The graph structure uses occupants, services, and environment entities as nodes, and their interactions and influence strengths as edges. Graph convolution operations propagate and aggregate feature information among neighboring nodes, gradually refining the high-level representations of nodes. The multi-layer convolutional structure captures association patterns at different scales in the graph, abstracting from direct to indirect relationships layer by layer. The node representation update process considers connection weights and relationship types, accurately reflecting the semantic relationships between elements in the graph. The service association graph visualizes the complex network relationships between occupant preferences, service functions, and environmental states. The random walk module applies a random walk algorithm to the service association graph, generating occupant preference path sequences and calculating service activation probabilities based on path weights. The random walk starts from the current interaction node and explores possible behavioral paths in the graph according to transition probabilities. The path sequence records the sequence of nodes traversed during the walk, reflecting the potential behavioral patterns of occupants. Path weights are calculated by accumulating the number of walks and edge weights, representing the popularity and importance of the path. Service activation probability quantifies the likelihood of a service being triggered in a given context. Probability calculations consider the combined effects of historical preferences and real-time context.

[0017] The dynamic adjustment module dynamically adjusts the service activation probability threshold based on real-time phase changes in the driving scenario, and selects service paths with probabilities higher than the threshold to generate personalized service instructions. Driving scenario phases are identified in real-time by vehicle sensors and the navigation system, including different conditions such as urban roads, highways, and parking. The threshold adjustment strategy is based on scenario safety requirements; the threshold is increased to reduce interference in complex road conditions, and decreased to enrich services in simple road conditions. Service path selection comprehensively considers probability values ​​and business priorities to ensure that the recommended results are both personalized and safe. Scenario adaptation logic is added to the instruction generation process to ensure that the service content format conforms to current driving needs. The execution control module converts personalized service instructions into specific operation signals through the onboard execution unit, controlling the information output devices to deliver the service. Instruction conversion is standardized according to device type and interface protocol to ensure signal compatibility. Operation signals include control information such as content parameters, display methods, and trigger timing. Information output devices include multimodal interaction interfaces such as displays, audio systems, and haptic feedback devices. The service delivery process monitors the execution effect in real time, providing feedback data for subsequent optimization. Through the synergistic effects of multimodal data fusion, graph correlation analysis, random walk exploration, and dynamic threshold adjustment, truly personalized in-vehicle information services are achieved. Temporal alignment ensures data quality, graph neural networks uncover deep correlations, random walks discover potential preferences, and dynamic threshold adjustment adapts to scenario requirements. This intelligent service system significantly improves the riding experience and human-machine interaction efficiency of autonomous vehicles. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the working principle of a human-computer interaction-based personalized information service system for autonomous vehicle occupants, as described in this invention. Figure 2 A flowchart generated for a time-aligned interactive dataset; Figure 3 A flowchart for outputting the service association graph; Figure 4 The correlation strength thermodynamic analysis diagram; Figure 5 A trend chart for multi-dimensional optimization of model performance. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please see Figure 1This invention provides a personalized information service system for autonomous vehicle occupants using human-computer interaction. The system includes: a data capture module that uses a high-frame-rate visual sensor and microphone array to capture multimodal sensor data streams in real time during occupant interaction with the vehicle interface. This data stream includes information such as facial expressions, gesture trajectories, and voice commands. The module divides the data into segments based on the occurrence time of the interaction event and generates a time-aligned interaction dataset. A feature extraction module processes the time-aligned interaction dataset, using a convolutional autoencoder to extract visual features, learning audio temporal patterns through a recurrent neural network, and simultaneously reading environmental sensor data from the vehicle bus system, ultimately generating occupant behavior feature vectors and a vehicle environment feature matrix. A graph neural network module receives the aforementioned feature vectors and matrices, constructs a graph structure with service type as nodes and service association strength as edges, aggregates neighbor node information and updates node embedding representations through multi-layer graph convolution operations, and outputs a stable service association graph. A random walk module starts from the root node on the service association graph, selects adjacent nodes based on transition probabilities to generate multiple paths, calculates the weight of each path and normalizes it to obtain the service activation probability, generating an occupant preference path sequence. The dynamic adjustment module dynamically adjusts the service activation probability threshold based on the driving scenario phase identifier obtained from the vehicle control system and the environmental complexity index. It filters service paths with probabilities higher than the threshold and maps them to specific service operations, generating personalized service instructions. The execution control module parses the personalized service instructions through the on-board execution unit, matches the service type parameters to the on-board device's driving protocol, and converts the timing data into pulse width modulation signals to control the display screen or speaker, achieving the final delivery of the service.

[0021] Example 1: See Figure 2In the specific implementation, the data capture module uses a high frame rate visual sensor and microphone array to simultaneously acquire occupant facial expression data, gesture trajectory data, and voice command data. The frame rate of the high frame rate visual sensor is set to no less than 120 frames per second, and the microphone array contains eight directional microphone units to form beamforming. In the specific implementation, the start and end points of interactive events are determined by detecting the start and end silence segments of voice commands, combined with the starting motion vector of gestures for joint determination. Multimodal data within a fixed time window before and after the event center is extracted, with the time window length set to 1.2 times the event duration. In the specific implementation, the extracted data segments undergo timestamp alignment processing, and a synchronization mechanism based on a network time protocol is used to eliminate timing deviations between different sensors, with the timing deviation controlled within 10 milliseconds. In the specific implementation, keyframes and audio features are extracted from each data segment. Keyframes are selected by calculating the sum of the absolute values ​​of the inter-frame differences of the image sequence to select extreme points. Mel-frequency cepstral coefficients are extracted from audio features and fused to generate a 256-dimensional multidimensional feature vector. In practice, the multidimensional feature vectors of all data segments are arranged in chronological order to construct a time-aligned interactive dataset. The time-aligned interactive dataset is stored in a circular buffer structure, retaining the interactive data within the most recent 300 seconds.

[0022] In the specific implementation, the feature extraction module uses a convolutional autoencoder to process the visual data segment. The encoder part of the convolutional autoencoder contains four convolutional layers and two fully connected layers, compressing the image size from 640x480 to a 128-dimensional feature vector. After extracting spatial features, it generates an occupant attention distribution vector, which represents the two-dimensional coordinate distribution of the occupant's gaze focus on the vehicle interface. In the specific implementation, a recurrent neural network is used to process the audio data segment. The recurrent neural network structure uses long short-term memory units with 128 hidden layer nodes to learn the temporal pattern of voice commands. The output layer generates an occupant intention encoding vector using a softmax function. The dimension of the occupant intention encoding vector is consistent with the number of predefined voice command categories. In the specific implementation, environmental sensor data is read from the vehicle bus system. The environmental sensor data includes vehicle speed, steering angle, and obstacle distance, with a sampling frequency of 100 Hz. Based on the environmental sensor data, an environmental complexity index is calculated, which is a weighted sum of the variance of vehicle speed, the mean absolute value of steering angle, and the mean obstacle distance. In the specific implementation, the occupant attention distribution vector and the occupant intention encoding vector are concatenated to form the occupant behavior feature vector, which has a total dimension of 256. In the specific implementation, the environmental complexity index is combined with historical environmental data within the time window to form a vehicle environmental feature matrix. The number of rows in the vehicle environmental feature matrix represents the number of time steps, and the number of columns represents the number of environmental parameter categories. The time window length of the historical environmental data is consistent with that of the interactive data segment.

[0023] In some embodiments, the high frame rate vision sensor is installed inside the rearview mirror base, covering the driver and passenger seat areas. The vision sensor employs a global shutter image sensor to reduce motion blur. In some embodiments, the microphone array has eight directional microphone units evenly distributed in a circle, each 15 cm in diameter, with a signal-to-noise ratio of at least 70 dB for each unit. The beamforming algorithm uses a minimum variance distortion-free response algorithm. It is understood that the convolutional autoencoder is pre-trained using a large-scale in-vehicle image dataset, with a training cycle of 100 epochs. The optimizer uses the Adam algorithm, and the learning rate is set to 0.001. It is understood that the long short-term memory unit of the recurrent neural network has an input sequence length of 50 time steps, with each time step corresponding to a 20-millisecond audio frame, and a dropout rate of 0.2 to prevent overfitting.

[0024] In the implementation, the circular buffer of the time-aligned interactive dataset adopts a first-in, first-out (FIFO) management strategy. When a new data segment is added, the oldest data segment is automatically overwritten. The buffer capacity is dynamically allocated through memory to ensure the storage of 300 seconds of data. In the implementation, the fusion process of multi-dimensional feature vectors adopts a feature-level fusion method. Visual feature vectors and audio feature vectors are subjected to L2 normalization before splicing to eliminate the influence of dimensions. In the implementation, gesture trajectory data acquisition utilizes continuous frame sequences output by a high-frame-rate visual sensor. The motion vectors of key hand points are calculated using optical flow, and the amplitude threshold of the motion vector is set to 10 pixels per frame to determine the start of the gesture. Endpoint detection of voice command data uses a dual-threshold comparison method. The short-time energy threshold is set to 3 times the average energy of the silent segment, and the zero-crossing rate threshold is set to 1.5 times the average zero-crossing rate of the silent segment. In the implementation, timestamp alignment processing uses a hardware synchronization signal to trigger all sensors to start acquiring data simultaneously. The synchronization signal is sent by the vehicle's main control unit once per second. In practice, the extraction parameters for the Mel frequency cepstral coefficients are set to 26 filter banks, 12 cepstral coefficients, a frame length of 25 milliseconds, and a frame shift of 10 milliseconds.

[0025] Example 2: See Figure 3In the specific implementation, the graph neural network module constructs a graph structure. Nodes in the graph structure represent different service types, including 15 categories such as navigation prompts, entertainment playback, and air conditioning control. Edges represent the association strength between services, initialized by historical co-occurrence frequency, calculated as the ratio of the number of consecutive activations of two services within the past 30 days. In the specific implementation, the initialized node features are occupant behavior feature vectors with a dimension of 256. Edge features are similarity scores of the vehicle environment feature matrix, obtained by calculating the cosine similarity of the row vectors in the vehicle environment feature matrix. The cosine similarity range is [-1, 1], with negative values ​​indicating dissimilarity. In the specific implementation, graph convolutional layer operations are performed. Each graph convolutional layer contains a normalized product of a feature transformation matrix and an adjacency matrix. The feature transformation matrix has a dimension of 128x128, and the adjacency matrix normalization uses a symmetric normalization method, i.e., left-multiplying and right-multiplying the inverse square root of the degree matrix by the adjacency matrix. In the specific implementation, neighbor node information is aggregated and the node embedding representation is updated. The node embedding dimension is set to 128 dimensions. The aggregation function uses mean pooling, and the update function uses the ReLU activation function. In the specific implementation, residual connections preserve the original features. The input and output of each graph convolution layer are added to prevent gradient vanishing. The weight factor of the residual connections is set to 0.5. In the specific implementation, after multiple iterations, convergence is determined when the L2 norm change rate of the node embedding is less than 0.001. A service association graph with stable node representations is output. The service association graph is stored as an adjacency list structure, containing node feature vectors and edge weight matrices.

[0026] In the implementation, the random walk module starts from the root node on the service association graph. The root node is selected as the currently active service node, determined by the most recently activated service type. In the implementation, neighboring nodes are selected based on transition probabilities, which are obtained by normalizing the edge weights using a softmax function with a temperature parameter set to 1.0. Multiple random walk paths are generated, with a fixed number of 100 paths and a fixed path length of 5 nodes. The weight of each path is calculated, determined by the inner product of the edge features and node features within the path. The inner product result is mapped to the 0-1 interval using a sigmoid function. The path weights are normalized using a min-max normalization method to scale the weights to a probability distribution, obtaining the activation probability of each path. In the implementation, the top k paths with the highest activation probabilities are selected as the passenger preference path sequence. The value of k is dynamically adjusted according to the current driving scenario, set to 5 for urban road scenarios and 3 for highway scenarios. In practice, the service activation probability is adjusted based on the cosine similarity between the current occupant behavior feature vector and the path node. When the similarity is greater than 0.8, the probability increases by 10%, and when the similarity is less than 0.3, the probability decreases by 15%.

[0027] In some embodiments, the graph structure construction process includes a node initialization step, in which a unique identifier is assigned to each service type. This identifier is encoded as a 4-bit binary number, and the initial values ​​of the edge weights are calculated based on historical interaction logs from offline analysis. In some embodiments, the graph convolution operation has 3 layers, with each layer maintaining a consistent output dimension of 128. Dropout layers are used between layers to prevent overfitting, with a dropout rate of 0.1. In some embodiments, the random walk path generation employs a biased random walk strategy, which prioritizes paths with larger edge weights, with a bias coefficient of 2.0. It is understood that the convergence determination of the service association graph is based on the stability of the node embeddings. The stability index is calculated as the average cosine similarity of node embeddings across consecutive iterations, with an average threshold of 0.995. It is understood that the path weight normalization process includes a smoothing step, in which a Laplace smoothing term is added to each weight, with a smoothing factor of 0.01.

[0028] In the implementation, edge features are calculated using the similarity score of the vehicle environment feature matrix. This similarity score is calculated based on the cosine similarity formula, which is the dot product of two vectors divided by their modulus product. Path weights are calculated using inner product operations, performed in parallel on a floating-point unit, batch processing 100 paths. Service activation probability adjustment includes truncation, forcibly limiting the adjusted probability value to between 0 and 1 to prevent overflow. The seed value for the random walk algorithm is generated by the system clock to ensure randomness in each execution, with an update frequency of once per second. Node embedding updates include gradient pruning, with a threshold of 1.0 to prevent gradient explosion. The adjacency matrix for graph convolution operations is sparsely stored, storing only non-zero elements and their indices to reduce memory usage. Residual connections are implemented using tensor addition, which is accelerated on a graphics processor. In the implementation, the normalization of path activation probabilities uses a numerically stabilized softmax function, with the input value of the softmax function subtracted from its maximum value to prevent exponential overflow. In the implementation, the cosine similarity calculation between occupant behavior feature vectors and path nodes is performed in batches, with a batch size of 15 corresponding to 15 service nodes. In the implementation, the edge weight matrix of the service association graph is periodically persisted to non-volatile memory at 24-hour intervals. In the implementation, the generation of random walk paths includes a termination condition: the path length reaches 5 or a node with an out-degree of 0 is encountered. In the implementation, the training phase of the graph neural network module uses the Adam optimizer with a learning rate of 0.001 and a training cycle of 100 rounds. In the implementation, the inner product calculation formula for path weights is:

[0029] in: Indicates path weight, This represents the sigmoid function. This represents the edge feature vector of the i-th edge. Let represent the node feature vector of the i-th node. Indicates path length. This represents the vector dot product operation. In the specific implementation, the path length L is fixed at 5, and the edge feature vector... The dimension is 128, and the node feature vector The dimension is 128. In the specific implementation, the summation operation starts from the first edge index 1 of the path and ends at the last edge index L-1, corresponding to the number of edges in the path. In the specific implementation, the output range of the sigmoid function is between 0 and 1, ensuring that the path weights are probability values.

[0030] In the implementation, the feature transformation matrix of the graph convolutional layer uses the Xavier initialization method, which sets the initial weight range based on the input and output dimensions. In the implementation, the diagonal elements of the degree matrix used for adjacency matrix normalization are the degrees of the nodes, calculated as the number of edges connected to that node. In the implementation, the L2 norm change rate of node embeddings is calculated as the average Euclidean distance between node embedding vectors between two iterations. In the implementation, the service association graph storage format includes a node index mapping table, which maps service type names to integer indices. In the implementation, random walk path generation uses a depth-first search strategy, with a stack depth limit of 5. In the implementation, the dynamic adjustment of path activation probabilities is based on historical sliding window data, with the sliding window size being the last 100 adjustment records.

[0031] See Figure 4 This heatmap is a visual representation of the service association graph output by the graph neural network module in the personalized information service system for autonomous vehicle occupants, intuitively quantifying the association strength between six core service categories. The graph uses color gradients to map association values; the closer the value is to 1, the stronger the association between services; the closer it is to 0, the weaker the association. This heatmap is a stable association graph generated by the graph neural network module through multi-layer graph convolution operations, aggregating the features of service nodes and edges. It provides core data support for the subsequent random walk module to generate occupant preference paths and dynamically adjust the module's service recommendation threshold optimization. It accurately depicts the network relationships of occupant service needs and is a key link in realizing personalized services across the entire chain from multimodal perception to intelligent services.

[0032] Example 3: In specific implementation, the dynamic adjustment module obtains driving scenario phase identifiers from the vehicle control system. These driving scenario phase identifiers include straight-ahead, turning, and parking phases. Each driving scenario phase corresponds to a different base threshold: 0.6 for straight-ahead, 0.7 for turning, and 0.8 for parking. In specific implementation, a base threshold is set for each driving scenario phase, and the threshold is dynamically offset based on an environmental complexity index. The environmental complexity index is calculated from the vehicle's environmental feature matrix. For every standard deviation increase in the environmental complexity index, the threshold increases by 0.05, with a maximum offset of 0.2. In specific implementation, the service activation probability is compared with the adjusted threshold. The service activation probability comes from the output of the random walk module. Qualified service paths are selected, requiring a probability greater than or equal to the current threshold and a path length of no more than 3 nodes. In specific implementation, qualified service paths are mapped to specific service operations, generating personalized service instructions containing service type and execution timing. The service type is encoded as a 4-bit binary number, and the timing parameters include start delay and execution duration.

[0033] In practical implementation, the execution control module parses the service type parameters in the personalized service instructions through the on-board execution unit and matches them with the on-board device driver protocol, which conforms to the ISO13400 standard. In practical implementation, service timing data is converted into pulse width modulation (PWM) signals. These PWM signals control the brightness changes of the display screen or the volume gradient of the speakers, with a pulse frequency of 1 kHz and a duty cycle linearly related to the service intensity parameter. In practical implementation, occupant feedback signals during service delivery are monitored in real time. These signals include facial expression change rate and voice response delay. The intensity of the operation signal is dynamically adjusted based on the occupant feedback signals, with an adjustment step size of 5% of the base intensity. In practical implementation, smooth service delivery is ensured by limiting the signal change rate to no more than 10% per second to achieve a gradual transition effect and avoid sudden state jumps.

[0034] In some embodiments, the acquisition of driving scenario phase identifiers is achieved through the Controller Area Network (CLAN) bus interface, which polls the vehicle control system's status register once per second. In some embodiments, the standard deviation of the environmental complexity index is calculated based on sliding window statistics, with the sliding window size being the most recent 100 sampling points, and the standard deviation calculation uses the Bessel correction formula. In some embodiments, the selection process for qualified service paths uses a parallel comparator circuit, which simultaneously compares the probability of all paths with a threshold and outputs a list of indexes for qualified paths. It is understood that the calculation of the threshold dynamic offset uses a linear model, and the coefficients of the linear model are calibrated based on historical data, with a calibration period of once every 7 days. It is understood that the 4-bit binary mapping table of service type codes is stored in read-only memory (ROM), with an address space of 16 entries corresponding to 15 service types and one reserved bit.

[0035] In the implementation, the base threshold of the dynamic adjustment module is stored in a lookup table. The key of the lookup table is the driving scenario phase identifier, and the value is the corresponding base threshold value. The lookup operation is triggered when a new driving scenario phase identifier is received. In the implementation, the standard deviation of the environmental complexity index is calculated as the difference between the current environmental complexity index value and the historical mean, divided by the historical standard deviation. The historical mean and historical standard deviation are calculated based on data from the past 1000 seconds. In the implementation, the threshold offset calculation includes a clamping operation to ensure that the offset does not exceed the maximum offset of 0.2. The offset calculation uses floating-point arithmetic. In the implementation, the comparison between the service activation probability and the threshold uses floating-point comparison instructions. The comparison result generates a Boolean mask, which is used to index qualified service paths. In the implementation, the path length check of qualified service paths is implemented by counting the number of nodes in the path; paths with more than 3 nodes are automatically discarded. In practice, the generation of personalized service instructions includes a serialization step. The serialization step packages the service type encoding and timing parameters into a 32-bit data packet. The first 4 bits of the data packet represent the service type, the middle 16 bits represent the start delay, and the last 12 bits represent the execution duration.

[0036] In specific implementations, the on-board execution unit of the execution control module includes a microcontroller. The microcontroller parses the data packets of personalized service instructions, extracts service type parameters, and matches the corresponding device driver by querying the driver protocol table. In specific implementations, a hardware timer is used to generate the pulse width modulation (PWM) signal. The hardware timer's counting frequency is set to 1 MHz, and the period of the PWM signal is fixed at 1 millisecond. In specific implementations, the service strength parameter is derived from the timing parameters of the personalized service instructions. The service strength parameter is calculated as the reciprocal of the execution duration and normalized to the 0-1 range. In specific implementations, the facial expression change rate of the occupant feedback signal is calculated using a high frame rate visual sensor. The facial expression change rate is the average distance of feature point movement between consecutive frames. In specific implementations, the voice response delay is detected by a microphone array. The voice response delay is the time interval from the start of service delivery to the start of the occupant's voice command. In specific implementations, the adjustment of the operation signal strength uses a proportional-integral (PI) controller. The PI controller's proportional gain is set to 0.05, and the integral time is set to 1 second. In practice, the signal rate of change is limited by a first-order low-pass filter, with the time constant of the first-order low-pass filter set to 10 seconds to ensure a smooth rate of change.

[0037] In practical implementation, the straight-ahead phase of the driving scenario phase identifier corresponds to a vehicle longitudinal acceleration of less than 0.5 m / s² and an absolute steering angle of less than 5 degrees; the turning phase corresponds to a steering angle of greater than 15 degrees; and the parking phase corresponds to a vehicle speed of less than 1 m / s² and an obstacle distance of less than 2 meters. In practical implementation, historical data of the environmental complexity index is stored in a circular buffer with a capacity of 1000 elements, each element being a sample value of the environmental complexity index. In practical implementation, the threshold adjustment is triggered when the change in the environmental complexity index exceeds 0.1 standard deviations to avoid jitter caused by frequent adjustments. In practical implementation, the input to the service activation probability comes from the output queue of the random walk module, which is updated once per second, containing the probability values ​​of the latest 100 paths. In practical implementation, the mapping table of qualified service paths is stored in flash memory, mapping the path node sequence to specific service operation codes, which are 8-bit integers. In practice, the starting delay unit for personalized service commands is milliseconds, with a range of 0 to 1000 milliseconds. The execution duration unit is seconds, with a range of 1 to 10 seconds.

[0038] See Figure 5 This figure illustrates the collaborative optimization trends of the model optimization module in a personalized information service system for autonomous vehicles across three core performance indicators: accuracy, inference latency, and memory usage. Model accuracy shows a continuous upward trend, reflecting the model's enhanced ability to recognize occupant intentions and scene complexity through iterative optimization. Inference latency shows a continuous downward trend, ensuring the real-time responsiveness of information services in autonomous driving scenarios. Memory usage shows a gradual decrease, improving the feasibility of deploying the model on in-vehicle hardware. The technical value of this figure lies in its verification of the effectiveness of techniques such as model lightweighting, inference engine acceleration, and feature distillation through multi-dimensional performance collaborative optimization. This provides performance assurance for the engineering implementation of personalized information service systems and represents a visual representation of the core achievements of the model optimization module.

[0039] Example 4: In the specific implementation, passenger feedback signals during real-time monitoring of the service delivery process are collected using a bio-radar sensor to acquire the passengers' micro-motion signals. The bio-radar sensor operates at a frequency of 5.8 GHz, with a sampling rate of 50 Hz and a detection range of 0.5 to 2 meters. It is installed in the center of the vehicle roof. In the specific implementation, time-frequency analysis is performed on the micro-motion signals using a short-time Fourier transform algorithm. The window function is a Hamming window with a window length of 256 sampling points and an overlap rate of 50%. Physiological rhythm patterns are extracted, including respiratory frequency harmonic components and heart rate variability spectral density. In the specific implementation, the physiological rhythm patterns are matched with baseline patterns in a pre-stored comfort pattern library. The comfort pattern library contains 100 baseline patterns, each a 30-second time-frequency feature sequence stored as a floating-point array. In the specific implementation, a pattern similarity score is calculated. A dynamic time warping algorithm is used to calculate the minimum path distance between the physiological rhythm pattern and the baseline pattern, mapping the distance value to a similarity score of 0-1. In practice, an occupant comfort index is generated based on the pattern similarity score. The occupant comfort index value is the moving average of the similarity scores, and the moving average window length is 10 seconds. In practice, the timing parameters of service commands are adjusted based on the occupant comfort index. These timing parameters include command execution delay and command interval period. For every 0.1 decrease in the occupant comfort index, the command execution delay increases by 100 milliseconds, and the command interval period increases by 200 milliseconds.

[0040] In practice, the timing parameters of service instructions are adjusted based on the occupant comfort index. When the occupant comfort index is lower than the dynamic comfort threshold (the 30th percentile of the historical comfort index), the instruction execution delay is increased and the instruction interval is extended, with the delay increment proportional to the index difference. Conversely, when the occupant comfort index is higher than the dynamic comfort threshold, the instruction execution delay is reduced and the instruction interval is shortened, with the interval shortening ranging from 10% to 30% of the baseline interval. The adjusted timing parameters and corresponding occupant comfort indices are recorded in real-time, updating the baseline patterns in the comfort mode library. The weight of the new baseline pattern is 0.9 times that of the old baseline pattern plus 0.1 times that of the new data. Furthermore, the pattern similarity score is recalculated based on the updated baseline pattern, and the parameters of the dynamic time warping algorithm are updated using incremental learning to iteratively optimize the timing parameter adjustment strategy.

[0041] In the specific implementation, data acquisition from the bio-radar sensor is transmitted to the main processor via a serial peripheral interface. The clock frequency of the serial peripheral interface is 2 MHz, and the data frame format is a 16-bit signed integer. In the specific implementation, the spectral resolution of the short-time Fourier transform is approximately 0.2 Hz, and the temporal resolution is approximately 5 seconds. The extracted respiratory frequency harmonic components include the fundamental and second harmonic amplitudes. In the specific implementation, the heart rate variability spectral density is calculated using the periodogram method, with the frequency range divided into ultra-low frequency band, low frequency band, and high frequency band, and the power spectral density of each band is calculated separately. In the specific implementation, the baseline patterns of the comfort mode library are stored in a circular buffer with a size of 100 pattern entries, each occupying 4 kilobytes of storage space. In the specific implementation, the path constraint of the dynamic time warping algorithm is set as a ramp constraint with a slope of 0.5 to 2, and the distance metric uses Euclidean distance. In the specific implementation, the mapping function for the pattern similarity score is an exponential decay function, and the distance value d is mapped to the similarity score exp(-d / 10). In the specific implementation, the moving average calculation of the occupant comfort index uses an exponentially weighted moving average with a decay factor set to 0.9. The adjustment range for command execution latency is limited to 0 to 2000 milliseconds, and the adjustment range for command interval is limited to 1 to 10 seconds. The dynamic comfort threshold is updated every minute, recalculating the 30th percentile of the most recent 300 comfort index samples during each update. The baseline pattern update process includes outlier detection, defined as patterns whose distance from the current pattern library average exceeds 3 standard deviations. The incremental learning method uses stochastic gradient descent with a learning rate set to 0.01, and each update uses only the 10 most recently acquired pattern samples.

[0042] In the specific implementation, referring to Table 1, the optimization objective of the timing parameter adjustment strategy is to minimize the variance of the comfort index. The optimization process uses gradient descent to find the optimal parameters. In the specific implementation, the calibration process of the bio-radar sensor is performed monthly. The calibration process corrects the system gain by measuring the micro-motion signals of a standard target. In the specific implementation, the extraction of physiological rhythm patterns includes a motion artifact removal algorithm, which detects and removes abnormal signal segments based on wavelet transform. In the specific implementation, the comfort pattern library is initialized using occupant data from the previous 10 days, and the training process uses K-means clustering to generate 100 typical patterns. In the specific implementation, the real-time requirement for pattern matching is a processing latency of less than 100 milliseconds to ensure the timeliness of timing parameter adjustments. In the specific implementation, timing parameter adjustment commands are sent to the execution control module via the controller area network bus, with a transmission rate of 500 kilobits per second. In the specific implementation, historical data of the occupant comfort index is stored in a circular log file, with a maximum capacity of 10,000 records. In the specific implementation, the adjustment step size for instruction execution delay is set to 50 milliseconds, and the adjustment step size for instruction interval period is set to 100 milliseconds to avoid parameter abrupt changes. In the specific implementation, the anomaly detection of the dynamic comfort threshold includes a rate-of-change limitation; a protection mechanism is triggered when the rate of change exceeds 0.1 per second. In the specific implementation, the weighting coefficients of the baseline mode update process are dynamically adjusted based on mode quality, which is evaluated using the signal-to-noise ratio.

[0043] Table 1: Timing Parameter Adjustment Correspondence Table

[0044] In the implementation, the bio-radar sensor's antenna is designed as a patch antenna array, containing four radiating elements with a beamwidth of 120 degrees, covering the entire seat area. The comfort mode library retrieval uses a KD-tree to accelerate nearest neighbor search, with the KD-tree being built daily. The calculation of the occupant comfort index includes a quality factor check; sampling points with a quality factor below 0.5 are excluded from the moving average calculation. The timing parameter adjustment instruction format is a 32-bit data packet containing delay adjustment values, period adjustment values, and timestamp information. The weighting in the baseline mode update process uses a hybrid floating-point operation with single-precision IEEE 754 precision.

[0045] In implementation, the real-time display of pattern similarity scores is provided through the in-vehicle interface, with an update frequency of 1 Hz, allowing occupants to monitor the system status. The bio-radar sensor's power management includes an automatic sleep mode, automatically entering a low-power state when no occupant is detected. The results of time-frequency analysis are cached in a temporary buffer, storing the spectrum data for the most recent 5 seconds for real-time display. The comfort mode library backup mechanism is executed daily, with backup data stored in non-volatile memory to prevent data loss. The dynamic comfort threshold is calculated using a fast selection algorithm with a time complexity of O(n). Historical records of timing parameter adjustments are saved to an audit log, used for system performance analysis and troubleshooting. The bio-radar sensor's temperature compensation circuit corrects for the impact of ambient temperature changes on measurement accuracy in real time.

[0046] Example 5: In the specific implementation, during the periodic model optimization phase, passenger feedback signal sequences and adjusted time-series parameter sequences are collected from historical service delivery records. The historical service delivery records span 30 days and contain 10,000 records. Each record includes a timestamp, passenger feedback signal values, and time-series parameter values. In the specific implementation, a model retraining dataset is constructed. The model retraining dataset extracts feature vectors from the historical service delivery records. The feature vectors have a 128-dimensional dimension and include statistical features of passenger feedback signals and trends in time-series parameter changes over the past week. In the specific implementation, the latest baseline pattern sequence is extracted from the comfort mode library. The latest baseline pattern sequence consists of 100 sets of baseline patterns updated most recently, and this latest baseline pattern sequence is injected into the graph neural network model as node features. In the specific implementation, graph convolution operations are performed to update the edge weights and node embeddings of the service association graph. The graph convolution operation undergoes 5 iterations, each iteration including feature transformation and neighbor node information aggregation. In the specific implementation, a random walk algorithm is used to verify the path consistency of the updated service association graph. The path consistency index is calculated as the correlation coefficient of path weights with the same start and end points. In practice, the service activation probability distribution is recalculated and updated every 24 hours. During the update process, the probability distribution of the previous period is retained as the prior distribution.

[0047] In implementation, historical service delivery records are stored using a time-series database. The database is partitioned by date, with each partition containing all service delivery records for that day. The sampling interval for occupant feedback signal sequences is 1 second, and each signal sample includes three dimensions: facial expression change rate, voice response delay, and physiological rhythm pattern similarity score. The adjusted time-series parameter sequence records the time point and adjustment value of each parameter adjustment, including instruction execution delay and instruction interval period. The model retraining dataset construction process includes a data cleaning step, which removes outlier records and incomplete records with more than 30% missing values. Feature vector extraction uses a sliding window method with a 7-day window size and a 1-day sliding step. The latest baseline pattern sequence is extracted by querying the comfort mode library's version management log to obtain the most recently updated pattern data. The node feature injection process concatenates the baseline pattern sequence with the original occupant behavior feature vectors to form an expanded node feature vector. In the implementation, the parameter updates for graph convolution operations use stochastic gradient descent with momentum, setting the momentum coefficient to 0.9 and the learning rate to 0.001. The edge weight updates for the service association graph are recalculated based on node feature similarity, using cosine similarity as the similarity metric. Path consistency verification generates 1000 random walk paths, calculating the weight correlation between these paths and historical paths. The recalculation of the service activation probability distribution employs a Bayesian update rule, combining the prior distribution with the new likelihood function to obtain the posterior distribution.

[0048] In implementation, historical service delivery records are collected via a distributed message queue with a throughput of 1000 messages per second and a message retention period of 30 days. Crew feedback signal sequences are stored in a columnar format to improve data retrieval efficiency, with each signal dimension stored separately. Adjusted time-series parameter sequences are associated with their corresponding service delivery records via transaction identifiers, which are globally unique 64-bit integers. The model retraining dataset is divided into training, validation, and test sets in a 7:2:1 ratio. The training set is used for model parameter updates, the validation set for hyperparameter tuning, and the test set for final evaluation. Feature vector normalization uses z-score standardization, with the mean and standard deviation of each feature dimension calculated from the training set. Version management of the latest baseline pattern sequence uses a git-like mechanism, recording the changes and timestamps for each pattern update. In the implementation, the graph neural network model with node feature injection expands the original input layer dimension, increasing the number of input layer neurons from 256 to 384. The number of layers in the graph convolution operation remains unchanged at 5, but the output dimension of each layer is adjusted to 192 dimensions to accommodate the expanded feature space. A decay factor is introduced into the edge weight update formula, set to 0.95, to maintain the smoothness of weight changes. Detailed reports are generated based on the path consistency verification results, including the weight change rate and consistency score for each path. The service activation probability distribution is stored using a probabilistic graphical model, where nodes represent service types and edges represent conditional probabilities.

[0049] In implementation, the trigger condition for the periodic model optimization phase is set to occur every 7 days, with the trigger time scheduled during the early morning hours when system load is low. Preprocessing of the occupant feedback signal sequence includes outlier detection using the Isolation Forest algorithm to remove signal points that significantly deviate from the normal pattern. The sampling frequency of the time-series parameter sequence is synchronized with the feedback signal sequence, both being once per second to ensure time alignment. The construction of the model retraining dataset is automated, with the automated process including data extraction, feature engineering, and dataset export. The comfort mode library's version rollback mechanism allows for reverting to a previous stable version when model performance degrades. Mutual information is used for feature selection before node feature injection, selecting the feature dimensions most relevant to occupant comfort. The training cycle for graph convolution operations is set to 100 epochs, with an early stopping patience value of 10 epochs to prevent overfitting. A visualization tool for the service association graph provides a graphical interface displaying the strength of relationships and feature distribution between nodes. In practice, the random walk algorithm's parameters are tuned using a grid search, which includes the walk step size and restart probability. The service activation probability distribution update log records detailed parameters for each change; this log is used for auditing and troubleshooting.

[0050] In practical implementation, the backup strategy for historical service delivery records adopts incremental backup, which is performed daily, while full backup is performed weekly. In practical implementation, the compressed storage of occupant feedback signal sequences uses a lossy compression algorithm with a compression ratio set to 10:1. In practical implementation, the index construction of the adjusted time-series parameter sequences uses B+ trees, with timestamps as the keys to improve query efficiency. In practical implementation, the version control of the model retraining dataset is bound to the model version, with each model version corresponding to a dataset snapshot. In practical implementation, access control for the comfort mode library uses role-based control, which distinguishes between system administrator and ordinary user permissions. In practical implementation, the real-time requirement for node feature injection is a processing latency of less than 50 milliseconds to ensure timely model updates. In practical implementation, parallel computation of graph convolution operations is accelerated using a graphics processing unit (GPU), with a computing power of at least 10 TFLOPS. In practical implementation, persistent storage of the service association graph uses a graph database, which supports complex graph query operations. In practice, the automated test suite for path consistency verification includes 100 test cases covering various typical driving scenarios. The service activation probability distribution monitoring dashboard displays the probability change curves in real time, with an update frequency of 1 Hz.

[0051] In practical implementation, the performance evaluation metrics during the periodic model optimization phase include three dimensions: model accuracy, inference latency, and memory usage. In practical implementation, the quality assessment of occupant feedback signal sequences uses the signal-to-noise ratio (SNR) metric; signal segments with an SNR below 20 dB are marked as low-quality data. In practical implementation, anomaly detection for time-series parameter sequences uses a change-point detection algorithm to identify moments of parameter mutation. In practical implementation, the distribution balance check of the model retraining dataset uses KL divergence; data resampling is triggered when the KL divergence exceeds a threshold. In practical implementation, the integrity verification of the comfort mode library uses cyclic redundancy check (CRC) codes, with the generator polynomial being 0xEDB88320. In practical implementation, dimensionality verification for node feature injection is performed before each operation; an anomaly handling process is triggered when dimensions do not match. In practical implementation, the gradient pruning threshold for graph convolution operations is set to 1.0 to prevent gradient explosion. In practical implementation, backup and recovery tests of the service association graph are performed monthly to ensure system reliability.

[0052] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A human-machine interaction autonomous vehicle occupant personalized information service system, characterized in that, The system comprises: a data capture module for capturing multi-modal sensor data streams in real time during the interaction of the occupant with the vehicle-mounted interface, and dividing data segments based on the occurrence time of the interaction event to generate a time-aligned interaction dataset; a feature extraction module for extracting features from the time-aligned interaction dataset to obtain an occupant behavior feature vector and a vehicle environment feature matrix; a graph neural network module for inputting the occupant behavior feature vector and the vehicle environment feature matrix into a pre-trained graph neural network model, updating node representations through multi-layer graph convolution operations, and outputting a service association graph; a random walk module for applying a random walk algorithm on the service association graph to generate an occupant preference path sequence and calculating a service activation probability based on path weights; a dynamic adjustment module for dynamically adjusting the threshold of the service activation probability according to the phase change of the real-time driving scene, and selecting a service path with a probability higher than the threshold to generate a personalized service instruction; an execution control module for converting the personalized service instruction into a specific operation signal through a vehicle-mounted execution unit, and controlling an information output device to deliver services.

2. The human-interactive, automated driving car passenger individualized information service system of claim 1, wherein, The real-time capturing of multi-modal sensor data streams during the interaction of the occupant with the vehicle-mounted interface, and the division of data segments based on the occurrence time of the interaction event to generate a time-aligned interaction dataset, comprises: synchronously collecting occupant facial expression data, gesture trajectory data and voice command data using high-frame-rate visual sensors and microphone arrays; detecting the starting point and ending point of the interaction event, and intercepting multi-modal data within a fixed time window before and after the event center as a reference; performing timestamp alignment processing on the intercepted data segments to eliminate the time sequence deviation between different sensors; extracting key frames and audio features in each data segment and fusing to generate multi-dimensional feature vectors; arranging the multi-dimensional feature vectors of all data segments in chronological order to construct the time-aligned interaction dataset.

3. The human-interactive, automated driving car passenger individualized information service system of claim 2, wherein, The feature extraction from the time-aligned interaction dataset to obtain an occupant behavior feature vector and a vehicle environment feature matrix, comprises: processing visual data segments using a convolutional autoencoder to compress image dimensions and extract spatial features, generating an occupant attention distribution vector; processing audio data segments through a recurrent neural network to learn the time sequence pattern of voice commands, generating an occupant intent encoding vector; reading environmental sensor data from the vehicle bus system, including vehicle speed, steering angle and obstacle distance, and calculating an environmental complexity index; concatenating the occupant attention distribution vector and the occupant intent encoding vector into an occupant behavior feature vector; combining the environmental complexity index with historical environmental data within the time window into a vehicle environment feature matrix.

4. The human-interactive, automated driving car passenger individualized information service system of claim 3, wherein, The inputting of the occupant behavior feature vector and the vehicle environment feature matrix into a pre-trained graph neural network model, the updating of node representations through multi-layer graph convolution operations, and the outputting of a service association graph, comprises: constructing a graph structure, wherein nodes represent different service types and edges represent the association strength between services; The initialization node feature is a passenger behavior feature vector, the edge feature is a similarity score of a vehicle environment feature matrix, a graph convolution layer operation is performed, neighbor node information is aggregated and node embedding representation is updated, an original feature is reserved through a residual connection to prevent gradient disappearance, and after multiple iterations, a stable node representation service association graph is output.

5. The human-interactive, automated driving car passenger individualized information service system of claim 4, wherein, The random walk algorithm is applied on the service association graph to generate a passenger preference path sequence, and a service activation probability is calculated based on a path weight, including: Starting from a root node of the service association graph, an adjacent node is selected with a transition probability to generate multiple random walk paths; A weight of each path is calculated, the weight is determined by an inner product of edge features and node features between nodes in the path, the path weight is normalized to obtain an activation probability of each path, the first k paths with the highest activation probability are selected as the passenger preference path sequence, and a service activation probability is adjusted according to a cosine similarity between a current passenger behavior feature vector and a path node.

6. The human-interactive, automated driving car passenger individualized information service system of claim 5, wherein, The threshold value of the service activation probability is dynamically adjusted according to a phase change of a real-time driving scene, and a personalized service instruction is generated by selecting a service path with a probability higher than the threshold value, including: A driving scene phase identifier is obtained from a vehicle control system, including a straight driving phase, a turning phase and a parking phase; A basic threshold value is set for each driving scene phase, and the threshold value is dynamically offset based on an environment complexity index; The service activation probability is compared with the adjusted threshold value, qualified service paths are screened, the qualified service paths are mapped to specific service operations, and a personalized service instruction containing a service type and an execution timing is generated.

7. The human-interactive, automated driving car passenger individualized information service system of claim 6, wherein, The personalized service instruction is converted into a specific operation signal by a vehicle-mounted execution unit, and a control information output device is controlled to deliver a service, including: A service type parameter in the personalized service instruction is parsed, and a vehicle-mounted device driving protocol is matched; Service timing data is converted into a pulse width modulation signal to control the brightness change of a display screen or the volume gradient of a loudspeaker; Passenger feedback signals in a service delivery process are monitored in real time, and the strength of an operation signal is dynamically adjusted according to the feedback signals to ensure service delivery smoothness.

8. The human-interactive, automated driving car passenger individualized information service system of claim 7, wherein, The real-time monitoring of the passenger feedback signals in the service delivery process further includes: collecting micro-motion signals of the passenger by a biological radar sensor, and extracting a physiological rhythm pattern through time-frequency analysis of the micro-motion signals; the physiological rhythm pattern is matched with a reference pattern in a pre-stored comfort pattern library to calculate a pattern similarity score; a passenger comfort index is generated according to the pattern similarity score; and the timing parameters of the service instruction, including instruction execution delay and instruction interval period, are adjusted according to the passenger comfort index.

9. The human-interactive, automated driving car passenger individualized information service system of claim 8, wherein, The adjustment of the timing parameters of the service instruction according to the passenger comfort index includes: when the passenger comfort index is lower than a dynamic comfort threshold value, increasing the instruction execution delay and prolonging the instruction interval period; when the passenger comfort index is higher than the dynamic comfort threshold value, reducing the instruction execution delay and shortening the instruction interval period; the adjusted timing parameters and the corresponding passenger comfort index are recorded in real time, and the reference pattern in the comfort pattern library is updated; the pattern similarity score is recalculated according to the updated reference pattern, and the timing parameter adjustment strategy is iteratively optimized.

10. The human-interactive, automated driving car passenger individualized information service system, according to claim 9, wherein, The method further comprises a periodic model optimization phase: collecting a sequence of passenger feedback signals and an adjusted sequence of timing parameters in historical service delivery records, constructing a model retraining data set; extracting the latest benchmark mode sequence from the comfort mode library and injecting it as a node feature into the graph neural network model; performing graph convolution operation to update the edge weight and node embedding of the service association graph; verifying the path consistency of the updated service association graph through the random walk algorithm, and recalculating the service activation probability distribution.

Images