Control method of vehicle-mounted multimedia system, electronic equipment and storage medium

Abstract

The invention discloses a control method of a vehicle-mounted multimedia system, electronic equipment and a storage medium, and relates to the technical field of vehicle-mounted multimedia, and the control method comprises a control module which is used for receiving a playing permission request generated by a target multimedia application and carrying out safety verification on the target multimedia application; the backup module is used for carrying out redundancy design on the control module, and when the control module breaks down, the backup module is switched to take over the multimedia system; monitoring the system state of the control module in real time, and identifying whether the system state of the control module is abnormal or not; through redundancy design and real-time monitoring, the stability and reliability of the system are improved, continuous operation of the vehicle-mounted multimedia system under various conditions is ensured, through fusion of multiple input modes, a user can more naturally interact with the system, the complexity and distraction of operation are reduced, and the recognition accuracy is improved; and the misrecognition risk possibly caused by a single mode is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle-mounted multimedia technology, and more specifically, to a control method, electronic equipment and storage medium of a vehicle-mounted multimedia system. Background Art

[0002] The in-vehicle multimedia system is an integrated infotainment system in modern cars, which provides a variety of functions and services to drivers and passengers through one or more display screens and corresponding software systems; The Chinese patent with publication number CN111332224B discloses a control method and device for an in-vehicle multimedia system, wherein the method includes: obtaining a user operation event for controlling a vehicle; adjusting the power mode of the vehicle according to the user operation event and the current vehicle state; wherein the power mode includes a sleep mode, a standby mode, a screen saver mode and a working mode; and controlling the multimedia system of the vehicle to work based on the adjusted power mode. The present invention can effectively improve the response speed of the multimedia system and reduce power consumption; However, the in-vehicle multimedia system usually processes multimedia data by a processor and sends the processed data to the multimedia interface. If the processor fails, the entire in-vehicle multimedia system will be difficult to operate, especially for systems with touch functions. The processor also needs to interact and process data with the touch components, and the possibility of failure of related components or lines is greater. The current control method of the in-vehicle multimedia system relies too much on a single processor. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a control method, an electronic device and a storage medium for a vehicle-mounted multimedia system.

[0004] The present invention provides a control method for a vehicle-mounted multimedia system, comprising the following steps: A control module, configured to receive a playback permission request generated by a target multimedia application and perform security verification on the target multimedia application; Control the reception of target permission status and notify the target multimedia application to obtain playback permission for playback; A backup module is used to perform redundancy design on the control module. When the control module fails, the backup module switches to take over the multimedia system. The system status of the control module is monitored in real time to identify whether the system status of the control module is abnormal. If abnormal, the control module is switched to the backup module, an alarm signal is generated, and the control module is restarted or self-checked and repaired.

[0005] Preferably, the control module further includes a collaborative interaction unit, which is used to fuse inputs from different modalities, and the specific working method is as follows: Collect input data from different modalities and pre-process the collected data; Convert data from different modalities into a computer-processable form and extract key features from the data of each modality; The extracted features are merged and corresponding operations are performed based on the merged information.

[0006] Preferably, the specific manner of merging the extracted features is as follows: Convert the extracted features into vector representation; Obtain the weight value of each mode, perform weighted merging, and obtain the merged feature vector; The user intention is identified based on the merged feature vector.

[0007] Preferably, the specific working mode of the backup module is as follows: Obtain the current load of the multimedia system; Collect historical failure data of multimedia systems and further analyze and identify the trend of system failures; According to the results of further analysis, the allocation of redundant resources is dynamically adjusted; the hardware and software architecture of the multimedia system is modularized so that each module can be flexibly switched and shared between the main control module and the backup module.

[0008] Preferably, the backup module further includes a perception switching unit, and the specific working method of the perception switching unit further includes: Collect vehicle status information in real time and conduct environmental assessment on the vehicle to determine whether the current status is safe; When the backup module executes the switching control module to take over the multimedia system, the current state is obtained. If it is safe, the switching is executed. If not, a signal is generated and transmitted to the user's display interface to prompt the vehicle to stop or switch at a low speed.

[0009] Preferably, the specific working method of the perception switching unit also includes: When the current state is confirmed to be safe and the switch is executed, the switch is seamlessly performed in real time; Set a threshold based on historical interaction data to determine the low-frequency period of user interaction; When the user enters a low-frequency period of interaction, switching is performed, and the switching control module is executed to take over the multimedia system; The design control module gradually fades out of the current interface and loads the backup module interface at the same time.

[0010] Preferably, the collaborative interaction unit specifically includes: By using deep learning models, data from different modalities are fused into a unified representation space; Achieve unified representation of multimodal input sequences at a fine-grained level; Pre-training enables unified multimodal expression, allowing the model to transfer knowledge learned in one modality to another.

[0011] An electronic device is also provided, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the control method of the in-vehicle multimedia system according to claim 1.

[0012] A storage medium is also provided, comprising a readable storage medium, wherein the readable storage medium is executed by the processor; When the processor executes the readable storage medium, the electronic device executes the control method of the in-vehicle multimedia system as claimed in claim 1.

[0013] Beneficial effects: Through redundant design and real-time monitoring, the stability and reliability of the system are improved, ensuring the continuous operation of the in-vehicle multimedia system in various situations. By integrating multiple input methods, users can interact with the system more naturally, reducing the complexity and distraction of operation and improving recognition accuracy: multi-modal input can complement each other and reduce the risk of misrecognition that may be caused by a single mode; By dynamically adjusting redundant resources, the system can maintain stable operation during high load or potential failure periods, reducing the risk of system crashes; resources are dynamically allocated based on real-time load and historical failure data to ensure that resources are available when needed, while avoiding over-configuration and improving resource utilization efficiency; the system can respond quickly to user requests and maintain a smooth user experience even under high load conditions; through predictive maintenance and automatic failover, the need for emergency repairs is reduced, thereby reducing maintenance costs and system downtime. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0015] Application scenarios: In-vehicle multimedia systems usually use a processor to process multimedia data and send the processed data to the multimedia interface. If the processor fails, the entire in-vehicle multimedia system will be difficult to operate, especially for systems with touch functions. The processor also needs to interact and process data with the touch components, and the possibility of related components or lines failing is greater. The current control method of in-vehicle multimedia systems relies too much on a single processor.

[0016] like Figure 1 As shown: A control method for a vehicle-mounted multimedia system comprises the following steps: A control module, used to receive a playback permission request generated by a target multimedia application and perform security verification on the target multimedia application; it should be noted that the control module is run by a processor of the vehicle multimedia system; Control the target permission status reception and notify the target multimedia application to obtain the play permission for playing; in this embodiment, the specific method is as follows: The control module monitors and receives playback permission requests from the target multimedia application, parses the received requests, extracts necessary information, such as the package name of the application, the requested playback content, etc., and prepares the data required for security verification, which may include the application's digital certificate, previous permission records, etc.; Verify that the digital certificate provided by the multimedia application is valid, ensure that the request comes from a trusted source, check that the request contains a valid digital signature to ensure the integrity and authenticity of the request, and check that the application has the right to send the playback request according to the system security policy, which may involve querying the application's permission list or whitelist; A backup module is used to perform redundancy design on the control module. When the control module fails, the backup module switches to take over the multimedia system. The system status of the control module is monitored in real time to identify whether the system status of the control module is abnormal. If abnormal, the control module is switched to the backup module, an alarm signal is generated, and the control module is restarted or self-checked and repaired.

[0017] Through redundant design and real-time monitoring, the stability and reliability of the system are improved, ensuring the continuous operation of the in-vehicle multimedia system under various circumstances; For example, during driving, if the control module of the in-vehicle multimedia system fails due to software failure or hardware damage, the backup module will immediately take over and continue to provide multimedia playback services to ensure that the entertainment experience of the driver and passengers is not affected. The system will monitor and prompt the fault status in real time, trigger an alarm and start the fault recovery process, such as restarting the control module or performing self-check and repair, so as to quickly restore the normal operation of the system; The in-vehicle multimedia system is an integrated infotainment system in modern cars, which provides a variety of functions and services to drivers and passengers through one or more display screens and corresponding software systems; In-vehicle multimedia systems usually use a processor to process multimedia data and send the processed data to the multimedia interface. If the processor fails, the entire in-vehicle multimedia system will be difficult to operate, especially for systems with touch functions. The processor also needs to interact and process data with the touch components, and the possibility of related components or lines failing is greater. The current control method of in-vehicle multimedia systems relies too much on a single processor.

[0018] As an optional embodiment: the control module further includes a collaborative interaction unit, and the collaborative interaction unit is used to fuse inputs from different modalities, and the specific working method is as follows: Collect input data from different modalities and pre-process the collected data, such as noise reduction, normalization, feature extraction, etc., to facilitate subsequent fusion processing; Convert data of different modalities into a computer-processable form and extract key features from the data of each modality; in this embodiment, CNN can be used to extract features from images, RNN or LSTM can be used to extract features from speech, and pre-trained models such as BERT can be used to extract features from text; The extracted features are merged and corresponding operations are performed based on the merged information.

[0019] For example, if a driver says "play my favorite music" through voice commands and swipes right to select the next song, the voice input will be collected and recognized first to extract the "play music" command. At the same time, the camera will capture the driver's gesture and recognize the "swipe right" action. The features of these two modalities will be combined to form a comprehensive user intent representation, and then the corresponding operation will be performed based on this intent. By integrating multiple input methods, users can interact with the system more naturally, reducing the complexity and distraction of operations and improving recognition accuracy: multimodal input can complement each other and reduce the risk of misrecognition that may be caused by a single modality.

[0020] As an optional embodiment: the specific manner of merging the extracted features is as follows: Convert the extracted features into vector representations; in this embodiment, for example, the extracted speech features are represented as vector v1, and the extracted gesture features are represented as vector v2; Obtain the weight value of each mode, perform weighted merging, and obtain the merged feature vector; it should be noted that, in this embodiment, the weight value of each mode is allocated according to the importance of each mode, and the sum of the weight values ​​is 1; For example, the weight of vector v1 is a, the weight of vector v2 is b, and a plus b is 1. The merged feature vector is obtained by multiplying a times v1 and b times v2. The user intention is identified based on the merged feature vector. It should be noted that, in this embodiment, the merged feature vector can be used to input into a classifier (such as a fully connected layer or a deep neural network) to identify the user's intention.

[0021] As an optional embodiment: the specific working mode of the backup module is as follows: The current load of the multimedia system is obtained; it should be noted that, in this embodiment, the current load can be obtained by collecting the CPU usage rate, memory usage rate, and I / O operation rate of the multimedia system. These indicators can be collected by a monitoring tool running on the processor of the vehicle multimedia system. A weighted average method is used, in which each performance indicator is assigned a different weight according to its influence on the system load. The weight can be determined according to the specific requirements and historical data of the system. The CPU and memory usage rates are the most important indicators, and the weights can be increased accordingly. The sum of the weights is 1; For example, suppose the in-vehicle multimedia system is running during a high-traffic period, with frequent user interactions, causing the CPU usage to increase to 75%, the memory usage to increase to 65%, and the I / O operation rate to increase to 200 times per second. The monitoring tool collects this data in real time and calculates the current load using the following weighted average formula; In actual usage, fixed-weight multimodal fusion may not be able to adapt to all situations, especially when user behavior patterns change; Dynamically adjust the weight of each modality based on the user's contextual information and historical behavior data; Reinforcement learning or online learning algorithms can be used to enable the system to adaptively adjust weights based on user feedback, specifically: set up is the weight of the ith mode, is the corresponding eigenvector, according to the formula , get the fused feature vector V; The weight It can be adjusted dynamically according to the following formula; , calculated, where is the preset learning rate coefficient, is the prediction deviation of the i-th mode; Collect historical fault data of the multimedia system, further analyze it, and identify the trend of system faults; in this embodiment, the historical fault data includes equipment fault reports, maintenance records, operator feedback and related documents. These data should contain information such as fault description, cause analysis, solution and repair steps. By analyzing these data, the patterns and trends of system faults and their relationship with system load and operating environment can be identified; It should also be noted that, in this embodiment, the risk priority number can be calculated based on the key elements in the historical fault data, specifically including the failure mode, impact, cause, severity, occurrence rate and detectability, and the risk priority can be evaluated by weighted sum of the key elements. The time period with a priority greater than a threshold is marked as a time period where a failure may occur. The specific weight of each key element can be determined according to the specific requirements of the system and historical data; Based on the results of further analysis, the allocation of redundant resources is dynamically adjusted; for example, if the system predicts a high-load period or a time period where failures may occur, processing power and storage resources can be automatically increased, such as load balancing, resource elasticity, high availability and failover, and predictive resource scheduling strategies.

[0022] The modular hardware and software architecture of the multimedia system allows each module to be flexibly switched and shared between the main control module and the backup module.

[0023] By dynamically adjusting redundant resources, the system can maintain stable operation during high load or potential failure periods, reducing the risk of system crashes; resources are dynamically allocated based on real-time load and historical failure data to ensure that resources are available when needed, while avoiding over-configuration and improving resource utilization efficiency; the system can respond quickly to user requests and maintain a smooth user experience even under high load conditions; through predictive maintenance and automatic failover, the need for emergency repairs is reduced, thereby reducing maintenance costs and system downtime.

[0024] As an optional embodiment: the backup module further includes a perception switching unit, and the specific working method of the perception switching unit further includes: Collect vehicle status information in real time and conduct environmental assessment of the vehicle to determine whether the current status is safe. It should be noted that the vehicle status information uses on-board sensors (such as GPS, accelerometer, gyroscope, etc.) to collect vehicle speed, acceleration, direction and other status information in real time. These data can be obtained through monitoring tools running on the processor of the on-board multimedia system. Environmental assessment of the vehicle, which can be achieved by analyzing the vehicle's surroundings, including traffic conditions, weather conditions, etc., such as using on-board cameras, radar or interfaces with external data services (such as traffic information providers); When the backup module executes the switching control module to take over the multimedia system, the current state is obtained. If it is safe, the switching is executed. If not, a signal is generated and transmitted to the user's display interface to prompt the vehicle to stop or switch at a low speed.

[0025] For example, if the vehicle is traveling at high speed, the vehicle system may display a message: "The current vehicle speed is too high. For safety reasons, please switch the multimedia system when the vehicle is stationary or traveling at a low speed." Such a prompt can ensure that the driver will not be distracted when switching the multimedia system, thereby avoiding possible accidents. The implementation of such safety measures can improve the safety of the vehicle in autonomous or manual driving states; Through real-time monitoring and environmental assessment, the switching operation is ensured to be carried out under safe conditions, reducing driving risks. Seamless switching is carried out during low-frequency periods of user interaction to ensure that the user experience is not affected and to avoid interference caused by switching. Through dynamic adjustment and real-time monitoring, the backup module can quickly take over when the main control module fails, ensuring the continuous operation of the multimedia system.

[0026] As an optional embodiment: the specific working method of the perception switching unit also includes: When the current state is confirmed to be safe and the switch is executed, the switch is seamlessly performed in real time; A threshold is set based on historical interaction data to determine the low-frequency period of user interaction. For example, if a user does not perform any operation within a certain period of time (such as 5 minutes), it is considered to have entered a low-frequency interaction period. When the user enters a low-frequency period of interaction, switching is performed, and the switching control module is executed to take over the multimedia system; Design an interface in which the control module gradually fades out of the current interface and loads the backup module at the same time; you can use UI transition animation design principles, such as transparency fade-in and fade-out, zoom animation, etc. to ensure the continuity and seamlessness of the user experience.

[0027] As an optional embodiment: the specific manner of the collaborative interaction unit also includes: although the system can process multimodal input, it may lack understanding of the user's intended context, for example, the user may use the same command or gesture in different contexts but expect different responses; This technical solution introduces a context-aware mechanism. The system not only needs to recognize the user's instructions and actions, but also needs to understand the context behind these instructions and actions. This can be achieved by analyzing the user's historical behavior, current environment, and possible intentions. For example, if the user says "go home" while driving, the system can automatically navigate to the user's home based on the current location and historical data, instead of playing a song called "Go Home"; By using deep learning models, data from different modalities can be fused into a unified representation space; such as Transformer or CNN, Achieve unified expression of multimodal input sequences at a fine-grained level; enable the model to better understand and process information with the same semantics Pre-training enables unified multi-modal expression, allowing the model to transfer knowledge learned in one modality to another, thus improving the generalization ability of the model.

[0028] An electronic device is also provided, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the control method of the in-vehicle multimedia system according to claim 1.

[0029] A storage medium is also provided, comprising a readable storage medium, wherein the readable storage medium is executed by the processor; When the processor executes the readable storage medium, the electronic device executes the control method of the in-vehicle multimedia system as claimed in claim 1.

[0030] How it works A control module, used to receive a playback permission request generated by a target multimedia application and perform security verification on the target multimedia application; it should be noted that the control module is run by a processor of the vehicle multimedia system; Control the target permission status reception and notify the target multimedia application to obtain the play permission for playing; in this embodiment, the specific method is as follows: The control module monitors and receives playback permission requests from the target multimedia application, parses the received requests, extracts necessary information, such as the package name of the application, the requested playback content, etc., and prepares the data required for security verification, which may include the application's digital certificate, previous permission records, etc.; Verify that the digital certificate provided by the multimedia application is valid, ensure that the request comes from a trusted source, check that the request contains a valid digital signature to ensure the integrity and authenticity of the request, and check that the application has the right to send the playback request according to the system security policy, which may involve querying the application's permission list or whitelist; A backup module is used to perform redundancy design on the control module. When the control module fails, the backup module switches to take over the multimedia system. The system status of the control module is monitored in real time to identify whether the system status of the control module is abnormal. If abnormal, the control module is switched to the backup module, an alarm signal is generated, and the control module is restarted or self-checked and repaired.

[0031] Through redundant design and real-time monitoring, the stability and reliability of the system are improved, ensuring the continuous operation of the in-vehicle multimedia system under various circumstances; For example, during driving, if the control module of the vehicle multimedia system fails due to software failure or hardware damage, the backup module will immediately take over and continue to provide multimedia playback services to ensure that the entertainment experience of the driver and passengers is not affected. The system will monitor and prompt the fault status in real time, trigger an alarm and start the fault recovery process, such as restarting the control module or performing self-inspection and repair, so as to quickly restore the normal operation of the system.

[0032] The above are only preferred implementations of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical staff in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of this template.

Claims

1. A control method for a vehicle multimedia system, characterized in that: The following steps are involved: A control module, configured to receive a playback permission request generated by a target multimedia application and perform security verification on the target multimedia application; Control the reception of target permission status and notify the target multimedia application to obtain playback permission for playback; A backup module is used to perform redundancy design on the control module. When the control module fails, the backup module switches to take over the multimedia system. The system status of the control module is monitored in real time to identify whether the system status of the control module is abnormal. If abnormal, the control module is switched to the backup module, an alarm signal is generated, and the control module is restarted or self-checked and repaired.

2. The control method of a vehicle multimedia system according to claim 1, characterized in that: The control module further includes a collaborative interaction unit, which is used to fuse inputs from different modalities. The specific working method is as follows: Collect input data from different modalities and pre-process the collected data; Convert data from different modalities into a computer-processable form and extract key features from the data of each modality; The extracted features are merged and corresponding operations are performed based on the merged information.

3. The control method of a vehicle multimedia system according to claim 2, characterized in that: The specific method of merging the extracted features is as follows: Convert the extracted features into vector representation; Obtain the weight value of each mode, perform weighted merging, and obtain the merged feature vector; The user intention is identified based on the merged feature vector.

4. The control method of a vehicle multimedia system according to claim 1, characterized in that: The specific working mode of the backup module is as follows: Obtain the current load of the multimedia system; Collect historical failure data of multimedia systems and further analyze and identify the trend of system failures; According to the results of further analysis, the allocation of redundant resources is dynamically adjusted; the hardware and software architecture of the multimedia system is modularized so that each module can be flexibly switched and shared between the main control module and the backup module.

5. The control method of the vehicle multimedia system according to claim 4, characterized in that: The backup module also includes a perception switching unit, and the specific working method of the perception switching unit also includes: Collect vehicle status information in real time and conduct environmental assessment on the vehicle to determine whether the current status is safe; When the backup module executes the switching control module to take over the multimedia system, the current state is obtained. If it is safe, the switching is executed. If not, a signal is generated and transmitted to the user's display interface to prompt the vehicle to stop or switch at a low speed.

6. The control method of the vehicle multimedia system according to claim 7, characterized in that: The specific working method of the perception switching unit also includes: When the current state is confirmed to be safe and the switch is executed, the switch is seamlessly performed in real time; Set a threshold based on historical interaction data to determine the low-frequency period of user interaction; When the user enters a low-frequency period of interaction, switching is performed, and the switching control module is executed to take over the multimedia system; The design control module gradually fades out of the current interface and loads the backup module interface at the same time.

7. The control method of a vehicle multimedia system according to claim 2, characterized in that: The specific method of the collaborative interaction unit also includes: By using deep learning models, data from different modalities are fused into a unified representation space; Achieve unified representation of multimodal input sequences at a fine-grained level; Pre-training enables unified multimodal expression, allowing the model to transfer knowledge learned in one modality to another.

8. An electronic device, applicable to the control method of a vehicle multimedia system according to claim 1, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the control method of the in-vehicle multimedia system according to claim 1.

9. A storage medium, applicable to the control method of a vehicle-mounted multimedia system according to claim 1, characterized in that: comprising a readable storage medium, the readable storage medium being executed by the processor; When the processor executes the readable storage medium, the electronic device executes the control method of the in-vehicle multimedia system as claimed in claim 1.

Images