Multi-screen interactive display and control method and system for vehicle-mounted multimedia host

Abstract

The present invention relates to the field of human-computer interaction, and discloses a multi-screen interactive display and control method and system of a vehicle-mounted multimedia host, the method comprising: constructing a multi-screen operating system for a vehicle-mounted screen, constructing a data custom protocol and synchronization logic code for the multi-screen operating system; calculating the multi-screen instant coefficient of the multi-screen operating system, and constructing an edge node of the vehicle corresponding to the vehicle of the vehicle-mounted multimedia host; constructing a multi-modal interactive system for the vehicle-mounted screen, and collecting interactive data of target users in the vehicle; analyzing the interactive tasks of the vehicle-mounted screen, and calculating the task priority coefficient of the interactive tasks, and constructing a task queue of the vehicle-mounted screen, using a trained user behavior analysis model and a multi-screen operating system, constructing a multi-screen interactive instruction for the vehicle-mounted screen, and realizing the multi-screen interactive display and control of the vehicle-mounted multimedia host through the multi-screen interactive instruction. The present invention can improve the effect of multi-screen interactive display and control.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction, and in particular to a multi-screen interactive display and control method and system for a vehicle-mounted multimedia host. Background Art

[0002] Multi-screen interactive display and control refers to the function of sharing, synchronous display and mutual control of content among multiple display screens within a system through the combination of software and hardware technologies. The purpose of multi-screen interactive display and control technology is to enhance user experience, enhance the interactivity and convenience of the system, and improve the efficiency of information processing.

[0003] Currently, the traditional method of multi-screen interactive display and control mainly relies on centralized computing processing and preset interaction modes. Usually, a central control unit is used to coordinate the display content of multiple screens, and user interaction relies on basic touch or physical button operations. This method is limited in processing speed, flexibility and personalized services, and it is difficult to meet the high requirements of modern in-vehicle infotainment systems for intelligence, efficiency and user experience, resulting in poor multi-screen interactive display and control effects. Summary of the invention

[0004] The present invention provides a multi-screen interactive display and control method and system for a vehicle-mounted multimedia host, the main purpose of which is to improve the effect of multi-screen interactive display and control.

[0005] To achieve the above-mentioned purpose, the present invention provides a multi-screen interactive display and control method of a vehicle-mounted multimedia host, comprising:

[0006] Acquire the in-vehicle screen of the in-vehicle multimedia host, determine the interaction requirements of the in-vehicle screen, and configure the multi-screen interaction environment and layered architecture of the in-vehicle screen according to the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer;

[0007] Construct a screen management module and a content distribution engine for the vehicle-mounted screen, and based on the screen management module, the content distribution engine, the multi-screen interactive environment and the layered architecture, construct a multi-screen operating system for the vehicle-mounted screen, and construct a data custom protocol and synchronization logic code for the multi-screen operating system;

[0008] Based on the data custom protocol and synchronization logic code, the multi-screen instant coefficient of the multi-screen operating system is calculated, and when the multi-screen instant coefficient meets the preset multi-screen instant threshold, the edge node of the vehicle corresponding to the vehicle-mounted multimedia host is constructed;

[0009] Constructing a multi-touch mechanism, a multi-language recognition mechanism, and a gesture recognition mechanism for the vehicle-mounted screen; constructing a multi-modal interaction system for the vehicle-mounted screen based on the multi-touch mechanism, the multi-language recognition mechanism, and the gesture recognition mechanism; and using the multi-modal interaction system to collect interaction data of a target user in the vehicle;

[0010] Based on the interaction data, the edge node is used to analyze the interaction tasks of the vehicle screen, and the task priority coefficient of the interaction task is calculated. The task queue of the vehicle screen is constructed through the task priority coefficient. Based on the task queue, the multi-screen interaction instructions of the vehicle screen are constructed using the trained user behavior analysis model and the multi-screen operating system. The multi-screen interactive display and control of the vehicle multimedia host are realized through the multi-screen interaction instructions.

[0011] Optionally, configuring the multi-screen interactive environment and layered architecture of the vehicle-mounted screen according to the interactive requirements includes:

[0012] Analyze the interaction requirements and functional requirements of the vehicle screen based on the interaction requirements;

[0013] Based on the interaction requirements, determining network indicators of the in-vehicle screen;

[0014] According to the network indicators, construct the network topology of the vehicle-mounted screen;

[0015] Based on the network topology, configuring a multi-screen interactive environment of the vehicle-mounted screen;

[0016] According to the functional requirements, the initial hierarchical architecture of the vehicle screen is configured;

[0017] Analyzing the scalability of the initial layered architecture;

[0018] When the scalability meets a preset scalability threshold, the initial hierarchical architecture is used as the hierarchical architecture of the vehicle-mounted screen.

[0019] Optionally, the screen management module and content distribution engine for constructing the vehicle-mounted screen include:

[0020] Identifying screen management function requirements, screen management performance requirements, content distribution function requirements, and content distribution performance requirements of the vehicle-mounted screen;

[0021] Constructing a screen management architecture of the vehicle-mounted screen according to the screen management function requirements and the screen management performance requirements, and configuring a screen management interface of the screen management architecture;

[0022] Based on the screen management architecture and the screen management interface, construct a screen management module for the vehicle-mounted screen;

[0023] According to the content distribution function requirements and content distribution performance requirements, construct the content distribution engine architecture of the vehicle screen, and configure the content distribution interface of the content distribution engine architecture;

[0024] The content distribution engine for the vehicle-mounted screen is constructed through the content distribution engine architecture and the content distribution interface.

[0025] Optionally, the data custom protocol and synchronization logic code for constructing the multi-screen operating system include:

[0026] Determining the data format and data structure of the multi-screen operating system;

[0027] Based on the data format and data structure, define a data transmission protocol of the multi-screen operating system;

[0028] Configuring an error handling mechanism of the multi-screen operating system;

[0029] By means of the data transmission protocol and error handling mechanism, a data custom protocol of the multi-screen operating system is constructed;

[0030] Designing synchronization trigger conditions and synchronization processes of the multi-screen operating system;

[0031] Based on the synchronization triggering conditions and synchronization process, construct a data update mechanism for the multi-screen operating system;

[0032] According to the synchronization triggering conditions, synchronization process and data update mechanism, the synchronization logic code of the multi-screen operating system is written.

[0033] Optionally, constructing a data update mechanism of the multi-screen operating system based on the synchronization trigger condition and the synchronization process includes:

[0034] Based on the synchronization triggering condition and the synchronization process, construct an initial data update mechanism of the multi-screen operating system;

[0035] Defining synchronization parameters of the initial data update mechanism, wherein the synchronization parameters include data update frequency, network transmission time, device processing time, data transmission stability factor, device synchronization factor and optimization coefficient;

[0036] The data update timeliness coefficient of the initial data update mechanism is calculated by the following formula through the synchronization parameters:

[0037]

[0038] in, represents the data update timeliness coefficient of the initial data update mechanism, Indicates the data update frequency corresponding to the synchronization parameters. Indicates the weight of the data update frequency corresponding to the synchronization parameter, Indicates the network transmission time corresponding to the synchronization parameters. Indicates the weight of the network transmission time corresponding to the synchronization parameter, Indicates the device processing time corresponding to the synchronization parameters. Indicates the weight of the device processing time corresponding to the synchronization parameter. Indicates the data transmission stability factor corresponding to the synchronization parameter, represents the weight of the data transmission stability factor corresponding to the synchronization parameter, Indicates the device synchronization factor corresponding to the synchronization parameter, Indicates the weight of the device synchronization factor corresponding to the synchronization parameter, Indicates the optimization coefficient corresponding to the synchronization parameter, Indicates the weight of the optimization coefficient corresponding to the synchronization parameter;

[0039] When the data update timeliness coefficient meets the preset data update timeliness threshold, the initial data update mechanism is used as the data update mechanism of the multi-screen operating system.

[0040] Optionally, the multi-touch mechanism, multi-language recognition mechanism and gesture recognition mechanism of the vehicle-mounted screen are constructed, including:

[0041] Configuring a multi-touch driver and a multi-touch application for the vehicle-mounted screen;

[0042] The multi-touch function of the vehicle-mounted screen is constructed using the following formula, wherein the multi-touch function:

[0043]

[0044] in, Indicates the coordinates of the touch point on the car screen corresponding to the user. Indicates the car screen The coordinates of the touch points, Indicates the car screen The pressure value of each touch point;

[0045] Constructing a multi-touch mechanism for the vehicle-mounted screen through a multi-touch driver, a multi-touch application, and a multi-touch function;

[0046] Determining a multilingual speech recognition engine for the vehicle-mounted screen and constructing a speech instruction set for the multilingual speech recognition engine;

[0047] Training a multilingual speech recognition model for the vehicle-mounted screen, and constructing a multilingual recognition mechanism for the vehicle-mounted screen through the multilingual speech recognition model, the multilingual speech recognition engine, and the speech instruction set;

[0048] Defining a gesture set and an image acquisition sensor for the vehicle-mounted screen;

[0049] Configuring a gesture recognition model for the vehicle-mounted screen;

[0050] The gesture recognition mechanism of the vehicle-mounted screen is constructed by using the gesture set, image acquisition sensor and gesture recognition model.

[0051] Optionally, configuring the gesture recognition model of the vehicle-mounted screen includes:

[0052] Collecting the voice signal sequence of the vehicle-mounted screen;

[0053] Analyzing model parameters of the vehicle-mounted screen;

[0054] Based on the model parameters and the speech signal sequence, the gesture recognition function of the vehicle screen is constructed using the following formula:

[0055]

[0056] in, represents the probability, Represents the gesture recognition function, represents the speech signal sequence, represents the model parameters, Indicates that under given conditions and the speech signal sequence corresponds to the previous observation value Under the condition of The conditional probability of Indicates that under given conditions Under the condition of The probability of

[0057] A gesture recognition model of the vehicle-mounted screen is constructed through the gesture recognition function.

[0058] Optionally, analyzing the interactive task of the vehicle-mounted screen by using the edge node based on the interactive data includes:

[0059] Preprocessing the interaction data to obtain preprocessed interaction data;

[0060] Select a target edge node among the edge nodes for processing the pre-processed interaction data;

[0061] By using the pre-processed interaction data, the target edge node corresponding intention analysis model is used to analyze the user intention of the target user corresponding to the vehicle screen;

[0062] According to the user intention, the target edge node corresponding rule engine is used to define the interactive task of the vehicle screen.

[0063] Optionally, based on the task queue, using a trained user behavior analysis model and a multi-screen operating system, constructing a multi-screen interaction instruction for the vehicle-mounted screen includes:

[0064] Identifying a queue structure of the task queue;

[0065] Analyzing the user interaction habits of the in-vehicle screen with respect to the target user using the user behavior analysis model;

[0066] Based on the queue structure and user interaction habits, constructing the multi-screen initial interaction instructions of the vehicle screen using the multi-screen operating system;

[0067] Simulating the multi-screen interaction result of the multi-screen initial interaction instruction on the vehicle screen;

[0068] Calculating the interaction seamlessness coefficient of the multi-screen interaction result;

[0069] When the interaction seamlessness coefficient meets a preset interaction seamlessness threshold, the multi-screen initial interaction instruction is used as the multi-screen interaction instruction of the vehicle-mounted screen.

[0070] In order to solve the above problems, the present invention also provides a multi-screen interactive display and control system for a vehicle-mounted multimedia host, the system comprising:

[0071] The vehicle screen acquisition module is used to acquire the vehicle screen of the vehicle multimedia host, determine the interaction requirements of the vehicle screen, and configure the multi-screen interaction environment and layered architecture of the vehicle screen according to the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer;

[0072] A multi-screen operating system construction module is used to construct a screen management module and a content distribution engine for the vehicle-mounted screen, and based on the screen management module, the content distribution engine, the multi-screen interactive environment and the layered architecture, the multi-screen operating system for the vehicle-mounted screen is constructed, and a data custom protocol and synchronization logic code for the multi-screen operating system are constructed;

[0073] An edge node arrangement module, for calculating the multi-screen instant coefficient of the multi-screen operating system based on the data custom protocol and the synchronization logic code, and constructing an edge node of the vehicle corresponding to the vehicle-mounted multimedia host when the multi-screen instant coefficient meets a preset multi-screen instant threshold;

[0074] An interactive data collection module, used to construct a multi-touch mechanism, a multi-language recognition mechanism and a gesture recognition mechanism for the vehicle-mounted screen, to construct a multi-modal interactive system for the vehicle-mounted screen based on the multi-touch mechanism, the multi-language recognition mechanism and the gesture recognition mechanism, and to collect interactive data of a target user in the vehicle using the multi-modal interactive system;

[0075] A multi-screen interaction control module is used to analyze the interaction tasks of the vehicle-mounted screen based on the interaction data using the edge node, and calculate the task priority coefficient of the interaction task. Through the task priority coefficient, a task queue of the vehicle-mounted screen is constructed. Based on the task queue, a multi-screen interaction instruction of the vehicle-mounted screen is constructed using a trained user behavior analysis model and a multi-screen operating system. The multi-screen interactive display and control of the vehicle-mounted multimedia host are realized through the multi-screen interaction instruction.

[0076] The present invention configures the multi-screen interactive environment and hierarchical architecture of the vehicle-mounted screen through the interactive requirements to achieve efficient and intelligent vehicle-mounted multi-screen interactive display and control; the present invention constructs the screen management module and content distribution engine of the vehicle-mounted screen to build a screen management module and content distribution engine that meet the functional requirements and performance requirements of the vehicle-mounted screen; further, the present invention constructs the data custom protocol and synchronization logic code of the multi-screen operating system to achieve the consistency of screen control, thereby improving the control effect of the vehicle-mounted screen, further, the present invention can construct a variety of interactive mechanisms of the vehicle-mounted screen for interaction by constructing the multi-touch mechanism, multi-language recognition mechanism and gesture recognition mechanism of the vehicle-mounted screen, thereby improving the interactive effect of the vehicle-mounted screen, further, based on the interactive data, the present invention uses the edge node to analyze the interactive tasks of the vehicle-mounted screen to effectively perform preliminary processing on real-time data, reduce dependence on the central server, and improve the response speed and efficiency of the system, and finally, the present invention can calculate the important interactive tasks first by calculating the task priority coefficient of the interactive task, thereby improving the effect of multi-screen interaction. Therefore, the present invention can improve the effect of multi-screen interactive display and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A schematic flow chart of a multi-screen interactive display and control method of a vehicle-mounted multimedia host provided by an embodiment of the present invention;

[0078] Figure 2 A functional module diagram of a multi-screen interactive display and control system of a vehicle-mounted multimedia host provided by an embodiment of the present invention;

[0079] Figure 3 A schematic diagram of the structure of an electronic device for a multi-screen interactive display and control system of a vehicle-mounted multimedia host provided by an embodiment of the present invention;

[0080] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0081] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0082] The embodiment of the present application provides a multi-screen interactive display and control method for a vehicle-mounted multimedia host. The execution subject of the multi-screen interactive display and control method of the vehicle-mounted multimedia host includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the multi-screen interactive display and control method of the vehicle-mounted multimedia host can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0083] Reference Figure 1 FIG. 1 is a flow chart of a multi-screen interactive display and control method of a vehicle-mounted multimedia host provided by an embodiment of the present invention. In this embodiment, the multi-screen interactive display and control method of the vehicle-mounted multimedia host includes:

[0084] S1. Obtain the in-vehicle screen of the in-vehicle multimedia host, determine the interaction requirements of the in-vehicle screen, and configure the multi-screen interaction environment and layered architecture of the in-vehicle screen based on the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer.

[0085] It should be explained that the vehicle-mounted screen refers to a screen installed inside the vehicle for displaying information and interacting with users, such as an instrument panel display, a central control touch screen, a rear entertainment screen, a head-up display, and a door control screen.

[0086] The present invention configures the multi-screen interactive environment and layered architecture of the vehicle-mounted screen through the interactive needs to achieve efficient and intelligent vehicle-mounted multi-screen interactive display and control.

[0087] In detail, configuring the multi-screen interactive environment and layered architecture of the vehicle-mounted screen according to the interactive requirements includes:

[0088] Analyze the interaction requirements and functional requirements of the vehicle screen based on the interaction requirements;

[0089] Based on the interaction requirements, determining network indicators of the in-vehicle screen;

[0090] According to the network indicators, construct the network topology of the vehicle-mounted screen;

[0091] Based on the network topology, configuring a multi-screen interactive environment of the vehicle-mounted screen;

[0092] According to the functional requirements, the initial hierarchical architecture of the vehicle screen is configured;

[0093] Analyzing the scalability of the initial layered architecture;

[0094] When the scalability meets a preset scalability threshold, the initial hierarchical architecture is used as the hierarchical architecture of the vehicle-mounted screen.

[0095] Among them, the interactive requirements refer to supporting multimodal interaction and providing intuitive and easy-to-use operation methods. The functional requirements include navigation function, entertainment function, communication function and assisted driving function. The network indicators refer to quantitative indicators that define network performance, such as bandwidth, delay, reliability, etc. The network topology refers to the physical and logical layout of network equipment. The multi-screen interactive environment refers to the realization of content sharing, synchronous display and interactive control between multiple screens. The initial layered architecture refers to the layered architecture designed in the early stage of system development. The scalability refers to the performance impact of evaluating the system when adding new functions or upgrading. The scalability threshold refers to the limitation of defining the performance and resource requirements of the system when expanding. The layered architecture includes application layer, service layer, middleware layer and hardware abstraction layer. The application layer refers to the layer that directly interacts with users and provides user interface and user interaction logic. The service layer refers to the layer that processes business logic and implements specific application functions. The middleware layer refers to the layer that provides services such as data processing, communication and system management. The hardware abstraction layer refers to the layer that hides the details of the underlying hardware and provides a unified hardware interface.

[0096] Furthermore, the multi-screen interactive environment configured on the vehicle screen based on the network topology can be realized through technologies such as vehicle Ethernet, Wi-Fi, and Bluetooth.

[0097] S2. Construct a screen management module and a content distribution engine for the vehicle-mounted screen. Based on the screen management module, the content distribution engine, the multi-screen interactive environment and the layered architecture, construct a multi-screen operating system for the vehicle-mounted screen, and construct a data customization protocol and synchronization logic code for the multi-screen operating system.

[0098] The screen management module and content distribution engine of the vehicle-mounted screen constructed by the present invention can construct a screen management module and content distribution engine that meet the functional requirements and performance requirements of the vehicle-mounted screen.

[0099] In detail, the screen management module and content distribution engine for constructing the vehicle-mounted screen include:

[0100] Identifying screen management function requirements, screen management performance requirements, content distribution function requirements, and content distribution performance requirements of the vehicle-mounted screen;

[0101] Constructing a screen management architecture of the vehicle-mounted screen according to the screen management function requirements and the screen management performance requirements, and configuring a screen management interface of the screen management architecture;

[0102] Based on the screen management architecture and the screen management interface, construct a screen management module for the vehicle-mounted screen;

[0103] According to the content distribution function requirements and content distribution performance requirements, construct the content distribution engine architecture of the vehicle screen, and configure the content distribution interface of the content distribution engine architecture;

[0104] The content distribution engine for the vehicle-mounted screen is constructed through the content distribution engine architecture and the content distribution interface.

[0105] Among them, the screen management function requirements refer to the requirements for the management functions of the vehicle-mounted screen, including functions such as screen identification, control, and synchronization; the screen management performance requirements refer to the performance requirements for the vehicle-mounted screen management module, including response time and processing speed; the content distribution function requirements refer to the requirements for vehicle-mounted screen content distribution, including content management and distribution; the content distribution performance requirements refer to the performance requirements for the vehicle-mounted screen content distribution engine, including transmission bandwidth and processing power; the screen management architecture refers to the architecture of the screen management module, including module division and hierarchical relationship; the screen management interface refers to the interface for interaction between screen management modules or between modules and external systems; the screen management module refers to the software component that implements the screen management function; the content distribution engine architecture refers to the software and hardware structure used to manage and distribute content (such as audio, video, images, text, etc.) to different screens in the multi-screen interactive system of the vehicle-mounted screen; the content distribution interface refers to the interface for interaction between content distribution modules or between modules and external systems; and the content distribution engine refers to the software component that implements the content distribution function.

[0106] Optionally, constructing the screen management architecture of the vehicle-mounted screen through the screen management function requirements and the screen management performance requirements, and configuring the screen management interface of the screen management architecture can be achieved through embedded system technology.

[0107] It should be explained that the multi-screen operating system refers to a system responsible for managing and coordinating the display content of the vehicle screen, user interaction, and communication with other parts of the vehicle system.

[0108] The present invention constructs a data custom protocol and a synchronization logic code of the multi-screen operating system to achieve consistency in screen control, thereby improving the control effect of the vehicle-mounted screen.

[0109] In detail, the data custom protocol and synchronization logic code for constructing the multi-screen operating system include:

[0110] Determining the data format and data structure of the multi-screen operating system;

[0111] Based on the data format and data structure, define a data transmission protocol of the multi-screen operating system;

[0112] Configuring an error handling mechanism of the multi-screen operating system;

[0113] By means of the data transmission protocol and error handling mechanism, a data custom protocol of the multi-screen operating system is constructed;

[0114] Designing synchronization trigger conditions and synchronization processes of the multi-screen operating system;

[0115] Based on the synchronization triggering conditions and synchronization process, construct a data update mechanism for the multi-screen operating system;

[0116] According to the synchronization triggering conditions, synchronization process and data update mechanism, the synchronization logic code of the multi-screen operating system is written.

[0117] Among them, the data format refers to the format of data transmission and storage, such as JSON, XML, Protocol Buffers, etc., the data structure refers to the internal layout and composition of data, such as the attributes of an object, the elements of an array, etc., the data transmission protocol refers to the rules and standards that define data transmission in the network, such as TCP / IP, HTTP, etc., the error handling mechanism refers to the mechanism for handling errors and exceptions during data transmission and processing, the data custom protocol refers to the data transmission and synchronization rules customized according to system requirements, the synchronization trigger condition refers to the condition that triggers data synchronization, such as user operation, scheduled task, etc., the synchronization process refers to the process of data synchronization, including data collection, packaging, transmission and other steps, the data update mechanism refers to the method and strategy for updating data, including the update timing and update strategy of data, and the synchronization logic code refers to the code that implements the synchronization trigger condition, synchronization process and data update mechanism.

[0118] Furthermore, the data update mechanism of the multi-screen operating system is constructed based on the synchronization trigger condition and the synchronization process, including:

[0119] Based on the synchronization triggering condition and the synchronization process, construct an initial data update mechanism of the multi-screen operating system;

[0120] Defining synchronization parameters of the initial data update mechanism, wherein the synchronization parameters include data update frequency, network transmission time, device processing time, data transmission stability factor, device synchronization factor and optimization coefficient;

[0121] The data update timeliness coefficient of the initial data update mechanism is calculated by the following formula through the synchronization parameters:

[0122]

[0123] in, represents the data update timeliness coefficient of the initial data update mechanism, Indicates the data update frequency corresponding to the synchronization parameters. Indicates the weight of the data update frequency corresponding to the synchronization parameter, Indicates the network transmission time corresponding to the synchronization parameters. Indicates the weight of the network transmission time corresponding to the synchronization parameter, Indicates the device processing time corresponding to the synchronization parameters. Indicates the weight of the device processing time corresponding to the synchronization parameter. Indicates the data transmission stability factor corresponding to the synchronization parameter, represents the weight of the data transmission stability factor corresponding to the synchronization parameter, Indicates the device synchronization factor corresponding to the synchronization parameter, Indicates the weight of the device synchronization factor corresponding to the synchronization parameter, Indicates the optimization coefficient corresponding to the synchronization parameter, Indicates the weight of the optimization coefficient corresponding to the synchronization parameter;

[0124] When the data update timeliness coefficient meets the preset data update timeliness threshold, the initial data update mechanism is used as the data update mechanism of the multi-screen operating system.

[0125] Among them, the initial data update mechanism refers to the data update mechanism defined when the system starts, which is the data update strategy in the initial state of the system. The data update frequency refers to the period of data update, such as every second, every minute, every hour, etc. The network transmission time refers to the time required for data to be transmitted from the source to the target device, including network delay and data packet size. The device processing time refers to the time required for the device to process data update requests and perform update operations. The data transmission stability factor refers to an indicator for measuring stability during data transmission, such as network fluctuations, data packet loss and other indicators. The device synchronization factor refers to an indicator for measuring synchronization performance between devices, such as screen update synchronization, data consistency, etc. The optimization coefficient refers to a coefficient used to balance the influence of different parameters. The data update timeliness threshold refers to the minimum standard for data update timeliness set by the system. The data update timeliness coefficient refers to an indicator for measuring the timeliness of the data update mechanism.

[0126] S3. Based on the data custom protocol and synchronization logic code, the multi-screen instant coefficient of the multi-screen operating system is calculated. When the multi-screen instant coefficient meets the preset multi-screen instant threshold, the edge node of the vehicle corresponding to the vehicle-mounted multimedia host is constructed.

[0127] It should be explained that the multi-screen instant coefficient refers to an indicator that measures the response speed and efficiency of the system. In detail, the multi-screen instant coefficient can be analyzed by analyzing the user operation time and data synchronization delay of the multi-screen operating system through the data custom protocol and the synchronization logic code. The edge node refers to a computing and storage device located inside the vehicle, which is located between data generation and data use, that is, after the data is generated from the source (such as a sensor), it is directly processed and analyzed inside the vehicle instead of transmitting all data to a remote data center. By building an edge node inside the vehicle, the performance of the vehicle multimedia host can be improved.

[0128] S4. Construct a multi-touch mechanism, a multi-language recognition mechanism, and a gesture recognition mechanism for the vehicle-mounted screen. Based on the multi-touch mechanism, the multi-language recognition mechanism, and the gesture recognition mechanism, construct a multimodal interaction system for the vehicle-mounted screen, and use the multimodal interaction system to collect interaction data of the target user in the vehicle.

[0129] The present invention can construct a variety of interaction mechanisms of the vehicle-mounted screen for interaction by constructing a multi-touch mechanism, a multi-language recognition mechanism and a gesture recognition mechanism of the vehicle-mounted screen, thereby improving the interaction effect of the vehicle-mounted screen.

[0130] In detail, the multi-touch mechanism, multi-language recognition mechanism and gesture recognition mechanism of the vehicle-mounted screen are constructed, including:

[0131] Configuring a multi-touch driver and a multi-touch application for the vehicle-mounted screen;

[0132] The multi-touch function of the vehicle-mounted screen is constructed using the following formula, wherein the multi-touch function:

[0133]

[0134] in, Indicates the coordinates of the touch point on the car screen corresponding to the user. Indicates the car screen The coordinates of the touch points, Indicates the car screen The pressure value of each touch point;

[0135] Constructing a multi-touch mechanism for the vehicle-mounted screen through a multi-touch driver, a multi-touch application, and a multi-touch function;

[0136] Determining a multilingual speech recognition engine for the vehicle-mounted screen and constructing a speech instruction set for the multilingual speech recognition engine;

[0137] Training a multilingual speech recognition model for the vehicle-mounted screen, and constructing a multilingual recognition mechanism for the vehicle-mounted screen through the multilingual speech recognition model, the multilingual speech recognition engine, and the speech instruction set;

[0138] Defining a gesture set and an image acquisition sensor for the vehicle-mounted screen;

[0139] Configuring a gesture recognition model for the vehicle-mounted screen;

[0140] The gesture recognition mechanism of the vehicle-mounted screen is constructed by using the gesture set, image acquisition sensor and gesture recognition model.

[0141] Among them, the multi-touch driver refers to a multi-touch driver, which is a software component that interacts directly with the hardware (touch screen controller) of the vehicle screen to capture and interpret multiple touch point inputs from the user; the multi-touch application refers to a multi-touch application that uses the information provided by the multi-touch driver to respond to user input. The multi-touch function refers to a mathematical model or algorithm for processing and interpreting data from multiple touch points on the vehicle screen; the multi-touch mechanism refers to the overall system for implementing the multi-touch function on the vehicle screen, including hardware, drivers, applications and multi-touch functions; the multi-language speech recognition engine refers to a multi-language speech recognition engine, which is a software system that can recognize and understand voice commands in multiple languages; the voice command set refers to predefined, A series of voice commands that can be understood by the voice recognition engine, the multilingual voice recognition model refers to a machine learning model that is trained to recognize and understand voices in multiple languages, the multilingual recognition mechanism refers to the overall system that implements the multilingual voice recognition function on the vehicle screen, including the voice recognition engine, the model and the instruction set, the gesture set refers to a series of predefined gestures that the vehicle screen can recognize and respond to, the image acquisition sensor refers to a hardware device for capturing images or videos of gestures performed by the user in front of the vehicle screen, the gesture recognition model refers to a machine learning model that is trained to recognize gestures from data captured by the image acquisition sensor, and the gesture recognition mechanism refers to the overall system that implements the gesture recognition function on the vehicle screen, including the gesture set, the image acquisition sensor and the gesture recognition model.

[0142] Furthermore, the configuration of the gesture recognition model of the vehicle-mounted screen includes:

[0143] Collecting the voice signal sequence of the vehicle-mounted screen;

[0144] Analyzing model parameters of the vehicle-mounted screen;

[0145] Based on the model parameters and the speech signal sequence, the gesture recognition function of the vehicle screen is constructed using the following formula:

[0146]

[0147] in, represents the probability, Represents the gesture recognition function, represents the speech signal sequence, represents the model parameters, Indicates that under given conditions and the speech signal sequence corresponds to the previous observation value Under the condition of The conditional probability of Indicates that under given conditions Under the condition of probability;

[0148] A gesture recognition model of the vehicle-mounted screen is constructed through the gesture recognition function.

[0149] The present invention constructs the multimodal interactive system of the vehicle screen based on the multi-touch mechanism, multi-language recognition mechanism and gesture recognition mechanism to realize efficient interaction between vehicle screens. The multimodal interactive system refers to a human-computer interaction system that can support multiple interaction modes, the target user refers to the user group targeted when the multimodal interactive system is designed and developed, and the interaction data refers to the data generated when the user interacts with the multimodal interactive system, including input data and system response data.

[0150] S5. Based on the interaction data, the edge node is used to analyze the interaction tasks of the vehicle screen, and the task priority coefficient of the interaction task is calculated. The task queue of the vehicle screen is constructed through the task priority coefficient. Based on the task queue, the multi-screen interaction instructions of the vehicle screen are constructed using the trained user behavior analysis model and the multi-screen operating system. The multi-screen interactive display and control of the vehicle multimedia host are realized through the multi-screen interaction instructions.

[0151] Based on the interactive data, the present invention utilizes the edge node to analyze the interactive tasks of the vehicle-mounted screen, which can effectively perform preliminary processing on real-time data, reduce dependence on the central server, and improve the response speed and efficiency of the system.

[0152] In detail, the analyzing the interactive task of the vehicle screen by using the edge node based on the interactive data includes:

[0153] Preprocessing the interaction data to obtain preprocessed interaction data;

[0154] Select a target edge node among the edge nodes for processing the pre-processed interaction data;

[0155] By using the pre-processed interaction data, the target edge node corresponding intention analysis model is used to analyze the user intention of the target user corresponding to the vehicle screen;

[0156] According to the user intention, the target edge node corresponding rule engine is used to define the interactive task of the vehicle screen.

[0157] Among them, the preprocessed interaction data refers to the interaction data that has been preliminarily processed and formatted, the target edge node refers to the computing node at the edge of the network responsible for processing specific preprocessed interaction data, the intent analysis model refers to a machine learning model that can analyze and infer the user's intention from the preprocessed interaction data, the user intent refers to the goal that the user wants to achieve or the operation that he wants to perform when interacting with the vehicle screen, the rule engine refers to a software system for implementing business logic, which can evaluate the input data (such as user intent) and make decisions according to predefined rules, and the interaction task refers to the specific operation or a set of operations performed by the vehicle screen according to the user's intention. The interaction task may include starting the music player, adjusting the volume, displaying the navigation route, etc.

[0158] Optionally, the intent analysis model is implemented through machine learning technology.

[0159] The present invention can calculate important interactive tasks first by calculating the task priority coefficient of the interactive task, thereby improving the effect of multi-screen interaction. In detail, the task priority coefficient refers to a quantitative indicator for evaluating the priority of the interactive task, and the task queue refers to a list of tasks to be executed sorted according to the task priority coefficient.

[0160] The present invention is based on the task queue, utilizes the trained user behavior analysis model and the multi-screen operating system, and constructs the multi-screen interaction instructions of the vehicle-mounted screen to achieve the multi-screen interaction effect.

[0161] In detail, based on the task queue, using the trained user behavior analysis model and the multi-screen operating system, constructing the multi-screen interaction instructions of the vehicle screen includes:

[0162] Identifying a queue structure of the task queue;

[0163] Analyzing the user interaction habits of the in-vehicle screen with respect to the target user using the user behavior analysis model;

[0164] Based on the queue structure and user interaction habits, constructing the multi-screen initial interaction instructions of the vehicle screen using the multi-screen operating system;

[0165] Simulating the multi-screen interaction result of the multi-screen initial interaction instruction on the vehicle screen;

[0166] Calculating the interaction seamlessness coefficient of the multi-screen interaction result;

[0167] When the interaction seamlessness coefficient meets a preset interaction seamlessness threshold, the multi-screen initial interaction instruction is used as the multi-screen interaction instruction of the vehicle-mounted screen.

[0168] Among them, the queue structure refers to the organization method of the task queue, which determines the order of task processing; the user interaction habit refers to the regular behavior pattern formed by the target user when interacting with the vehicle screen; the user behavior analysis model refers to a machine learning model that analyzes user behavior patterns and preferences; the multi-screen initial interaction instruction refers to the initial instruction issued by the vehicle screen when starting the multi-screen interaction; the multi-screen interaction result refers to the output or state generated after the vehicle screen executes the multi-screen initial interaction instruction; the interaction seamless coefficient refers to an indicator for measuring the smoothness of multi-screen interaction and user experience; the interaction seamless threshold refers to a preset standard used to judge whether the interaction seamless coefficient reaches an acceptable level; the multi-screen interaction instruction refers to the specific instruction executed by the vehicle screen in the multi-screen operating system to achieve specific interaction tasks.

[0169] Optionally, the multi-screen interaction result of simulating the multi-screen initial interaction instruction on the vehicle screen can be implemented by a simulator or emulator.

[0170] The present invention configures the multi-screen interactive environment and hierarchical architecture of the vehicle-mounted screen through the interactive requirements to achieve efficient and intelligent vehicle-mounted multi-screen interactive display and control; the present invention constructs the screen management module and content distribution engine of the vehicle-mounted screen to build a screen management module and content distribution engine that meet the functional requirements and performance requirements of the vehicle-mounted screen; further, the present invention constructs the data custom protocol and synchronization logic code of the multi-screen operating system to achieve the consistency of screen control, thereby improving the control effect of the vehicle-mounted screen, further, the present invention can construct a variety of interactive mechanisms of the vehicle-mounted screen for interaction by constructing the multi-touch mechanism, multi-language recognition mechanism and gesture recognition mechanism of the vehicle-mounted screen, thereby improving the interactive effect of the vehicle-mounted screen, further, based on the interactive data, the present invention uses the edge node to analyze the interactive tasks of the vehicle-mounted screen to effectively perform preliminary processing on real-time data, reduce dependence on the central server, and improve the response speed and efficiency of the system, and finally, the present invention can calculate the important interactive tasks first by calculating the task priority coefficient of the interactive task, thereby improving the effect of multi-screen interaction. Therefore, the present invention can improve the effect of multi-screen interactive display and control.

[0171] like Figure 2 , which is a functional module diagram of a multi-screen interactive display and control system of a vehicle-mounted multimedia host provided by an embodiment of the present invention.

[0172] The multi-screen interactive display and control system 200 of the vehicle-mounted multimedia host of the present invention can be installed in an electronic device. According to the functions to be implemented, the multi-screen interactive display and control system 200 of the vehicle-mounted multimedia host may include a vehicle-mounted screen acquisition module 201, a multi-screen operating system construction module 202, an edge node arrangement module 203, an interactive data acquisition module 204 and a multi-screen interactive control module 205. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0173] In this embodiment, the functions of each module / unit are as follows:

[0174] The vehicle screen acquisition module 201 is used to acquire the vehicle screen of the vehicle multimedia host, determine the interaction requirements of the vehicle screen, and configure the multi-screen interaction environment and layered architecture of the vehicle screen according to the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer;

[0175] The multi-screen operating system construction module 202 is used to construct the screen management module and content distribution engine of the vehicle screen, and to construct the multi-screen operating system of the vehicle screen based on the screen management module, content distribution engine, multi-screen interactive environment and layered architecture, and to construct the data custom protocol and synchronization logic code of the multi-screen operating system;

[0176] The edge node arrangement module 203 is used to calculate the multi-screen instant coefficient of the multi-screen operating system based on the data custom protocol and the synchronization logic code, and when the multi-screen instant coefficient meets the preset multi-screen instant threshold, construct the edge node of the vehicle corresponding to the vehicle-mounted multimedia host;

[0177] The interactive data collection module 204 is used to construct a multi-touch mechanism, a multi-language recognition mechanism and a gesture recognition mechanism for the vehicle screen, and based on the multi-touch mechanism, the multi-language recognition mechanism and the gesture recognition mechanism, construct a multi-modal interactive system for the vehicle screen, and use the multi-modal interactive system to collect interactive data of the target user in the vehicle;

[0178] The multi-screen interaction control module 205 is used to analyze the interaction tasks of the vehicle-mounted screen based on the interaction data using the edge node, and calculate the task priority coefficient of the interaction task. Through the task priority coefficient, a task queue of the vehicle-mounted screen is constructed. Based on the task queue, a trained user behavior analysis model and a multi-screen operating system are used to construct multi-screen interaction instructions for the vehicle-mounted screen. The multi-screen interactive display and control of the vehicle-mounted multimedia host are realized through the multi-screen interaction instructions.

[0179] In detail, the modules described in the multi-screen interactive display and control system 200 of the vehicle-mounted multimedia host in the embodiment of the present invention adopt the same technical means as the multi-screen interactive display and control method of the vehicle-mounted multimedia host described in the accompanying drawings when in use, and can produce the same technical effects, which will not be repeated here.

[0180] An embodiment of the present invention provides an electronic device for implementing a multi-screen interactive display and control method of an in-vehicle multimedia host.

[0181] See also Figure 3 As shown, the electronic device may include a processor 30, a memory 31, a communication bus 32 and a communication interface 33, and may also include a computer program stored in the memory 31 and executable on the processor 30, such as a multi-screen interactive display and control method program of a vehicle-mounted multimedia host.

[0182] In some embodiments, the processor may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips, etc. The processor is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing programs or modules stored in the memory (for example, executing multi-screen interactive display and control programs of the vehicle-mounted multimedia host, etc.), and calling data stored in the memory.

[0183] The memory includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory may also include both an internal storage unit of the electronic device and an external storage device. The memory can not only be used to store application software and various types of data installed in the electronic device, such as the code of the multi-screen interactive display and control program based on the vehicle-mounted multimedia host, but also can be used to temporarily store data that has been output or is to be output.

[0184] The communication bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory and at least one processor, etc.

[0185] The communication interface is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0186] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor through a power management system, so that the power management system can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0187] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0188] The multi-screen interactive display and control program of the vehicle-mounted multimedia host stored in the memory of the electronic device is a combination of multiple instructions. When running in the processor, it can achieve:

[0189] Acquire the in-vehicle screen of the in-vehicle multimedia host, determine the interaction requirements of the in-vehicle screen, and configure the multi-screen interaction environment and layered architecture of the in-vehicle screen according to the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer;

[0190] Construct a screen management module and a content distribution engine for the vehicle-mounted screen, and based on the screen management module, the content distribution engine, the multi-screen interactive environment and the layered architecture, construct a multi-screen operating system for the vehicle-mounted screen, and construct a data custom protocol and synchronization logic code for the multi-screen operating system;

[0191] Based on the data custom protocol and synchronization logic code, the multi-screen instant coefficient of the multi-screen operating system is calculated, and when the multi-screen instant coefficient meets the preset multi-screen instant threshold, the edge node of the vehicle corresponding to the vehicle-mounted multimedia host is constructed;

[0192] Constructing a multi-touch mechanism, a multi-language recognition mechanism, and a gesture recognition mechanism for the vehicle-mounted screen; constructing a multi-modal interaction system for the vehicle-mounted screen based on the multi-touch mechanism, the multi-language recognition mechanism, and the gesture recognition mechanism; and using the multi-modal interaction system to collect interaction data of a target user in the vehicle;

[0193] Based on the interaction data, the edge node is used to analyze the interaction tasks of the vehicle screen, and the task priority coefficient of the interaction task is calculated. The task queue of the vehicle screen is constructed through the task priority coefficient. Based on the task queue, the multi-screen interaction instructions of the vehicle screen are constructed using the trained user behavior analysis model and the multi-screen operating system. The multi-screen interactive display and control of the vehicle multimedia host are realized through the multi-screen interaction instructions.

[0194] Specifically, the specific implementation method of the processor for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0195] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or system that can carry the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0196] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0197] Acquire the in-vehicle screen of the in-vehicle multimedia host, determine the interaction requirements of the in-vehicle screen, and configure the multi-screen interaction environment and layered architecture of the in-vehicle screen according to the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer;

[0198] Construct a screen management module and a content distribution engine for the vehicle-mounted screen, and based on the screen management module, the content distribution engine, the multi-screen interactive environment and the layered architecture, construct a multi-screen operating system for the vehicle-mounted screen, and construct a data custom protocol and synchronization logic code for the multi-screen operating system;

[0199] Based on the data custom protocol and synchronization logic code, the multi-screen instant coefficient of the multi-screen operating system is calculated, and when the multi-screen instant coefficient meets the preset multi-screen instant threshold, the edge node of the vehicle corresponding to the vehicle-mounted multimedia host is constructed;

[0200] Constructing a multi-touch mechanism, a multi-language recognition mechanism, and a gesture recognition mechanism for the vehicle-mounted screen; constructing a multi-modal interaction system for the vehicle-mounted screen based on the multi-touch mechanism, the multi-language recognition mechanism, and the gesture recognition mechanism; and using the multi-modal interaction system to collect interaction data of a target user in the vehicle;

[0201] Based on the interaction data, the edge node is used to analyze the interaction tasks of the vehicle screen, and the task priority coefficient of the interaction task is calculated. The task queue of the vehicle screen is constructed through the task priority coefficient. Based on the task queue, the multi-screen interaction instructions of the vehicle screen are constructed using the trained user behavior analysis model and the multi-screen operating system. The multi-screen interactive display and control of the vehicle multimedia host are realized through the multi-screen interaction instructions.

[0202] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0203] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0205] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0206] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0207] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0208] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0209] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A multi-screen interactive display and control method for a vehicle-mounted multimedia host, characterized in that: The method comprises: Acquire the in-vehicle screen of the in-vehicle multimedia host, determine the interaction requirements of the in-vehicle screen, and configure the multi-screen interaction environment and layered architecture of the in-vehicle screen according to the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer; Construct a screen management module and a content distribution engine for the vehicle-mounted screen; construct a multi-screen operating system for the vehicle-mounted screen based on the screen management module, the content distribution engine, the multi-screen interactive environment, and the layered architecture; determine the data format and data structure of the multi-screen operating system; define the data transmission protocol of the multi-screen operating system based on the data format and data structure; configure the error handling mechanism of the multi-screen operating system; construct a data customization protocol for the multi-screen operating system through the data transmission protocol and the error handling mechanism; design the synchronization trigger conditions and synchronization process of the multi-screen operating system; construct an initial data update mechanism for the multi-screen operating system based on the synchronization trigger conditions and the synchronization process; define the synchronization parameters of the initial data update mechanism; wherein the synchronization parameters include data update frequency, network transmission time, device processing time, data transmission stability factor, device synchronization factor, and optimization coefficient; and calculate the data update timeliness coefficient of the initial data update mechanism through the synchronization parameters using the following formula: in, represents the data update timeliness coefficient of the initial data update mechanism, Indicates the data update frequency corresponding to the synchronization parameters. Indicates the weight of the data update frequency corresponding to the synchronization parameter, Indicates the network transmission time corresponding to the synchronization parameters. Indicates the weight of the network transmission time corresponding to the synchronization parameter, Indicates the device processing time corresponding to the synchronization parameters. Indicates the weight of the device processing time corresponding to the synchronization parameter. Indicates the data transmission stability factor corresponding to the synchronization parameter, represents the weight of the data transmission stability factor corresponding to the synchronization parameter, Indicates the device synchronization factor corresponding to the synchronization parameter, Indicates the weight of the device synchronization factor corresponding to the synchronization parameter, Indicates the optimization coefficient corresponding to the synchronization parameter, Indicates the weight of the optimization coefficient corresponding to the synchronization parameter. When the data update timeliness coefficient meets the preset data update timeliness threshold, the initial data update mechanism is used as the data update mechanism of the multi-screen operating system. According to the synchronization trigger condition, synchronization process and data update mechanism, the synchronization logic code of the multi-screen operating system is written; Based on the data custom protocol and synchronization logic code, the multi-screen instant coefficient of the multi-screen operating system is calculated, and when the multi-screen instant coefficient meets the preset multi-screen instant threshold, the edge node of the vehicle corresponding to the vehicle-mounted multimedia host is constructed; The multi-touch driver and multi-touch application of the vehicle-mounted screen are configured, and the multi-touch function of the vehicle-mounted screen is constructed using the following formula, wherein the multi-touch function is: in, Indicates the coordinates of the touch point on the car screen corresponding to the user. Indicates the car screen The coordinates of the touch points, Indicates the car screen The invention relates to a method for determining the pressure value of each touch point, constructing a multi-touch mechanism of the vehicle screen through a multi-touch driver, a multi-touch application and a multi-touch function, determining a multi-language speech recognition engine for the vehicle screen, and constructing a speech instruction set for the multi-language speech recognition engine, training a multi-language speech recognition model for the vehicle screen, and constructing a multi-language recognition mechanism for the vehicle screen through the multi-language speech recognition model, the multi-language speech recognition engine and the speech instruction set, defining a gesture set and an image acquisition sensor for the vehicle screen, configuring a gesture recognition model for the vehicle screen, and constructing a gesture recognition mechanism for the vehicle screen through the gesture set, the image acquisition sensor and the gesture recognition model, and constructing a multi-modal interaction system for the vehicle screen based on the multi-touch mechanism, the multi-language recognition mechanism and the gesture recognition mechanism, and using the multi-modal interaction system to collect interaction data of a target user in the vehicle; Based on the interaction data, the edge node is used to analyze the interaction tasks of the vehicle screen, and the task priority coefficient of the interaction task is calculated. The task queue of the vehicle screen is constructed through the task priority coefficient. Based on the task queue, the multi-screen interaction instructions of the vehicle screen are constructed using the trained user behavior analysis model and the multi-screen operating system. The multi-screen interactive display and control of the vehicle multimedia host are realized through the multi-screen interaction instructions.

2. The multi-screen interactive display and control method of the vehicle-mounted multimedia host according to claim 1, characterized in that: The multi-screen interactive environment and layered architecture of the vehicle-mounted screen are configured according to the interactive requirements, including: Analyze the interaction requirements and functional requirements of the vehicle screen based on the interaction requirements; Based on the interaction requirements, determining network indicators of the in-vehicle screen; According to the network indicators, construct the network topology of the vehicle-mounted screen; Based on the network topology, configuring a multi-screen interactive environment of the vehicle-mounted screen; According to the functional requirements, the initial hierarchical architecture of the vehicle screen is configured; Analyzing the scalability of the initial layered architecture; When the scalability meets a preset scalability threshold, the initial hierarchical architecture is used as the hierarchical architecture of the vehicle-mounted screen.

3. The multi-screen interactive display and control method of the vehicle-mounted multimedia host according to claim 2, characterized in that: The screen management module and content distribution engine for constructing the vehicle-mounted screen include: Identifying screen management function requirements, screen management performance requirements, content distribution function requirements, and content distribution performance requirements of the vehicle-mounted screen; Constructing a screen management architecture of the vehicle-mounted screen according to the screen management function requirements and the screen management performance requirements, and configuring a screen management interface of the screen management architecture; Based on the screen management architecture and the screen management interface, construct a screen management module for the vehicle-mounted screen; According to the content distribution function requirements and content distribution performance requirements, construct the content distribution engine architecture of the vehicle screen, and configure the content distribution interface of the content distribution engine architecture; The content distribution engine for the vehicle-mounted screen is constructed through the content distribution engine architecture and the content distribution interface.

4. The multi-screen interactive display and control method of the vehicle-mounted multimedia host according to claim 3, characterized in that: The configuration of the gesture recognition model of the vehicle-mounted screen includes: Collecting the voice signal sequence of the vehicle-mounted screen; Analyzing model parameters of the vehicle-mounted screen; Based on the model parameters and the speech signal sequence, the gesture recognition function of the vehicle screen is constructed using the following formula: in, represents the probability, Represents the gesture recognition function, represents the speech signal sequence, represents the model parameters, Indicates that under the given conditions and the speech signal sequence corresponds to the previous observation value Under the condition of The conditional probability of Indicates that under given conditions Under the condition of probability; A gesture recognition model of the vehicle-mounted screen is constructed through the gesture recognition function.

5. The multi-screen interactive display and control method of the vehicle-mounted multimedia host according to claim 4, characterized in that: The analyzing the interactive task of the vehicle-mounted screen by using the edge node based on the interactive data includes: Preprocessing the interaction data to obtain preprocessed interaction data; Select a target edge node among the edge nodes for processing the pre-processed interaction data; By using the pre-processed interaction data, the target edge node corresponding intention analysis model is used to analyze the user intention of the target user corresponding to the vehicle screen; According to the user intention, the target edge node corresponding rule engine is used to define the interactive task of the vehicle screen.

6. The multi-screen interactive display and control method of the vehicle-mounted multimedia host according to claim 5, characterized in that: The multi-screen interaction instructions of the vehicle screen are constructed based on the task queue and using the trained user behavior analysis model and the multi-screen operating system, including: Identifying a queue structure of the task queue; Analyzing the user interaction habits of the in-vehicle screen with respect to the target user using the user behavior analysis model; Based on the queue structure and user interaction habits, constructing the multi-screen initial interaction instructions of the vehicle screen using the multi-screen operating system; Simulating the multi-screen interaction result of the multi-screen initial interaction instruction on the vehicle screen; Calculating the interaction seamlessness coefficient of the multi-screen interaction result; When the interaction seamlessness coefficient meets a preset interaction seamlessness threshold, the multi-screen initial interaction instruction is used as the multi-screen interaction instruction of the vehicle-mounted screen.

7. A multi-screen interactive display and control system for a vehicle-mounted multimedia host, characterized in that: A system for executing a multi-screen interactive display and control method for a vehicle-mounted multimedia host according to any one of claims 1 to 6, the system comprising: The vehicle screen acquisition module is used to acquire the vehicle screen of the vehicle multimedia host, determine the interaction requirements of the vehicle screen, and configure the multi-screen interaction environment and layered architecture of the vehicle screen according to the interaction requirements, wherein the layered architecture includes an application layer, a service layer, a middleware layer, and a hardware abstraction layer; A multi-screen operating system construction module is used to construct a screen management module and a content distribution engine for the vehicle-mounted screen. Based on the screen management module, the content distribution engine, the multi-screen interactive environment and the layered architecture, the multi-screen operating system for the vehicle-mounted screen is constructed, the data format and data structure of the multi-screen operating system are determined, and based on the data format and data structure, the data transmission protocol of the multi-screen operating system is defined, and the error handling mechanism of the multi-screen operating system is configured. Through the data transmission protocol and the error handling mechanism, a data customization protocol for the multi-screen operating system is constructed, and the synchronization trigger conditions and synchronization process of the multi-screen operating system are designed. Based on the synchronization trigger conditions and the synchronization process, an initial data update mechanism for the multi-screen operating system is constructed, and synchronization parameters of the initial data update mechanism are defined, wherein the synchronization parameters include data update frequency, network transmission time, device processing time, data transmission stability factor, device synchronization factor and optimization coefficient. Through the synchronization parameters, the data update timeliness coefficient of the initial data update mechanism is calculated using the following formula: in, represents the data update timeliness coefficient of the initial data update mechanism, Indicates the data update frequency corresponding to the synchronization parameters. Indicates the weight of the data update frequency corresponding to the synchronization parameter, Indicates the network transmission time corresponding to the synchronization parameters. Indicates the weight of the network transmission time corresponding to the synchronization parameter, Indicates the device processing time corresponding to the synchronization parameters. Indicates the weight of the device processing time corresponding to the synchronization parameter. Indicates the data transmission stability factor corresponding to the synchronization parameter, represents the weight of the data transmission stability factor corresponding to the synchronization parameter, Indicates the device synchronization factor corresponding to the synchronization parameter, Indicates the weight of the device synchronization factor corresponding to the synchronization parameter, Indicates the optimization coefficient corresponding to the synchronization parameter, Indicates the weight of the optimization coefficient corresponding to the synchronization parameter. When the data update timeliness coefficient meets the preset data update timeliness threshold, the initial data update mechanism is used as the data update mechanism of the multi-screen operating system. According to the synchronization trigger condition, synchronization process and data update mechanism, the synchronization logic code of the multi-screen operating system is written; An edge node arrangement module, for calculating the multi-screen instant coefficient of the multi-screen operating system based on the data custom protocol and the synchronization logic code, and constructing an edge node of the vehicle corresponding to the vehicle-mounted multimedia host when the multi-screen instant coefficient meets a preset multi-screen instant threshold; The interactive data acquisition module is used to configure the multi-touch driver and multi-touch application of the vehicle-mounted screen, and construct the multi-touch function of the vehicle-mounted screen using the following formula, wherein the multi-touch function is: in, Indicates the coordinates of the touch point on the car screen corresponding to the user. Indicates the car screen The coordinates of the touch points, Indicates the car screen The invention relates to a method for determining the pressure value of each touch point, constructing a multi-touch mechanism of the vehicle screen through a multi-touch driver, a multi-touch application and a multi-touch function, determining a multi-language speech recognition engine for the vehicle screen, and constructing a speech instruction set for the multi-language speech recognition engine, training a multi-language speech recognition model for the vehicle screen, and constructing a multi-language recognition mechanism for the vehicle screen through the multi-language speech recognition model, the multi-language speech recognition engine and the speech instruction set, defining a gesture set and an image acquisition sensor for the vehicle screen, configuring a gesture recognition model for the vehicle screen, and constructing a gesture recognition mechanism for the vehicle screen through the gesture set, the image acquisition sensor and the gesture recognition model, and constructing a multi-modal interaction system for the vehicle screen based on the multi-touch mechanism, the multi-language recognition mechanism and the gesture recognition mechanism, and using the multi-modal interaction system to collect interaction data of a target user in the vehicle; A multi-screen interaction control module is used to analyze the interaction tasks of the vehicle-mounted screen based on the interaction data using the edge node, and calculate the task priority coefficient of the interaction task. Through the task priority coefficient, a task queue of the vehicle-mounted screen is constructed. Based on the task queue, a multi-screen interaction instruction of the vehicle-mounted screen is constructed using a trained user behavior analysis model and a multi-screen operating system. The multi-screen interactive display and control of the vehicle-mounted multimedia host are realized through the multi-screen interaction instruction.

Images