Vehicle-mounted man-machine interaction method and system based on dynamic partition of integrated display screen

Abstract

The present application belongs to the technical field of vehicle-mounted human-computer interaction, and particularly relates to a vehicle-mounted human-computer interaction method and system based on dynamic partitioning of an integrated display screen. The method comprises initializing display of an integrated display screen that dynamically displays three functional areas; obtaining user input data, vehicle state parameters, and environmental and user information in a multi-modal human-computer interaction mode, and determining a current target scene from a preset scene database using a scene decision algorithm, and adjusting the proportion of each functional area according to the current target scene; receiving a user operation request, identifying user identity information and real-time vehicle state, querying a preset permission matrix, and verifying the legality of the user operation request, thereby realizing vehicle-mounted human-computer interaction with dynamic partitioning of the integrated display screen. Through the integrated display screen and software-defined dynamic logic partitioning technology, the present application realizes direct sharing of multi-user information, effectively eliminating the information fragmentation problem caused by traditional decentralized multi-screen layout.

Description

Technical Field

[0001] This invention belongs to the field of vehicle-mounted human-machine interaction technology, and particularly relates to a vehicle-mounted human-machine interaction method and system based on dynamic partitioning of an integrated display screen. Background Technology

[0002] Currently, mainstream smart car cockpits generally adopt a distributed multi-screen layout. Existing technology includes an in-vehicle multi-screen interaction system where a central touchscreen (typically 10 to 15 inches in size) is located to the right of the driver, handling navigation, vehicle control, and multimedia playback; a separate passenger screen (typically 7 to 10 inches in size) is located in front of the front passenger, primarily providing audio-visual entertainment and information query services; some high-end models also feature a fully digital instrument cluster and rear-seat entertainment screens, forming a multi-screen, separate display architecture.

[0003] In the above scheme, the hardware interfaces and software systems of each screen are independent of each other, and there is no direct data sharing channel between the screens. When multiple passengers need to work together to complete tasks such as itinerary planning, information transmission must be completed through verbal communication or third-party devices (such as mobile phones), which is cumbersome and the efficiency of information flow between multiple users is low, resulting in obvious information fragmentation problems. Summary of the Invention

[0004] To overcome the shortcomings of the prior art, this invention provides an in-vehicle human-machine interaction method and system based on dynamic partitioning of an integrated display screen. By using an integrated display screen and software-defined dynamic logical partitioning technology, direct sharing of information among multiple users can be achieved, effectively eliminating the information fragmentation problem caused by traditional distributed multi-screen layouts.

[0005] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a vehicle-mounted human-machine interaction method based on dynamic partitioning of an integrated display screen.

[0006] The in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen includes the following steps: Initialize the display of the integrated display screen that dynamically displays three functional areas: the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area; The system acquires user input data, vehicle status parameters, and environmental and user information from multimodal human-computer interaction methods, and uses a scenario decision algorithm to determine the current target scenario from a preset scenario database, and drives the adjustment of the proportion of each functional area based on the current target scenario. Receive user operation requests, identify user identity information and real-time vehicle status, query the preset permission matrix, and perform legality verification on user operation requests based on user identity information and real-time vehicle status. For operations that pass verification, the corresponding function is executed; for operations that fail verification, a prompt is displayed in the function area where the initiator is located, realizing in-vehicle human-machine interaction with dynamic partitioning of the integrated display screen.

[0007] The second aspect of the present invention provides an in-vehicle human-machine interaction system based on dynamic partitioning of an integrated display screen.

[0008] The in-vehicle human-machine interaction system based on dynamic partitioning of an integrated display screen includes: The initialization module is configured to initialize the display of an integrated display screen that dynamically displays three functional areas: the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area. The first human-computer interaction module is configured to: acquire user input data, vehicle status parameters and environmental and user information in multimodal human-computer interaction methods, and use a scenario decision algorithm to determine the current target scenario from a preset scenario database, and drive the adjustment of the proportion of each functional area according to the current target scenario; The operation verification module is configured to: receive user operation requests, identify user identity information and real-time vehicle status, query a preset permission matrix, and perform legality verification on user operation requests based on user identity information and real-time vehicle status. The second human-machine interaction module is configured to execute the corresponding function for operations that pass verification, and to provide a prompt in the function area where the initiator is located for operations that fail verification, thereby realizing the in-vehicle human-machine interaction of dynamic partitioning of the integrated display screen.

[0009] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in the first aspect of the present invention.

[0010] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in the first aspect of the present invention.

[0011] The above one or more technical solutions have the following beneficial effects: This invention provides a vehicle-mounted human-machine interaction method and system based on dynamic partitioning of an integrated display screen. Through the physical integration and logical partitioning of the integrated screen, it achieves seamless flow of driving, sharing, and entertainment information, significantly improving multi-user collaboration efficiency and overcoming the "information silos" defects of traditional distributed screens, thus solving the problem of information fragmentation. Contextualized dynamic adaptation and hierarchical permission management ensure that driving safety is not disturbed by entertainment while driving, and that the passenger seat provides an immersive experience when parked, resolving the contradiction between safety and experience in existing technologies and achieving a balance between the two. Contextualized adaptation of multimodal interaction provides users with the experience of "operating in the right way in the right scenario," reducing driving distraction and overcoming the limitations of single touch interaction, improving the naturalness of interaction. Unified interaction logic and visual language reduce the cognitive burden of users operating across regions, solving the problem of fragmented experience in traditional multi-screen solutions and enhancing experience consistency. A self-learning scene engine supports continuous optimization of scene matching accuracy via OTA, adapting to changes in user habits, solving the defect that static systems cannot meet long-term usage needs, and possessing scene scalability.

[0012] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0013] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0014] Figure 1 This is a flowchart illustrating the steps of Example 1.

[0015] Figure 2 This is a data flow diagram for Example 1.

[0016] Figure 3 This is a schematic diagram of the integrated display screen area division and dynamic adaptation in Embodiment 1.

[0017] Figure 4 This is a hierarchical architecture diagram of the system in Example 2. Detailed Implementation

[0018] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0020] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0021] Example 1 With the rapid development of new energy vehicles and intelligent connected vehicle technologies, the human-machine interaction system in the vehicle cockpit has become an important indicator of the overall vehicle's intelligence level. Users' demands for cockpit information access, entertainment experiences, and multi-user collaborative operation are becoming increasingly diverse, driving the continuous evolution of in-vehicle display and interaction systems towards multi-functionality and multi-user capabilities. Against this backdrop, how to rationally arrange display devices and coordinate multi-user operation needs within the limited cockpit space has become a crucial research topic in the field of in-vehicle human-machine interaction.

[0022] In addition to the aforementioned background technology section, where the hardware interfaces and software systems of each screen are independent and there is no direct data sharing channel between screens, when multiple passengers need to collaborate on tasks such as itinerary planning, information transmission must be completed through verbal communication or third-party devices (such as mobile phones). This process is cumbersome, the efficiency of information flow between multiple users is low, and there is a significant problem of information fragmentation. Furthermore, the following issues also exist: Regarding interaction methods, existing distributed multi-screen solutions primarily rely on touch operation, supplemented by basic voice commands (such as controlling the air conditioning or making phone calls). A few models support simple gesture operations (such as adjusting volume with air gestures). Chinese patent CN 113459975 A discloses an in-vehicle multimodal human-machine interaction method, but its voice interaction module processes all voice commands from all users in the vehicle indiscriminately, failing to differentiate between the driver's and front passenger's intentions based on the direction of the voice source. This can easily lead to command confusion in scenarios where multiple users issue voice commands simultaneously. Furthermore, existing touch interaction solutions use uniform sensitivity parameters for the driving and entertainment areas, failing to differentiate between driving and parked states. This poses a risk of driver distraction due to accidental touch operations while driving. Additionally, the high-brightness entertainment content (such as video) displayed on the front passenger screen can visually interfere with the driver's vision while driving, increasing the frequency with which the driver's gaze deviates from the road ahead.

[0023] Regarding display layout and scene adaptation, Chinese patent CN 117719428 A discloses a smart cockpit display control method. This solution adopts a static fixed layout, where the functional allocation and display ratio of each display area remain unchanged after system initialization. It cannot dynamically adjust the priority and area proportion of the displayed content according to different usage states such as driving, parking, and charging. For example, in a parking waiting scenario, the user's demand for entertainment content increases significantly, while in a high-speed driving scenario, the display priority of navigation and vehicle status information should be correspondingly increased. Static layout solutions cannot meet these dynamic scenario requirements.

[0024] Regarding user access control, existing in-vehicle human-machine interaction systems generally lack a sophisticated mechanism for distinguishing the operating permissions of drivers and front-seat passengers. The system fails to verify the legitimacy of operation requests based on user identity and real-time vehicle status, leading to a safety risk that front-seat passengers may accidentally activate driving control functions (such as switching driving modes) while driving. At the same time, drivers may also overuse entertainment functions due to a lack of operational restrictions, affecting driving safety.

[0025] Therefore, existing in-vehicle human-machine interaction systems have shortcomings in terms of multi-user information sharing efficiency, dynamic adaptation capability of display layout, fine management of multi-user operation permissions, and multi-modal interaction collaboration, requiring a new technical solution to improve them.

[0026] Based on this, this embodiment provides a vehicle-mounted human-machine interaction method based on dynamic partitioning of an integrated display screen to solve problems such as low efficiency of multi-user information flow, prominent driving safety hazards, insufficient adaptation of display layout scenarios, and lack of operation permission management in the existing technology.

[0027] like Figure 1 As shown, the in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen includes the following steps: Initialize the display of the integrated display screen that dynamically displays three functional areas: the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area; The system acquires user input data, vehicle status parameters, and environmental and user information from multimodal human-computer interaction methods, and uses a scenario decision algorithm to determine the current target scenario from a preset scenario database, and drives the adjustment of the proportion of each functional area based on the current target scenario. Receive user operation requests, identify user identity information and real-time vehicle status, query the preset permission matrix, and perform legality verification on user operation requests based on user identity information and real-time vehicle status. For operations that pass verification, the corresponding function is executed; for operations that fail verification, a prompt is displayed in the function area where the initiator is located, realizing in-vehicle human-machine interaction with dynamic partitioning of the integrated display screen.

[0028] Further: In short, current smart car cockpit display systems are trending towards multi-screen design, with mainstream solutions including: 1. Distributed multi-screen layout: The central control screen (10-15 inches) is located on the right side of the driver and is responsible for navigation, vehicle control, multimedia and other functions; the passenger screen (7-10 inches) is independent of the central control screen and only provides basic services such as audio-visual entertainment and information query; some models are additionally equipped with an instrument panel (full LCD) and a rear entertainment screen, forming a "four-screen separation" architecture.

[0029] 2. Limited interaction methods: The core is touch operation, supplemented by basic voice commands (such as "turn on the air conditioner"). A few high-end models support simple gesture recognition (such as adjusting the volume with the air), but a multimodal collaborative interaction system has not been formed.

[0030] 3. Fixed functional boundaries: The hardware interfaces and software systems of the central control screen and the passenger screen are independent of each other, and data cannot be directly shared. For example, the passenger in the front seat needs to take a picture with his mobile phone or describe it verbally to synchronize the information of the attractions along the way to the driver.

[0031] 4. Poor scene adaptation: Most systems adopt a "static layout". Regardless of whether the vehicle is driving, parked or charging, the screen display ratio and function priority remain unchanged, which makes it difficult to meet the needs of dynamic scenes.

[0032] The defects and shortcomings of existing technologies include: 1. Low information flow efficiency: The decentralized layout leads to "information silos". Collaborative operations among multiple users (such as family travel route planning) need to be completed through non-systematic methods (language, mobile phone), which takes 2-3 times longer on average than the integrated solution and is prone to information errors.

[0033] 2. Significant driving safety hazards: The high brightness of entertainment content (such as movies and games) on the passenger-side screen easily attracts the driver's peripheral vision. According to tests, the frequency of drivers' eyes deviating from the road while driving increases by 35% due to the presence of the passenger-side screen; at the same time, the time it takes for the driver to shift their gaze when operating the central control screen (an average of 2.3 seconds) far exceeds the safety threshold (1.5 seconds).

[0034] 3. Poor user experience consistency: The interaction logic (such as swiping direction and button layout) and visual style (color and icons) of the central control screen and the passenger screen are often developed by different teams. Users need to readjust when operating across screens, which increases the learning cost by more than 40%.

[0035] 4. Insufficient scene responsiveness: For example, when parked, users want the passenger screen to expand to "cinema mode", but the existing solution cannot achieve this due to hardware size limitations; when charging, it is necessary to display charging progress and surrounding service information at the same time, but it is difficult to balance the information display priority of the separate screens.

[0036] 5. Lack of access control: The operating permissions of the driver and the front passenger are not clearly distinguished, which poses a risk that the front passenger may accidentally touch the driving control function (such as switching driving modes) or the driver may over-operate the entertainment function while driving.

[0037] This embodiment aims to address issues in existing technologies such as information fragmentation, security vulnerabilities, inconsistent user experience, insufficient scenario adaptation, and lack of access control. Specifically, it includes: 1. How to achieve efficient information flow and sharing between the central control and passenger areas through hardware integration and logical partitioning; 2. How to design a scenario-based dynamic adaptation mechanism so that the display layout and interaction logic can be adjusted in real time according to vehicle status and user needs; 3. How to build a unified interaction system to reduce the cognitive cost for users operating across regions; 4. How to maximize the entertainment value of the passenger seat while ensuring driving safety through hierarchical permission management; 5. How to integrate multimodal interaction technology to provide natural and efficient operation entry points for different users and different scenarios.

[0038] The key technical points and protection points of this embodiment are as follows: 1. Dynamic partitioning technology of integrated display module: The proportion of multiple areas is dynamically adjusted through software-defined virtual dividing lines, which solves the problem that fixed layout cannot adapt to multiple scenarios. The protection points include partitioning logic, proportion adjustment algorithm, and scene-layout mapping relationship.

[0039] 2. Multimodal interaction scenario-based adaptation technology: dynamically adjust the sensitivity, response priority and available function set of each interaction method according to the scenario. Protection points include interaction parameter adjustment rules, sound source directional response mechanism and gesture permission control logic.

[0040] 3. Scene decision engine based on multi-dimensional data: a scene matching algorithm and self-learning optimization mechanism that integrates vehicle status, user data and environmental information. Protection points include scene triggering conditions, weighted scoring model and user behavior iteration logic.

[0041] 4. Identity and status-based hierarchical permission management mechanism: User identity is confirmed through 3D facial recognition, and fine-grained permission control is achieved by combining vehicle status. Protection points include permission matrix design, verification process, and illegal operation interception and prompting methods.

[0042] 5. Cross-regional information flow and collaborative interaction technology: Supports multi-user joint operation and targeted information push in shared collaboration areas. Protection points include information push triggering conditions, collaborative operation synchronization mechanism, and cross-regional data interaction protocol.

[0043] This embodiment can achieve the following: (1) By using an integrated display screen and software-defined dynamic logical partitioning technology, direct sharing of information among multiple users is achieved. The operation time for the co-driver to push route suggestions to the driver is reduced from 15 seconds to 3 seconds, the information flow efficiency is improved by about 80%, and the error rate of multi-person collaborative operation is reduced from 25% to 5%, effectively eliminating the information fragmentation problem caused by the traditional distributed multi-screen layout. (2) When driving, the dynamic expansion of the core driving interaction area and the shrinking of the passenger seat immersive entertainment area reduce the frequency of the driver's eyes deviating from the road by about 40%; multimodal interaction increases the proportion of voice and gesture operation from 30% to 70%, and the time for eye transfer caused by touch operation is shortened from 2.3 seconds to 0.8 seconds; the multi-user permission isolation module has a 100% interception rate for illegal operations, which significantly improves driving safety. (3) The scenario-based dynamic adaptation engine integrates multi-dimensional parameters, uses a weighted scoring algorithm to match the target scenario in real time, and improves the scenario matching accuracy by 5% to 8% per month through a self-learning optimization mechanism. The satisfaction rate of parking and watching movies has increased from 68% to 92%, fully meeting the dynamic adaptation needs of different scenarios. (4) The unified interaction logic and visual language reduce the learning cost of cross-regional operation by about 65%, and the adaptation time for new users is shortened from 10 minutes to 3.5 minutes; the average system response time is no more than 500ms, which is better than the industry average of 1000ms, and the user experience is fully optimized. (5) Dynamic partitioning and scene engine support the addition of new scene types through remote upgrades without the need for hardware modifications, effectively extending the product life cycle and enhancing system scalability.

[0044] The present invention adopts the following technical solution: This embodiment provides a vehicle-mounted human-machine interaction method based on dynamic logical partitioning of an integrated display screen. This method is applied to a vehicle-mounted human-machine interaction system, which includes a display screen covering the driver's operating area and the passenger's operating area. The entire method is executed cyclically in the following order: initialization step S1, data acquisition step S2, scene adaptation step S3, and permission verification and execution step S4. The operation result of step S4 is fed back to step S2 in real time, forming a complete closed-loop control process.

[0045] Step S1: Initialize as follows Figure 2 As shown, after receiving the start signal, the system enters the initialization step S1.

[0046] Before detailing this step, let's first explain the specific design of the integrated display screen: (a) Hardware parameters of the integrated display screen: It adopts an ultra-wide flexible OLED display with a physical size of 1200-1500mm in length (covering the right side of the driver to the front of the passenger seat) and 200-300mm in height, a resolution of ≥3840×1080 (supporting 4K display), a peak brightness of ≥1000nit (ensuring visibility under strong light), a response time of ≤5ms, and supports 10-point touch and HDR display.

[0047] (ii) Logical partitioning design: (1) Driving core interaction area (left side, initial occupancy 35%): Display content includes: vehicle status (vehicle speed, battery level, remaining range, driving mode), navigation information (lane-level route, real-time traffic conditions, traffic light countdown), ADAS prompts (collision warning, lane departure warning), and key vehicle controls (air conditioning fan speed, seat heating shortcut button).

[0048] Interface features: The interface adopts a "black background with white text and high-saturation warning colors" color scheme, with a text font size of ≥16pt (to ensure fast recognition), and the information hierarchy is arranged according to "safety priority" (vehicle speed > navigation > auxiliary prompts).

[0049] (2) Shared Collaboration Area (Central Area, Initially 30%): Display content includes: multi-person trip planning (add / delete waypoints), point of interest recommendations (gas stations, restaurants), vehicle public settings (ambient light color, audio equalizer), and cross-user information push (route suggestions pushed from the passenger to the driver).

[0050] Interactive features: Supports two-finger zoom (map), drag-and-drop sorting (trip points), and uses a fade-in animation when updating information (to avoid interfering with the driver's line of sight).

[0051] (3) Passenger seat immersive entertainment area (right side, initial occupancy 35%): Display content includes: 4K movies and TV shows, online music, interactive games, social applications (video calls, Moments), and personalized information (news, weather).

[0052] Interface features: Supports full-screen / window mode switching, and automatically reduces light reflection on the driver's area when loading entertainment content (through a brightness adaptive algorithm).

[0053] (III) Dynamic Partitioning Mechanism: The boundaries of each area are marked by virtual dividing lines (1px semi-transparent lines). These dividing lines can be dynamically moved according to scene requirements to adjust their proportions. Driving scenario: The driving area is expanded to 40%-50%, and the entertainment area is reduced to 25%-30%; Parking entertainment scene: The entertainment area is expanded to 70%-90%, and the driving area is shrunk to 5%-10% (only parking status is displayed); Charging scenarios: The shared area is expanded to 40% (displaying charging progress + surrounding services), the driving area to 20%, and the entertainment area to 40%.

[0054] It is understood that the integrated display screen in this embodiment is not limited to OLED material; other display technologies such as LCD and Mini LED are also applicable. The core lies in the logical partitioning and dynamic adaptation method.

[0055] In this step, the system logically divides the display screen into three functional areas using software-defined virtual dividing lines: a core driver interaction area, a shared collaboration area, and a passenger-side immersive entertainment area. These virtual dividing lines are presented on the screen as 1-pixel semi-transparent lines, and their positions can be dynamically moved according to subsequent scene requirements, thereby changing the proportion of each functional area.

[0056] In its initial state, the core driver interaction area occupies 30% to 40% of the screen and is located on the left side of the display. It mainly displays vehicle self-check information, including key parameters such as the state of charge (SOC) of the power battery, tire pressure, and driving mode. The shared collaboration area occupies 25% to 35% of the screen and is located in the center of the display. It displays the default welcome screen and suggested commuting routes for the day. The passenger immersive entertainment area occupies 30% to 40% of the screen and is located on the right side of the display. It displays a list of recently used applications for the passenger to quickly access.

[0057] Taking specific figures as an example, the initial proportion of the core driving interaction area is 35%, the initial proportion of the shared collaboration area is 30%, and the initial proportion of the passenger immersive entertainment area is 35%. The total proportion of the three functional areas is 100%, covering the entire physical area of ​​the display screen.

[0058] Meanwhile, the system completes the following operations in initialization step S1: A communication link is established between the multimodal interaction unit and the display screen, and the response delay of the communication link does not exceed 100ms, so as to ensure that user operation commands can be reflected in the changes of the display content in a timely manner. The identity recognition unit is activated, and the identity recognition unit begins to continuously detect the user identity of the driver's seat and the front passenger seat. The initial identity verification is completed within 5 seconds after the system starts up. It supports wake-up-free recognition and does not require active user operation. Initialize each interaction unit of the multimodal interaction unit, including the touch interaction unit, voice interaction unit, gesture recognition unit, and physical button unit, so that each unit enters standby mode and is ready to receive user input data.

[0059] Specifically, the multimodal interaction unit is configured as follows: (1) Touch interaction unit: 1. Zone Sensitivity Control: The touch sensitivity of the driving zone is reduced by 30% while driving (requiring a slightly firmer press), and the range of swipe operations is limited (to avoid distracting with large gestures); the entertainment zone maintains high sensitivity (supporting light touch operations).

[0060] 2. Anti-accidental touch design: Recognizes "accidental hand touch" (ignores operation when the contact area is >50cm²), and sets an "anti-accidental touch buffer zone" (5cm wide) at the edge of the driver's area.

[0061] (2) Voice interaction unit: 1. Hardware configuration: 6 microphone array (arranged in the roof lining), supporting 360° sound source localization (accuracy ±0.3m), noise suppression capability ≥20dB (can filter engine and wind noise).

[0062] 2. Directional Response: Distinguish between driver and passenger commands by sound source localization. For example, if the driver says "Navigate to the airport", only the driver's area will be operated, and if the passenger says "Play music", only the entertainment area will be controlled.

[0063] 3. Enhanced Interaction: Supports contextual understanding (e.g., "Next song" refers to the currently playing content by default), dialect recognition (covering 8 major dialects), and offline command library (supports basic operations when there is no network).

[0064] (3) Gesture recognition unit: 1. Hardware configuration: 2 infrared depth cameras (placed on both sides of the rearview mirror inside the vehicle), frame rate 30fps, recognition distance 0.5-2.5m.

[0065] 2. Gesture library design: Driver-specific gestures: "Clench fist → Release" to switch driving modes (Sport / Comfort), "Slide horizontally" to switch navigation views (2D / 3D); Passenger-specific gestures: "Slide up and down" to adjust volume, "Pinch → Open" to zoom in / out on entertainment content, "Wave" to pause / play; Scenario limitations: Only respond to driver gestures while driving; enable full gestures when parked.

[0066] (4) Physical button unit: 1. Retain core physical buttons: The left side of the steering wheel integrates driving-related buttons (cruise control, ADAS switch), and the right side integrates multimedia shortcut keys (volume, track skipping); the lower part of the center console is equipped with air conditioning adjustment knobs (with press confirmation function) and emergency buttons (hazard lights, SOS).

[0067] 2. Blind operation design: The button surface uses different textures to distinguish them (e.g., raised dots correspond to volume +, and grooves correspond to volume -), and the pressing feedback force is ≥500g (to ensure a sense of confirmation of operation).

[0068] The hardware configuration of the multimodal interaction unit can be adjusted according to cost requirements. Entry-level models can be simplified to a combination of "touch + voice + physical buttons", while high-end models retain full functionality.

[0069] After initialization step S1 is completed, preprocessed initial interaction data and status information are generated and transmitted to data acquisition step S2 to trigger the execution of subsequent processes.

[0070] Step S2: Data collection as follows Figure 2 As shown, the data acquisition step S2 continues to run after the initialization step S1 is completed, and at the same time receives the operation result feedback from the permission verification and execution step S4 to update the system status in real time.

[0071] In this step, the multimodal interaction unit collects three types of data in real time, and stores the collected raw data in a local cache after preprocessing. The local cache retains the data records of the most recent 30 minutes for use by the scene decision algorithm and the permission verification module.

[0072] (1) The first type of data is user input data.

[0073] The touch interaction unit collects the user's touch coordinates, accurate to 1 pixel; The voice interaction unit collects user voice commands through the microphone array sound source localization unit, converts the voice waveform into text, and records the sound source direction information simultaneously. The microphone array sound source localization unit includes no less than 6 microphones, which are arranged in the roof lining and support 360° sound source localization. The sound source localization accuracy is better than ±0.5m, and the noise suppression capability is no less than 20dB. It can effectively filter engine noise and wind noise. The gesture recognition unit collects the user's gesture type and parses the gesture into standardized operation commands; the physical button unit collects the button identifier and press duration signal.

[0074] During the user input data collection process, the microphone array sound source localization unit distinguishes the source of the driver's and passenger's voice commands using a sound source localization algorithm. When a voice command is detected to originate from the driver's seat, the multimodal interaction unit routes the voice command to the core driver interaction area, driving the execution of relevant functions in the driver area. When a voice command is detected to be coming from the direction of the passenger seat, the multimodal interaction unit routes the voice command to the passenger seat immersive entertainment area, driving the relevant functions of the entertainment area to be executed.

[0075] The aforementioned directional routing mechanism ensures that no command confusion occurs when multiple users issue voice commands simultaneously, fundamentally solving the command confusion problem caused by the unified processing of multiple user voice commands in existing technologies.

[0076] (2) The second type of data is vehicle status parameters.

[0077] The following vehicle status parameters are collected via the vehicle bus (CAN bus, Controller Area Network): vehicle speed, with an accuracy of ±1km / h; gear position, including P, R, N, and D; state of charge (SOC) of the power battery, with an accuracy of 1%; and charging gun connection status, including both connected and disconnected states.

[0078] In addition, the number of passengers is collected through seat pressure sensors. When the seat pressure sensor detects that the seat pressure exceeds a preset threshold (usually 15kg), it is determined that there is a passenger sitting in that seat. The vehicle location information is collected through a satellite positioning system (GPS, Global Positioning System), with a positioning accuracy better than ±5m. At the same time, it is combined with high-precision map data to determine the current road type (urban main road, highway, parking lot, etc.).

[0079] (3) The third type of data is environmental information.

[0080] Collect current time information (accurate to the minute), vehicle location information, and user identity information provided by the identity recognition unit.

[0081] The identity recognition unit includes 3D structured light cameras located inside the A-pillars and facing the driver's seat and passenger seat respectively. The identity recognition distance is 0.8m to 1.5m, and the false recognition rate is less than 0.001%. When the identity recognition unit fails to recognize the user, the system defaults to processing the user's operation request as a guest. Guest permissions only allow basic entertainment control and information query operations, and prohibit the execution of driving control functions.

[0082] During the data preprocessing stage, the system performs noise reduction on the collected raw data, filters out single-point touch signals generated by accidental touches (when the detected touch contact area exceeds the preset area threshold of 50cm², the system determines it as a palm touch and ignores the touch input), and converts the voice waveform into a standardized text format for use by subsequent scene decision algorithms.

[0083] After preprocessing, the data acquisition step S2 outputs the processed data to the scene adaptation step S3 and the permission verification and execution step S4, respectively, as the input basis for scene matching and permission verification.

[0084] Step S3: Scene adaptation, such as Figure 2 As shown, the scenario adaptation step S3 is triggered when changes in user operation, vehicle status, or environmental information are detected.

[0085] In this step, the preprocessed data output from the data acquisition step S2 is read through the scenario-based dynamic adaptation engine, the current target scenario is determined from the preset scenario database through the scenario decision algorithm, and the display layout adjustment and multimodal interaction configuration update are driven according to the target scenario.

[0086] (i) The preset scene database contains no less than 12 typical scenes, and each scene includes three elements: triggering conditions, display layout parameters, and interaction method priority.

[0087] For ease of demonstration and explanation, a table showing a portion of the data from the scenario database is provided below: Table 1: Partial data in the scenario database:

[0088] The following lists the main scenarios and their parameters: In urban commuting scenarios, the triggering conditions are weekday morning and evening rush hours (7:00 to 9:00 or 17:00 to 19:00) and the vehicle speed is greater than the preset urban driving speed threshold (30km / h). The core driving interaction area accounts for 40% to 50%, the shared collaboration area accounts for 20% to 30%, and the passenger seat immersive entertainment area accounts for 25% to 35%. The interaction priority is voice interaction, which is higher than physical buttons, which is higher than touch interaction. In high-speed driving scenarios, the triggering conditions are a vehicle speed of not less than 100km / h and a duration of not less than 5 minutes. The core driving interaction area accounts for not less than 50% (emphasizing lane-level navigation and real-time traffic conditions), the shared collaboration area accounts for no more than 15%, the passenger immersive entertainment area accounts for 35% to 45%, and the interaction priority is physical buttons, voice interaction, and gesture interaction. In the parking movie-watching scenario, the trigger conditions are that the vehicle speed is zero, the gear is in P gear, and the user selects the cinema mode. The passenger seat immersive entertainment area accounts for no less than 70% (maximum expandable to 90%), the core driving interaction area accounts for 5% to 10% (only displaying the parking status), and the shared collaboration area accounts for 5% to 20%. The interaction priority is gesture interaction, which is higher than touch interaction, which is higher than voice interaction. In the charging waiting scenario, the triggering conditions are that the charging gun is connected and the state of charge of the power battery is lower than the preset threshold (80%). The shared collaboration area accounts for no less than 35% (the left side displays the charging progress and the estimated full charge time, and the right side displays the nearby points of interest recommendations), the core driving interaction area accounts for 15% to 25%, and the passenger immersive entertainment area accounts for 35% to 45%. In family travel scenarios, the trigger conditions are that at least 3 passengers are detected and it is a non-working day. The shared collaboration area accounts for no less than 30% (used for multi-person trip voting and collaborative planning), the core driving interaction area accounts for 30% to 40%, and the passenger seat immersive entertainment area accounts for 25% to 35%. In parking and waiting scenarios, the trigger conditions are that the vehicle speed is zero, the gear is in P gear, and the cinema mode is not selected. The proportion of each area is restored to the initial value, and the interaction methods are fully open. In nighttime driving scenarios, the trigger condition is that the vehicle's light sensor detects that the ambient brightness is lower than a preset threshold and the vehicle is in motion. The overall brightness of the core driver interaction area is reduced to 60% of that during the day, and the brightness of the entertainment area is reduced to 40% of that during the day, in order to reduce the interference of strong light at night on the driver's vision. In severe weather scenarios, the triggering conditions are that the windshield wipers are running at high speed and the vehicle speed is below 60km / h. The core driver interaction area is expanded to no less than 55%, with ADAS prompts and visibility warning information prominently displayed. In the business travel scenario, the trigger condition is that a passenger is detected in the back seat and the driver is a business user. The shared collaboration area is expanded to 35%, displaying calendar, meeting information and destination navigation. In the sport driving scenario, the trigger condition is that the user actively selects the sport driving mode and the vehicle speed exceeds 80km / h. The core driving interaction area is expanded to 55%, focusing on displaying G-force, power output and track-level navigation information. In the campus pick-up and drop-off scenario, the trigger conditions are that the time is from 15:00 to 17:30 and the location is within 500m of the school. The shared collaboration area displays pick-up and drop-off reminders and parking space information. In the camping mode scenario, the trigger condition is that the user actively selects the camping mode and the vehicle is in a parked state. The entertainment area is expanded to 65%, and the shared area displays information about the surrounding area of ​​the campsite and vehicle energy consumption statistics.

[0089] Figure 3 The results of adapting the basic layout and dynamic scenes to the graphic display screen are shown.

[0090] It is understood that the scenario database in this embodiment can be customized according to the vehicle type positioning (such as commercial vehicle or family car). For example, a "meeting mode" can be added to commercial vehicles (the shared area is expanded into a video conferencing interface).

[0091] (ii) In this embodiment, a scenario decision-making algorithm was designed: Input parameters: vehicle status parameters (vehicle speed, gear, battery level, charging status), user data (identity, number of people, historical behavior), and environmental information (time, location, weather).

[0092] Decision logic: A weighted scoring method is used to assign a score (0-100) to the matching degree of each scenario, and the scenario with the highest score is selected for execution. For example: Saturday 19:00 + shopping mall parking lot + parked car + 2 passengers → "Parking and watching a movie" scene (rating 92); Monday 8:00 AM + main urban road + speed 50 km / h + 1 driver → “Urban Commuting” scenario (score 95).

[0093] The scenario decision-making algorithm uses a weighted scoring method to calculate a matching score for each type of scenario in the preset scenario database. The score ranges from 0 to 100, and the scenario with the highest score is selected as the current target scenario.

[0094] The matching score is calculated based on a weighted combination of vehicle status parameters, user identity data, and environmental information, with the weights of each dimension preset according to the scenario type. Taking an urban commuting scenario as an example: When the system detects that the time is Monday 8:00 AM, the vehicle is located on a main urban road, the speed is 50 km / h, the driver's identity has been verified, and there is 1 passenger: The matching score for urban commuting scenarios is 95 points; The score for the high-speed driving scenario is 12 points because the vehicle speed did not reach 100km / h; The parking and movie-watching scenario receives 0 points because the vehicle speed is not zero. The family travel scenario received a score of 8 because there were fewer than 3 passengers. Therefore, the system determines that the current target scenario is an urban commuting scenario.

[0095] After determining the target scenario, scenario adaptation step S3 performs the following two configuration updates: (1) The first item is the display layout adjustment. The scene-based dynamic adaptation engine drives the integrated display module to adjust the position of the virtual dividing line, thereby changing the proportion of each functional area and updating the display content of each functional area simultaneously.

[0096] Taking the transition from the initial state to an urban commuting scenario as an example: The core driving interaction area has been expanded from the initial 35% to 45%, and the physical width has increased by about 120mm (taking a 1250mm screen as an example). The content of the area has been updated to a real-time traffic overlay map and traffic light countdown information. The shared collaboration area has been reduced from the initial 30% to 25%, and the content of the area has been updated to the estimated travel time for the day. The passenger-side immersive entertainment area has been reduced from its initial 35% to 30%, and the content in the area has been switched to a morning news text version.

[0097] The above-mentioned area switching process is presented using a gradual transition animation, with a transition time of no more than 300ms, to avoid sudden changes causing visual interference to the driver.

[0098] (2) The second item is the multimodal interaction configuration update.

[0099] Dynamically adjust touch sensitivity, gesture recognition range, and response priority of each interaction method according to the current target scenario.

[0100] The specific rules are as follows: In high-speed driving scenarios, the wake-up sensitivity of voice interaction is increased by 20% to adapt to the in-vehicle noise environment at high speeds (in-vehicle wind noise is about 65dB, engine noise is about 70dB), the touch response time of the core driver interaction area is extended to 200ms to reduce the probability of misoperation, and the gesture recognition unit only enables the driver's preset gesture command set (including "slide horizontally" to switch navigation views and "clench and release fist" to switch driving modes), and disables other gesture commands to avoid accidental triggering. In the scenario of watching movies while parked, gesture interaction takes priority over voice interaction, and the full set of gesture commands for the driver and front passenger is enabled, while the touch interaction unit is restored to high sensitivity mode. In urban commuting scenarios, voice interaction is set to the highest priority, the touch sensitivity of the core driving interaction area is reduced by 30%, and the anti-accidental touch buffer is maintained at a width of 5cm.

[0101] In addition, the touch sensitivity of the core driving interaction unit in driving mode is lower than that in parking mode, and an anti-mistouch buffer is set at the edge of the core driving interaction area with a width of 5cm. When the detected touch contact area exceeds the preset area threshold (50cm²), the system determines it as a palm touch and ignores the touch input.

[0102] (III) The scenario-based dynamic adaptation engine also possesses a self-learning optimization mechanism based on user behavior data. Self-learning optimization: (1) Data collection: Record the user's manual switching of scenes (such as forcibly switching from "commuting" to "entertainment") and the click rate of recommended content.

[0103] (2) Iteration mechanism: The scene trigger threshold is updated weekly based on user behavior data (e.g., if users often watch movies while parked on Friday nights, the trigger threshold for "parking and watching movies" during that time period is lowered), so that the scene matching accuracy rate is increased by 5%-8% per month.

[0104] The self-learning optimization mechanism records the user's manual switching of scenes and the user's response data to recommended content (including click-through rate, dwell time, etc.). According to the preset time period (once a week), it updates the trigger threshold weight of each scene based on the user behavior data, so that the scene matching accuracy continues to improve with the usage time, with a monthly increase of 5% to 8%.

[0105] For example, if the system detects that a user frequently switches to the parking and movie-watching scenario while parking on Friday evenings, the self-learning mechanism will lower the triggering score threshold for the parking and movie-watching scenario during that period in the next iteration, so that the system can proactively recommend and switch to the parking and movie-watching scenario under the same conditions, reducing the number of times the user manually operates.

[0106] Step S4: Permission verification and execution as follows Figure 2 As shown, the permission verification and execution step S4 simultaneously receives user input data from the data acquisition step S2 and target scene instructions from the scene adaptation step S3, verifies the legality of the user operation request, and executes the corresponding operation or returns an interception prompt based on the verification result.

[0107] During the operation request parsing phase, the system converts user input (such as touch coordinates, voice commands, or gesture types) into standardized operation instructions. These standardized operation instructions include three elements: the operation object (such as the full-screen button in the entertainment area), the function type (such as the full-screen entertainment operation), and the initiator's identity (provided by the identity recognition unit).

[0108] The object being manipulated is determined by mapping touch coordinates to a functional area map; the function type is determined by the object's functional attribute tags; and the initiator's identity is determined by the real-time identification result of the identity recognition unit. Specifically, the identity recognition unit: (1) Hardware: Two 3D structured light cameras are installed on the inside of the A-pillar (facing the driver's seat and the passenger seat respectively), with a recognition distance of 0.8-1.5m and a false recognition rate of <0.001%.

[0109] (2) Identification logic: The system completes authentication within 3 seconds after startup and supports "wake-up-free" identification (no user operation required); if no user is identified, it will be treated as "guest" by default.

[0110] During the permission verification phase, the multi-user permission isolation module determines the legality of the operation by querying a preset permission matrix based on the function type and initiator identity in the standardized operation command, combined with the real-time vehicle status (vehicle speed, gear). The permission matrix design is shown in Table 2 below: Table 2 shows the permission matrix:

[0111] The preset permission matrix is ​​configured according to two dimensions: user identity and vehicle status. The specific rules are as follows: While in motion (vehicle speed greater than 0 km / h), the driver is allowed to perform driving control functions (such as switching driving modes) and basic navigation operations (such as setting a destination), but is prohibited from performing full-screen entertainment operations; When the vehicle is parked (vehicle speed is 0 and gear is P), the driver is allowed to perform driving control functions, full-function navigation operation, and full-screen entertainment operation. The front passenger is prohibited from performing driving control functions under any vehicle status. They are allowed to perform full-screen entertainment operations, and navigation operations are limited to pushing route suggestions to the core driver interaction area. Direct modification of the navigation destination is not allowed. Guest users are only allowed to perform basic entertainment control and information query operations under any circumstances.

[0112] Access control supports user-defined configurations (security authentication required), for example, the driver can authorize the passenger to adjust the air conditioning in the whole vehicle while driving (by default, only the passenger area can be adjusted).

[0113] The permission verification process is briefly described below: 1. User initiates an action (e.g., clicking the "Full Screen" button in the entertainment area); 2. The system obtains the user's identity (through camera recognition) and vehicle status (speed, gear). 3. Match the permission matrix to determine the legality of the operation; 4. If the operation is legal, it will be executed; otherwise, a prompt will pop up (such as "Full-screen mode is prohibited while driving. Please pay attention to safety"). The prompt will be placed in the current user's operating area (to avoid disturbing others).

[0114] Let's take a specific scenario as an example to illustrate the permission verification process: When the front passenger clicks the "full screen" button in the entertainment area while the vehicle is in motion (vehicle speed 50km / h), the system obtains the front passenger's identity information (registered front passenger) and the current vehicle speed (50km / h, greater than 0km / h), generates a standardized operation command (function type: entertainment full screen operation, initiator: front passenger, vehicle status: in motion), and after querying the permission matrix, confirms that the verification result of this operation type is "prohibited", so the verification fails. When the driver clicks the "Full Screen" button in the entertainment area while the vehicle is parked (vehicle speed 0 km / h, gear P), the system obtains the driver's identity information and the current vehicle status (vehicle speed 0 km / h, gear P), generates a standardized operation command (function type: entertainment full screen operation, initiator: driver, vehicle status: parked), and after querying the permission matrix, confirms that the verification result of this operation type is "allowed", so the verification passes.

[0115] During the result execution and feedback phase, the system performs the following operations based on the permission verification results: For operations that pass verification, the system executes the corresponding function and displays a lightweight feedback prompt in the operation area (such as "Full Screen" which disappears automatically after 3 seconds). For operations that fail verification, the system displays the reason for the operation blocking in a prominent prompt message in the function area where the initiator is located (such as "Full screen is prohibited while driving, please pay attention to safety"). The prompt message is only displayed in the function area where the initiator is located and does not spread to other function areas to avoid disturbing other users. At the same time, the prompt content is broadcast synchronously through the voice output unit of the multimodal interaction module.

[0116] After the permission verification and execution step S4 is completed, the result of this operation (including the execution result or interception record) is fed back to the data collection step S2 in real time to update the current status of the system and provide an accurate status basis for the next scenario judgment and permission verification, thus forming a complete closed-loop control process of "data collection → scenario adaptation → permission verification and execution → data collection".

[0117] This embodiment achieves efficient and secure interaction through a four-step closed loop of "initialization - data acquisition - adaptation - verification": 1. Initialization Phase (S1) After the system starts up, the integrated display screen completes three configurations using default parameters: (1) Zone division: 35% for driver core interaction zone, 30% for shared collaboration zone, and 35% for passenger immersive entertainment zone; (2) Content loading: The driving area displays vehicle self-check information (battery level, tire pressure), the sharing area displays the default welcome screen, and the entertainment area displays a list of recently used applications; (3) Module linkage: Establish a communication link between the multimodal interaction unit and the display module (response delay ≤ 100ms), and activate the identity recognition unit to start user detection.

[0118] 2. Data Acquisition Phase (S2) (1) The multimodal interaction module collects three types of data in real time: 1. User input: Touch coordinates (accurate to 1px), voice commands (text conversion + sound source direction), gesture type (parsed into operation commands), physical button signals (button ID + press duration); 2. Vehicle status: Vehicle speed (accuracy ±1km / h), gear (P / R / N / D), battery level (SOC value), and charging status (connected / disconnected) are obtained via CAN bus. 3. Environmental and User Information: Time (accurate to the minute), Location (GPS positioning, accuracy ±5m), User Identity (camera recognition results), Number of Passengers (seat pressure sensor detection).

[0119] (2) Data preprocessing: Noise reduction (such as filtering out accidental single-point touches) and format conversion (such as converting speech waveforms into text) are performed on the collected data, and the data is stored in the local cache (keeping the most recent 30 minutes of data).

[0120] 3. Scenario-based dynamic adaptation stage (S3) (1) Scene matching: The scenario-based dynamic adaptation engine reads the preprocessed data from S2, inputs it into the scenario decision algorithm, calculates the matching score for each scenario, and selects the scenario with the highest score as the current target scenario.

[0121] (2) Layout adjustment: Based on preset parameters for the target scenario, the integrated display module adjusts its area proportion, for example, when switching from "Initialization" to "Urban Commuting": The driver's area has been expanded from 35% to 45% (width increased by 120mm); The shared area has been reduced from 30% to 25%. The entertainment area was reduced from 35% to 30%; The content of each area is updated synchronously (the driving area loads real-time traffic conditions, and the shared area displays the schedule).

[0122] (3) Interaction configuration: Adjust the response parameters of the multimodal interaction module, for example, in the "high-speed driving" scenario: The wake-up sensitivity of voice interaction is improved by 20% (adapting to high-speed noise). The touch response time in the driver's area has been extended to 200ms (reducing accidental operation). Gesture recognition is only enabled for the driver's "switch navigation view" gesture.

[0123] 4. Permission verification and execution phase (S4) (1) Operation request parsing: Convert user input (such as "click the full screen button in the entertainment area") into standardized operation instructions (including operation object, function type, and initiator identity).

[0124] (2) Permission verification: The multi-user permission isolation module queries the permission matrix based on the operation command and judges the validity in combination with the current vehicle status: Example 1: The front passenger clicks "Drive Mode Switch" while the vehicle is in motion → "Front Passenger Prohibited" in the permission matrix → Verification failed; Example 2: When the driver is parked, click "Entertainment Area Full Screen" → "Driver Parking Allowed" in the permission matrix → Verification passed.

[0125] (3) Results feedback: Legitimate operation: Perform the function (such as displaying a movie in full screen) and display lightweight feedback in the operation area (such as a "Full screen" prompt that disappears after 3 seconds); Illegal operation: A prompt box (red border + text description) will pop up in the initiator's operation area, and a voice prompt will be given (such as "This function can only be operated by the driver").

[0126] The following sections will use several specific application scenarios as examples to explain in detail the specific implementation steps of the above method.

[0127] (1) Scene 1: Taking the complete interaction process in an urban commuting scenario as an example, the practical application of the above four-step closed-loop method is illustrated: At 7:30 on a certain weekday, the driver starts the vehicle, and the system enters the initialization step S1, completing the division of the three areas (driving core interaction area 35%, shared collaboration area 30%, and passenger immersive entertainment area 35%) and the establishment of module linkage (communication link response delay 85ms, meeting the requirement of not exceeding 100ms). The identity recognition unit completes the driver's identity verification within 3 seconds. When the vehicle travels to the main urban road and the speed reaches 50km / h, data collection step S2 collects data such as the driver's voice command "Navigate to the company" (the microphone array confirms that the sound source is from the driver's seat, and the sound source location is ±0.2m away from the driver's seat), time 7:40, and location on the main urban road. The driver's voice command is then routed to the core driving interaction area. In the scene adaptation step S3, the matching score for the urban commuting scene is calculated to be 95 points. The core driving interaction area is expanded to 45% and real-time traffic conditions and traffic light countdown are loaded. The shared collaboration area is shrunk to 25% and the estimated travel time is displayed. At the same time, voice interaction is set to the highest priority and the touch sensitivity of the core driving interaction area is reduced by 30%. The front passenger then clicks the "Full Screen Video" button in the entertainment area. The permission verification and execution step S4 queries the permission matrix and confirms that the front passenger's full screen operation in the entertainment area is prohibited while driving. An interception prompt pops up in the entertainment area: "Full screen is prohibited while driving. Please pay attention to safety." The operation result (interception record) is fed back to the data collection step S2 to complete this round of closed loop.

[0128] (2) Scene 2: Taking the complete interaction process in a parking entertainment scenario as an example, the practical application of the above four-step closed-loop method is illustrated: 1. Scene Trigger: At 19:00, the vehicle enters the shopping mall parking lot (speed 0, P gear), and S2 collects "parked + front passenger present + time 19:00".

[0129] 2. Scene adaptation: S3 matches "parking entertainment scene", drive display adjustment: entertainment area expanded to 80% (width 1000mm), driving area 5% (displaying battery 70%), sharing area 15% (displaying "cinema mode: brightness 100%, volume 50%)".

[0130] 3. Multimodal interaction: The front passenger makes an "open" gesture → the gesture recognition unit interprets it as "zoom in" → the entertainment area screen zooms in from 70% to 100%; The driver says "Play the movie I didn't finish watching last week" → The voice unit locates the driver's seat and verifies that permissions are allowed while the vehicle is parked → The entertainment area loads the movie content; The passenger can increase the volume from 50% to 70% by touching the "Volume+" button in the shared area, then verifying permissions and allowing access.

[0131] 4. End process: Driver engages D gear → S2 collects "driving status" → S3 switches to "city commuting" scene → entertainment area shrinks to 30%, driving area recovers to 45%.

[0132] (3) Scene 3: Taking the complete interaction process in a charging scenario as an example, the practical application of the above four-step closed-loop method is illustrated: 1. Scenario trigger: The vehicle is connected to a fast charging station (SOC 30%), and S2 collects "charging status + location charging station + time 14:00".

[0133] 2. Scene adaptation: S3 matches "charging scene" and the display is adjusted as follows: Shared area 40% (left side shows charging progress 30% → estimated full charge time 45 minutes, right side shows nearby shopping malls / restaurant recommendations), driving area 20% (displays battery temperature and charging current), entertainment area 40% (displays video applications).

[0134] 3. Collaborative Operation: The passenger clicks on "Shopping Mall A" in the shared area → permission verification is allowed → the shared area displays a floor plan of Shopping Mall A, and at the same time pushes a "Go to Shopping Mall A?" prompt to the driver's area; The driver confirms the selection via a physical button on the steering wheel → the driving area records the selection, and the shared area updates the navigation route (charging station → shopping mall A). In the passenger seat, select "TV series" in the entertainment area and play it in full screen (if permissions allow), while the charging information will remain displayed in the driver's area.

[0135] This embodiment has the following specific technical advantages: (a) Direct technical effects: 1. Improved information interaction efficiency: The integrated screen enables direct information sharing. The operation time for the co-pilot to push route suggestions to the driver is reduced from 15 seconds in the traditional solution to 3 seconds, improving efficiency by 80%. The error rate of collaborative operation in multi-person trip planning has been reduced from 25% to 5%.

[0136] 2. Enhanced driving safety: With the driving area expanding and the entertainment area shrinking while driving, the frequency of the driver's gaze deviating from the road decreases by 40%. Multimodal interaction increased the proportion of voice / gesture operations from 30% to 70%, and reduced the eye-shift time caused by touch operations from 2.3 seconds to 0.8 seconds; The access control interception rate reached 100% (no cases of the passenger accidentally operating the driving function occurred).

[0137] 3. User experience optimization: Unified interaction logic reduces the learning cost of cross-regional operation by 65% ​​(new user adaptation time is reduced from 10 minutes to 3.5 minutes). Scenario-based adaptation meets different needs, and the expanded entertainment area when parked increases the satisfaction rate of the movie viewing experience to 92% (compared to 68% for the traditional passenger-side screen). The average response time for multimodal interaction is ≤500ms (industry average 1000ms), significantly reducing the user's waiting experience.

[0138] (ii) Indirect technical effects: 1. Enhanced product competitiveness: This design increased the "intelligence" and "personalization" ratings of the cockpit interaction by 35% in user surveys, helping to enhance the vehicle's premium pricing power (the final selling price is expected to increase by 5%-8%).

[0139] 2. Optimized development efficiency: The unified hardware and software platform of the integrated screen reduces the development workload of multi-screen adaptation, shortens the development cycle of the cockpit interaction system by 20%, and reduces testing costs by 15%.

[0140] 3. Enhanced functional scalability: Dynamic partitioning and scene engine support the addition of new scenes (such as "camping mode" and "office mode") via OTA upgrades without requiring hardware modifications, thus extending the product lifecycle.

[0141] Example 2 This embodiment discloses an in-vehicle human-machine interaction system based on dynamic partitioning of an integrated display screen.

[0142] The in-vehicle human-machine interaction system based on dynamic partitioning of an integrated display screen includes: The initialization module is configured to initialize the display of an integrated display screen that dynamically displays three functional areas: the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area. The first human-computer interaction module is configured to: acquire user input data, vehicle status parameters and environmental and user information in multimodal human-computer interaction methods, and use a scenario decision algorithm to determine the current target scenario from a preset scenario database, and drive the adjustment of the proportion of each functional area according to the current target scenario; The operation verification module is configured to: receive user operation requests, identify user identity information and real-time vehicle status, query a preset permission matrix, and perform legality verification on user operation requests based on user identity information and real-time vehicle status. The second human-machine interaction module is configured to execute the corresponding function for operations that pass verification, and to provide a prompt in the function area where the initiator is located for operations that fail verification, thereby realizing the in-vehicle human-machine interaction of dynamic partitioning of the integrated display screen.

[0143] exist Figure 4 The document showcases the specific layered architecture of integrated display, multimodal interaction, and hardware. The integrated display dynamically partitions the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area; the multimodal interaction includes multiple modal interaction units, a scenario-based dynamic adaptation engine, and multi-user permission isolation.

[0144] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0145] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in Embodiment 1 of this disclosure.

[0146] Example 4 The purpose of this embodiment is to provide an electronic device.

[0147] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in Embodiment 1 of this disclosure.

[0148] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0149] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0150] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A vehicle-mounted human-machine interaction method based on dynamic partitioning of an integrated display screen, characterized in that: Includes the following steps: Initialize the display of the integrated display screen that dynamically displays three functional areas: the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area; The system acquires user input data, vehicle status parameters, and environmental and user information from multimodal human-computer interaction methods, and uses a scenario decision algorithm to determine the current target scenario from a preset scenario database, and drives the adjustment of the proportion of each functional area based on the current target scenario. Receive user operation requests, identify user identity information and real-time vehicle status, query the preset permission matrix, and perform legality verification on user operation requests based on user identity information and real-time vehicle status. For operations that pass verification, the corresponding function is executed; for operations that fail verification, a prompt is displayed in the function area where the initiator is located, realizing in-vehicle human-machine interaction with dynamic partitioning of the integrated display screen.

2. The in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in claim 1, characterized in that, The shared collaboration area supports multi-user collaborative operation, including: The driver and front passenger can jointly add and delete points along the route. The passenger sends route suggestions to the core driver interaction area. The driver can confirm or ignore the pushed content via physical buttons or voice commands; The content updates in the shared collaboration area are presented using a gradual transition method. or, During dynamic partitioning: The display screen is divided into three functional areas by software-defined virtual dividing lines: the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area. The position of the virtual dividing lines can be dynamically adjusted to change the proportion of each functional area. or, During the initial display process, the initial proportion of the driving core interaction area is 30% to 40%, the initial proportion of the shared collaboration area is 25% to 35%, and the initial proportion of the passenger immersive entertainment area is 30% to 40%.

3. The in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in claim 1, characterized in that, The multimodal human-computer interaction methods include touch interaction, voice interaction, gesture recognition interaction, and physical button interaction; The user input data in the multimodal human-computer interaction method includes touch coordinates, voice commands, gesture types that can be parsed into operation commands, and physical button signals; The vehicle status parameters include vehicle speed, gear, battery level, and charging status; The environmental and user information includes time, location, weather information, user identity, and number of passengers; The scenario decision-making algorithm adopts a weighted scoring method, specifically: For each type of scenario in the preset scenario database, a matching score is calculated based on a weighted combination of user input data, vehicle status parameters, and environmental information, and the scenario with the highest score is selected as the current target scenario. The weights for each dimension are preset according to the scenario type.

4. The in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in claim 2, characterized in that, The preset scene database contains no fewer than 12 typical scenes, each scene including trigger conditions, display layout parameters and interaction method priority; after determining the current target scene, the position of the virtual dividing line is adjusted to change the proportion of each functional area, and the display content of each functional area is updated synchronously. The preset scene database contains 12 typical scenes, including the following scenes and their corresponding display layout parameters: In urban commuting scenarios, the triggering conditions are weekday morning and evening rush hours and vehicle speed exceeding the preset urban driving speed threshold, with the core driving interaction area accounting for 40% to 50%; In high-speed driving scenarios, the triggering conditions are a vehicle speed of not less than 100 km / h and a duration of not less than 5 minutes, with the core driving interaction area accounting for not less than 50%; In a parked car movie-watching scenario, the trigger conditions are that the vehicle speed is zero, the gear is in P gear, and the user selects cinema mode, with the passenger seat's immersive entertainment area accounting for no less than 70%; In the charging waiting scenario, the triggering conditions are that the charging gun is connected and the state of charge of the power battery is lower than a preset threshold, and the proportion of the shared collaboration area is not less than 35%. For family travel scenarios, the triggering conditions are that at least 3 passengers are detected and it is a non-working day, with the shared collaboration area accounting for no less than 30%.

5. The in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in claim 3, characterized in that, It also includes updating the multimodal interaction configuration based on the current target scene, wherein the multimodal interaction configuration includes touch sensitivity, gesture recognition range, and response priority of each interaction method; The multimodal interaction configuration update specifically includes: In high-speed driving scenarios, improve the wake-up sensitivity of voice interaction, extend the touch response time of the core driving interaction area, and only enable the driver's preset gesture command set; In the scenario of watching movies while parked, gesture interaction takes priority over voice interaction, and the full set of gesture commands for the driver and front passenger is available. In the multimodal interaction configuration update, the touch sensitivity of the driving core interaction area in the driving state is lower than that in the parking state, and an anti-mistouch buffer is set at the edge of the driving core interaction area; when the detected touch contact area exceeds the preset area threshold, the current touch input is ignored.

6. The in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in claim 1, characterized in that, The user operation requests include driving control, navigation operation, full-screen entertainment, vehicle settings, and editing of shared content; The preset permission matrix is ​​configured according to two dimensions: user identity information and real-time vehicle status, including: While in motion, the driver is allowed to perform driving control functions and basic navigation operations, but is prohibited from performing full-screen entertainment operations. When the vehicle is parked, the driver is allowed to perform driving control functions, full-function navigation operation, and full-screen entertainment operation; The front passenger is prohibited from performing driving control functions under any vehicle status, but is allowed to perform full-screen entertainment operations, and navigation operations are limited to pushing route suggestions to the core driving interaction area.

7. The in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in claim 1, characterized in that, When the verification fails, the reason for the operation block will be displayed in the function area where the initiator is located with an identified prompt message, and the prompt content will be broadcast simultaneously. or, It also includes self-learning optimization, specifically: Acquire user behavior data, including user actions of manually switching scenes and user response data to recommended content; According to the preset time period, the trigger threshold weights of each scenario are updated based on user behavior data, so that the scenario matching accuracy continues to improve with the usage time.

8. A vehicle-mounted human-machine interaction system based on dynamic partitioning of an integrated display screen, characterized in that: include: The initialization module is configured to initialize the display of an integrated display screen that dynamically displays three functional areas: the core driver interaction area, the shared collaboration area, and the passenger immersive entertainment area. The first human-computer interaction module is configured to: acquire user input data, vehicle status parameters and environmental and user information in multimodal human-computer interaction methods, and use a scenario decision algorithm to determine the current target scenario from a preset scenario database, and drive the adjustment of the proportion of each functional area according to the current target scenario; The operation verification module is configured to: receive user operation requests, identify user identity information and real-time vehicle status, query a preset permission matrix, and perform legality verification on user operation requests based on user identity information and real-time vehicle status. The second human-machine interaction module is configured to execute the corresponding function for operations that pass verification, and to provide a prompt in the function area where the initiator is located for operations that fail verification, thereby realizing the in-vehicle human-machine interaction of dynamic partitioning of the integrated display screen.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the in-vehicle human-machine interaction method based on dynamic partitioning of an integrated display screen as described in any one of claims 1-7.

Images